From bc7842e301f39df5cbe1bbb1da29831021fd7a7f Mon Sep 17 00:00:00 2001 From: Ch40zz Date: Wed, 23 Jun 2021 18:17:36 +0200 Subject: [PATCH] Allow unwinding of qvmcall64 call-stub for msvc/masm qvmcall64 has its own custom calling convention due to pushing all non-volatile registers to the stack. The game uses set/longjmp which on Windows uses "RtlUnwindEx" to unwind the callstack. qvmcall64 cannot be unwound by default due to the custom calling convention. To allow unwinding, we need to add custom SEH unwind data to the function. --- code/asm/vm_x86_64.asm | 18 +++++++++++++++++- 1 file changed, 17 insertions(+), 1 deletion(-) diff --git a/code/asm/vm_x86_64.asm b/code/asm/vm_x86_64.asm index f39289ec..f84eb8f8 100644 --- a/code/asm/vm_x86_64.asm +++ b/code/asm/vm_x86_64.asm @@ -28,19 +28,35 @@ ; Call to compiled code after setting up the register environment for the VM ; prototype: ; uint8_t qvmcall64(int *programStack, int *opStack, intptr_t *instructionPointers, byte *dataBase); +; +; This call-stub has its own custom calling convention due to pushing all non-volatile registers +; to the stack. The game uses set/longjmp which on Windows uses "RtlUnwindEx" to unwind the callstack. +; This function cannot be unwound by default due to the custom calling convention. +; To allow unwinding, we need to add custom SEH unwind data to the function. -qvmcall64 PROC +qvmcall64 PROC FRAME push r12 ; push all non-volatile registers to stack + .pushreg r12 push r13 + .pushreg r13 push r14 + .pushreg r14 push r15 + .pushreg r15 push rdi + .pushreg rdi push rsi + .pushreg rsi push rbx + .pushreg rbx push rbp + .pushreg rbp ; need to save pointer in rcx so we can write back the programData value to caller push rcx + .pushreg rcx + + .endprolog ; custom unwind data ends here ; registers r8 and r9 have correct value already thanx to __fastcall xor rbx, rbx ; opStackOfs starts out being 0