View Full Version : Anti debugging technique?

October 20th, 2013, 19:09
I was playing a CTF which had a binary (PE32 exec) in which there's a hidden flag. There's quite a few anti-debug/disasm/logic things in there.. which make life difficult. I got a few of them but am now stuck . As in... I can keep poking and have not tried all possibilities but wanted to check with you guys if this was a common technique. I'll try and explain.

There's a lot of self modifying code in there which I kinda managed to unpack, I think. It extracts itself into memory in the 370000 range. Here there are a few more problems:

a) I can see an entire PE Exec in memory. So I tried Binary copy in Olly from MZ till the last byte before 00 00 00 starts. But that flopped and Olly said .. 'This is not an EXE want to load it anyway? And opened a terminal with ntvdm.exe... whatever that means (Not yet Googled ). Tried dumping using a couple of plugins...but must be doing something wrong. What's the best way to proceed? It's all in memory mind you... so bit of a pain to unpack each each time by setting 1 million breakpoints and F8 little by little.

b) If then seems to dump an address onto the stack as follows and use that address as the 2nd argument to VirtualAlloc later on. Then it tries MOV to this address + 3C or something... which is non existent... so I NOP'd that. Then there is the VirtualAlloc call which succeeds..


.... but after that there is a REP MOV EDI ESI type instruction and it tries to copy a huge huge number of bytes into the newly allocated memory. Eventually I get an access violation error. I tried playing with the sizes and NOPping bits out .. and patching EAX, EDX etc at runtime but it keeps failing at different points. So the question is... is this a common anti-debugging technique too? And if yes.. what's the best way to approach it .. apart from F8

There's a while to go for the CTF to end.. so I can't yet upload the binary...but I will once it's over. Before that.. any help you guys can give would be great.

I attached a screenshot of what I see as teh value of ECX just before the Copy of the unpacked code. The number is insanely huge; I tried looking at memory...calculating the size and using that size instead..by patching ECX before the REP.. but that caused the VirtualAlloc to fail. This was the code.


I'll keep trying and update you all if I find a solution. In the meanwhile ..



October 20th, 2013, 22:38
wow thats playing blind like a blind from what you describe it looks like you are a true follower of the maxim

when you have a nop every byte looks like a patch

ntvdm = new technology virtual dos manager

it allows old dos and ne com executables to run in nt (a virtual machine that traps INT XX calls and emulates them as Nt does not allow access to interrupts from

user mode)

so the code executing might be a dos exe

+3c is usually a very very strong indicator of elfaw_new in IMAGE_DOS_DIRECTORY

it usually points to the PE header offset

and repmovsb/w/d/ are simple assembly instuctions to copy a range of bytes from somewhere to somewhere else

esi holds source buffer addres
edi holds destination buffer address
ecx holds the count

copies ecx no(b/w/d) of source at esi (b/w/d) to destination at edi (b/w/d)

it probably allocates memory transfers the decrypted content to newly allocated memory edits pe header and executes the code from memory

you would need to trap the entry point and dump from there and possibly look for code that corrupts the headers prior to execution of new code in order to thwart dumping

nopping is not a solution to every problem in hand

try analysing and making notes one block at a time and create a flow chart you should have to understand the basic principle and solve it not apply lame craker techniques like nop and jump

it might be far smarter than your nop and jumps

from the snap it seems ecx got its value from [ebp-24] (local variable) find who initialises it and where and how ?

October 21st, 2013, 00:36
Thanks blabberer . I didn't get the blind blind joke though

It's not that I went on a NOP spree.. I did analyze and was NOPping only specific parts that were pointing to non existent mem locations. It's a PE header for sure.. I confirmed that but the copy and paste didn't kinda work... I'll check again why. It's a good point about +3C thanks... I'll play with the addresses accordingly.

You're also bang on that it's a decryptor/decompresser... VirtualAlloc succeeds and copies a huge bunch of code to the allocated address. Only it copies too much and crashes the program with an access violation... and the way it's getting the length is just weird. It gets it at the start of the routine...stores in EBP-44...and copies it from there at the time of the call. Nothing changes inbetween.

It still copies encrypted code though... which is all fine if I can proceed. But it only crashes

October 21st, 2013, 02:35
I noticed that 0x7c809b49 is the address of the final RET in VirtualAllocEx in XPsp3. What is interesting is that it is by default the value in ECX and is pushed onto the stack from there as the return value from the immediately preceding subcall, and *remains* as the ECX value even after return from VirtualAlloc -> VirtualAllocEx.

Put a BP on VirtualAllocEx in notepad (then Open a new file to trigger) for example to see what I mean.

So if you're also using XPsp3, having ECX=0x7c809b49 after a VirtualAlloc call seems normal, until it gets changed. The question then is, why isn't it changed before the rep movsd copy routine? Seems like that might be a nice little trick to play on someone who has triggered an antidebug flag somewhere previously

October 21st, 2013, 02:47
ah you didn't get the joke then ask google god to shower you with the information pray hammer nail phrase

so it gets ecx from start of routine and stores it at [ebp-44]
that should be something like

func_make_donkey (int monkey , bool)
int donkey = default;
if (bool == false)
donkey != monkey;
donkey == ass;


look at the previous routine (call stack / breaking earlier / tracing / logging / leeching / tedious single stepping / whatever it takes )
whatever it takes methodology includes hand modifying ecx to be say size of virtual alloc and letting the function succeed and getting to know where else it starts crashing / toggling the flags one at at time / simply bypassing the routine hand filling the buffer with asdfsffdfdfdfdfdf and seeing if it executes your asdffhfhf blah blah blah blah)

October 21st, 2013, 10:31
@Kayaker: I am using XP-SP3 yes . I'll check with Notepad too..thnx. Isn't the 2nd argument to VirtualAlloc supposed to be a size argument? As in... how much should it copy from src to dest? And ECX before the REP MOV? That's size too..rt? Why is it then a return address? Quoting MSDN:

dwSize [in]

The size of the region, in bytes. If the lpAddress parameter is NULL, this value is rounded up to the next page boundary. Otherwise, the allocated pages include all pages containing one or more bytes in the range from lpAddress to lpAddress+dwSize. This means that a 2-byte range straddling a page boundary causes both pages to be included in the allocated region.

But you're right..maybe I've triggered some other debugging flag and have to start F8ing again.

@blabberer: Haha got it now....no no I wasn't doing something as blind as that. I'll check for a routine like that at the start. Although..all I could see is was this..it comes to an instruction, pops it on to the stack and uses that later. There's some antidebug just after the pop which I fixed (else it goes into an access violation loop again). Here's a screenshot of the POP.

... and then it gets used in the VirtualAlloc as is...

... and points to a non existent address

I'll continue in another reply as it's not letting me attach more screenshots

October 21st, 2013, 10:36
...now if I do a little nopping to force it to go on..the virtual alloc call succeeds for a start.. and returns a 900000 which is filled with zeros.

... go on.. ECX filled with the address as Kayaker mentioned - 7C809B49. Then the REP starts.. 900000 gets filled up with lot of content as expected. ECX decrements to 7C7Fe2e7 and then crashes.

NOt letting me put more screenshots though.Weird. ANyway..hope it's clear. Sorry

October 22nd, 2013, 10:50
whatever you posted seems to look crystal clear
@91 seems to start with push ebp

so this should be the start of the procedure / block / xyz noramlly (not fpo omitted so arguments and local variables should be available without any problems)

it creates space in stack for 0x13 local variables (sub esp,4c 4c /4 == 0x13)

and executes a magic function which might be void mymaggie()

could be something like

int mymagicarray[0x13];
or some thing like
Mystruct crap;
or could be long a ; ulong c; quad d; etc etc etc
call Mymaggie();

Mymagggie pssibly sets a local variable to some value which is popped ecx (the jump je will never be taken it is junk instruction)

before popping [ebp-4] holds 3000 and after popping [ebp-44] will hold 37079e

which seems to be bogus value clearly find out what else [ebp-44] can contain and you will get your clue

November 26th, 2013, 01:18
Thank you all for all your help. I finally got time to complete the challenge. The application was unpacking stuff into memory, and then searching for a PE header in memory. It searched for 5A 4D in the unpacked code. There were 2 occurences of 5A 4D. I was stuck on the first one last time and didn't get into the 2nd occurence. I'd set a conditional breakpoint too but it wasn't getting triggered so I was wondering...why.

Then I found out that at the point of the second occurence, due to my putting a breakpoint at CMP EDX, 5A4D or just above (didnt check exact location) - EDX was getting a value 5A CC and the CMP was failing. So I patched EDX after the compare.. switched the zero flag and everything worked after that

It turned out to be reasonably simple...but I quite enjoyed it.

I wrote a little bit about it here. Do pass on feedback if any - http://ardsec.blogspot.com/2013/11/emc-defenders-ctf-week-3-contest-14.html