Results 1 to 12 of 12

Thread: x64 Memory Mapped File Execution Issue

  1. #1

    Exclamation x64 Memory Mapped File Execution Issue

    Hello all,

    I am wondering why my example code below works perfectly fine on x86 Windows NT OS (XP SP2 and above) compiled as a 32-bit binary and when compiled as a 64-bit binary it does not work on x64 OS (Testing with Windows 7 Ultimate x64). It appears that doing this causes a seg fault in x64, removing stdcall also has no effect. Oddly enough, if I were to point to a native API such as NtClose it works perfectly on x64 but not other functions in other DLL modules. Anyone know what is going on here with this 64-bit difference?

    #define FILE_MAP_EXECUTE 0
      LPVOID __stdcall MapModuleToMemory(LPCSTR lpFileName) 
    	HANDLE hFile = CreateFile(lpFileName, GENERIC_READ | GENERIC_EXECUTE,
    	if (hFile == INVALID_HANDLE_VALUE)
    	    	return NULL;
    	HANDLE hMappedFile = CreateFileMapping(hFile, NULL, SEC_IMAGE | PAGE_EXECUTE_READ,
    		0, 0, NULL);
    	if (!hMappedFile)
    	      	return NULL;
    	LPVOID lpMap = MapViewOfFile(hMappedFile, FILE_MAP_READ | FILE_MAP_EXECUTE, 0,
    		0, 0);
    	return lpMap;
    typedef void (__stdcall *PSleep )(ULONG Milliseconds);
        PSleep MySleep = NULL;
    int main(int argc, char *argv[])
        char Buf[1024] = {0};
        GetSystemDirectory(Buf, MAX_PATH);
        lstrcat(Buf, "\\kernel32.dll");
        LPVOID lpMapBase = MapModuleToMemory(Buf);
        if (lpMapBase)
         ULONG_PTR SleepPtr = (ULONG_PTR)GetProcAddress(GetModuleHandle(Buf), "Sleep");
         SleepPtr -= (ULONG_PTR)GetModuleHandle(Buf);
         MySleep = (PSleep)(SleepPtr + (ULONG_PTR)lpMapBase);  
        printf("Error: Could not map module!"); 
        return 0;
    Last edited by cpuZ; January 23rd, 2012 at 21:16.

  2. #2


    Can anyone lend a hand or at least compile the code in a 64-bit c/c++ compiler to test? I'd truly appreciate any insight you experts can give me, seriously! I really don't see why using documented Windows APIs (yes, executing code from an executable file mapping is mentioned on MSDN) works on x86 OSs for all modules but not on x64. Can someone please help me understand what I need to do to fix this or if it's simply not plausible due to internal changes with calling conventions in x64 and so forth. It's driving me


  3. #3
    Does CreateFileMapping with SEC_IMAGE call the DLL entry point? Load DLLs the DLL depends on? From the examples of using CreateFileMapping with SEC_IMAGE I can find, I would guess no. In which case it wouldn't be guaranteed to work for running code from a DLL unless you've written the DLL yourself and know that it won't need any feature of the normal DLL loader that you've skipped by just mapping it.
    I promise that I have read the FAQ and tried to use the Search to answer my question.

  4. #4


    Hello Baffe,

    What you said is accurate and I realize that... to answer your questions

    "Does CreateFileMapping with SEC_IMAGE call the DLL entry point?" - No, much like LoadLibraryEx(hMod, 0, DONT_RESOLVE_DLL_REFERENCES) but the image mapped is still fully executable as if the entrypoint were called and sections are executable similar to a PE loader mapping all sections in and allowing for executable regions all without calling the entrypoint or main function.

    "Load DLLs the DLL depends on?" - No, but read below.

    I think the true question is, why does this work flawlessly for 90%+ of Windows DLLs yet not on x64 OS using the same exact concept. I doubt DLL initialization code would create differences in API behavior under these circumstances, even so an exact mapping to the last return opcode would function as the original function you're emulating. In my example Sleep and MySleep match every instruction perfectly until the last instruction, I don't see the problem. Even if this were an unresolved DLL dependency loaded/called along the way the original emulated API would have to have already mapped this dependency meaning no issue should really occur. If you replace emulating Sleep with any other Windows OS API exported from a DLL in x86 you'd see that this is no issue. Why does literally any API mapped and called like this fail in x64 except direct SYSCALLs performed by native APIs (ntdll native APIs such as NtClose / NtTerminateProcess etc) ?

    Last edited by cpuZ; January 27th, 2012 at 02:58.

  5. #5
    if i remember correctly when mapping like that, the MZ header etc started at 1 for the memory address, not a zero. so perhaps. alignment is the issue in x64

  6. #6

    Question Hmmm

    I don't see this alignment issue, all bytes seem to match up perfectly on x86 and x64 to the last transfer control instruction


  7. #7
    Teach, Not Flame Kayaker's Avatar
    Join Date
    Oct 2000
    Blog Entries

    It is strange that it works for Nt* calls but nothing else. When exactly is the crash occurring and can you not simply step through with WinDbg to find the fault? Does a crash dump analysis give anything useful and what was it that pointed to a seg fault?

  8. #8


    Hello Kayaker,

    I will create a dump of what is happening later today. Honestly, I am still baffled really. My example in c works perfectly fine on x86 XP SP2 / Server 2003 SP1 and up. If you test it you can literally replace the API call with any other function and it works alright too!


  9. #9

    As promised, I am back with some information about this crash. Also, I have written a small utility with built-in SEH handling to show the exception info. I used WinDbg and wrote the demo so that it does a ReadLn (pauses execution) until you're ready to execute the crash prone mapped Sleep function, making it easier to attach to the process and step from there. Also I have mapped NtClose to show it works fine unlike Sleep.

    The demo can be recompiled with Delphi XE2 (Win64 Release Mode) or even Lazarus 64, I have included the binary for those who are trusting of course. I wrote this in Delphi to see if this was some strange 64-bit c compiler issue. It's definitely not since this happens with Delphi 64 too.

    Results seem to be this:

    Access Violation (0xC0000005)
    At Addresss 0x000007FF38881140
    Write of Address 0x000007FF38881140

    That address appears to be well out of both ntdll and kernel32's address space from a quick glance. Any feedback would be fantastic

    Attached Files Attached Files

  10. #10
    Teach, Not Flame Kayaker's Avatar
    Join Date
    Oct 2000
    Blog Entries
    Hi cpuZ,

    Unfortunately I don't have an x64 setup to test it on, else I probably would have already tried your code because I've been curious-by-proxy about it since you first posted

    I've been reading about all the issues involved when porting to x64, such as type conversions and such, and figured the problem must be due to something like that. But it still comes back to - why does it work with the Nt* calls then?

    Not to beat a dead horse, but just to find other clues in case you got unlucky with the Sleep() API, do you also get the crash if trying another API say in advapi or shell, or a non-Nt* API in ntdll, such as one of the ntdll math or Rtl functions? Just curious if the problem is universal *except* for the Nt* API's. Does your x64 code work if you run it in 32-bit compatibility mode?, though that's probably a useless suggestion.

    Sorry I can't help practically.


  11. #11

    I'll test some other functions later today, I do recall testing a few other kernel APIs with the same conclusion. I'll test other common system libs today and post back. Personally, I am not too fond of 64-bit OSs whatsoever, since I've written 32-bit code for what seems like forever and there didn't seem to be as many caveats, but this is me being too comfortable I suppose with what seems to be obsolete sooner than later. I'm just literally baffled as to *why* this works so effectively in x86 with practically any API in any DLL, even code which I export from the main executable has no issue. By the way, since I have failed to mention why this is important to me... Please glance over this article if you're interested, it is basically what I am trying to achieve in x64 (unpacking related) If you scroll down mid page you can see a similar concept

    Thank you again for a very quick response, your technical knowledge is definitely appreciated as is your insight on this strange matter


  12. #12
    Seems to be no real rhyme or reason as to why some functions work and others fail/crash. I tested RtlZeroMemory and all worked fine in ntdll, then I tried LdrUnloadDll and it was back to crashing. Very strange but I'll continue investigating this week.


Similar Threads

  1. Semi-Automated Input Crafting by Symbolic Execution, with an Application to Automatic
    By Gunther in forum Advanced Reversing and Programming
    Replies: 0
    Last Post: March 6th, 2012, 22:31
  2. JAR Signing Issue
    By Velos in forum The Newbie Forum
    Replies: 10
    Last Post: June 7th, 2010, 12:21
  3. what about XPSP2 "Execution Protection"
    By 0rp in forum The Newbie Forum
    Replies: 12
    Last Post: March 8th, 2004, 19:19
  4. Memory Breakpoint In Execution Only?
    By Ricardo Narvaja in forum Malware Analysis and Unpacking Forum
    Replies: 9
    Last Post: December 16th, 2003, 15:48


Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts