From Collaborative RCE Tool Library

Jump to: navigation, search


Tool name: D3DLookingGlass
Rating: 0.0 (0 votes)
Author: Admiral                        
Current version: 0.1
Last updated: May 5, 2008
Direct D/L link: Locally archived copy
License type: Free / Open Source
Description: The topic of debugging full-screen Direct3D applications came up a little while ago. If you’ve ever tried it on a single-monitor setup (or even multi-monitor if the app wasn’t designed to handle it) then you’ll know how much of a pain it is. Windows just can’t handle focus being stolen from a suspended exclusive-mode program. The solution’s exactly what you’d expect - to intercept the relevant window- and device-creation calls and coax the debuggee into running in a window. This works, but fiddling with the calls manually each time you restart the process quickly gets boring. So here’s my first attempt at a generic solution.

D3DLookingGlass is a DLL which, if injected into a Direct3D process early enough, will make sure that all video devices are created in windowed mode, allowing the hosting process to coexist with a debugger without any bother. If you can inject this DLL into the target process before the first call to CreateWindow, then everything should go smoothly. I think. Any later than this and your mileage may vary.

I’ve also written a ‘loader’ program that installs the DLL as a system-wide CBT hook, so that you don’t need to inject it manually. This kind of worked for my limited set of test-cases, but there seems to be no Windows-hooks method of injecting a DLL globally and beating the call to CreateWindow. Windows installs the DLL containing the hook at the latest possible moment for its function, and I can find no type of hook that needs to be around before a window is created. I’d love for somebody to prove me wrong (or suggest another way to install the DLL system-wide), but by the looks of things, my loader is of limited use.

In particular, I recall a situation where the game (Call Of Duty 4 Demo, I think) creates a non-overlapped window, which works fine for full-screen mode, but causes problems when you try force the device to bind as windowed. This will still be a problem unless the call to CreateWindow can be intercepted (and a well-formed window induced), which means that D3DLookingGlassLoader will struggle. Confirmation would be nice.

Here’s the small-print:

* The DLL hooks CreateWindowExW and ShowWindow in its DLLMain. I think this is kosher in terms of loader-lock, but it’s obviously not too cool with regard to system stability. Especially if it’s being installed in every running process. If d3d9.dll isn’t found in the address-space then the hooks fall straight through, so that shouldn’t be too much of a problem. But if it is found then all attempts to create or show (or hide) a window will be overridden - possibly to the demise of the process if it’s doing anything but the basic behaviour. So in all cases, watch out, and make sure you aren’t running anything important in the background (in particular, I’ve noticed that it doesn’t play nice with Firefox).
* The loader uses a system-wide hook, and you hate system-wide hooks. I trust that anybody who needs this tool has some degree of technical expertise and is aware of the stability concerns inherent in installing somebody else’s barely-tested system-wide hook.
* This was harder to put together than I anticipated, and that’s probably evident from the slightly shabby code. Again, I intend for this only to be used for debugging purposes, so you’ll have to forgive me for the sub-production-quality code.
* Despite my focus on Direct3D, I’m not really a gamer and I don’t actually have any commercial games installed on this machine. So I only got a chance to test this against my own programs. Obviously, there are several ways to skin the metaphorical Direct3D-initialisation cat, so please leave a comment when you find a game that this chokes on.
Related URLs:
Forum discussion thread:

RSS feed Feed containing all updates for this tool.

You are welcome to add your own useful notes about this tool, for others to see!

If you find that any information for the tool above is missing, outdated or incorrect, please edit it!
(please also edit it if you think it fits well in some additional category, since this can also be controlled)