AMD processors "undocumented" debugging features and MSRs (DbgCtlMSR2 & al.)

[Czernobyl]

Hi! I've been seeking in vain for detailed information about AMD processors' supplementary debug facilities that are controlled by (undocumented, password protected!) MSR # C001_1024 "DbgCTLMSR2" and subsequent "DR0_Data_Match", "DR0_Data_Mask" & DR0_Addr_Mask (are there more of those ?)

Tantalisingly the register names which were leaked hint at how useful as a productivity tool the undocumented functions may be to programmers and reversers; unfortunately little more is available (the password itself is easily found by exhaustive search once the register numbers are known...). I've been trying to put those registers to work as a soothing since my dearest one passed away, but there is little hope of success without knowing at least the functions of the bits in the Control register (the low 8 bits are settable on my X32 Sempron 2400+ The registers /do/ affect DR0-based debugging, unfortunately I have not obtained more than the occasional crash or hang)... The data_match and Data_mask have full 32-bit width, while Address_match has the low 12-bits only settable (wild guess, an offset within "page" ?).

I find it unconceivable that such features be kept secret - especially so many years after they were introduced ! Doesn't someone in this select circle have access to the information, either from professional activity or personal reversing ?

It would be nice to have it published here either in the forum or the wiki.

Therefore we have set ourselves to solve this enigma and disclose our findings for the benefit of the community... Please keep visiting this forum thread and the results pages at my blog or the Collaborative RCE knowledge library

Best...

--
Czerno

[disavowed]

Lots of info here: http://www.google.com/search?q=9C5A203A+OR+9C5A203Ah+OR+09C5A203A+OR+09C5A203Ah+OR+0x9C5A203A

[Czernobyl]

Hi Disavowed! Sorry but your search suggestion is not very specific & yields nothing - the "secret" password is shared with a bunch of MSRs having nothing to do with extended debugging. You can bet I have searched, and not just Google! I even tried to remember the late Fravia's precious lessons in searching - to no avail. There is /no mention/ of this "thing" on the net, certainly not in AMD's publicly available PDFs, not even in the BIOS_Guides which are freely accessible for the Athlon 64 and later (while strangely they never released them for the K7).

The only place on the net it was mentionned before this thread is where I learnt about the existence of debugctlmsr2, etc. way back : an extended list of AMD documented and so-called undocumented MSR gathered by the author of the CBID tweaking utility, formerly at http://cbid.amdclub.ru/, now retrievable only from the web archive hosted at http://cbid.softnology.biz/html/undocmsrs.html.

I exchanged emails with the author a few years ago, he had no clues beyond the names and numbers for those MSRs As he wrote to me, he had gained knowledge of their existence by them being mentionned on a restricted AMD partners forum, but even there there was no further disclosure.

It is my guess AMD must have application notes which they communicated only under extremely strict NDAs. Though I can't accept nor even understand /why/ such tight secrecy applies - or, who knows, post 9/11 paranoia ?

Presumably /some/ developers received the specs yet beyond AMD's own, and tis my hope someone lurking here is able to clue us.

[Czernobyl]

Guys, I've reversed this in part... breakpoints defined in DR0 can be made to fire only on data match (under optional mask), plus masking of any or all of 12 low address bits ! Works also for I/O break points, provided CR4_DE is set, of course !

I may start a tuto if some are really interested (which has not been clear until now). The above results were obtained with bit 1 (weight=2) of DebugCTLMSR2 set.

Interested folk, if any, might also help reversing further features. Bit 0 set definitely has an effect, but has only "locked up" the victim machine under the (limited) test conditions. Bits 2 to 7 haven't had any perceivable effect, need more testing.

Come on, please consider the challenge of completing the reversal...

[dELTA]

Hardware conditional breakpoints? Sounds very interesting to me (and any debugger coder)! We would love to have your tutorial about this, and we could also store it in the CRCEKL.

PS.
I think the "Poll" thing might scare people away from the thread...

[Czernobyl]

Hardware conditional breakpoints? Sounds very interesting to me (and any debugger coder)! We would love to have your tutorial about this, and we could also store it in the CRCEKL.

Very well, how do we add pages to the CRCEKL ? What I have at the moment is already opening very nice possibilities for inclusion in a debugger indeed. I was thinking about writing an add-on for ye old Turbo Debugger (they document the interface for HW based debugging). And this is only using control bit n°1, I still hope to reverse control bit zero in addition...

May I ask which particular debugger you have in mind, dELTA ? Excuse my question as I have not been following the scene for many years...

I've got another question by the way, do you know if there is some debugger, preferably with open source or at least open interfaces, which breaks into System Management Mode (using ACPI chipset facilties) ?

I think the "Poll" thing might scare people away from the thread...

And I thought it would be good as an attention catcher :=)

[dELTA]

Very well, how do we add pages to the CRCEKL?

The main page (http://www.woodmann.com/collaborative/knowledge) has some good info about how to use the CRCEKL, but here's a summary of the hopefully-not-too-complex process for you :

• Traverse the catgeory tree to the left to find the most suitable category for your submission, and go to it (if there are several, pick one and see below for more info). (for you in this case I would e.g. suggest: http://www.woodmann.com/collaborative/knowledge/index.php/Category:Windows_Internals_Articles)
• Click the button at the bottom of the selected category's page, which says it will let you add an item.
• Enter all relevant info for your item in the presented form fields. And if there are more categories than one that you wanted the item to be listed in, select these in the "Item categories" section of this form.
• Click the submit button, and enjoy the exposure and historical preservation of your item. And also feel free to update the entry for any updates you make to the item in question.

What I have at the moment is already opening very nice possibilities for inclusion in a debugger indeed. I was thinking about writing an add-on for ye old Turbo Debugger (they document the interface for HW based debugging). And this is only using control bit n°1, I still hope to reverse control bit zero in addition...

Sounds great indeed! Extending Turbo Debugger might be quite a practical waste though (except possibly for academic purposes of course), since very few people actually use that old relic anymore. Extending OllyDbg or Syser would be much more appreciated and useful to the community I think.

May I ask which particular debugger you have in mind, dELTA ? Excuse my question as I have not been following the scene for many years...

All I said is that it might be interesting for debugger developers to implement your findings in your products (if I understood you correctly). This should apply to any of them, but the big ones like OllyDbg etc would of course be extra nice if they got extended with this.

I've got another question by the way, do you know if there is some debugger, preferably with open source or at least open interfaces, which breaks into System Management Mode (using ACPI chipset facilties) ?

I must refer this question to the system low-level pro members, Kayaker? Intuitive guesses would be WinDbg or Syser though, but I really don't have any actual idea.

[Czernobyl]

. . . .[*]Traverse the catgeory tree to the left to find the most suitable category for your submission, and go to it (if there are several, pick one and see below for more info). (for you in this case I would e.g. suggest: Category:Windows_Internals_Articles

Thanks for the hints. Though, this has nothing to do with Windows or another existing category, could a new category be created, maybe "processors internals" ?

And, oh, I've now reversed bit zero of the control MSR ! In conjunction with bit 1, this simply reverses the condition for beakpoints on data, i.e. break on non match instead of on match !

I'm finding this stuff simply fantastic, and the most amazing is that AMD kept it under such strict secret.

but the big ones like OllyDbg etc would of course be extra nice if they got extended with this.

Right, I want to publish what I can find and hope developers of free and or open debuggers go on and implement the stuff in their products.

I must refer this question to the system low-level pro members, Kayaker? Intuitive guesses would be WinDbg or Syser though, but I really don't have any actual idea.

SMM seems ideal for a powerful systems level debugger, having most of the capabilities of hardware ICE but without the high price onus !

...C U later !

[dELTA]

Thanks for the hints. Though, this has nothing to do with Windows or another existing category, could a new category be created, maybe "processors internals" ?

Ah, sorry, my (stupid) mistake. Here you go:

http://www.woodmann.com/collaborative/knowledge/index.php/Category:X86_Internals_Articles

Is this applicable to both x86 and x64 btw?

And, oh, I've now reversed bit zero of the control MSR ! In conjunction with bit 1, this simply reverses the condition for beakpoints on data, i.e. break on non match instead of on match !

I'm finding this stuff simply fantastic, and the most amazing is that AMD kept it under such strict secret.

Very cool, sounds like this will be a nice article for the CRCEKL indeed.

[FrankRizzo]

I can address the SMM question.

Debuggers - None that I know of, other than an actual ICE. The problem is that most modern BIOSes copy the code to SMRAM, and then set the D_LOCK (D_LCK?) bit, which stops you from being able to modify that memory. (It reverts back to video RAM).

There was an exploit on a VERY specific Intel MB, but that hole was patched and now if you happen upon that very specific board, there's no guarantee that you'll get one with the hole still open. (The exploit was a cache based exploit).

Now, if you were to get in before BIOS locked memory. You'd have to figure out how to plug your code in, and how to get it called.

[GamingMasteR]

Hi,

Maybe this source code will help (ps2+smm+apic HOOK):
http://www.debugman.com/read.php?tid=5562

[Kayaker]

Hi Czerno,

Nice to see you around again, up to your old (new?) tricks

I think the problem with working with SMM nowadays is that most new bioses probably lock the SMRAM control register to deter possible exploits such as the publicized SMM rootkits. (as FrankRizzo mentioned)

I don't know of any ad hoc debuggers which will work with SMM, other than a hardware based system such as Trace32

http://www.lauterbach.com/


If you *can* install your own SMI handler, "debugging" it (and presumably other aspects of the system) is fairly straightforward. You can communicate information over the COM port (WRITE_PORT_BUFFER_UCHAR / WRITE_PORT_UCHAR) using a serial cable and pick it up with something as simple as HyperTerminal. I had made good use of an old Win95 box for that.

Actually, I still have an article and example code on that...attached...

Am486® and ÉlanSC4xx Microcontrollers Simple Remote Debug Kernel CodeKit Software


Regards,
Kayaker

[Czernobyl]

(dELTA Is this applicable to both x86 and x64 btw?

I expect it is, since the same interesting MSRs are listed on Vitaliy's (CBID) page in the X64 section. However as I don't own an X64 processor, I expect other members to check for that soon...

(FrankRizzo+Kayaker most new BIOSes probably lock the SMRAM control register ...

My BIOS doesn't lock it !... and if it did, it would be a simple matter to init/reset the processor programmatically and take control before the BIOS does - I'm sure you know the old trick (like, reprogram memory controller and/or A20-gate so invalid code will be fetched from address FFFFFFF0 upon reset, and catch the subsequent invalid Op exception). Maybe new Mobos have been protected against such hijacking now, but I doubt it somehow...

[Czernobyl]

Here you go:

http://www.woodmann.com/collaborative/knowledge/index.php/Category:X86_Internals_Articles...

I'm ready to start an article but at the above URL it says

X86 Internals Articles
No items can be added directly to this category, please rather select one of its sub-categories above to submit an item!

... and I don't see any listed subcategory to choose from ! Unless it's a browser compatibility problem (I tried 2! Scripts allowed) or PEBKAC of course, an administrator might have kindly to intervene.

[dELTA]

Sorry, my mistake. It's fixed now, and I also added equivalent categories for x64 too, just in case.