NTFS MFT Internals

[WaxfordSqueers]

I can hear all the Olly users going Arrrrgggghh!!!!

I am having severe brain lock. I fired up softice because I want to trace a file creation through kernel mode to see if I can get access to where the system accesses the NTFS MFT structure on disk. When I set a bpx on creatfileA and go to double click my file, createfile fires off on everything....of course. So, I need to use an IF statement in my BPX expression.

If I look at createfileA in kernel32, I see:

PUSH DWORD PTR [EBP+hTemplateFile]
PUSH DWORD PTR [EBP+dwFlagsAndAtrributes]
PUSH DWORD PTR [EBP+dwCreationDisposition]
PUSH DWORD PTR [EBP+lpSecurityAttributes]
PUSH DWORD PTR [EBP+dwSharedMode]
PUSH DWORD PTR [EBP+dwDesiredAccess]
PUSH DWORD PTR [EAX+04]
CALL _CreateFileW@28

[EAX+04] points to the drive, path and filename of the file I want to open.

How can I enter that as a BPX in softice so createfile will only go off when I activate my file?

I am thinking BPX creatfilew IF [EAX+04] = 'c:\(path)\(file)'

I get that flagged as an invalid expression.

Extra questions:

1)should createfile look on the disk via the NTFS MFT table to find the file...I think it should.
2)which form of createfile do I use? The call that uses the parameters above is createfileW but it has that @28 attached.

[WaxfordSqueers]

I am thinking BPX creatfilew IF [EAX+04] = 'c:\(path)\(file)'

I get that flagged as an invalid expression

Played with it for a while and got past the invalid expression warning.

I used BPX createfilew EAX+4 == 'drive\path\file' ...ie. no brackets on EAX+4 and == instead of =

It now calls the drive\path\file expression an invalid character constant but it doesn't seem to mind the rest of the expression. I tried it with double quotes and it goes back to invalid expression.

I have double checked the path a dozen times and it is dead on. I have checked the context and creatfilea and createfilew are both listed under the exp command.

I tried a BPX on readfile and got it to break but I think readfile is too far down the chain for what I want. By the time readfile gets into the act, the file seems to have already been located. I need to get at it as createfile is trying to open it and finding it on the disk.

[Kayaker]

Hey Waxford,

It sounds like you want to do a system break when you double click your file, in which case the first break would be in Explorer. You could set a bp on NtCreateFile.

:TABLE ntoskrnl
:bpx _NtCreateFile do "dd esp"

Now you can start building the conditional expression.

NTSTATUS NtCreateFile(
  _Out_     PHANDLE FileHandle,
  _In_      ACCESS_MASK DesiredAccess,
  _In_      POBJECT_ATTRIBUTES ObjectAttributes,
 ...
);

The 3rd parameter of POBJECT_ATTRIBUTES is PUNICODE_STRING ObjectName, so when you get the basic break on NtCreateFile:

:dd *(esp+c)
will point to POBJECT_ATTRIBUTES. Dereference that to get PUNICODE_STRING

:dd *(*(esp+c)+8)
then one more dereference to get UNICODE_STRING.Buffer

:d *(*(*(esp+c)+8)+4)
and we see (for notepad):

0023:025D5418 003F005C  005C003F  003A0043  0057005C      \.?.?.\.C.:.\.W.
0023:025D5428 004E0049  004F0044  00530057  0073005C      I.N.D.O.W.S.\.s.
0023:025D5438 00730079  00650074  0033006D  005C0032      y.s.t.e.m.3.2.\.
0023:025D5448 006F006E  00650074  00610070  002E0064      n.o.t.e.p.a.d...
0023:025D5458 00780065  00000065  002E0064  00780065      e.x.e.

Now you need to get creative, there's a character limit in Softice of ULONGULONG or UQUAD (which explains your problem in your second post; as well I thought that '\' characters need to be double escaped to '\\'), so you can't do the whole string, but you could specify the string offset where "notepad.exe" starts. As you can see in this case it's +0x30 into the buffer, so a working breakpoint could be:

:BPX _NtCreateFile IF *(*(*(*(esp+c)+8)+4)+30) == 006f006e

And we get success:

Break due to BP 00: BPX _NtCreateFile IF *(*(*(*(esp+c)+8)+4)+30) == 006f006e (ET=3.80 seconds)

Sorry, can't answer extra question #1, and #2 is moot but the same principles apply.

[WaxfordSqueers]

Hey Waxford,

It sounds like you want to do a system break when you double click your file, in which case the first break would be in Explorer. You could set a bp on NtCreateFile.

Hey, Kayaker, thanks for response and good to hear from you. It's scary how much you know about this stuff.

I need time to digest what you wrote but nothing you said is mystifying. It's late and I'll have a go tomorrow.

I did resort to placing my file in the root directory to simplify the path. No good, but I may do that using your suggestion.

I have been reading a lot on the MFT file (Master File Table) in NTFS and it appears Windows reads the MFT initially, when the drive is first accessed, and puts certain parameters from the MFT in memory for later access. I had hoped to read the MFT right from the beginning but I may have to settle for just reading the ROOT attribute, which is essentially the ROOT directory. If I can find that using softice I can then try finding a file further down the B-tree. I know how it's done now but there just isn't much detail out there on how to read an MFT.

NTFS uses a B-tree structure to store files and the first node of the B-tree is in the $ROOT attribute. I would think the system would have to read the B-tree structure to find any file being opened by Createfile.

It would be nice to find a way to set a breakpoint on an external drive so it would fire when I plugged the drive into the USB port. I'd like to trace it from the time Windows detects the drive being plugged in through to the point where it first reads the MFT to initialize the file system.

I am getting ahead of myself here but maybe I could use your suggestions in some way to get softice to break when the drive is plugged in. Simplifying the process so I did not have to wade through scads of plug & play and other code not essential to my problem is something for me to think about later.

Thanks again for the suggestions.

[WaxfordSqueers]

:d *(*(*(esp+c)+8)+4)

Kayaker...have followed your examples up to the one listed and I can see the path and filename. However, I am still having grief with _ntcreatefile firing off before I can double-click the file I want to trap. For example, _ntcreatefile was going off on my NIC card (\Device\NetBT_Tcpip), so I disabled it. Then it went off on a pipe (\pipe\gpxhwxt.....) belonging to ASP.Net in NET Framework version 2. I have looked into uninstalling Net 2 but it is apparently tied in with versions 3 and 3.5.

In case I am not being clear, I have set a BPX on _NtCreateFile, but when I hit X, then Enter, to get out of softice, softice comes back up again with a BP on _NtCreateFile before I can activate the mouse to start the app I want.

The solution I have in mind is to run softice in VMWare, to get away from all the garbage running in the background.

I don't recall having such issues in the past. If I set a BPX on Createfile, or Readfile, I was always able to get out of softice and activate an app.

[Kayaker]

In theory the IF statement is supposed to prevent premature ebreakulation. Again, the final +0x30 offset is the unicode string offset to the filename itself, in the case of a file in the system root directory. Maybe you're still trying to determine that offset for your own app is I guess what you're getting at.

You could try executing it from /system32 and hope that my derived offset would be the same as yours and write your IF statement the same as mine, just changing the unicode hex bytes to match the first 2 letters of your app.

Or, if it is Explorer that you want to start tracing the file execution from you could use the PID to get the break only in that address context at least. i.e.

:BPX _NtCreateFile IF PID == pidofExplorer

You may have to hit F5 a few times until your app name comes up but that should eliminate most of the white noise breaks.

That's all I really did to check that initially, under VMWare with not many background processes running I set
:BPX _ntcreatefile do "dd *(*(*(esp+c)+8)+4)"

and just kept hitting F5 until I could click my app, then F5 again until I saw the 'notepad' string show in the data window. It just so happened that Explorer was the address context that occurred in.

Maybe try the PID == trick as a start.

Make sense?

[WaxfordSqueers]

just kept hitting F5 until I could click my app, then F5 again until I saw the 'notepad' string show in the data window. It just so happened that Explorer was the address context that occurred in.

Maybe try the PID == trick as a start.

Make sense?

Makes eminent sense...thanks.

Meantime, I have been using SPYXX and watching the mouse messages sent when I double-click my app. I was hoping to get the process started right after a mouse-click, which has worked for me many times in the past. Then, when the app is processing the mouse message, I can set a BPX on NtCreateFile. I was hoping that if I was in the context of the file process that the NtCreateFile hit would be mine.

Problem so far is that I need WM_LBUTTONDBLCLK and all I am finding sent for messages is WM_LBUTTONDOWN and WM_LBUTTONUP. The double-click mouse message could be a user-defined message, which complicates matters.

I am also working on my van out front and getting a but frustrated, so maybe I'll put this away for the night.

Thanks for taking the time to reply.

[blabberer]

throw away soft-ice it is old creaky and cranky (hiding from Kayaker removing myself from linked list)

get windbg its is young creaky and cranky

so you have nt!NtCreateFile (.....,POBJECT_ATTRIBUTES oa,.....) at DWORD PTR SS:[esp+c] lets first make a masm expression out of it

@@masm(poi(@esp+c) this is a pointer to _OBJECT_ATTRIBUTE so lets make a c++ expression for it

((nt!_OBJECT_ATTRIBUTES *)@@masm(poi(@esp+c)))
it is a structure and ObjectName is Member in this structure and Object Name is a Pointer to another Structure _UNICODE_STRING lets make an expression to this

((nt!_UNICODE_STRING *)((nt!_OBJECT_ATTRIBUTES *)@@masm(poi(@esp+c)))->ObjectName

Buffer is member in this Structure which is a pointer to a wide character string (wchar_t *)

so lets express ify and c++ ify and alias ify the whole expression so that we can use it as a pattern

as /mu ${/v:instr} @@c++((wchar_t *)(((nt!_UNICODE_STRING *)((nt!_OBJECT_ATTRIBUTES *)@@masm(poi(@esp+c)))->ObjectName)->Buffer)) ;

you can now compare this alias with any wild characterized pattern and / or do whatever you can imagine

lets search for a pattern say *note* as in notepad.exe

.block { r $t0 = $spat( "${instr}", "*${$arg1}*" ) ; } ;

it is simple you are looking for a pattern that you will specify in run-time and assigns the result to a pseudo register so you can use this script as a generic script
( instead of ======note you can use NETBT or waxf or squeer========

lets compare results and break when we are matching is what is left

lets do it

.if (@$t0 !=1) {gc} .else {.echo matched ${instr} } ;

whole script below

as /mu ${/v:instr} @@c++((wchar_t *)(((nt!_UNICODE_STRING *)((nt!_OBJECT_ATTRIBUTES *)@@masm(poi(@esp+c)))->ObjectName)->Buffer)) ;
.block {  r $t0 = $spat( "${instr}", "*${$arg1}*" ) ; } ;
.if (@$t0 !=1) {gc} .else {.echo matched ${instr} } ;

copy paste to "filepatternSearcher.wds" and put it in a folder where windbg can access it (.wds is a random extension name is supposed to mean windbg script it can be .txt too

now open windbg and set a break on nt!NtCreateFile

kd> bl
 0 e 8056cdc0     0001 (0001) nt!NtCreateFile "$$>a<filepattern.txt note"

notice the argument passed note at the end you are all set for breaking on NtCreateFile when the path to NtCreateFile contains the pattern note

if you have a folder called note where your notebad.exe resides and you open badnote.exe in that folder this will break because it is in blind love with a partial word note
you may need to correct its near or long vision with proper spectacles and if it breaks too much you can curtail it with /p = _EPROCESS address of process

like


bp /p 0x12345678 nt!ntCreateFile " $$>a< filepattern.txt ntdll.dll"



a result from a debug spew enabled build of the above script

\Device\Tcp6
\Device\Tcp6
\??\C:\Documents and Settings\admin\Desktop\g
\Device\NetBT_Tcpip_{CC71BAFF-4258-4B62-91F8-58D3C1FDD1EC}
\??\C:\WINDOWS\Prefetch\RUNDLL32.EXE-39944CC2.pf
\DEVICE\HARDDISKVOLUME1\PROGRAM FILES\INTERNET EXPLORER\IEXPLORE.EXE
\DEVICE\HARDDISKVOLUME1\PROGRAM FILES\WINDOWS MEDIA PLAYER\WMPLAYER.EXE
\DEVICE\HARDDISKVOLUME1\PROGRAM FILES\WINDOWS NT\ACCESSORIES\WORDPAD.EXE
\DEVICE\HARDDISKVOLUME1\WINDOWS\APPPATCH\ACGENRAL.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\APPPATCH\SYSMAIN.SDB
\DEVICE\HARDDISKVOLUME1\WINDOWS\REGISTRATION\R000000000007.CLB
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\ADVAPI32.DLL
\Device\Tcp6
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\APPHELP.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CLBCATQ.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\COMCTL32.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\COMRES.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CONFIG\SOFTWARE
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CONFIG\SYSTEM
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CSCDLL.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CSCUI.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CTYPE.NLS
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\GDI32.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\IMAGEHLP.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\KERNEL32.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\LOCALE.NLS
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\MSACM32.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\MSPAINT.EXE
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\MSVCRT.DLL
matched \DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\NOTEPAD.EXE
nt!NtCreateFile:
8056cdc0 8bff            mov     edi,edi
kd> bl
 0 e 8056cdc0     0001 (0001) nt!NtCreateFile "$$>a<filepattern.txt note"

[Kayaker]

"Old, creaky and cranky"

Yep, I resemble that remark fully

I admit, a tool able to pattern match like that is powerful and elegant. Cryptic at first, but elegant. Nice one.

[naides]

"IF statement is supposed to prevent premature ebreakulation"

This statement. . .
This lone statement. . .

Is the gem of what makes this forum a classic of literature!

[WaxfordSqueers]

throw away soft-ice it is old creaky and cranky (hiding from Kayaker removing myself from linked list)

get windbg its is young creaky and cranky

Thanks Blabberer, I need time to go through what you wrote and it's way too late tonight.

I agree that I need to get into windbg but the learning curve is slowing me down. Besides, softice is working like a charm on my xp box and I have Elenil's hiding apparatus along with sten's old hider.

I tried and tried to use NtCreateFile but it would not settle down. I removed Net Framework, all version, and that cut the garbage dramatically and now I am down to the rasacd (remote service related to Telephony) driver popping up NtCreateFile all the time. I shut off the service...no go. Even with the driver popping up, in between popups, NtCreateFile pops up with nothing there...just question marks in lieu of the filename. I can't get it to stop long enough to double-click my file.

F5 is a shortcut in softice to run the app and I hit F5 over 600 times to see what would happen. The BPX on NtCreateFile went off over 600 times without stopping. Kayaker claims he got it to stop in VM with less overhead. I have it set up on an XP virtual machine but the rust is thick. I have not tried the IF statement yet, which should solve the problem. I just don't understand what is setting off NtCreateFile.

I looked it up and it's a user mode function, so I switched to ZwCreateFile, and bingo, I'm right into kernel-mode and all it's goodies. Found some interesting functions that sound NTFSish, like ZwQueryDirectoryFile, ZwQueryVolumeInformation and CcPfPrefetchMetadata. MFT Metadata is what I am looking for, but I selected Notepad and it has a prefetch shortcut. I have to read up on prefetch versus finding a file that prefetch doesn't know about, but I am getting somewhere. I think prefetch files are kept elsewhere, other than in the MFT structure.

Saw the IRP being assigned, so I must be close to the actual NTFS MFT disk data that is hidden from the user-side OS. I'll get back when I get some rest.

Thanks again.

[Kayaker]

Lol Naides. I don't know about a gem.., but it might be good on the back of a Woodmann Swag tshirt

[Elenil]

from what i know things of this get to over filter driver to ntfs.sys ?

[WaxfordSqueers]

from what i know things of this get to over filter driver to ntfs.sys ?

Hey, Elenil....I am just reading about that now but from what I understand filter drivers are related to OEM manufacturers who want to divert and modify code specific to their equipment, in this case a hard drive. It might not hurt to find the filter driver, if it's there, and see if I can set a BP on one of it's functions. There are no drivers supplied with my drive, however, which is an external USB type from Western Digital. They have likely supplied a driver to Microsoft and had it incorporated into Msofts kernel. Don't know.

I am taking some time right now to read about functions I have encountered in ntoskrnl and win32k. In fact, I am reviewing the entire Windows API since I have become quite rusty. I figure it would be easier to trace through the kernel if I recognized functions and knew which ones to jump over and which ones to trace.

I am already fairly comfortable tracing through the kernel and I know that HAL is the interface between the disk and the OS. I would imagine filter drivers would lead to HAL but I don't want to make presumptions right now, at least not till I get an idea where I am going in the code.

[WaxfordSqueers]

In theory the IF statement is supposed to prevent premature ebreakulation.

Kayaker...found reasonable explanation for why NtCreateFile is failing on me and contributing to premature ebreakulation. If only it would explain the other kind.

Normally, a system service request from user mode requires an Int 2e on earlier Pentiums or a sysenter on more recent models. However, if the caller is already in kernel mode, the int or sysenter is not required. Apparently, drivers are not supposed to access system calls directly, even if they have a proper privelege level. At least, that's my understanding, which is still a bit foggy.

If NtCreateFile is called directly, the kernel notes the previous mode value and sees that the call was initiated in u-mode but that the desired address is in k-mode. It fails the call. I have noted that you were able to get in eventually but I tried several hundred times without NtCreateFile pausing to let me double-click my app. Perhaps the system service interrupt is set up differently in a VM.

The way around that is for a driver to call ZwCreateFile. Apparently the Zw version is not an alias, or a wrapper around NtCreateFile. It does not generate an INT or a Sysenter, rather, it builds a fake interrupt stack then calls KiSystemService directly. In other words, it emulates a CPU interrupt. The handler executes as if the call came from u-mode, but it detects the privelege level the call came from and sets the previous mode to kernel.

NtCreateFile now sees the call came from k-mode and accepts it.

That's my story and I'm sticking to it...for now, at least.

Keep in mind we are talking about drivers and I don't know how that might apply to the softice driver.

Anyway, the moment I set a BP on ZwCreateFile, there was no more premature ebreakulation.