Welcome to the new Woodmann RCE Messageboards Regroupment
Please be patient while the rest of the site is restored.

To all Members of the old RCE Forums:
In order to log in, it will be necessary to reset your forum login password ("I forgot my password") using the original email address you registered with. You will be sent an email with a link to reset your password for that member account.

The old vBulletin forum was converted to phpBB format, requiring the passwords to be reset. If this is a problem for some because of a forgotten email address, please feel free to re-register with a new username. We are happy to welcome old and new members back to the forums! Thanks.

All new accounts are manually activated before you can post. Any questions can be PM'ed to Kayaker.

NTFS MFT Internals

Interesting low-level stuff, operating system related issues, packer/vx acrobatics, drivers and non-newbie programming in general, including win32 assembly and whatever else.
Locked
WaxfordSqueers
Senior Member
Posts: 1000
Joined: Tue Apr 06, 2004 11:00 am

NTFS MFT Internals

Post by WaxfordSqueers »

I can hear all the Olly users going Arrrrgggghh!!!!

I am having severe brain lock. I fired up softice because I want to trace a file creation through kernel mode to see if I can get access to where the system accesses the NTFS MFT structure on disk. When I set a bpx on creatfileA and go to double click my file, createfile fires off on everything....of course. So, I need to use an IF statement in my BPX expression.

If I look at createfileA in kernel32, I see:

PUSH DWORD PTR [EBP+hTemplateFile]
PUSH DWORD PTR [EBP+dwFlagsAndAtrributes]
PUSH DWORD PTR [EBP+dwCreationDisposition]
PUSH DWORD PTR [EBP+lpSecurityAttributes]
PUSH DWORD PTR [EBP+dwSharedMode]
PUSH DWORD PTR [EBP+dwDesiredAccess]
PUSH DWORD PTR [EAX+04]
CALL [email protected]

[EAX+04] points to the drive, path and filename of the file I want to open.

How can I enter that as a BPX in softice so createfile will only go off when I activate my file?

I am thinking BPX creatfilew IF [EAX+04] = 'c:\(path)\(file)'

I get that flagged as an invalid expression.

Extra questions:

1)should createfile look on the disk via the NTFS MFT table to find the file...I think it should.
2)which form of createfile do I use? The call that uses the parameters above is createfileW but it has that @28 attached.
WaxfordSqueers
Senior Member
Posts: 1000
Joined: Tue Apr 06, 2004 11:00 am

Post by WaxfordSqueers »

WaxfordSqueers wrote:
I am thinking BPX creatfilew IF [EAX+04] = 'c:\(path)\(file)'

I get that flagged as an invalid expression
Played with it for a while and got past the invalid expression warning.

I used BPX createfilew EAX+4 == 'drive\path\file' ...ie. no brackets on EAX+4 and == instead of =

It now calls the drive\path\file expression an invalid character constant but it doesn't seem to mind the rest of the expression. I tried it with double quotes and it goes back to invalid expression.

I have double checked the path a dozen times and it is dead on. I have checked the context and creatfilea and createfilew are both listed under the exp command.

I tried a BPX on readfile and got it to break but I think readfile is too far down the chain for what I want. By the time readfile gets into the act, the file seems to have already been located. I need to get at it as createfile is trying to open it and finding it on the disk.
User avatar
Kayaker
Posts: 4169
Joined: Thu Oct 26, 2000 11:00 am

Post by Kayaker »

Hey Waxford,

It sounds like you want to do a system break when you double click your file, in which case the first break would be in Explorer. You could set a bp on NtCreateFile.

:TABLE ntoskrnl
:bpx _NtCreateFile do "dd esp"

Now you can start building the conditional expression.

Code: Select all

NTSTATUS NtCreateFile(
  _Out_     PHANDLE FileHandle,
  _In_      ACCESS_MASK DesiredAccess,
  _In_      POBJECT_ATTRIBUTES ObjectAttributes,
 ...
);
The 3rd parameter of POBJECT_ATTRIBUTES is PUNICODE_STRING ObjectName, so when you get the basic break on NtCreateFile:

:dd *(esp+c)
will point to POBJECT_ATTRIBUTES. Dereference that to get PUNICODE_STRING

:dd *(*(esp+c)+8)
then one more dereference to get UNICODE_STRING.Buffer

:d *(*(*(esp+c)+8)+4)
and we see (for notepad):

Code: Select all

0023:025D5418 003F005C  005C003F  003A0043  0057005C      \.?.?.\.C.:.\.W.
0023:025D5428 004E0049  004F0044  00530057  0073005C      I.N.D.O.W.S.\.s.
0023:025D5438 00730079  00650074  0033006D  005C0032      y.s.t.e.m.3.2.\.
0023:025D5448 006F006E  00650074  00610070  002E0064      n.o.t.e.p.a.d...
0023:025D5458 00780065  00000065  002E0064  00780065      e.x.e.

Now you need to get creative, there's a character limit in Softice of ULONGULONG or UQUAD (which explains your problem in your second post; as well I thought that '\' characters need to be double escaped to '\\'), so you can't do the whole string, but you could specify the string offset where "notepad.exe" starts. As you can see in this case it's +0x30 into the buffer, so a working breakpoint could be:

:BPX _NtCreateFile IF *(*(*(*(esp+c)+8)+4)+30) == 006f006e

And we get success:

Break due to BP 00: BPX _NtCreateFile IF *(*(*(*(esp+c)+8)+4)+30) == 006f006e (ET=3.80 seconds)

Sorry, can't answer extra question #1, and #2 is moot but the same principles apply.
WaxfordSqueers
Senior Member
Posts: 1000
Joined: Tue Apr 06, 2004 11:00 am

Post by WaxfordSqueers »

Kayaker wrote:Hey Waxford,

It sounds like you want to do a system break when you double click your file, in which case the first break would be in Explorer. You could set a bp on NtCreateFile.
Hey, Kayaker, thanks for response and good to hear from you. It's scary how much you know about this stuff.

I need time to digest what you wrote but nothing you said is mystifying. It's late and I'll have a go tomorrow.

I did resort to placing my file in the root directory to simplify the path. No good, but I may do that using your suggestion.

I have been reading a lot on the MFT file (Master File Table) in NTFS and it appears Windows reads the MFT initially, when the drive is first accessed, and puts certain parameters from the MFT in memory for later access. I had hoped to read the MFT right from the beginning but I may have to settle for just reading the ROOT attribute, which is essentially the ROOT directory. If I can find that using softice I can then try finding a file further down the B-tree. I know how it's done now but there just isn't much detail out there on how to read an MFT.

NTFS uses a B-tree structure to store files and the first node of the B-tree is in the $ROOT attribute. I would think the system would have to read the B-tree structure to find any file being opened by Createfile.

It would be nice to find a way to set a breakpoint on an external drive so it would fire when I plugged the drive into the USB port. I'd like to trace it from the time Windows detects the drive being plugged in through to the point where it first reads the MFT to initialize the file system.

I am getting ahead of myself here but maybe I could use your suggestions in some way to get softice to break when the drive is plugged in. Simplifying the process so I did not have to wade through scads of plug & play and other code not essential to my problem is something for me to think about later.

Thanks again for the suggestions.
WaxfordSqueers
Senior Member
Posts: 1000
Joined: Tue Apr 06, 2004 11:00 am

Post by WaxfordSqueers »

Kayaker wrote::d *(*(*(esp+c)+8)+4)
Kayaker...have followed your examples up to the one listed and I can see the path and filename. However, I am still having grief with _ntcreatefile firing off before I can double-click the file I want to trap. For example, _ntcreatefile was going off on my NIC card (\Device\NetBT_Tcpip), so I disabled it. Then it went off on a pipe (\pipe\gpxhwxt.....) belonging to ASP.Net in NET Framework version 2. I have looked into uninstalling Net 2 but it is apparently tied in with versions 3 and 3.5.

In case I am not being clear, I have set a BPX on _NtCreateFile, but when I hit X, then Enter, to get out of softice, softice comes back up again with a BP on _NtCreateFile before I can activate the mouse to start the app I want.

The solution I have in mind is to run softice in VMWare, to get away from all the garbage running in the background.

I don't recall having such issues in the past. If I set a BPX on Createfile, or Readfile, I was always able to get out of softice and activate an app.
User avatar
Kayaker
Posts: 4169
Joined: Thu Oct 26, 2000 11:00 am

Post by Kayaker »

In theory the IF statement is supposed to prevent premature ebreakulation. Again, the final +0x30 offset is the unicode string offset to the filename itself, in the case of a file in the system root directory. Maybe you're still trying to determine that offset for your own app is I guess what you're getting at.

You could try executing it from /system32 and hope that my derived offset would be the same as yours and write your IF statement the same as mine, just changing the unicode hex bytes to match the first 2 letters of your app.

Or, if it is Explorer that you want to start tracing the file execution from you could use the PID to get the break only in that address context at least. i.e.

:BPX _NtCreateFile IF PID == pidofExplorer

You may have to hit F5 a few times until your app name comes up but that should eliminate most of the white noise breaks.

That's all I really did to check that initially, under VMWare with not many background processes running I set
:BPX _ntcreatefile do "dd *(*(*(esp+c)+8)+4)"

and just kept hitting F5 until I could click my app, then F5 again until I saw the 'notepad' string show in the data window. It just so happened that Explorer was the address context that occurred in.

Maybe try the PID == trick as a start.

Make sense?
WaxfordSqueers
Senior Member
Posts: 1000
Joined: Tue Apr 06, 2004 11:00 am

Post by WaxfordSqueers »

Kayaker wrote:just kept hitting F5 until I could click my app, then F5 again until I saw the 'notepad' string show in the data window. It just so happened that Explorer was the address context that occurred in.

Maybe try the PID == trick as a start.

Make sense?
Makes eminent sense...thanks.

Meantime, I have been using SPYXX and watching the mouse messages sent when I double-click my app. I was hoping to get the process started right after a mouse-click, which has worked for me many times in the past. Then, when the app is processing the mouse message, I can set a BPX on NtCreateFile. I was hoping that if I was in the context of the file process that the NtCreateFile hit would be mine.

Problem so far is that I need WM_LBUTTONDBLCLK and all I am finding sent for messages is WM_LBUTTONDOWN and WM_LBUTTONUP. The double-click mouse message could be a user-defined message, which complicates matters.

I am also working on my van out front and getting a but frustrated, so maybe I'll put this away for the night.

Thanks for taking the time to reply.
blabberer
Senior Member
Posts: 1535
Joined: Wed Dec 08, 2004 11:12 am

Post by blabberer »

throw away soft-ice it is old creaky and cranky :) (hiding from Kayaker removing myself from linked list)

get windbg its is young creaky and cranky :)

so you have nt!NtCreateFile (.....,POBJECT_ATTRIBUTES oa,.....) at DWORD PTR SS:[esp+c] lets first make a masm expression out of it

@@masm(poi(@esp+c) this is a pointer to _OBJECT_ATTRIBUTE so lets make a c++ expression for it

((nt!_OBJECT_ATTRIBUTES *)@@masm(poi(@esp+c)))
it is a structure and ObjectName is Member in this structure and Object Name is a Pointer to another Structure _UNICODE_STRING lets make an expression to this

((nt!_UNICODE_STRING *)((nt!_OBJECT_ATTRIBUTES *)@@masm(poi(@esp+c)))->ObjectName

Buffer is member in this Structure which is a pointer to a wide character string (wchar_t *)

so lets express ify and c++ ify and alias ify the whole expression so that we can use it as a pattern

as /mu ${/v:instr} @@c++((wchar_t *)(((nt!_UNICODE_STRING *)((nt!_OBJECT_ATTRIBUTES *)@@masm(poi(@esp+c)))->ObjectName)->Buffer)) ;

you can now compare this alias with any wild characterized pattern and / or do whatever you can imagine

lets search for a pattern say *note* as in notepad.exe

.block { r $t0 = $spat( "${instr}", "*${$arg1}*" ) ; } ;

it is simple you are looking for a pattern that you will specify in run-time and assigns the result to a pseudo register so you can use this script as a generic script
( instead of ======note you can use NETBT or waxf or squeer========

lets compare results and break when we are matching is what is left

lets do it

.if (@$t0 !=1) {gc} .else {.echo matched ${instr} } ;

whole script below

Code: Select all

[b]

as /mu ${/v:instr} @@c++((wchar_t *)(((nt!_UNICODE_STRING *)((nt!_OBJECT_ATTRIBUTES *)@@masm(poi(@esp+c)))->ObjectName)->Buffer)) ;
.block {  r $t0 = $spat( "${instr}", "*${$arg1}*" ) ; } ;
.if (@$t0 !=1) {gc} .else {.echo matched ${instr} } ;
[/b]
copy paste to "filepatternSearcher.wds" and put it in a folder where windbg can access it (.wds is a random extension name is supposed to mean windbg script it can be .txt too :)

now open windbg and set a break on nt!NtCreateFile

Code: Select all

[b]
kd> bl
 0 e 8056cdc0     0001 (0001) nt!NtCreateFile "$$>a<filepattern.txt note"
[/b]
notice the argument passed note at the end you are all set for breaking on NtCreateFile when the path to NtCreateFile contains the pattern note

if you have a folder called note where your notebad.exe resides and you open badnote.exe in that folder this will break because it is in blind love with a partial word note
you may need to correct its near or long vision with proper spectacles and if it breaks too much you can curtail it with /p = _EPROCESS address of process

like


bp /p 0x12345678 nt!ntCreateFile " $$>a< filepattern.txt ntdll.dll"



a result from a debug spew enabled build of the above script

Code: Select all


\Device\Tcp6
\Device\Tcp6
\??\C:\Documents and Settings\admin\Desktop\g
\Device\NetBT_Tcpip_{CC71BAFF-4258-4B62-91F8-58D3C1FDD1EC}
\??\C:\WINDOWS\Prefetch\RUNDLL32.EXE-39944CC2.pf
\DEVICE\HARDDISKVOLUME1\PROGRAM FILES\INTERNET EXPLORER\IEXPLORE.EXE
\DEVICE\HARDDISKVOLUME1\PROGRAM FILES\WINDOWS MEDIA PLAYER\WMPLAYER.EXE
\DEVICE\HARDDISKVOLUME1\PROGRAM FILES\WINDOWS NT\ACCESSORIES\WORDPAD.EXE
\DEVICE\HARDDISKVOLUME1\WINDOWS\APPPATCH\ACGENRAL.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\APPPATCH\SYSMAIN.SDB
\DEVICE\HARDDISKVOLUME1\WINDOWS\REGISTRATION\R000000000007.CLB
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\ADVAPI32.DLL
\Device\Tcp6
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\APPHELP.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CLBCATQ.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\COMCTL32.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\COMRES.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CONFIG\SOFTWARE
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CONFIG\SYSTEM
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CSCDLL.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CSCUI.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\CTYPE.NLS
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\GDI32.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\IMAGEHLP.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\KERNEL32.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\LOCALE.NLS
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\MSACM32.DLL
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\MSPAINT.EXE
\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\MSVCRT.DLL
[b]matched [/b]\DEVICE\HARDDISKVOLUME1\WINDOWS\SYSTEM32\[b]NOTE[/b]PAD.EXE
nt!NtCreateFile:
8056cdc0 8bff            mov     edi,edi
kd> bl
 0 e 8056cdc0     0001 (0001) nt!NtCreateFile "$$>a<filepattern.txt note"
User avatar
Kayaker
Posts: 4169
Joined: Thu Oct 26, 2000 11:00 am

Post by Kayaker »

"Old, creaky and cranky"

Yep, I resemble that remark fully :p

I admit, a tool able to pattern match like that is powerful and elegant. Cryptic at first, but elegant. Nice one.
naides
Posts: 1655
Joined: Sat Jan 12, 2002 12:00 pm
Location: Planet Earth

Post by naides »

"IF statement is supposed to prevent premature ebreakulation"

This statement. . .
This lone statement. . .

Is the gem of what makes this forum a classic of literature!
WaxfordSqueers
Senior Member
Posts: 1000
Joined: Tue Apr 06, 2004 11:00 am

Post by WaxfordSqueers »

blabberer wrote:throw away soft-ice it is old creaky and cranky :) (hiding from Kayaker removing myself from linked list)

get windbg its is young creaky and cranky :)
Thanks Blabberer, I need time to go through what you wrote and it's way too late tonight.

I agree that I need to get into windbg but the learning curve is slowing me down. Besides, softice is working like a charm on my xp box and I have Elenil's hiding apparatus along with sten's old hider.

I tried and tried to use NtCreateFile but it would not settle down. I removed Net Framework, all version, and that cut the garbage dramatically and now I am down to the rasacd (remote service related to Telephony) driver popping up NtCreateFile all the time. I shut off the service...no go. Even with the driver popping up, in between popups, NtCreateFile pops up with nothing there...just question marks in lieu of the filename. I can't get it to stop long enough to double-click my file.

F5 is a shortcut in softice to run the app and I hit F5 over 600 times to see what would happen. The BPX on NtCreateFile went off over 600 times without stopping. Kayaker claims he got it to stop in VM with less overhead. I have it set up on an XP virtual machine but the rust is thick. I have not tried the IF statement yet, which should solve the problem. I just don't understand what is setting off NtCreateFile.

I looked it up and it's a user mode function, so I switched to ZwCreateFile, and bingo, I'm right into kernel-mode and all it's goodies. Found some interesting functions that sound NTFSish, like ZwQueryDirectoryFile, ZwQueryVolumeInformation and CcPfPrefetchMetadata. MFT Metadata is what I am looking for, but I selected Notepad and it has a prefetch shortcut. I have to read up on prefetch versus finding a file that prefetch doesn't know about, but I am getting somewhere. I think prefetch files are kept elsewhere, other than in the MFT structure.

Saw the IRP being assigned, so I must be close to the actual NTFS MFT disk data that is hidden from the user-side OS. I'll get back when I get some rest.

Thanks again.
User avatar
Kayaker
Posts: 4169
Joined: Thu Oct 26, 2000 11:00 am

Post by Kayaker »

Lol Naides. I don't know about a gem.., but it might be good on the back of a Woodmann Swag tshirt :D
Elenil
Senior Member
Posts: 140
Joined: Tue Sep 30, 2008 7:53 pm

Post by Elenil »

from what i know things of this get to over filter driver to ntfs.sys ?
WaxfordSqueers
Senior Member
Posts: 1000
Joined: Tue Apr 06, 2004 11:00 am

Post by WaxfordSqueers »

Elenil wrote:from what i know things of this get to over filter driver to ntfs.sys ?
Hey, Elenil....I am just reading about that now but from what I understand filter drivers are related to OEM manufacturers who want to divert and modify code specific to their equipment, in this case a hard drive. It might not hurt to find the filter driver, if it's there, and see if I can set a BP on one of it's functions. There are no drivers supplied with my drive, however, which is an external USB type from Western Digital. They have likely supplied a driver to Microsoft and had it incorporated into Msofts kernel. Don't know.

I am taking some time right now to read about functions I have encountered in ntoskrnl and win32k. In fact, I am reviewing the entire Windows API since I have become quite rusty. I figure it would be easier to trace through the kernel if I recognized functions and knew which ones to jump over and which ones to trace.

I am already fairly comfortable tracing through the kernel and I know that HAL is the interface between the disk and the OS. I would imagine filter drivers would lead to HAL but I don't want to make presumptions right now, at least not till I get an idea where I am going in the code.
WaxfordSqueers
Senior Member
Posts: 1000
Joined: Tue Apr 06, 2004 11:00 am

Post by WaxfordSqueers »

Kayaker wrote:In theory the IF statement is supposed to prevent premature ebreakulation.
Kayaker...found reasonable explanation for why NtCreateFile is failing on me and contributing to premature ebreakulation. If only it would explain the other kind.

Normally, a system service request from user mode requires an Int 2e on earlier Pentiums or a sysenter on more recent models. However, if the caller is already in kernel mode, the int or sysenter is not required. Apparently, drivers are not supposed to access system calls directly, even if they have a proper privelege level. At least, that's my understanding, which is still a bit foggy.

If NtCreateFile is called directly, the kernel notes the previous mode value and sees that the call was initiated in u-mode but that the desired address is in k-mode. It fails the call. I have noted that you were able to get in eventually but I tried several hundred times without NtCreateFile pausing to let me double-click my app. Perhaps the system service interrupt is set up differently in a VM.

The way around that is for a driver to call ZwCreateFile. Apparently the Zw version is not an alias, or a wrapper around NtCreateFile. It does not generate an INT or a Sysenter, rather, it builds a fake interrupt stack then calls KiSystemService directly. In other words, it emulates a CPU interrupt. The handler executes as if the call came from u-mode, but it detects the privelege level the call came from and sets the previous mode to kernel.

NtCreateFile now sees the call came from k-mode and accepts it.

That's my story and I'm sticking to it...for now, at least. :devil:

Keep in mind we are talking about drivers and I don't know how that might apply to the softice driver.

Anyway, the moment I set a BP on ZwCreateFile, there was no more premature ebreakulation.
Locked