Page 6 of 11 FirstFirst 1234567891011 LastLast
Results 76 to 90 of 154

Thread: NTFS MFT Internals

  1. #76
    Quote Originally Posted by blabberer View Post
    might then as well add the completed pdf half of which i posted earlier in this thread
    Attachment 2807
    I went back and found your original post at

    http://www.woodmann.com/forum/showthread.php?15188-Softice-and-createfile/page3 ....mid page.

    You may think I have been ignoring you or that I am terminally stupid, but I have been going in so many direction on this problem that I don't know which way is up at times. Between trying to learn windbg and trying to setup softice on a VM in which I could not set BPs. then having to re-install, I have forgotten much of what has already been posted. On top of that, I took a week's vacation and rebuilt two computers, spending hours trying to get a new motherboard to recognize the existing disks.

    A couple of posts ago, I discovered on my own what you had already written. Sorry about that. However, it gets me not much closer to my initial reason for this thread and that's to find a way to repair a broken MFT record.

    We now know how to get from a file record for a specific file to the file's MZ header. The MFT record at Inode 5, I think it is, is called the dot file record because it represents the root directory. I am going to explore in there to see if it gives a clue as to how the first sub-directory can be accessed and to find a semblance to a b-tree linkage.

    That's what I really want to do, take an existing MFT that has been truncated and rebuild it...if possible. Obviously, files are missing and I want to see if it's possible to splice the tree back together so windows will read the rest. There are scads of files on the disk that I have recovered using another app, but that app finds them using known file signatures. I would like to rebuild the directory structure, placing the files back in their original directories.

  2. #77
    Quote Originally Posted by Kayaker View Post
    Might as well add these three NTFS powerpoint docs to the list of reading material. Ntfs, Ntfs Recovery, Ntfs Encryption.
    K., here are two docs that I posted earlier:

    http://dubeyko.com/development/FileSystems/NTFS/ntfsdoc.pdf

    http://grayscale-research.org/new/pdfs/NTFS%20forensics.pdf

  3. #78
    Quote Originally Posted by WaxfordSqueers View Post
    K., here are two docs that I posted earlier:]
    Here's an interesting ntfs-related project that may interest the advanced programming abilities of Kayaker and Blabs.

    http://www.codeproject.com/Articles/81456/An-NTFS-Parser-Lib

  4. #79
    Super Moderator
    Join Date
    Dec 2004
    Posts
    1,486
    Blog Entries
    15
    You may think I have been ignoring you or that I am terminally stupid,
    why should i judge you by looking from my peep hole ? no

    i am still not sure what exactly you are aiming at

    i posted the pdf and i posted how to extract a file from a file0 signature

    the only link to a parent from that structure is the parent directory entry
    you would need to follow the trail from there only

    read the post again which you linked get a hang of the structures and look at offset a192b0 you will see 0x60ed this is the parent of devtext.xxxxx

    Code:
    Filename Attrib: devtest_g.hxi
      parnt mft: 0x000060ed [24813]
    now if you follow this parent you will see it has a parent

    Code:
    Path: [root]\WinDDK\7600.16385.1\help
    
    File Record: 
      MFT entry: 0x000060ed [24813]
    
    Filename Attrib: help
      parnt mft: 0x000060ca [24778]
    which again has a parent

    Code:
    Path: [root]\WinDDK\7600.16385.1
    
    File Record: 
      MFT entry: 0x000060ca [24778]
      Seq Num:   0x007c [124]
      Type:      directory
    
    Filename Attrib: 7600.16385.1
      parnt mft: 0x000060c9 [24777]
    again a parent but this time it is the root

    Code:
    Path: [root]\WinDDK
    
    File Record: 
      MFT entry: 0x000060c9 [24777]
    
    Filename Attrib: WinDDK
      parnt mft: 0x00000005 [5] <------ inode 5 dot directory
    
    Path: [root]
    
    File Record: 
      MFT entry: 0x00000005 [5]
    it points to itself as parent ( i am you you are me them are us and they are we)

    so it is all intertwined

    so if you find the parent directory you would need to find its cluster runs and carve out each file
    so no file that points to a parent directory is left

    now you need to climb one step up since this is a directory you would need to carve all the directories first and follow its subdirectories till you reach a file entry and so on

    the thing is like a single celled ameoba splitting itself into two at regular intervals and the children ameoba spliiting itself into two again at regular intervals

    the 2 ^ X

    simplyfying greatly leaving alone corner cases optimizations gotchas special cases

    each directory can have sub directories and every directory has got only one great greatest grandest father the inode 5 also called dot directory

    it is not possible to find the ameoba that first split into two when you are looking at a culture some hours after you left a single ameoba in the petri dish you would need to stop the grandchildren from proliferating and do a dna sequencing sort them into hierachy and label each of them to come anywhere nearer the first single celled organism

  5. #80
    Quote Originally Posted by blabberer View Post
    i am still not sure what exactly you are aiming at
    You have hit on a lot of it in this reply. I need to look into it more. The b-tree/amoeba comparison is apt. I have done similar stuff studying digital theory in electronics with Karnagh maps. Not quite the same but tracing through digital logic can be like tracing a tree structure.

    Right now I am immersed in an interesting find. Kayaker posted some code the other day with calls to _NtfsReadFileRecord and _NtfsReadMftRecord. Sandwiched in between is a call to _NtfsFindCachedFileRecord, which looks to see if the file is in the file cache. When it return it does a test al, al, which has always been 0 when I passed through and that always lead to _NtfsReadMftRecord, which always had the file record number.

    This time, instead of following the normal trail, I forced a 1 into al and altered the Z flag so a jump was forced. I had no idea where it was going. It spat at me and complained by sending me through an error function which complained further of corrupted files and setting 'dirty tags', presumably for chkdsk to handle.

    I was expecting a program termination with an error message box but instead, it sent me back through NtfsReadFileRecord. Bingo...I hit the holy grail...the memory location of the $MFT record. Now I am going through routines that check the attributes in each FILE record and that could get lengthy if it checks all of the 18 or so base metafiles.

    Anyway, it is revealing a few things such as offset number 4 in each attribute being the attribute length. It compares that to 0x400 which is the size of each FILE record. So far, we have checked the $VOLUME record and I have no idea where this is heading but it's interesting.

    I have read that the MFT is loaded into memory during the volume initialization at boot time. I am just hoping it will reload the file I am using from the disk.

  6. #81
    Super Moderator
    Join Date
    Dec 2004
    Posts
    1,486
    Blog Entries
    15
    Sandwiched in between is a call to _NtfsFindCachedFileRecord,
    if you get bored from tracing and would want to get an overview of all the calls that xyz apis make directly
    you can ask windbg to tell you with

    uf /c <symbol / addr/ @eip>

    for example ntfsReadFile call all these ddi's directly

    uf /c Ntfs!NtfsReadFileRecord

    Code:
    Ntfs!NtfsReadFileRecord (bae93b5c)
      Ntfs!NtfsFsdClose+0x3b9 (bae9248d):
        call to Ntfs!_SEH_epilog (bae6d300)
      Ntfs!NtfsLookupInFileRecord+0x1ad (bae93aa5):
        call to Ntfs!NtfsFindInFileRecord (bae94156)
      Ntfs!NtfsReadFileRecord+0x1a (bae93b76):
        call to Ntfs!NtfsFindCachedFileRecord (bae6f5d0)
      Ntfs!NtfsReadFileRecord+0x75 (bae93b91):
        call to Ntfs!NtfsReadMftRecord (bae93c03)
      Ntfs!NtfsFindInFileRecord+0xf5 (bae946aa):
        call to nt!RtlCompareMemory (804dab8a)
      Ntfs!NtfsLookupInFileRecord+0xf0 (bae99d53):
        call to Ntfs!NtfsMapAttributeValue (bae99fb8)
      Ntfs!NtfsLookupInFileRecord+0x142 (bae99d96):
        call to Ntfs!NtfsLookupExternalAttribute (bae99da5)
      Ntfs!NtfsFindInFileRecord+0xab (baeae0be):
        call to Ntfs!NtfsCollateNames (bae945c1)
      Ntfs!NtfsFindInFileRecord+0x77 (baeae0e4):
        call to nt!FsRtlAreNamesEqual (8056c959)
      Ntfs!NtfsFsdClose+0x81 (baeb16db):
        call to Ntfs!NtfsCompleteRequest (bae6d638)
      Ntfs!NtfsReadFileRecord+0x89 (baec15df):
        call to nt!CcUnpinData (8056d72d)
      Ntfs!NtfsReadFileRecord+0x9d (baec15f3):
        call to Ntfs!NtfsRaiseStatus (bae72661)
      Ntfs!NtfsFindInFileRecord+0xc7 (baec1606):
        call to Ntfs!NtfsCollateNames (bae945c1)
      Ntfs!NtfsFindInFileRecord+0x16a (baec1618):
        call to Ntfs!NtfsRaiseStatus (bae72661)
      Ntfs!NtfsLookupInFileRecord+0x11f (baec164a):
        call to nt!CcUnpinData (8056d72d)
      Ntfs!NtfsLookupInFileRecord+0x1c3 (baec1663):
        call to Ntfs!NtfsRaiseStatus (bae72661)
    these are not sequential calls but will be called based on results of earlier calls for example
    RaiseStatus may be called from failure path or never called at all

    any way you wrote earlier that you would want to trace through file creation

    then try Ntfs!NtfsCreateNewFile:

    set process specific breakpoint first

    one way to curtail spurious breaks could be to

    open a command prompt
    and type copy con blah.txt



    !process 0 0 cmd.exe

    bp /p <EPROCESS of cmd.exe FROM above command > ntfs!ntfsCreatenewFile
    g

    now if you start writing to the file thsi will break in proper context

    btw this ddi takes 20 parameters should be fun to unravel it have FHUN

    a call stack

    Code:
    kd> kb
    ChildEBP RetAddr  Args to Child              
    f9afd638 bae95b32 ffa09718 81592e48 81592fd8 Ntfs!NtfsCreateNewFile
    f9afd88c bae92f2d ffa09718 81592e48 f9afd8e4 Ntfs!NtfsCommonCreate+0x12ce
    f9afd970 804e37f7 812721c0 81592e48 806ef2a4 Ntfs!NtfsFsdCreate+0x1dc
    f9afd980 8066bec5 f9af0080 ffb531e8 81592e48 nt!IopfCallDriver+0x31
    f9afd9a4 baf16876 812c8dd0 812cbc50 ffb53100 nt!IovCallDriver+0xa0
    f9afd9f0 804e37f7 812c8e88 00000001 806ef2a4 sr!SrCreate+0x150
    f9afda00 8066bec5 81592e58 81592e48 ffb531e8 nt!IopfCallDriver+0x31
    f9afda24 8056c712 ffb3a968 ffa09830 f9afdc04 nt!IovCallDriver+0xa0
    f9afdb04 80577b38 812c8dd0 00000000 ffac0b48 nt!IopParseDevice+0xa12
    f9afdb3c 805686f5 ffb3a968 00000000 ffac0b48 nt!IopParseFile+0x46
    f9afdbc4 805684da 0000000c f9afdc04 00000040 nt!ObpLookupObjectName+0x119
    f9afdc18 8056cbeb 00000000 00000000 43005c01 nt!ObOpenObjectByName+0xeb
    f9afdc94 8056ccba 0013f350 80100080 0013f2f0 nt!IopCreateFile+0x407
    f9afdcf0 8056cdf0 0013f350 80100080 0013f2f0 nt!IoCreateFile+0x8e
    f9afdd30 804de7ec 0013f350 80100080 0013f2f0 nt!NtCreateFile+0x30
    f9afdd30 7c90e4f4 0013f350 80100080 0013f2f0 nt!KiFastCallEntry+0xf8
    0013f2ac 7c90d09c 7c8109a6 0013f350 80100080 ntdll!KiFastSystemCallRet
    0013f2b0 7c8109a6 0013f350 80100080 0013f2f0 ntdll!ZwCreateFile+0xc
    0013f348 4ad16281 00000000 80000000 00000001 kernel32!CreateFileW+0x35f
    0013f598 4ad0c6a5 001587c0 00158758 0013fa34 cmd!get_dest_name+0x3c1
    0013fc40 4ad0c9b5 00153ae0 00158758 001586a0 cmd!do_normal_copy+0x4a7
    0013fc60 4ad0c901 00156b70 0013fe9c 4ad05a92 cmd!copy+0xad
    0013fc6c 4ad05a92 001586a0 00000000 001586a0 cmd!eCopy+0x10
    0013fe9c 4ad013eb 001586a0 001586a0 00000002 cmd!FindFixAndRun+0x1f5
    0013fee0 4ad0f138 00000000 00000001 00000000 cmd!Dispatch+0x137
    0013ff44 4ad05154 00000001 00034100 00032960 cmd!main+0x216
    0013ffc0 7c817067 80000001 011bda28 7ffdf000 cmd!mainCRTStartup+0x125
    0013fff0 00000000 4ad05046 00000000 78746341 kernel32!BaseProcessStart+0x23
    the third parameter to ZwCreateFile is OBJECT_ATTRIBUTES check it

    kd> !obja 0013f2f0
    Obja +0013f2f0 at 0013f2f0:
    Name is blah.txt
    OBJ_CASE_INSENSITIVE

    the first param is an undocumented IRP_CONTEXT structure whose 0x24 param is a pointer to IRP Structure
    which contains a file object

    kd> dt -r nt!_IRP ta..or->Fi* @@masm(poi(poi(@esp+4)+24))
    +0x040 Tail :
    +0x000 Overlay :
    +0x024 OriginalFileObject :
    +0x01c FinalStatus : 0n0
    +0x030 FileName : _UNICODE_STRING "\Documents and Settings\admin\blah.txt"


    Code:
    kd> !irp @@masm(poi(poi(@esp+4)+24)) 1
    
    Irp is active with 9 stacks 9 is current (= 0x81592fd8)
     No Mdl: No System Buffer: Thread ffb0cb28:  Irp stack trace.  
    Flags = 40000884
    ThreadListEntry.Flink = ffb0cd38
    ThreadListEntry.Blink = ffb0cd38
    IoStatus.Status = 00000000
    IoStatus.Information = 00000000
    RequestorMode = 00000001
    Cancel = 00
    CancelIrql = 0
    ApcEnvironment = 00
    UserIosb = f9afdac0
    UserEvent = 00000000
    Overlay.AsynchronousParameters.UserApcRoutine = 00000000
    Overlay.AsynchronousParameters.UserApcContext = 00000000
    Overlay.AllocationSize = 00000000 - 00000000
    CancelRoutine = 00000000   
    UserBuffer = 00000000
    &Tail.Overlay.DeviceQueueEntry = 81592e88
    Tail.Overlay.Thread = ffb0cb28
    Tail.Overlay.AuxiliaryBuffer = 00000000
    Tail.Overlay.ListEntry.Flink = 00000000
    Tail.Overlay.ListEntry.Blink = 00000000
    Tail.Overlay.CurrentStackLocation = 81592fd8
    Tail.Overlay.OriginalFileObject = ffb531e8
    Tail.Apc = 00000000
    Tail.CompletionKey = 00000000
         cmd  flg cl Device   File     Completion-Context
     [  0, 0]   0  0 00000000 00000000 00000000-00000000    
    
    			Args: 00000000 00000000 00000000 00000000
     [  0, 0]   0  0 00000000 00000000 00000000-00000000    
    
    			Args: 00000000 00000000 00000000 00000000
     [  0, 0]   0  0 00000000 00000000 00000000-00000000    
    
    			Args: 00000000 00000000 00000000 00000000
     [  0, 0]   0  0 00000000 00000000 00000000-00000000    
    
    			Args: 00000000 00000000 00000000 00000000
     [  0, 0]   0  0 00000000 00000000 00000000-00000000    
    
    			Args: 00000000 00000000 00000000 00000000
     [  0, 0]   0  0 00000000 00000000 00000000-00000000    
    
    			Args: 00000000 00000000 00000000 00000000
     [  0, 0]   0  0 00000000 00000000 00000000-00000000    
    
    			Args: 00000000 00000000 00000000 00000000
     [  0, 0]   0  0 00000000 00000000 00000000-00000000    
    
    			Args: 00000000 00000000 00000000 00000000
    >[  0, 0]   0  0 812721c0 ffb531e8 00000000-00000000    
    	       \FileSystem\Ntfs
    			Args: f9afda50 01000060 00010080 00000000
    kd> !devobj 812721c0
    Device object (812721c0) is for:
      \FileSystem\Ntfs DriverObject 812cb2f0
    Current Irp 00000000 RefCount 0 Type 00000008 Flags 00000000
    DevExt 81272278 DevObjExt 81272a20 
    ExtensionFlags (0x80000000)  DOE_DESIGNATED_FDO
    AttachedDevice (Upper) 812c8dd0 \FileSystem\sr
    Device queue is not busy.
    it is an IRP_MJ_CREATE [0,0]

    the MajorNames are in an array whose 0th element foo[0] ==

    kd> dpa nt!IrpMajorNames L1
    8069c118 8069c3dc "IRP_MJ_CREATE"

    sizeof(IRP_CONTEXT) == 0x44 dwords (see NtfsInitializeMFT -> ZeroMemory(&..,0,sizzeof(...)) Constant) and it will contain a pointer to the MFT record of current directory structure in my case the admin directory

    Code:
    kd> .load domdbg
    dom WinDBG extension v0.3 loaded
    kd> !grep -e "FILE0" -c "dpa edi l44"
    ffa097e8  c7716000 "FILE0"

    Code:
    kd> dt Ntfs!_NTFSMFT c7716000
       +0x000 MAGIC            : 0x454c4946
       +0x004 UpdateSeqOffset  : 0x30
       +0x006 FixupArrayEntries : 3
       +0x008 $LogFileSeqNo    : 0x829d4ec
       +0x010 SequenceNumber   : 2
       +0x012 HardLinkCount    : 1
       +0x014 AttributeOffset  : 0x38
       +0x016 Flags            : 3
       +0x018 MftUsed          : 0x1d8
       +0x01c MftAlloc         : 0x400
       +0x020 FileRefernace    : 0
       +0x028 NextAttributeID  : 6
       +0x02a AlignNext4B      : 0
       +0x02c ThisMFTRecordNumber : 0x2858
       +0x030 UpdateSequence   : [8]  "+"
    
    
    kd> dt Ntfs!_ATTRIBUTE_HEADER (0xc7716000 + 38)
       +0x000 AttributeType    : 0x10
       +0x004 AttributeLength  : 0x60
       +0x008 Resident         : 0 ''
       +0x009 NameLength       : 0 ''
       +0x00a NameOffset       : 0
       +0x00c Flags            : 0
       +0x00e AttributeNumber  : 0
       +0x010 AttributeContentLength : 0x48
       +0x014 AttributeContentStartOffset : 0x18
       +0x016 unk              : 0
    
    
    kd> ?? (wchar_t*)((Ntfs!_FILE_INFO_ATTRIBUTE_RECORD *)@@masm((0xc7716000 + 38+60+18 )))->Filename
    wchar_t * 0xc77160f2
     "admin"
    Last edited by blabberer; September 6th, 2013 at 06:34.

  7. #82
    Quote Originally Posted by blabberer View Post
    if you get bored from tracing and would want to get an overview of all the calls that xyz apis make directly
    you can ask windbg
    Thanks for reply and I will check it later, I have to go out. I just want to say that I tried windbg the other night in the VM with !filecache and it complained that it could not find any vacbs, I think it was. I tried it in debug mode (adding /debug in boot.ini) as well with the same results.

    I also tried ntfswalk and it has reported scads of directories and files in the unallocated region. Oddly, I cannot find any of the 'FILE' records with pertinent material. The main system records are there up to about file 16 but that is followed by up to 0x255 seemingly empty records with the 'FILE' signature. It's like something has written over the existing files.

    I was reading last night that the files may still be there. Apparently files are never deleted completely, just like the FAT system, they are just marked as not there by the $BitMap attribute. I need to explore that attribute to see if turning bits (clusters) on and off does anything. For all I know, all my information is there and just hidden.

  8. #83
    Quote Originally Posted by blabberer View Post
    i am still not sure what exactly you are aiming at ....read the post again which you linked get a hang of the structures and look at offset a192b0 you will see 0x60ed this is the parent of devtext.xxxxx
    I have done a lot more reading and I am getting a better understanding of the MFT structure.

    In your reference above, you mention the parent directory 0x60ed. That is actually the record number of the file record.

    Here's how I understand it. The MFT begins with record $MFT which is file record 0, or inode 0. The $MFT record is followed by 16 system file records and user file records begin at about the 24th record. So, all file records begin with the signature 'FILE'. In your case file record 0x60ed is the decimal 24813 record from the beginning of the MFT table, where $MFT is file 0.

    At inode 5, the 0x5 file record, is the 'dot' or '.' file record, which represents the root directory of the directory/sub-directory system. The b-tree lives in this record which is regarded as another file in NTFS terminology. Or, at least, node 0 of the b-tree is here and most of the rest is non-resident and pointed to in this record.

    To summarize that, file record # 5, which is inode 5, begins with signature 'FILE', and is regarded as just another file in the MFT structure. However, the function of that file is to layout the directory/sub-directory index, which may contain other files, and whose files use the signature INDX.

    Here is a copy from my system:

    Code:
        Offset    |  0  1  2  3  4  5  6  7 -  8  9  A  B  C  D  E  F |       ASCII      
    -----------------------------------------------------------------------------------
      00C0001400  | 46 49 4C 45 30 00 03 00   CF 0A 00 0C 00 00 00 00 | FILE0...........
      00C0001410  | 05 00 01 00 38 00 03 00   38 02 00 00 00 04 00 00 | ....8...8.......
      00C0001420  | 00 00 00 00 00 00 00 00   19 00 00 00 05 00 00 00 | ................
      00C0001430  | 12 00 05 00 00 00 00 00   10 00 00 00 60 00 00 00 | ............`...
      00C0001440  | 00 00 18 00 00 00 00 00   48 00 00 00 18 00 00 00 | ........H.......
      00C0001450  | CF CB C8 CC F3 3B CE 01   04 52 87 46 4D AC CE 01 | .....;...R.FM...
      00C0001460  | 04 52 87 46 4D AC CE 01   04 52 87 46 4D AC CE 01 | .R.FM....R.FM...
      00C0001470  | 06 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001480  | 00 00 00 00 0D 01 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001490  | 00 00 00 00 00 00 00 00   30 00 00 00 60 00 00 00 | ........0...`...
      00C00014A0  | 00 00 18 00 00 00 01 00   44 00 00 00 18 00 01 00 | ........D.......
      00C00014B0  | 05 00 00 00 00 00 05 00   CF CB C8 CC F3 3B CE 01 | .............;..
      00C00014C0  | 57 BA 96 5C CB 3C CE 01   57 BA 96 5C CB 3C CE 01 | W..\.<..W..\.<..
      00C00014D0  | 57 BA 96 5C CB 3C CE 01   00 00 00 00 00 00 00 00 | W..\.<..........
      00C00014E0  | 00 00 00 00 00 00 00 00   06 00 00 10 00 00 00 00 | ................
      00C00014F0  | 01 03 2E 00 00 00 00 00   90 00 00 00 58 00 00 00 | ............X...
      00C0001500  | 00 04 18 00 00 00 16 00   38 00 00 00 20 00 00 00 | ........8... ...
      00C0001510  | 24 00 49 00 33 00 30 00   30 00 00 00 01 00 00 00 | $.I.3.0.0.......
      00C0001520  | 00 10 00 00 01 00 00 00   10 00 00 00 28 00 00 00 | ............(...
      00C0001530  | 28 00 00 00 01 00 00 00   00 00 00 00 00 00 00 00 | (...............
      00C0001540  | 18 00 00 00 03 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001550  | A0 00 00 00 50 00 00 00   01 04 40 00 00 00 18 00 | ....P.....@.....
      00C0001560  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001570  | 48 00 00 00 00 00 00 00   00 10 00 00 00 00 00 00 | H...............
      00C0001580  | 00 10 00 00 00 00 00 00   00 10 00 00 00 00 00 00 | ................
      00C0001590  | 24 00 49 00 33 00 30 00   11 01 2C 00 00 00 00 00 | $.I.3.0...,.....
      00C00015A0  | B0 00 00 00 28 00 00 00   00 04 18 00 00 00 17 00 | ....(...........
      00C00015B0  | 08 00 00 00 20 00 00 00   24 00 49 00 33 00 30 00 | .... ...$.I.3.0.
      00C00015C0  | 01 00 00 00 00 00 00 00   00 01 00 00 68 00 00 00 | ............h...
      00C00015D0  | 00 09 18 00 00 00 09 00   38 00 00 00 30 00 00 00 | ........8...0...
      00C00015E0  | 24 00 54 00 58 00 46 00   5F 00 44 00 41 00 54 00 | $.T.X.F._.D.A.T.
      00C00015F0  | 41 00 00 00 00 00 00 00   05 00 00 00 00 00 12 00 | A...............
      00C0001600  | 01 00 00 00 01 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001610  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001620  | 00 00 00 00 00 00 00 00   02 00 00 00 00 00 00 00 | ................
      00C0001630  | FF FF FF FF 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001640  | 18 00 00 00 03 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001650  | A0 00 00 00 50 00 00 00   01 04 40 00 00 00 18 00 | ....P.....@.....
      00C0001660  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001670  | 48 00 00 00 00 00 00 00   00 10 00 00 00 00 00 00 | H...............
      00C0001680  | 00 10 00 00 00 00 00 00   00 10 00 00 00 00 00 00 | ................
      00C0001690  | 24 00 49 00 33 00 30 00   11 01 2C 00 00 00 00 00 | $.I.3.0...,.....
      00C00016A0  | B0 00 00 00 28 00 00 00   00 04 18 00 00 00 17 00 | ....(...........
      00C00016B0  | 08 00 00 00 20 00 00 00   24 00 49 00 33 00 30 00 | .... ...$.I.3.0.
      00C00016C0  | 01 00 00 00 00 00 00 00   00 01 00 00 68 00 00 00 | ............h...
      00C00016D0  | 00 09 18 00 00 00 09 00   38 00 00 00 30 00 00 00 | ........8...0...
      00C00016E0  | 24 00 54 00 58 00 46 00   5F 00 44 00 41 00 54 00 | $.T.X.F._.D.A.T.
      00C00016F0  | 41 00 00 00 00 00 00 00   05 00 00 00 00 00 05 00 | A...............
      00C0001700  | 01 00 00 00 01 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001710  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001720  | 00 00 00 00 00 00 00 00   02 00 00 00 00 00 00 00 | ................
      00C0001730  | FF FF FF FF 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001740  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001750  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001760  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001770  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001780  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C0001790  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C00017A0  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C00017B0  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C00017C0  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C00017D0  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C00017E0  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      00C00017F0  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 12 00 | ................
    Offset 000 = file signature = FILE
    Offset 004 = offset to update sequence = 0x30
    Offset 006 = update sequence size in words = 3
    Offset 008 = $Logfile sequence number (LSN) = C000ACF = 201,329,359
    Offset 010 = Sequence number = 5
    Offset 012 = Hard Link count = 1
    Offset 014 = offset to the first attribute = 0x38 **********************
    Offset 016 = Flags = 3 00 (in use and a directory)
    Offset 018 = Real size of the file record = 0X238 = 568 bytes
    Offset 01C = Allocated size of the file record = 0x0400 = 1024 bytes
    Offset 020 = Base File record = 0 ($MFT)
    Offset 028 = Next attribute ID = 0x19 = 25
    Offset 02C = ID of this record = 5 ********************************
    Offset 030 = Update Sequence Number = 12 00
    Offset 032 = Update Sequence Array = 05 00 00 00

    Offset 014 told us the first attribute was located at offset 0x38

    Offset 038 = Attribute $10 = $STANDARD_INFORMATION (contains file time/date etc.)

    Inside attribute $10 it tells us the offset to the next attribute is 0x18
    Also, it tells us the length of the $10 attribute is 0x60, including header.
    Adding 0x60 to Offset 038 gives us Offset 098

    Offset 098 = Attribute $30 = $FILE_NAME, which tells us the file name of record 5 is 0x2e, or . (ie. dot)

    That attributes size is also 0x60, so adding that to Offset 096 gives us Offset 0F8

    Offset 0F8 = attribute $90 = $INDEX_ROOT

    Bingo!! Here is the root directory attribute. If the directory index is really small, it can all be described here. If not, pointers are required to a non-resident index described by the $0A attribute = $INDEX_ALLOCATION.

    The word 'allocation' gives us a hint. On a volume, space is either allocated (in use) or not. Another attribute $B0 - $BITMAP, keeps tract of allocated and unallocated space by marking a cluster in a table as allocated or not allocated. So, $INDEX_ROOT, $INDEX_ALLOCATION and $BITMAP work together to keep tract of directory/sub-directory indexing.

    In the dot directory (file record 5), if the directory structure is small, there is no need for the $ALLOCATION_INDEX attribute because everything is resident within the 1024 bytes allocated to file record 5.

    Getting back to:

    Offset 098 = $90 attribute ($INDEX_ROOT), it tells us that the name of the directory is at 0X18 = $I30. All directories are called $I30. It also tells us that the directory data is at 0x20. There you will find 4 x 64 bit time/date structures, file created, file modified, record changed and last access time.

    Here's where it gets tricky for me. The next attribute at Offset 150 is the $INDEX_ALLOCATION attribute, which describes non-resident directory info:

    Offset 150 = $A0
    Offset 154 = Length including header = 0x80
    Offset 158 = Non-resident flag = 1 (means data that follows is not in the MFT).
    Offset 159 = Name length = 4 (name is $I30...another directory)
    Offset 15A = Name offset = 0x40 ($I30 @ offset 190)
    Offset 15C = Flags = 00 00 = (not compressed, not encrypted, not sparse)
    Offset 15D = Attribute ID = 24 (???)

    Now for the good stuff:

    Offset 160 = First VCN = 0
    Offset 168 = Last VCN = 0 (not sure what this means yet)
    Offset 170 = Data Runs Offset = 0

    This could be referring to $INDEX_ROOT which has no data runs offset or VCN offset. (ie. it's right here)

    Fast forward to:

    Offset 198 = $INDEX_ALLOCATION = Data run

    At 198, you see the sequence 11 01 2C, meaning size = 0x11, cluster count = 01 and first cluster is at 0x2C. I am still not sure what size refers to but the cluster count is the number of times a standard sized cluster can be divided into the file length I have verified that with another situation.

    So, my C: directory should be at cluster offset 2C from the beginning of the partition. Let's check:

    Sure enough, at offset 2C000, there is an INDX signature.

    I wont print out the whole thing, just the header:
    Code:
        Offset    |  0  1  2  3  4  5  6  7 -  8  9  A  B  C  D  E  F |       ASCII      
    -----------------------------------------------------------------------------------
      000002C000  | 49 4E 44 58 28 00 09 00   91 0A 00 0C 00 00 00 00 | INDX(...........
      000002C010  | 00 00 00 00 00 00 00 00   40 00 00 00 A8 06 00 00 | ........@.......
      000002C020  | E8 0F 00 00 00 00 00 00   45 00 05 00 CE 01 00 00 | ........E.......
      000002C030  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      000002C040  | 00 00 00 00 00 00 00 00   00 00 00 00 00 00 00 00 | ................
      000002C050  | 00 00 00 00 00 00 00 00   04 00 00 00 00 00 04 00 | ................
      000002C060  | 68 00 52 00 00 00 00 00   05 00 00 00 00 00 05 00 | h.R.............
      000002C070  | CF CB C8 CC F3 3B CE 01   CF CB C8 CC F3 3B CE 01 | .....;.......;..
      000002C080  | CF CB C8 CC F3 3B CE 01   CF CB C8 CC F3 3B CE 01 | .....;.......;..
      000002C090  | 00 10 00 00 00 00 00 00   00 0A 00 00 00 00 00 00 | ................
      000002C0A0  | 06 00 00 00 00 00 00 00   08 03 24 00 41 00 74 00 | ..........$.A.t.
      000002C0B0  | 74 00 72 00 44 00 65 00   66 00 00 00 00 00 00 00 | t.r.D.e.f.......
    You can see the $AttrDef file which is an MFT file listed in the C:\ directory but hidden. Also, later in the record, you can see all the other MFT system files like $Mft, $MftMirr, etc., plus the Recycle bin.

    At this time, I don't have a structure for the INDX files and I am working on that. Your reference to parent and child interactions applies to the nodes in the b-tree which is made up of $I30 directories and INDX files.
    Last edited by WaxfordSqueers; September 10th, 2013 at 02:11.

  9. #84
    Quote Originally Posted by WaxfordSqueers View Post
    I have done a lot more reading....
    I am now proceeding to make myself dangerous. You know how directories like System Volume Information are inaccessible? Not anymore, I just turned off the file access flags that make it system and hidden and now it is fully accessible.

    On my system SVI is file record 38 (0x26) and it has three attributes that control the permissions. There is one $10 ($STANDARD_INFORMATION) and 2 x $30 ($FILE_NAME). There are two $30s I think because one is for DOS names.

    Anyway, in $10, at Offset 0x70 (from beginning of file record at FILE signature), there is a file permissions string. It was 06 00 00 00 which marks it for system and hidden. Changing it to 20 00 00 00 makes it an archive.

    In both $30s, at offset 0x50 from the beginning of the attribute (from the 0x30 signature) there is a flag string. It is normally set at 06 00 00 10, which marks it as system, hidden and a directory. The 10 at the end holds the directory bit while the 6 holds the hidden and system bits. Changing both $30 attributes to 20 00 00 10 marks them as archives with a directory.

    When I went to the file manager and double-clicked System Volume Information, I got a message box telling me I did not have file access and that I'd have to consent to get it. I hit OK, or whatever it was, and bingo, SVI was fully accessible.

    As I said, could be dangerous. I presume the same can be done to make the $MFT, $MftMirr, and other hidden system files visible.

    I wonder if it works making rootkits and other hidden malware visible? One would think a rootkit has to register itself in the MFT, otherwise how would it operate within an NTFS system? Same with hidden directories as might be revealed by rootkitrevealer or Gmer. Just change the file/directory attributes and have a look inside.

    Same with registry keys I suppose, since the registry is just another file in the MFT. I have seen Windbg used to access the registry with a plugin. That would be more elegant than searching through it via the MFT but it's nice to know you could possibly access hidden registry keys via the MFT.
    Last edited by WaxfordSqueers; September 10th, 2013 at 07:58.

  10. #85
    Super Moderator
    Join Date
    Dec 2004
    Posts
    1,486
    Blog Entries
    15
    At this time, I don't have a structure for the INDX files and I am working on that.
    INDX is structured like below it is similar to other attributes variable length structure consisting of a size and offset to information

    Code:
    kd> bl
     0 e bae942fe     0001 (0001) Ntfs!NtfsCheckIndexBuffer "da poi(@esp+8);dt Ntfs!_NTATTR_STANDARD_INDEX_HEADER  poi(@esp+8); dt Ntfs!_NTATTR_INDEX_RECORD_ENTRY poi(esp+8)+40;"
    
    kd> g
    c16a6000  "INDX("
       +0x000 magicNumber      : [4]  "INDX"
       +0x004 updateSeqOffs    : 0x28
       +0x006 sizeOfUpdateSequenceNumberInWords : 9
       +0x008 logFileSeqNum    : 0x24fb639
       +0x010 vcnOfINDX        : 0x26
       +0x018 indexEntryOffs   : 0x28
       +0x01c sizeOFEntries    : 0x920
       +0x020 sizeOfEntryAlloc : 0xfe8
       +0x024 flags            : 0x1 ''
       +0x025 padding          : [3]  ""
       +0x028 updateSeq        : 0x37
    
    
       +0x000 mftReference     : 0x10000`00000aba
       +0x008 sizeOfIndexEntry : 0x70
       +0x00a sizeofStream     : 0x58
       +0x00c flags            : 1
       +0x00e padding          : [2]  ""
       +0x010 mftFileReferenceOfParent : 0x10000`0000001d
       +0x018 creationTime     : 0x1c8148e`3ceaee00
       +0x020 lastModified     : 0x1c8148e`3ceaee00
       +0x028 lastModifiedForFileRecord : 0x1ce4c4c`ae483d60
       +0x030 lastAccessTime   : 0x1ce4c26`66282a20
       +0x038 allocatedSizeOfFile : 0x173000
       +0x040 realFileSize     : 0x172418
       +0x048 fileFlags        : 0x20
       +0x050 fNameLength      : 0xb ''
       +0x051 filenameNamespace : 0x3 ''
       +0x052 FileName         : [10]  "msjet40.dl"
    eax=812e0101 ebx=e13fe0d0 ecx=8057021c edx=00000000 esi=f9ce875c edi=c16a6000
    eip=bae942fe esp=f9ce85e8 ebp=f9ce8600 iopl=0         nv up ei pl nz na po nc
    cs=0008  ss=0010  ds=0023  es=0023  fs=0030  gs=0000             efl=00000202
    Ntfs!NtfsCheckIndexBuffer:
    bae942fe 8bff            mov     edi,edi

    I have seen Windbg used to access the registry with a plugin.
    do you have a name ? or a link ? i know windbg can natively parse through the registry files in raw format
    but haven't seen this plugin yet

    Code:
    kd> !reg findkcb \registry\machine\software\microsoft\windows\currentversion
    
    
    Found KCB = e15ad198 :: \REGISTRY\MACHINE\SOFTWARE\MICROSOFT\WINDOWS\CURRENTVERSION

  11. #86
    Quote Originally Posted by blabberer View Post
    INDX is structured like below it is similar to other attributes variable length structure consisting of a size and offset to information
    Thanks for info, I'll check it out.


    Quote Originally Posted by blabberer View Post
    do you have a name ? or a link ? i know windbg can natively parse through the registry files in raw format
    but haven't seen this plugin yet
    I should not have called it a plugin, it's a script.

    Actually, after all my goofing around, the script is right here at RCE (credit to Lionel Hauenens for script):

    http://www.woodmann.com/collaborative/tools/index.php/WinDbg_Script

    I got the lead from this paper on hunting rootkits with windbg:

    http://www.reconstructer.org/papers/Hunting%20rootkits%20with%20Windbg.pdf

    It's got some good stuff on registry searches. Although the script is not related to a registry search per se, I seem to remember using it in a related manner.

    Sorry if my faulty memory has mislead you.

  12. #87
    Quote Originally Posted by blabberer View Post
    Code:
    kd> bl
     0 e bae942fe     0001 (0001) Ntfs!NtfsCheckIndexBuffer "da poi(@esp+8);dt Ntfs!_NTATTR_STANDARD_INDEX_HEADER  poi(@esp+8); dt Ntfs!_NTATTR_INDEX_RECORD_ENTRY poi(esp+8)+40;"
    I am still trying to get my head around what you are doing here. You wrote a similar input line in an earlier post with the 0001 (0001).

    Is this statement meant to be kd>bl 0 e bae942fe or is it the whole line?

    questions:
    1)where did you get bae942fe from?

    2)Where are you getting the values like Ntfs!NtfsCheckIndexBuffer?

    Is this from the SDK or DDK?

  13. #88
    windbg command maybe
    thats the advantage of a newer debugger compared to softice it also shows local variables
    but thats almost all what is the advantage , maybe it can find the call positions for jmps/calls but that isnt useful when file is crypted

    however there was some project what tryed to fix this
    http://www.woodmann.com/collaborative/tools/index.php/IDA2SICE

    but it doesnt give a command to iceext what would be useful

    it make it easy to re-construct things what make it very simple to understand for lerned programmer , the same count for idas "create pseudocode" function what shows a c++ like code
    instead of having the understanding how this really works

  14. #89
    Teach, Not Flame Kayaker's Avatar
    Join Date
    Oct 2000
    Posts
    4,079
    Blog Entries
    5
    I'll see if I can babelfish that.

    >bl is list breakpoints
    The second line is the breakpoint set, address is derived from the symbol for Ntfs!NtfsCheckIndexBuffer from the standard ntfs pdb file
    >bp Ntfs!NtfsCheckIndexBuffer would be the basic breakpoint

    The fancy blabberese breakpoint annotations include 'dt Ntfs!_NTATTR_*' portions which are typeinfo additions added to the standard pdb file. Blabberer mentioned in an earlier post about having a secret stash of ntfs pdb's

    Code:
    |       \---addtypeinfotontfs.pdb
    |               addtypeinfot.bat
    |               ntfs.c
    |               ntfs.obj
    |               ntfs.pdb
    His procedure I believe is outlined here

    http://www.woodmann.com/forum/showthread.php?10295-Mysteries-of-win32k-amp-GDI&p=72632&viewfull=1#post72632
    or
    http://www.woodmann.com/forum/entry.php?227-How-To-Add-TypeInfo-So-That-Dt-Commands-Work-Properly-In-Windbg

    /remove fish from ear

  15. #90
    Super Moderator
    Join Date
    Dec 2004
    Posts
    1,486
    Blog Entries
    15
    bl = list breakpoints

    in windbg you can set 4 types of breakpoints

    bp = permanent breakpoint to set this you need an address like 0xbadd00d or a resolved symbol
    bu = unresolved breakpoint this bp can be set in advance and doesnt matter if the load address changes due to aslr etc
    the break will be set correctly on the address but having many of them can slow down windbg because windbg will load each modules symbols too as and when a new module is loaded and will try to resolve the symbol

    ba = access breakpoint or data brekpoints or hardware breakpoints (read write execute byte word dword)

    bm = memory breakpoints (not much usefull for reversing as you need private pdb with full typeinfo)

    breakpoints can be disabled and enabled

    bl will list all the breakpoints with their current status

    where did you get bae942fe from?
    windbg resolved it for me


    Where are you getting the values like Ntfs!NtfsCheckIndexBuffer?
    if you have the nms for ntfs.sys

    typing exp ntfs!*che*ind* in softice should get it for you too

    Code:
    kd> .reload /f ntfs.sys
    kd> x ntfs!*che*ind*
    bae942fe Ntfs!NtfsCheckIndexBuffer = <no type information>
    baeab3dd Ntfs!NtfsCheckIndexForAddOrDelete = <no type information>
    bae94505 Ntfs!NtfsCheckIndexRoot = <no type information>
    bae94277 Ntfs!NtfsCheckIndexHeader = <no type information>
    below are some steps that you can try reproducing

    Code:
    kd> bl
    
    kd> $there are no breakpoints
    kd> $ lets set a breakpoint
    kd> bp Ntfs!NtfsCheckIndexBuffer
    kd> bl
     0 e bae942fe     0001 (0001) Ntfs!NtfsCheckIndexBuffer
    
    kd> $now we have a breakpoint bl has listed it for us
    kd> $ 0 = breakpoint nunber 0 based index this index can be used for deleting disabling and in scripting 
    kd> $ e = enabled the status of the breakpoint is enabled
    kd> $ the address of Ntfs!NtfsCheckIndexBuffer which windbg resolved for us
    kd> $ 0001 = number of passes  left before this break point will be hit
    kd> $ the initial pass number you can ask windbg to break only after n hits
    kd> $ the symbol
    kd> $ lets disable this and list
    kd> bd 0
    kd> bl
     0 d bae942fe     0001 (0001) Ntfs!NtfsCheckIndexBuffer
    
    kd> $ e became d disabled 
    kd> $ lets set a bp with pass
    kd> bp Ntfs!NtfsCheckIndexBuffer 3
    breakpoint 0 redefined
    kd> bl
     0 e bae942fe     0003 (0003) Ntfs!NtfsCheckIndexBuffer
    
    kd> $ three initial pass three passes left so this break will hit only after three passes
    kd> lets set an execute hardware break on this symbol
        ^ Syntax error in 'lets set an execute hardware break on this symbol'
    kd> $ lets set an execute hardware break on this symbol
    kd> ba e1 Ntfs!NtfsCheckIndexBuffer
    kd> bl
     0 e bae942fe     0003 (0003) Ntfs!NtfsCheckIndexBuffer
     1 e bae942fe e 1 0001 (0001) Ntfs!NtfsCheckIndexBuffer
    
    kd> $ see the e and 1 it is an execute breakpoint and execute bps are always 1 byte wide
    kd> bu Ntfs!NtfsCheckIndexBuffer 8 ".echo foo;gc"
    breakpoint 0 redefined
    kd> bl
     0 e bae942fe     0008 (0008) Ntfs!NtfsCheckIndexBuffer ".echo foo;gc"
     1 e bae942fe e 1 0001 (0001) Ntfs!NtfsCheckIndexBuffer
    
    kd> $ the first bp is modified to hit only after 8 passes and a conditional statement will also execute 
    kd> g
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    foo
    Ntfs!NtfsCheckIndexBuffer:
    bae942fe 8bff            mov     edi,edi
    kd> bl
     0 e bae942fe     0001 (0008) Ntfs!NtfsCheckIndexBuffer ".echo foo;gc"
     1 e bae942fe e 1 0001 (0001) Ntfs!NtfsCheckIndexBuffer
    
    kd> $ oh boy we created a spurious bp :) our handler bypassed our pass count see the gc go from conditional bp ?
    here is an unresolved breakpoint notice the u

    Code:
    kd> bc *
    kd> bu NtfsCheckIndixbuffet
    kd> bl
     0 eu             0001 (0001) (NtfsCheckIndixbuffet)
    Last edited by blabberer; September 11th, 2013 at 07:07.

Similar Threads

  1. NTFS reversing
    By WaxfordSqueers in forum The Newbie Forum
    Replies: 21
    Last Post: April 28th, 2013, 00:56
  2. Qt Internals & Reversing
    By Daniel Pistelli in forum Blogs Forum
    Replies: 11
    Last Post: December 5th, 2008, 04:12
  3. problem with NTFS file encryption
    By Hero in forum The Newbie Forum
    Replies: 10
    Last Post: October 22nd, 2004, 03:49
  4. New project: RSA-65 analysis on GetDataBack for NTFS
    By Lbolt99 in forum RCE Cryptographics
    Replies: 6
    Last Post: August 1st, 2002, 14:48
  5. Write to NTFS
    By tentakkel in forum Malware Analysis and Unpacking Forum
    Replies: 7
    Last Post: October 8th, 2001, 17:18

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •