Page 9 of 11 FirstFirst ... 234567891011 LastLast
Results 121 to 135 of 154

Thread: NTFS MFT Internals

  1. #121
    Quote Originally Posted by blabberer View Post
    i still dont believe shell and ole ole plays significant role in the process the object is created in R0 with ObCreateObjName & sisters and the handle is created with obpCreateObHandle and brothers does shell and ole send some undocumented DeviceIoControl to ntfs.sys
    I know this is getting dated Blabs but I was thinking of opening up my investigation again, albeit temporarily, and I was reviewing the structures I know I'll encounter if I BPX on a file in explorer. Came across this concise but dated explanation of the process:

    http://netez.com/2xExplorer/shellFAQ/bg_shell.html

    The author explains that parsing a file path had to be extended due to the inclusion of objects that were not file objects, like printers, URLs, and virtual objects like the Desktop, Recycle Bin, etc. For that reason, different objects were given the ability to be opened by conventional means in a file manager. The means of opening them, executing them, printing them, etc., are referred to as verbs, which are defined in the registry. Also, you will find verbs as a parameter in ShellExecuteEx, the main function for processing the paths. As you trace through the code, you encounter Urlmon, which is related to processing URLs.

    Of course, when you try to trace through this stuff, it is not going to process just the stuff you want, like opening an executable file. It seems to check and test for every conceivable object that can be opened by a file manager.

    In the article, they stick to the convention of referring to an Item ID List, an IDL, as a PIDL, which is totally screwed. Even Microsoft does it. A PIDL is a POINTER to an IDL and a pointer is an address, not a list. To me, it is exceedingly sloppy to use the acronym PIDL in reference to an IDL, especially when you have already claimed that an item ID list is an IDL. A PIDL should be the location at which the IDL is found, not the list itself.

    Although Shell and Ole may not play direct roles in the process, you have to get through their code unless you know where to go on the other side. I have tried jumping over the code to reach functions in NTFS.sys only to find the file has already been opened.

    I am guessing that somewhere in an IDL, there is a reference to the MFT as an object. In fact, I seem to recall such a reference. Perhaps the MFT is being accessed early in the process as a file object, via Shell and Ole long before NTFS.sys comes into it.

    Also, as I have pointed out in the past, there are intervening processes, depending on whether the file has already been moved to a cache. That includes the MFT or parts of it. During initial access to the disk volume, the MFT is moved to a filecache. Of course, it doesn't seem to move the entire MFT to a cache, so even if you end up in a cache, eventually it should go back to disk in order to refill the cache.

    To complicate matters even more, Windows has different levels of caching, like WriteBehind caches. It also has search indexing. NTFS also writes to logfiles that are used by Chkdsk to rebuild the MFT should a power fail or crash happen before data from the WriteBehind cache can be written to disk. That's why I turn of that feature on a hard drive, especially when I am tracing.

    I have already experienced a BSOD while tracing after which my system would not boot. However, running Chkdsk restored the system.

    It's little wonder that mere mortals can make head or tail of the NTFS, especially in the deep code woods.

  2. #122
    Super Moderator
    Join Date
    Dec 2004
    Posts
    1,487
    Blog Entries
    15
    @waxf
    wasnt around so couldnt reply early tx for the link let me check it.

    and if you want a collaborative trial setup a vm with say xp-sp3 and hook up windbg to it.
    that way we can both go through the same path and see the same things albeit with different eyes. and others may join in too because of reproducibility in thier end at leisure

    suppose you want to trace file creation through ntfs of a file name willy.billy.magicext
    we can start from fopen(willybilly.magicext,"wb") or rather oldschoolish copy con blah_blah.foo ctrl+z or go with the youth of today by
    'loo'sing shitstem.dognasties

  3. #123
    Quote Originally Posted by blabberer View Post
    and if you want a collaborative trial setup a vm with say xp-sp3 and hook up windbg to it.
    Just so I don't have to re-invent the wheel:

    1)am I hooking windbg up so it runs in XP3 or in the VM?
    2)If I get a pipe set up between XP3 and VM, how am I initiating the app...say notepad? Do I start the app in windbg?

    I think it's better to use an exe file because other apps like a txt file will require opening up an intermediate app to run it.

    I had the basics of windbg down a year ago but i have not tried it as of late and rust has set in.

    We could start with a BP on fopen but I'm pretty sure the app will be open by then.

  4. #124
    Quote Originally Posted by blabberer View Post
    ... hook up windbg to it.
    another question...I have configured the VM as \\.\pipe\com_1, This end is server, The other end is an application (should that be a virtual machine??) and Yield CPU on poll.

    I set the boot.ini file up in the VM version of XP with "Debug" /fastdetect /debugport=com1 /baudrate=115200

    When I boot into the VM now using the Debug selection in the boot menu, the mouse is acting up. It stays as an hour glass then disappears.

    Have a feeling the mouse is using com1, even though it is a USB mouse. I wonder if the boot.ini should be changed to debugport=com_1 or the VM setting should be changed to \\.\pipe\com1

  5. #125
    Teach, Not Flame Kayaker's Avatar
    Join Date
    Oct 2000
    Posts
    4,085
    Blog Entries
    5
    Do yourself a favor W and use VirtualKD to set it all up. I just reset everything up on my x64 system -> XP VMWare and it took like 2 seconds.

    I forget how to start a user app remotely though, this did nothing

    >.create c:\windows\system32\notepad.exe
    Create will proceed with next execution
    > g

  6. #126
    Super Moderator
    Join Date
    Dec 2004
    Posts
    1,487
    Blog Entries
    15
    boot . ini in vm os /debug /debugport=com1 /baudrate=XXXX

    virtual maching settings
    remove the printer service which normally hogs com port 1 in wmware iirc (atleast i remember vmware player 4.01 hogging serial port 1 )
    add serial port using named pipe leave all the defaults iirc \\.\pipe\com_1 select this end is server / other end is appllication

    i dont know what differnce it makes by ticking yield cpu (it is not ticked by default and i leave it at default)

    open windbg in host with

    windbg.exe -k comipe,port=\\.\pipe\com_1,resets=0,reconnect


    you should have a valid connection ( i dont remember any problems with mouses disappearing probably the clock struck one and hickory dickory dock ran down

  7. #127
    Quote Originally Posted by Kayaker View Post
    Do yourself a favor W and use VirtualKD to set it all up.
    Thanks for tip Kayaker. After spending multi hours trying the conventional approach, and getting nowhere, I tried Virtual KD. No better luck. I'm beginning to feel like Eagle-eyed Fleagle, walking around with a perpetual black cloud over my head. That might be a step up from Fearless Fosdick, who usually had a cannonball hole in his chest, but it didn't seem to bother him.

    I got everything set up ok but when my vm boots, it freezes immediately rather than at the windows icon they suggest. Here's the virtialKD log:

    *******************************************************************************
    *VirtualKD patcher DLL successfully loaded. Patching the GuestRPC mechanism...*
    *******************************************************************************
    Searching patch database for information about current executable...
    No information found.
    Waiting for VMWare to initialize (5954 ms more to wait)
    Analyzing VMWARE-VMX executable...
    Building list of EXE sections... 11687K of data found.
    Scanning for RPC command name strings...
    Finished scanning. Found 41 strings.
    Searching for string references...
    Found 27 string references.
    Found 1 structures resemblant to RPC dispatcher table.
    (address 00B0EBE0, matched pointers: 27)
    Analyzing potential RPC dispatcher tables...
    Potential RPC table analyzis complete. Found 1 candidates.
    (address 00B0E640, entries: 74, free entries: 7)
    Using RPC dispatcher table at 0x00B0E640 (74 entries)
    Waiting for RPC table to be initialized by VMWare...
    RPC table initialized. Patching it...
    Successfully patched entry #1
    VMWare reset monitor activated...

    and here's the windbg log:

    Microsoft (R) Windows Debugger Version 6.11.0001.404 X86
    Copyright (c) Microsoft Corporation. All rights reserved.

    Opened \\.\pipe\kd_Windows_XP_Professional
    Waiting to reconnect...
    Connected to Windows XP 2600 x86 compatible target at (Thu Mar 20 04:59:36.562 2014 (GMT-8)), ptr64 FALSE
    Kernel Debugger connection established. (Initial Breakpoint requested)
    Symbol search path is: f:\winxp\symbols;srv*f:\winxp\symbols*http://msdl.microsoft.com/download/symbols
    Executable search path is:
    Windows XP Kernel Version 2600 UP Free x86 compatible
    Built by: 2600.xpsp.080413-2111
    Machine Name:
    Kernel base = 0x804d7000 PsLoadedModuleList = 0x8055b1c0
    System Uptime: not available
    Break instruction exception - code 80000003 (first chance)
    *******************************************************************************
    * *
    * You are seeing this message because you pressed either *
    * CTRL+C (if you run kd.exe) or, *
    * CTRL+BREAK (if you run WinDBG), *
    * on your debugger machine's keyboard. *
    * *
    * THIS IS NOT A BUG OR A SYSTEM CRASH *
    * *
    * If you did not intend to break into the debugger, press the "g" key, then *
    * press the "Enter" key now. This message might immediately reappear. If it *
    * does, press "g" and "Enter" again. *
    * *
    *******************************************************************************
    nt!RtlpBreakWithStatusInstruction:
    804e3592 cc int 3

    I did not press CTRL -C or CTRL + Break.

    I ran the app mentioned in the instructions inside the VM and they indicated everything was ok. Then I shut the VM down and booted the Virtual machine Monitor from the host. I tried various ways of starting the vm again but none of them worked. I can see a 'yes' under OS and Debugger but the vm just freezes.

  8. #128
    Quote Originally Posted by blabberer View Post
    virtual maching settings
    remove the printer service which normally hogs com port 1 in wmware iirc (atleast i remember vmware player 4.01 hogging serial port 1 )
    add serial port using named pipe leave all the defaults iirc \\.\pipe\com_1 select this end is server / other end is appllication
    thanks for help, blabs. I removed the printer service, no dice. If the player was an issue, I don't think the vm would boot with a regular start up never mind debug. I'll keep it in mind.

    The thing that really bugs me is setting com1 (serial0) on a pipe then setting the boot.ini file to com1 for debugport. If windbg is using com1 for a pipe how does windoze in the vm use it? I tried setting it to com2 and I even configured the vm for 4 serial ports but no luck. I say that because com1 on the vm is supposed to be mapped to the com1 irq/mem address of the host.

    Somehow the mouse is getting onto com1 even though it's on a USB port on the host machine. It looks to me like an old-style IRQ conflict but I don't see how the mouse is getting onto irq 4 where com1 is located.

    Mind you, the vm insists on loading both the PS/2 mouse and the HID variety via USB. It wont let me uninstal the ps/2 driver.

    Part of the problem may be that my Intel mobo only supplies one com port.

  9. #129
    Teach, Not Flame Kayaker's Avatar
    Join Date
    Oct 2000
    Posts
    4,085
    Blog Entries
    5
    Hmm, it sounds like you were 100% successful. That's exactly what I get. Now you should be under Windbg control but need to press F5 (g) to allow the VM to continue loading. Soon you should be able to interact with the VM. Back in Windbg, issue Debug/Break and the VM will "freeze" again but you can use Windbg commands, i.e. !process 0 0 should give a list of all VM processes running.

  10. #130
    Quote Originally Posted by Kayaker View Post
    Now you should be under Windbg control but need to press F5 (g)....Back in Windbg, issue Debug/Break and the VM will "freeze" again but you can use Windbg commands, i.e. !process 0 0 should give a list of all VM processes running.
    Sheer genius, kayaker...as usual. No one mentioned F5 and I'm too stupid to remember what blabs told me about it.

    Did the F5 and the vm continued loading. Waited till it was fully booted and windbg changed from busy. Did a cntl - break but did not notice anything. When I did the !process 0 0 it gave me the list.

    The vm just sits there, however, with an hour glass where the mouse cursor is. If I move it, the cursor disappears.

    I was at the same point with the old method so maybe all I had to do was boot windbg and follow the same procedure. I'll try it.

    I had the notion that I needed to use the vm. I presume I go into windbg and issue an attach to process but I don't have notepad running in the vm and there's no way to start it till I sort out the mouse issue.

    Need to spend more time with it.

  11. #131
    Super Moderator
    Join Date
    Dec 2004
    Posts
    1,487
    Blog Entries
    15
    f5 = g == go same from good old dos days old debug.com uses g = go
    Code:
    C:\WINDOWS\system32>debug command.com
    -g
    Microsoft(R) Windows DOS
    (C)Copyright Microsoft Corp 1990-2001.
    
    C:\WINDOWS\SYSTEM32>
    windbg printed that for you when it broke and asked you to do it if you didnt intend to break didnt it ?

    anyway now that you are into windbg after you pressed f5 or typed g and the vm booted up fully

    if you intend to break again press ctrl+break

    when broken when you want to run the target again type g and hit enter or press f5

    also note what you are now into is kernel debugging not your usual user mode debugging

    you do not attach to process from here (only remote debugging will let you either create or attach to process and that is a different beast

    i noticed kayaker doing .create

    no .create or .attach wont work in this setup
    ( this needs -premote option with debuging tools runnning on both machines one as a server and other as a client or a dbgsrv.exe or remote.exe in commandline )

    in this setup you need to set breakpoints and start the process in vm for windbg to break on process creation

    for example

    do ctrl+break in windbg

    type bp nt!NtCreateProcessEx ; g and hit enter

    now in vm click notepad.exe and windbg should break again

  12. #132
    Quote Originally Posted by blabberer View Post
    windbg printed that for you when it broke and asked you to do it if you didnt intend to break didnt it ?
    mighty decent of it, I'd say.

    Quote Originally Posted by blabberer View Post
    now in vm click notepad.exe and windbg should break again
    I can't get into vm and I have to find out why. I had the same issue with softice if I tried to start in debug mode. Don't ask why I wanted to do that...just curious.

    I have several things to investigate. For one, my bios can be updated and there may be things in there that are affecting the overall operation. Right now, I am trying to foolproof the bios update using a USB thumb drive but that's another experience by itself.

  13. #133
    Super Moderator
    Join Date
    Dec 2004
    Posts
    1,487
    Blog Entries
    15
    I had the same issue with softice if I tried to start in debug mode.
    do i infer that you have softice installed in the vm ? and you have connected teh vm to windbg too ?

    if so ctrl+break will be caught by softice

    you either need a neat and clean vm or you have to muck with softice i3here off

  14. #134
    Quote Originally Posted by blabberer View Post
    do i infer that you have softice installed in the vm
    I don't run softice from boot, I start it once I get to the desktop. I double-click it to run it and it goes through it's initialization and loads all it's crap. It's driver is loaded and that may be creating problems. Kayaker would know more.

    I don't think it would be trapping anything but I could clone my present system and remove SI. Driverstudio is another matter. I have never used it and have no idea what its drivers are up to.

    My problems are occurring before windbg is started. Before I loaded the app Kayaker suggested I was trying to launch kernel debug mode and that's when the mouse started to freeze. I thought it might be due to the statements required in the vmx file to let SI run but defeating them did nothing.

    I think my problem is a conflict between the mouse and com1 but I will definitely look into the SI matter.

    Thanks.

  15. #135
    Quote Originally Posted by blabberer View Post
    do i infer that you have softice installed in the vm ?
    Cloned the VM amd completely removed SI. Still having mouse trouble.

    Mouse seems OK till desktop appears, then within a few secs an hourglass replaces arrow. Any movement of cursor causes it to disappear.

    Noted that when I press g in windbg, I get the following:

    kd> g
    ERROR: DavReadRegistryValues/RegQueryValueExW(4). WStatus = 5
    ERROR: DavReadRegistryValues/RegQueryValueExW(5). WStatus = 5
    ERROR: DavReadRegistryValues/RegQueryValueExW(6). WStatus = 5
    Single step exception - code 80000004 (first chance)
    First chance exceptions are reported before any exception handling.
    This exception may be expected and handled.
    001b:006cdad2 8d55b8 lea edx,[ebp-48h]


    It seems to be indicating an exception and I wonder if it is related.

Similar Threads

  1. NTFS reversing
    By WaxfordSqueers in forum The Newbie Forum
    Replies: 21
    Last Post: April 28th, 2013, 00:56
  2. Qt Internals & Reversing
    By Daniel Pistelli in forum Blogs Forum
    Replies: 11
    Last Post: December 5th, 2008, 04:12
  3. problem with NTFS file encryption
    By Hero in forum The Newbie Forum
    Replies: 10
    Last Post: October 22nd, 2004, 03:49
  4. New project: RSA-65 analysis on GetDataBack for NTFS
    By Lbolt99 in forum RCE Cryptographics
    Replies: 6
    Last Post: August 1st, 2002, 14:48
  5. Write to NTFS
    By tentakkel in forum Malware Analysis and Unpacking Forum
    Replies: 7
    Last Post: October 8th, 2001, 17:18

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •