Kayaker wrote:.... whether you notice any difference if you defrag the MFT, or if that might even make the convoluted path you're trying to follow for your specific test files any easier if by chance their entries happen to be fragmented across MFT sections.
Thanks for links Kayaker. I don't see how at this point an mft defrag would help. The MFT lives as a database with it's origin at a specific address. Microsoft claims it has no permanent address so it can be moved in the case of a bad cluster. What I am trying to do is find where the MFT is called to go find a file on disk. You supplied an NTFS function earlier, I think it was _NtfsCheckFile Record, or something like that. I did explore that but all it was doing was testing the file record in the MFT for veracity. It seemed to return to shell32 without finding the file. I presume the link I seek came before that or after that.
On both my systems, with 512 bytes/sector and 8 sectors/cluster, (laptop win7 and desktop XP) the MFT is located at cluster offset 0xC0000. On my systems that is byte offset 0XC0000000. It's likely the same on yours if you have the same cluster factor and 512 bytes/sector, and you can see a file signature of FILE at that address. If you look down the file a short ways, you will see a reference to $MFT. That is the $MFT file record which is a reference to MFT itself. However, many people, apparently including Russinovich, the author of Contig, are using it in an odd manner.
***I have made an error here but I will leave it and correct it later. My error shows how easy it is to get confused in the MFT***
That may sound arrogant but consider the definitions supplied by Microsoft. The first 16 file records in the MFT are system records and that's why they have the $ sign in front of them. The very first record is $MFT and it is record 0. When Russinovich writes C:\$MFT::$BITMAP, he is suggesting that $BITMAP is contained in $MFT and it is not. $BITMAP is a record in the overall MFT but it is not contained in $MFT.
***I have confused $Bitmap the file record with $Bitmap the attribute...explained below**********
An interesting aside. Microsoft describes the MFT as a database with rows of file records and columns of attributes. That does make sense since it begins at file record 0 in the first row and each record has a header with attributes numbered from 0x10 (STANDARD_INFORMATION) upwards of 0xB0 in columns. You could visualize the columns as file header, attribute 1, attribute 2, etc. Not every file has the same attributes, however, so some columns would have no data in them. The data (header and attributes) in columns are telling you things about the file record in that row. Every file on the system is in its own row and the rows are numbered as file record numbers.
In the root directory, normally C:\, the metafiles beginning with $ are listed in the C:\ directory but their attributes are hidden and system. The attribs cannot be changed using attrib from a DOS prompt. At this point, I don't know if the $MFT is a reference to the entire MFT or just to record 0 in the MFT. Along with $MFT can be found $MftMirr and several other system MFT metafiles all beginning with $.
Problem is, none of them are in the C:\ directory even though they are listed there. As I said above, the MFT database is located at 0xC0000000 on my system and my C:\ directory is at 0x2C000. It seems the references to the metafiles are hard links just as the recycle bin is a virtual file pointed at in C:\. Anyway, at 0xC0000000, is the $MFT file record with it's file header and attributes, the rest of the file records, including $MftMirr follow with each of the following 16 records being system records. Here are the system records in the MFT, all rows of the database:
0 = $Mft - the record with information about the MFT only. It contains a $Bitmap attribute ($B0) hence $Mft::$Bitmap.
1 = $MftMirr
2 = $LogFile
3 = $Volume
4 = $AttDef
5 = no name...referred to as the dot record, or '.'. It is the directory system root.
6 = $Bitmap - note that $Bitmap is a separate file and not contained in $Mft <<<<-------------------------
7 = $Boot - the actual boot file is at the beginning of the partition with a copy at the end of the partition.
8 = $BadClus
9 = $Secure
10 = $Upcase
11 = $Extend
12 - 15 - reserved
The description given on my reference for $Mft claims it contains one base file record for each file and folder on the NTFS volume. Horsefeathers!! If you look in $MFT there is nothing there but a reference to itself. There is nary a mention of any other file.
It has a standard header with 4 attributes $10, $30, $80 and $A0. $10 is standard information. $30 is $File_Name which tells you it's name is $MFT. $80 is more interesting , it is $Data and is presumably a file's data. It gives the first and last VCN and tells you the size of the MFT. However, the data reference is to this $MFT record only and to no other file on the disk.
$80 has two data runs. The first tells you the MFT begins at cluster 0xC0000, which is also listed in the NTFS boot record at the start of the partition. The second data run is presumably a pointer to the other half of my split MFT. When I go to that address, sure enough, there is a FILE signature and the continuation of the MFT.
$B0 = the $Bitmap 'attribute', not the $Bitmap 'file record'. Confusing??? So I am wrong, there is a $MFT::$Bitmap reference but the $Mft is a file record reference and the $Bitmap reference is to an attribute. I interpret C:\$Mft::$Bitmap as meaning the $Bitmap attribute in the $Mft file record.
The $BITMAP record is apparently a record of available clusters on the system.
Anyway, it has two data runs in the attribute and data runs must be non-resident, or external to the MFT. The first is at 0xBFFFF and extends for 1 cluster, which is 0x1000. That means that portion of the bitmap extends from one cluster below the MFT to the MFT at 0XC0000. It is full of FFs, an indications of bit fields for each cluster on the drive.
The second $Bitmap data run is at 0xDCA85 and extends for 6 clusters (0x6000). That bit field is far more sparse than the first one as might be expected since it represents a region on disk that has more free space.
To summarize, the name $MFT seems to be used incorrectly by many people. $Mft does not mean MFT. $Mft is one record, the first, in the MFT database and its sole purpose is to give information about the MFT such as its name, it's time/date information (when it was changed, created, etc.), it's security attributes, it's location, its size and the clusters it occupies. $MFT tells you nothing about any other file on the system. That information comes from the user file records beginning after the system $-records and referred to in the index records in the dot file record at file record 0x5.
That seems to be how you locate a file in the MFT. You trace the b-tree with it's root node in the 'dot' file record 0x5. The b-tree is a sorted index based on the directory/file name, and once you have the path, you can find the file reference in the index. At that point, there is apparently a pointer to the associated file record.
After having explored the MFT to a decent depth and seeing its complexity I don't think I'd want to run something like contig on my main system. I tried that once with a boot disk from Comodo written in Linux and it completely screwed my system, installing a Linux folder while writing over my directories including Program Files.
I think part of the problem I am having now is based on what Comodo backup did on my external drive. Ultimately, it was a bonehead error on my part, mistaking a clone operation for an image operation. Comodo started cloning to my external disk without a warning that all data would be lost. That would have clued me into my error. Although it only wrote for a few seconds before I aborted it, the damage was done. It seems to have written over my external drive's MFT but why it would begin in mid disk with an MFT is not clear. I would think a sector by sector clone would work from sector 1 outward.
Age has been harsh to me.