NtQuerySystemInformation and open handles

[omega_red]

I'll forward here a post from microsoft.public.win32.programmer.kernel newsgroup (Message-ID: <cgkcte$1kcp$1@mamut.aster.pl>)

NtQuerySystemInformation can return all opened handles, but in all infos I
can find in net, returned array provides 16-bit handle values and handles
are pointer-precision values (I tested my XP for handle limit and I was able
to open much more then 2^16 files). Are docs found on the net invalid or
native system calls are not very compatible..?

...and my reply:

I've just written quick and dirty code to check what this 'handles'
actually are. All of them are 0 (on w2k sp4). It returns valid type and
kernel object address, so maybe this is the way.

Interestingly, there is some sample code in the book "Windows NT/2000
native API reference" for enumerating any process' open handles. Firstly it
calls NtQuerySystemInformation with class SystemHandleInformation, and then
uses ZwDuplicateObject for each handle to get private copy of it. However,
foreign handle supplied to ZwDuplicateObject is the same abovementioned
handle - so how it's supposed to work if all of them are null?

#include <windows.h>
#include <stdio.h>

typedef DWORD (NTAPI *pNtQuerySystemInformation)(DWORD info_class, void *out, DWORD size, DWORD *out_size);

struct SYSTEM_HANDLE_INFORMATION
{
	ULONG	ProcessId;
	UCHAR	ObjectTypeNumber;
	UCHAR	Flags;
	USHORT	Handle;
	PVOID	Object;
	ACCESS_MASK	GrantedAccess;
};

SYSTEM_HANDLE_INFORMATION buf[100000];

void main()
{
	DWORD size;
	pNtQuerySystemInformation NtQuerySystemInformation = 
		(pNtQuerySystemInformation) GetProcAddress(GetModuleHandle("ntdll.dll"), "NtQuerySystemInformation");
	NtQuerySystemInformation(16, buf, sizeof(buf), &size);
	DWORD n = size/sizeof(SYSTEM_HANDLE_INFORMATION);
	for (DWORD i=0; i<n; i++)
	{
		printf("%08x: PID: %08x, type: %02x, handle: %04x, ptr: %08x\n", i,
			buf[i].ProcessId, buf[i].ObjectTypeNumber, buf[i].Handle, buf[i].Object);
	}
	DebugBreak();
}

Anyone got some info about this?

[Kayaker]

Hi,

All I can say at the moment after checking out what is returned from SystemHandleInformation (an array of SYSTEM_HANDLE_INFORMATION structures), is that "handles" are indeed 16 bit values, they begin numbering from 04h for each PID and increase to the total number of (open?) handles for each process, 0004h, 0008h, 000Ch,...

If you type PROC -x <process> in Softice you'll get further Handle info including the total number of handles. This number matches the number of SYSTEM_HANDLE_INFORMATION structures returned for each process.

As far as I can tell, everything looks fine. I'm not sure what 'pointer-precision values' are supposed to be or why these 16 bit handles don't match what this guy thinks they should be.

Kayaker

[omega_red]

Well, HANDLE type is 32-bit, but NtQuerySystemInformation returns 16-bit "handles", which are anyway all NULL (at least on my system).

00000000: PID: 00001041, type: 08, handle: 0000, ptr: 00040005
00000001: PID: 001f0fff, type: 08, handle: 0000, ptr: 00080006
00000002: PID: 001f03ff, type: 08, handle: 0000, ptr: 000c0012
00000003: PID: 00000000, type: 08, handle: 0000, ptr: 00100012
00000004: PID: 00020019, type: 08, handle: 0000, ptr: 00140012
00000005: PID: 0002001f, type: 08, handle: 0000, ptr: 00180012
00000006: PID: 0002001f, type: 08, handle: 0000, ptr: 001c0012
00000007: PID: 00000020, type: 08, handle: 0000, ptr: 00200012
00000008: PID: 00020019, type: 08, handle: 0000, ptr: 00240012
00000009: PID: 0002001f, type: 08, handle: 0000, ptr: 00280008
0000000a: PID: 001f0003, type: 08, handle: 0000, ptr: 002c0012
0000000b: PID: 0002001f, type: 08, handle: 0000, ptr: 00300008
0000000c: PID: 001f0003, type: 08, handle: 0000, ptr: 00340012
0000000d: PID: 0002001f, type: 08, handle: 0000, ptr: 00380006
0000000e: PID: 001f03ff, type: 08, handle: 0000, ptr: 003c0006
0000000f: PID: 001f03ff, type: 08, handle: 0000, ptr: 00400006
...

[edit]
The results seem strange anyway, what those PIDs suppose to be? Have I made some silly error in my code?

[Kayaker]

Hi

I can tell right off that you seem to be offset by 1 DWORD. The 1st DWORD returned by NtQuerySystemInformation/SystemHandleInformation is the *number* of SYSTEM_HANDLE_INFORMATION structures that follow, an array of arrays.

The first dword you show is 1041, this is the number of structures to follow, offset your output by 1 dword and it should make sense.

The PID you have should probably be 00000008 (default System pid)
ObjectTypeNumber should be your 5, 6 ,12...
Flags can be 00 or 01 I think
Handle is 0004, 0008, 000c, etc.
GrantedAccess will be values like 001f0fff

If you type PROC in Softice you'll see the pid's of all the processes, these seem to be in the same order (load order?) as the "groups" of SYSTEM_HANDLE_INFORMATION strucs returned by SystemHandleInformation.


Further comment:
It would appear then that Object Handles as we know them (passed from API calls for example) are zero-extended 16 bit values from 0004h - 0FFFFh that the system assigns sequentially for each process. This should give a total of 16,383 handles available for each process. Logically enough I suppose, MS decided that was enough (Bugcheck protected?) So what will happen with 64bit processors, zero-extended dword values passed as 64bit? ;-)

[omega_red]

Oh well *slaps his head*. One dword and such a mess
It also lightened the original question: why this handles are 16-bit, although there can be more than 65536 handles per system.

00000000: PID: 00000008, type: 05, handle: 0004, ptr: 81ad63c0
00000073: PID: 00000000, type: 1a, handle: 0004, ptr: 81530b08
000000b5: PID: 000000ac, type: 11, handle: 0004, ptr: e134a830
000000d6: PID: 000000c4, type: 11, handle: 0004, ptr: e1332910
...

Handle values are not systemwide unique.

[Four-F]

The kernel object handles are NOT 16 bit values.

First, open your copy of "Inside Microsof Windows 2000" and look at Figure 3-14 "Process handle table architecture in Windows 2000". Then go deep into ExpLookupHandleTableEntry and you will see something like this:

//
//  Decode the handle index into its separate table indicies
//

l = (Handle.Index >> 24) & 255;
i = (Handle.Index >> 16) & 255;
j = (Handle.Index >> 8)  & 255;
k = (Handle.Index)       & 255;

[omega_red]

So how do you explain this? Handles returned by NtQuery.. ARE 16bit...

[0rp]

So how do you explain this? Handles returned by NtQuery.. ARE 16bit...

the several components of a 32bit handle are used to select one handletable

    if ( l != 0 ) return NULL;
    if (HandleTable->Table[i] == NULL) return NULL;
    if (HandleTable->Table[i][j] == NULL) return NULL;
    return &(HandleTable->Table[i][j][k]);

and handles returned by NtQuery.. are probably always Table[i] handles

[Kayaker]

Hi, thanks for clarifying that those are actually index values.

[omega_red]

Yes, it makes sense now and explains why they are not unique.

[Four-F]

The right answer is: ObpCaptureHandleInformation callback truncates handle values down to USHORT.

(*HandleEntryInfo)->HandleValue = (USHORT)((ULONG_PTR)(HandleIndex));

I've attached the simple msvc6 project. It creates 16384 named events. Run handle -a -p handles at appropriate moment and you will see something like this:

 handle -a -p handles

 ffe8: Event         \BaseNamedObjects\MyEvent00016360
 ffec: Event         \BaseNamedObjects\MyEvent00016361
 fff0: Event         \BaseNamedObjects\MyEvent00016362
 fff4: Event         \BaseNamedObjects\MyEvent00016363
 fff8: Event         \BaseNamedObjects\MyEvent00016364
 fffc: Event         \BaseNamedObjects\MyEvent00016365
    0: Event
    4: Event
    8: Event
    c: Event
   10: Event
   14: Event
   18: Event
   1c: Event
   20: Event
   24: Event
   28: Event
   2c: Event
   30: Event
   34: Event
   38: Event
   3c: Event
   40: Event
   44: Event         <- This is the last created event with the name
                        "\BaseNamedObjects\MyEvent00016383".  And its
                        handle is 0x10044