Daniel Pistelli

  1. IDAQ: The result of 7 months at Hex-Rays

    It is not a mistery that Hex-Rays is preparing for the IDA 6.0 beta program. In this post I'll write a bit about my personal, behind the scenes, experience with the project.

    It took me 7 months to port/rewrite the old VCL GUI of IDA Pro. The new GUI, as it had been already anticipated months ago on the official blog, is Qt based.

    The main difficulties I have faced were mostly not of technical nature, although it was a complex task, but psychological ones. It took a lot of patience and it was very difficult every morning to go to work and to have to see an unfinished product with the old GUI reminding myself how much was still to do.

    What follows is a rough roadmap of my work, I'll mention only the milestones and not the hundreds of smaller parts. It has to be noted that at least for what concerns the docking I wrote most of it before joining Hex-Rays to accelerate the development of the actual GUI once in the company. While Qt has a docking system, it is not as advanced as the one used by the VCL GUI, which is a commercial control. So, I wrote a docking system myself in order to offer all the advanced features the old GUI had.

    January: first impact with the code. Took me a week to grasp the initial concepts to start. Basically at the end of the month I could display disassembly and graph mode of a file. Also, hints, graph overview and disassembly arrows were implemented.

    February: implemented chooser and forms (which I actually completely changed internally, that's why I had to improve them again later on to obtain better backwards compatibility).

    March: marathon month. Implemented every day one or more dialogs/views such as: hex view, cpu regs view, enum view, struct view, options, navigation band, colors, etc. etc. More than 30, some very easy, some advanced controls such as the hex view or the cpu regs view.

    April: two weeks to finish the docking and smaller things.

    May: two weeks to implement the desktop part (the ability to save/restore layouts and options) and smaller things.

    June: fixes, help system and improved the forms implementation.

    July: Hundreds of fixes for the beta.

    While there will be still bugs to fix, I consider the project as completed and I wrote this post to close a chapter for myself.
    Categories
    Uncategorized
  2. My first month at Hex-Rays

    It is not a technical post, but I think I should put it here, because after all I read the Hex-Rays hiring announcement on the blogs of this forum.

    At the beginning of September I started looking for a job. I actually wanted a job to work from remote. Despite the fact that I got several offers, all of them required relocation. So in the end I saw the Hex-Rays hiring announcement on Woodmann and sent out my résumé. From all the relocations, Belgium was the nearest and best connected one and of course it's a very good job.

    The first month at Hex-Rays has been tough on all fronts. Mainly because of the relocation and getting used to work in an office. Now work is proceeding well, but the rest is still difficult. Having one day of sun here in Belgium would help, by the way.

    Musil wrote in his life's work that modern man is spending his life always increasing his level of expertise, remaining with a millimeter of specialistic knowledge which only few people in the world could really understand. The others, talking about his millimeter would only say stupid things and he himself can't move from his own millimeter without running into the same problem.
    I think I found my millimeter in the IT world. However, I can't stand still on it. I always keep moving with exasperated restlessness.

    I have written in the last 2 years at least 5 programs of bigger size which are almost complete, but as I'm now working at Hex-Rays they will have to wait. When I was writing mostly software on my own I did it almost entirely to be active in something. Now that I'm working on IDA every day I feel that my need of being active is fulfilled and I don't feel the need to write more code when I'm at home. Instead, I feel the need of art.

    Why haven't I tried producing art instead of programming already? Because I have always been capable of judging my own work objectively and I know when the time is not right.

    What's the difference between programming and art? Both need experience. The difference is that one can build one's technical experience alone in one's room, without the help of events, social interaction, etc. In a technical field it is possible to make the time needed for experience advance faster. Viceversa, in one's reflections about life one has to actually follow the time of his own life.

    I feel that something is changing about that.
    Categories
    Uncategorized
  3. Filter Monitor 1.0.1

    This week, after months of development of bigger projects, I found some time to windbg "ntoskrnl.exe" and write a utility. It is called Filter Monitor and shows some key filters installed by kernel mode components.

    http://www.youtube.com/watch?v=5iIeISGoHxM

    “As you probably all know the Service Descriptor Table has been a playground on x86 for all sorts of things: rootkits, anti-viruses, system monitors etc. On x64 modifying the Service Descriptor Table is no longer possible, at least not without subverting the Patch Guard technology.

    Thus, programs have now to rely on the filtering/notification technologies provided by Microsoft. And that’s why I wrote this little utility which monitors some key filters.

    Since I haven’t signed the driver of my utility, you have to press F8 at boot time and then select the “Disable Driver Signature Enforcement” option. If you have a multiple boot screen like myself, then you can take your time. Otherwise you have to press F8 frenetically to not miss right moment.

    A disclaimer: the boot process can be a bit annoying, but the utility should be used on virtualized systems anyway, as I haven’t fully tested it yet. I doubt that it will crash your system, I guess the worst scenario is that it won’t list some filters. It should work on any Windows system starting from Vista RTM and I have provided an x86 version and an x64 version. But the truth is that I have tested only the x64 version on Windows 7 RTM. Last but not least, I can’t guarantee that this utility will work on future versions of Windows, it relies heavily on system internals.

    Now, let’s run it. The supported filters/notifications at the time are these: Registry, Create Process, Create Thread and Load Image. “Registry” stands for CmRegisterCallback filters. “Create Process” for PsSetCreateProcessNotifyRoutine callbacks. “Create Thread” for PsSetCreateThreadNotifyRoutine callbacks. And “Load Image” for PsSetLoadImageNotifyRoutine callbacks.

    The “Additional Info” in the list view provides internal information like the address of the callback function.

    There are some default filters registered by system components, but, as you can notice, there are also Kaspersky components. That’s because some filters (like the registry filter) are not used by system components and I needed a tool which would make use of these filters for my little demonstration.

    The version of Kaspersky I have installed is the latest one available on the internet which is: 9.0.0.463.

    I created for this demonstration a little executable called “k-test” (what you see on the desktop are three copies of the same executable) which copies itself in a directory called “borda” in the “Roaming” directory of the operating system. It then creates a value in the Run key of the registry to execute itself at each start-up. Finally, it launches itself from the “Roaming” directory and ends.

    This is a typical malware behavior. Beware that the signature of the application itself is not contained in the databases of Kaspersky as I have written it on the fly, but it detects the suspicious behavior, stops execution and deletes the file. And it does this every time I launch the test application.

    Now let’s get to the part where I show an additional functionality of the Filter Monitor which is the ability to remove registered filters and see what happens if I remove the filters installed by klif.sys, which is the “Kaspersky Lab Interceptor and Filter” driver. As the name suggests, this driver intercepts and filters: it installs all four of typologies of filters listed by the Filter Monitor. On x86 instead of calling CmRegisterCallback it additionally hooks about 60 functions of the Service Descriptor Table (which is a lot), but that’s no longer possible on x64.

    So, let’s remove the filters and re-launch k-test. It works now.

    Final disclaimer: It is not my intent to comment on security features of anti-viruses, I just wanted to present my new tool and show its functionalities. I was already familiar with the internals of Kaspersky before writing this utility.

    I hope you enjoyed the presentation.”


    P.S. A huge thanks goes to Alessandro Gario for providing me with all the different versions of ntoskrnl.exe.
    Categories
    Uncategorized
  4. Native Blocks Pre-Alpha

    Hello, I wanted to show you my new tool. I called it Native Blocks. It's a re-assembler basically. Since a written presentation would take me too much time I prepared a video presentation.

    http://www.youtube.com/watch?v=pDlwStN3KZY
    http://www.youtube.com/watch?v=ene6hvE5qVQ

    Again, this is a pre-alpha. This tool will soon support other technologies such as java, actionscript and maybe even x86. Right now it only supports .NET (and even .NET support will be hugely improved, like for instance supporting the direct modification of assemblies without having to use Rebel.NET).

    The development of this tool depends mainly on the interest of people.

    As I can be considered a student from now on, I would like to earn some extra money by writing tools such as this one. I have still my job as consultant, but it's a very limited partime, because I just became a student.

    This tool is in my opinion pretty good, it is not only good for deobfuscation purposes but also patching and assembling on the fly.

    If this tool can be sold, then the support of technologies will depend on requests. I think I'll add Java immediately and after that maybe x86/x64. Again it depends.

    Suggestions and comments are welcome.

    Updated July 3rd, 2009 at 16:36 by Daniel Pistelli

    Categories
    Uncategorized
  5. x64 SEH & Explorer Suite Update

    Yesterday I took a bit of time and updated the Explorer Suite. One important new feauture is the addition of the Exception Directory. I'm no longer working on the old CFF Explorer. However, I thought this feature was too important for people to wait for the new CFF Explorer. Here's a screenshot of the Exception Directory UI:



    If you have no idea how the x64 Structured Exception Handling works, you can briefly read this article on <a href="http://www.osronline.com/article.cfm?id=469">osronline</a> or my article about Vista x64. There's also a pretty in depth quantity of information in a series of posts on <a href="http://www.nynaeve.net/?p=99">Ken Johnson's blog</a>. However, don't hope to find too much information on the web about the real physical layout of the Exception Directory. The MSDN information is incomplete if not wrong and even the SDK doesn't help. This post isn't a complete guide to x64 exceptions, I just want to explain how to analyze them inside the CFF Explorer.

    In the screenshot above you can see two arrays of tables. The first one is an array of RUNTIME_FUNCTION structures. The last column isn't part of this structure though: it shows the five high bits of the first byte of the UNWIND_INFO structure refrenced by the UnwindData member of RUNTIME_FUNCTION. This is the declaration of UNWIND_INFO:

    Code:
    typedef struct _UNWIND_INFO {
        UBYTE Version       : 3;
        UBYTE Flags         : 5;
        UBYTE SizeOfProlog;
        UBYTE CountOfCodes;
        UBYTE FrameRegister : 4;
        UBYTE FrameOffset   : 4;
        UNWIND_CODE UnwindCode[1];
    /*  UNWIND_CODE MoreUnwindCode[((CountOfCodes + 1) & ~1) - 1];
    *   union {
    *       OPTIONAL ULONG ExceptionHandler;
    *       OPTIONAL ULONG FunctionEntry;
    *   };
    *   OPTIONAL ULONG ExceptionData[]; */
    } UNWIND_INFO, *PUNWIND_INFO;
    The flags represent the type of handlers. An exception flag represents __try/__except blocks, while the termination flag represents __try/__finally blocks.

    The second is an array of scope records. An UNWIND_INFO can contain more than one scope records. Let's consider this little code sample:

    Code:
    __try
    {
    	__try
    	{
    		// code
    	}
    	__finally
    	{
    		// code
    	}
    	
    	__try
    	{
    		// code
    	}
    	__finally
    	{
    		// code
    	}
    }
    __except (EXCEPTION_EXECUTE_HANDLER)
    {
    	// code
    }
    As you can see from the screenshot, it results in 3 scope records. The HandlerAddress in a scope record structure can be an RVA to a C_exception_handler function. Or it can be a simple value like EXCEPTION_EXECUTE_HANDLER (which is 1). The last scope record represents the __except statement. Don't confuse the exception handler (or filter) with its code.

    The JumpTarget member, if not 0, is an RVA to the exception code. It's possible to see if a particular address has an entry inside the Exception Directory by right clicking on the first table and then clicking 'Is Address Handled' in the pop-up menu. Nevertheless, remember that exception handlers can be added at runtime with APIs like RtlAddFunctionTable and RtlInstallFunctionTableCallback.

    I fixed some minor bugs in the CFF Explorer and one major bug in the Task Explorer. I noticed this bug years ago but never took time to fix it. It showed only when trying to dump the region of an x86 process using the 64 bit version of the Task Explorer. However, x64 is becoming very used and so the bug is now fixed. Also, I thought it a good idea on 64-bit platforms to install a 32-bit version of the Task Explorer and a 64-bit one. Thus, the installer now behaves accordingly.

    Updated January 18th, 2009 at 19:16 by Daniel Pistelli

    Categories
    Uncategorized
  6. Dynamic C++ Proposal

    As anticipated, I just finished my Dynamic C++ proposal. This is not a reversing related article, but a good knowledge of C++ internals and assembly is necessary to read it. I'm glad I'm it's over, frankly. I'd like to thank my friend Quake2 for all his support during the last month.

    http://ntcore.com/Files/dyncpp.htm

    I wrote this document because I needed to express myself about this subject. Despite the fact that C++ is one of the most used programming language, especially for serious projects, it gets much criticism for being messy, bloated, complicate etc. I believe these critics miss the point. Yes, C++ is a very powerful programming language and that's why it is difficult. And this is also why sometimes C++ source codes are poorly written. I don't believe in improvements of C++ resulting in a new programming language. All attempts in that direction have failed. I think that C++ is here to stay for many reasons. Not only because of the amount of code already available in C++, but also because at the moment there isn't a better programming language for large projects. The only thing I want is for C++ to evolve, but not by losing compatibility with older code or by removing some features. No, I'd like C++ to evolve in a healthy and compatible way. This paper contains the suggestions to achieve this and I will demonstrate technically how it can be implemented at low level.

    Everybody should be warned that the material contained in this paper is purely theoretical. The first idea behind this paper came out while working on a particular project. At the time I discovered myself in need of particular dynamic features. So, for many months I had some ideas in the background of my mind and decided eventually to write them down. So, in this paper I'm going to talk about the current status of dynamism for C++, why dynamism is important and what could be done. At the time of writing (November 2008) the new C++0x (or C++09) standard has not yet been introduced. However, I will talk about it throughout this paper when the related topic is affected by it.

    Updated December 17th, 2008 at 18:01 by Daniel Pistelli

    Categories
    Uncategorized
  7. Qt Internals & Reversing

    Today I took a break from the larger article I'm currently writing. To relax, I wrote a smaller article about the Qt framework. I hope you enjoy.

    Qt Internals & Reversing

    Half of the text of this article comes from my larger paper "Dynamic C++ Proposal". I decided that it was useful to take the part about Qt internals, put it into another article and extend it by adding a reversing part. Because of its nature, this is not the usual kind of article I write. In fact, I wrote the reversing part in less than a day. So, this is a very easy one. However, I think it is useful for people who need to reverse a Qt application and certainly wouldn't consider reading my other paper about Dynamic C++, which doesn't sound like a paper about Qt and, in fact, isn't a paper about Qt: the paragraph about Qt is only one among many others. Moreover, I haven't seen serious articles about this subject.

    The first thing which needs to be considered when reversing Qt applications is what Qt brought to the C++ language. Events (inside the Qt framework) are just virtual functions, so nothing new there. This is not a C++ reversing guide. What is new in Qt are signals and slots, which rely on the dynamism of the Qt framework.

    So, first thing I'm going to show how this dynamism works. The second part focus on reversing and, at that point, I will show how to obtain all the metadata one needs when disassembling a "Q_OBJECT" class.

    Updated November 27th, 2008 at 19:42 by Daniel Pistelli

    Categories
    Uncategorized
  8. Small Devices & RCE

    Didn't want to go off-topic in the other thread, that's why I'm opening a new one. I wanted to add some thoughts about the IDA-on-IPhone news.

    Good news for real iPhone fans: we ported IDA to iPhone! It can handle any application and provides the same analysis as on other platforms. It is funny to see IDA on a such small device:

    http://hexblog.com/2008/07/ida_on_iphone.html

    Ilfak Guilfanov
    I think it's awesome.

    It's also funny, because in theory the new CFF Explorer will be compilable for mac os (being written in Qt), thus also IPhone. The only problem is the small display of such devices and I'm not sure if there's a possibility to reduce the needed space, but I'm quite optimistic.

    I mention this because the new CFF Explorer will support elf and other formats (lib, object, symbian etc), making it useful also for other systems and it might become part of a new generation of cross platform/device tools. It would be encouraging to know that in the future it will be possible to do reversing stuff on such a small device. The new CFF will also have zoom in/out features for the hex editor, making it very useful on devices with a small (or big) display.

    I hope that other programmers will follow the same lead.

    The main problem is writing cross platform applications and reorganizing GUIs for small displays.

    I want to share something I read on wikipedia some time ago:

    Microsoft software is also presented as a "safe" choice for IT managers purchasing software systems. In an internal memo for senior management Microsoft's head of C++ development, Aaron Contorer, stated:[7]

    “The Windows API is so broad, so deep, and so functional that most Independent Software Vendors would be crazy not to use it. And it is so deeply embedded in the source code of many Windows apps that there is a huge switching cost to using a different operating system instead... It is this switching cost that has given the customers the patience to stick with Windows through all our mistakes, our buggy drivers, our high TCO (total cost of ownership), our lack of a sexy vision at times, and many other difficulties [...] Customers constantly evaluate other desktop platforms, [but] it would be so much work to move over that they hope we just improve Windows rather than force them to move. In short, without this exclusive franchise called the Windows API, we would have been dead a long time ago.
    Companies such as Apple and Microsoft are very conscious of the strategic importance of hard binding applications to their propretary API. That's why Apple pushes cocoa and Microsoft .NET. They don't want cross platform development environments (oh and don't tell me that .NET is cross-platform, before doing so, show me a .NET GUI with more than a button in it on a system which isn't Windows), because it would make possible for users to switch to another system without losing his tools.

    However, "the times they are a changin'". Nowadays, developers are more conscious about this problem and prefer not to bind their application to only one platform. You can notice this if you pay attention to the names of newer applications. Ten years ago there were lots of windows applications which contained the word "win" in them. Winhex, WinDvd, Winzip, WinRar, WinAce, Winamp etc. etc. etc. Have you noticed that this trend has stopped? It's interesting, right now a struggle between developers and OS producers is taking place. OS producers want to ever more bind (even more than before) developers to their platform. Why do I say more than before? Well, consider that .NET implements its own languages, you can't simply share real C++ code with the managed one (yes, you can rely on pinvoke, but not for everything). Well, it's a bit more complicate than that, I know, but unsafe code is not encouraged in the .NET environment. Meanwhile, Apple pushes Obj-C. I want to know how this ends. Speaking for myself, I refuse to take a side and will stick with my beloved C++ (the real one).

    I hope this post won't generate a big controversy like the one about Windows Vista.

    Updated July 25th, 2008 at 08:52 by Daniel Pistelli

    Categories
    Uncategorized
  9. Phoenix Protector 1.3.0.1

    http://ntcore.com/phoenix.php

    This application is now freeware for various reasons. I first wrote the core of the Phoenix Protector for a company when I was 19. That project didn't work out for internal reasons not related to the quality of the product itself. I then wrote the Phoenix Protector, which was basically a new GUI for the now improved core. However, during the years (one has to consider that I was quite young when I developed the .NET obfuscator) I became ever more conscious that I didn't want to spend my life writing protections and also that I was not convinced by protections for the .NET technology in the first place. That's partly why I wrote Rebel.NET. By combining Rebel.NET with the DisasMSIL engine it is very easy to write a code obfuscator for .NET assemblies. The only thing which would be missing is name obfuscation and string encryption, which are even easier as protections. That's why I'm releasing the Phoenix Protector for free: nowadays, writing a commercial obfuscator doesn't make much sense to me. The code obfuscation provided by the Phoenix Protector is quite good when compared to other commercial obfuscators. I noticed that most obfuscator provide a very easy to reverse code obfuscation scheme. I'm not saying that the Phoenix Protector's code obfuscation can't be reversed. Every .NET code obfuscation scheme can be reversed somehow and the rebuilding task becomes very easy through Rebel.NET.
    Categories
    Uncategorized
  10. .NET Internals and Native Compiling

    http://ntcore.com/Files/netint_native.htm

    Strictly speaking it means converting the MSIL code of a .NET assembly to native machine code and then removing the MSIL code from that assembly, making it impossible to decompile it in a straightforward way. The only existing tool to native compile .NET assemblies is the Salamander.NET linker which relies on native images to do its job. The "native images" (which in this article I called "Native Framework Deployment") technique is quite distant from .NET internals: one doesn't need a good knowledge of .NET internals to implement it. But, as the topic is, I might say, quite popular, I'm going to show to the reader how to write his Native Framework Deployment tool if he wishes to. However, the article will go further than that by introducing Native Injection, which means nothing else than taking the JIT's place. Even though this is not useful for commercial protections (or whatever), it's a good way to play with JIT internals. I'm also going to introduce Native Decompiling, which is the result of an understanding of .NET internals. I'm also trying to address another topic: .NET Virtual Machine Protections.

    I hope you'll enjoy this.

    P.S. As always, if you notice typos, please report them.

    Updated May 25th, 2008 at 10:53 by Daniel Pistelli

    Categories
    Uncategorized
  11. .NET Internals and Code Injection

    http://ntcore.com/Files/netint_injection.htm

    The first article of the two is out. The next will be about .NET native compiling.

    If you notice typos in the text, please do tell me. I'm a bit wasted, as you can see it's a long article.

    The content should be quite a new thing. I hope you enjoy the journey into the .NET internals from the perspective of a reverser.

    Also the applications of this can be many.
    Categories
    Uncategorized
  12. DisasMSIL and CFF Explorer

    Today I wrote a free/open disasm engine for Microsoft's Intermediate Language (MSIL).

    http://ntcore.com/Files/disasmsil.htm

    You can use it any context you wish. There are no license restrictions. The only thing I ask you to do is to send me your bug fixes (if any).

    I also added the MSIL disasm engine to the CFF Explorer which is now able to disassemble the methods' code directly from the MetaData tables.



    I hope you aren't already fed up with me and my updates =)
    Categories
    Uncategorized
  13. My next 2 articles

    Not that I want to make a big announcement, just wanted to tell you what those articles will be about.

    The first as I already told you is about code injection.

    The second one is about .NET native linking (no MSIL left). And even WITHOUT .NET native images, something I came up just today and already discovered that it can absolutely be done.

    I expect these articles to be released in the next 10 days. I decided to split the two of them up because they treat different things and the native linking is just "too hot".

    Just wanted to let you know...

    Updated April 28th, 2008 at 03:13 by Daniel Pistelli

    Categories
    Uncategorized
  14. Rebel.NET

    As promised, I'm hereby releasing the Rebe.NET software.

    http://ntcore.com/rebelnet.php
    http://ntcore.com/Files/rebelnet.htm

    Rebel.NET is a rebuilding tool for .NET assemblies which is capable of adding and replacing methods and streams. It's possible to replace only a limited number of methods or every method contained in a .NET assembly. The simplicity of Rebel.NET consists in the replacing process: one can choose what to replace. Rebel.NET is, mainly, a very solid base to overcome every .NET protection and to re-create a fully decompilable .NET assembly. As such, Rebel.NET has to be considered a research project, not an encouragement to violate licensing terms.

    As I have written the software and the article in this week when during my sickness (fever), I'm expecting bugs and typos. Please report them.

    Of course, I've tested the Rebel.NET with more advanced .NET assemblies than those presented in the guide.

    Updated April 25th, 2008 at 14:59 by Daniel Pistelli

    Categories
    Uncategorized
  15. DynLogger

    http://ntcore.com/dynlogger.php

    DynLogger logs all dynamically retrieved functions by reporting the module name and the requested function. It can come very handy when one wants to know a "hidden" function used by an application.

    I recycled the code of a bigger project to write this little application. It's a very small utility, but it might be of use after all. It was tested on XP and Vista, both x86 and x64. It works for .NET application as well. Just start the logging process, the log will be saved after you quit the monitored application.

    I wasn't really sure if I should have posted it here or not. But a friend of mine needed it, so I figured out that maybe even other people might need it.
    Categories
    Uncategorized
Page 1 of 2 12 LastLast