Results 1 to 15 of 15

Thread: How best to approach encrypted files?

  1. #1
    VeeDub
    Guest

    How best to approach encrypted files?

    Hi,

    I have a target that stores configuration data in separate files. The target application has some annoying limitations due to some assumptions by the developer. I believe these limitations can be circumvented by modification of the configuration data. Unfortunately the configurations files are "encrypted", which means it is not possible to edit the configuration files directly using a text editor.

    My original intention was to trace the target, identify the encryption algorithm, then port that logic into a separate program (in a higher-level language) so that I could convert the data file to text - modify the contents - and then convert the data file back so that the application could read it.

    I have traced the target and identified the encryption algorithm, unfortunately it's fairly lengthy, does many operations, references the stack quite a bit - and basically trying to follow the logic - so that I can reproduce the logic in my program is doing my head in.

    I'm now wondering if there might be an easier way to go about this. One option I know is to use the existing assembly logic, and once the file is converted in memory, write the buffer out as a text file, but I am a novice at this level and while I can probably manage that - it's not what I consider to be a user-friendly option.

    Conceptually what I would like is some sort of "386 virtual machine", where I can enter the encryption logic in assembler. Then use a higher-level language to load the data file, call the "virtual machine", then write the output to a text file.

    Do these sort of tools exist?

    Or am I going about this the wrong way, and is there a simpler approach?

    Thanks

    VeeDub
    I promise that I have read the FAQ and tried to use the Search to answer my question.

  2. #2
    1. Are you sure the configuration files are encrypted text files? They might be in proprietary binary format...

    2. Is it a standard encryption algorithm, or a custom-made one? With a standard encryption algorithm like DES/AES/RC4 etc. sourcecode can be found on the Internet, and you can just extract the key from the program.

    If you know what portions of the code handle the decryption/encryption, you can just copy+paste entire sections into a new source file, adjusting the code as necessary to fit memory locations etc. and add the connecting code, then assemble and test. This should be the simplest approach... if you want to produce a separate program to do the decryption.

  3. #3
    VeeDub
    Guest
    Hi LLXX,

    Quote Originally Posted by LLXX
    1. Are you sure the configuration files are encrypted text files? They might be in proprietary binary format...
    That's a good point, I looked up the file-type on a "file type" Web site that I found on Google. I can't recall the URL, but the only application recognised was the one that I am using - so it's most likely a proprietary format and possibly even binary. I don't think that will matter though for what I'm trying to achieve. It's a bit difficult to explain without going into specifics, but essentially I want to copy the majority of an existing job definition and just make one alteration (which the program won't permit) and which their support think is a good idea - but the program doesn't currently support (don't I know it!). The field that I want to modify is in Unicode when the file is decoded in memory so as long as I preserve the length I should be OK. Anyway time will tell.

    Quote Originally Posted by LLXX
    2. Is it a standard encryption algorithm, or a custom-made one? With a standard encryption algorithm like DES/AES/RC4 etc. sourcecode can be found on the Internet, and you can just extract the key from the program.
    About that I have no idea, although I have been interested in reversing for a while, this is my first real attempt at actually tracing a program. So I don't recognise what the code is doing. FWIW I don't think it is that "smart", it reads in two blocks of 4 bytes at a time, then goes through a few loops (which on a trace count goes for about 450 steps). The 8 bytes get converted to 8 bytes and then the next block is processed. It is not going to be impossible to follow the logic, it is just a lot more involved than I had anticipated. A lot of the Olly tutorials I have read have been far more straightforward, and I guess at this point I was just pondering whether maybe there was an easier approach. I have been mulling this over in my head since I made the initial post, the issue really is the number of operations that are performed, I'm probably just going to have to get a notepad and work through the routine slowly. A PITA.

    Quote Originally Posted by LLXX
    If you know what portions of the code handle the decryption/encryption, you can just copy+paste entire sections into a new source file, adjusting the code as necessary to fit memory locations etc. and add the connecting code, then assemble and test. This should be the simplest approach... if you want to produce a separate program to do the decryption.
    Yes well this raises two issues. I understand that conceptually this is probably the way to approach the problem as this way I can use the existing assembly code. However the last time I actually programmed in assembly was with an Apple II (and funnily enough I can still remember a lot of the calls - we used to code directly - no need for an assembler). But I don't even have a copy of MASM, yet alone know how to use it. This is why I have a preference for "understanding" the algorithm so that I can implement in a higher-level language that I do understand. Or a way to "run" the assembly code from a higher-level language. I was really hoping rather than expecting a positive answer on this. The second issue, which I will be able to solve, is at the moment I haven't gone looking for the "encrypt" algorithm. I had assumed that I would be able to understand the "decrypt" algorithm and just reverse it myself.

    Given the nature of the application, I'm surprised that the converting the files contents is so complex, I guess someone had their reasons. Just my luck not to get some simple bit arithmetic conversion. Shit happens.

    Thanks for your thoughts, they help me to clarify my thinking.

    Cheers

    VeeDub
    I promise that I have read the FAQ and tried to use the Search to answer my question.

  4. #4
    hi,

    just a "stupid" tip - a quick and rather dumb check would be to load your
    app into PEiD, and run the crypto scan plugin against it. maybe you are lucky?

    so you can focus on understanding the algos more. by just looking at the
    encrypted file itself its a bit hard to give any advices.

    regards, 0xf001

  5. #5
    VeeDub
    Guest
    Hi,

    That's an interesting suggestion, I have a copy of PEiD, it told me that the app was written in Visual C++ 6.0. But that's about as far as I went with PEiD, I was not aware that there was a crypto plug-in.

    I should clarify though, it's not the app that is encrypted, it's the data files that the app uses that have been scrambled, but what you're saying is that PEiD can scan the app for crypto algorithms?

    I am definitely going to look for that crypto plug-in before I move forward.

    Most likely though I think I am going to stick with the original plan, but now rather than try and follow the algorithm, I'll just reproduce it in my "config file editor" application. Perhaps as I work through the code it might become clearer to me what the algorithm is actually doing - in which case I might be able to simplify the code - and write the encrypt algorithm myself. If not I'll find the encrypt algorithm and repeat the process.

    Cheers
    VeeDub
    I promise that I have read the FAQ and tried to use the Search to answer my question.

  6. #6
    Registered User
    Join Date
    Mar 2004
    Location
    maze of twisty little passages, all alike
    Posts
    133
    It's been a very long time since I downloaded my copy of PEID, but as I recall, there's no need to go looking for the plugin. The "Krypto ANALyzer" plugin comes with PEID if you downloaded the official release. From the main dialogue, hit the -> button and look under the plugins flyout menu.

    The crypto plugin will search for the "thumbprints" of most well-known crypto algorithms. For example, most hash algorithms make use of some very specific constant values to initialize the hashing rounds. Using MD5 as an example, the psuedo code will always contain something like this:
    Code:
    h0 = 0x67452301
    h1 = 0xEFCDAB89
    h2 = 0x98BADCFE
    h3 = 0x10325476
    So if you see those constants in close proximity to one another, it's very likely to be part of the MD5 algorithm's prologue.

    So the short answer is that the crypto analyzer searches for encryption algorithms, not encrypted data. It will find locations where the program makes use of encryption, even if the program isn't en/decrypting itself.
    The debugging of a thousand lines of code begins with a single-step.

    "It has always therefore been one of my main endeavors as a teacher to persuade the young that first-hand knowledge is not only more worth acquiring than second-hand knowledge, but is usually much easier and more delightful to acquire." -- C.S. Lewis

    I think I can, I think I can, I think I can...

  7. #7
    VeeDub
    Guest
    Hi TBone,

    I think there must have been some issue with the forum as some of my (and others) earlier posts to this thread have gone missing.

    As you explained PEiD includes the KANAL plug-in (which sigint33 explained as well). I ran the analysis on the exe and it reported 7 signatures:

    Adler32
    Blowfish Sbox 2
    CRC32
    MD4
    MD5
    PI fraction (NIMBUS / BLOWFISH)
    ZLIB deflate [long]
    I promise that I have read the FAQ and tried to use the Search to answer my question.

  8. #8
    Your data files are probably encrypted with Blowfish and may be compressed with zlib as well.

  9. #9
    VeeDub
    Guest
    Quote Originally Posted by LLXX
    Your data files are probably encrypted with Blowfish and may be compressed with zlib as well.
    If I accept that this is in fact the case, shouldn't the addresses referenced in the algorithm I am analysing correspond with the addresses reported by PEiD? for Blowfish (Because they don't).

    Or should the algorithm refer to the addresses reported by PEiD, because the plug-in has identified some constants? (Again the algorithm I am looking at never refers to the address reported by the plug-in for the Blowfish).

    I have an open mind about this. But at this stage, if either of my questions above should be answered in the affirmative, then the algorithm I am analysing does not use Blowfish.

    Actually, I have just had a brainwave. The application has the option to encrypt the files that it copies - using blowfish, it also has the option to compress these files, that's why those references are present.

    So Kanal is correct in it's analysis, but so am I, the program's own data files are not encrypted with blowfish.

    Which at the end of the day leaves me with working through the algorithm. This is proving to be slow going.

    Say I wanted to somehow extract and use the existing decryption code, would I be copy & pasting the relevant sections into MASM for instance?

    Another option is to spend a little bit more time researching the file type of the config file, which given I know have a better handle on what's involved if I can't find a config file reader, I think is worth doing.

    I never expected this task to be easy, but nevertheless it is turning out to be more involved than I anticipated.
    I promise that I have read the FAQ and tried to use the Search to answer my question.

  10. #10
    Quote Originally Posted by VeeDub
    Say I wanted to somehow extract and use the existing decryption code, would I be copy & pasting the relevant sections into MASM for instance?
    Into a source file, along with some support code which you must write yourself. Of course, addresses and the such must be changed in order for the code to work, and any data tables will have to be copied as well.

  11. #11
    VeeDub
    Guest
    Quote Originally Posted by LLXX
    Into a source file, along with some support code which you must write yourself. Of course, addresses and the such must be changed in order for the code to work, and any data tables will have to be copied as well.
    I had no luck with the file type, basically it looks unique to this program. Also I have spent more time analysing the code and it's making fairly extensive use of the stack. As quite a few of the entries on the stack have been set before the algorithm is called, that means there is actually more "setup" code required to decrypt the files, in addition to the algorithm I have been examining.

    I think the only practical option is to reproduce the relevant code in a custom app in assembler. This approach has the advantage that I can just copy the relevant chunks of existing "decrypt" code.

    The problem is that I'm going to have to learn how to use Masm and also write some front-end and linking code to get this all to work. This has turned out to be a fairly challenging first up choice, still I have learnt about PEiD.

    I'll post back in the future if I'm successful!

    Thanks for your advice, it has been very helpful. (I may return asking some questions about Masm, but hopefully I'll be able to work that out myself).

    Cheers,
    VW
    I promise that I have read the FAQ and tried to use the Search to answer my question.

  12. #12
    hi,

    its probably this file uses blowfish but you can only tell it by looking into the code yourself. peid gives you the fingerprint of some algos, as part of those algos are numeric constants (arays) which can easily
    be identified. zlib basically means zip compression.

    now, this is only a little but also not unimportant step to identify the algo. now that you have code references, you could check those functions and verify (ie those constants, you see them in peid kanal plugin source). you can better do that by grabbing the source of the algos, or passively:

    how about checking input / output parameters? feed it a block of known values), etc ...

    does the app ask for a key, or not? if not, then the key to decrypt is accessed in the execution flow, around those addresses listed by peid.

    decrypting unknown stuff - there's not a real general way, if you could
    explain the situation a bit more, i could better answer.

    does this app ask for a key, or keyfile, or .... are just the files protected to be viewed from outside the app etc ...?

    regards, 0xf001

  13. #13
    VeeDub
    Guest
    G'day ,

    Quote Originally Posted by 0xf001
    its probably this file uses blowfish but you can only tell it by looking into the code yourself. peid gives you the fingerprint of some algos, as part of those algos are numeric constants (arays) which can easily
    be identified. zlib basically means zip compression.
    The application does provide options for encrypting (you supply the key) and compressing files that it acts upon. This activity however doesn't concern me, as I am more interested in the application's own configuration files.

    decrypting unknown stuff - there's not a real general way, if you could explain the situation a bit more, i could better answer.

    does this app ask for a key, or keyfile, or .... are just the files protected to be viewed from outside the app etc ...?
    The app does not ask for a key, all the logic (and constants) to decode the configuration files is internal to the app.

    I think conceptually the method to approach the target is straight-forward enough. Trace the target from the time it reads the config files into memory and decrypts the data, and copy that code into a separate stand-alone app. Repeat the process with the encryption algorithm / write the encrypted config file back to disk. Then I can read the original config files, write them out to a separate file unencrypted - make changes to the unencrypted config file - then re-encrypt the modified config file for the app to use.

    The challenge is in the actual doing as I don't have any experience in Win32 assembly programming, so at the moment I'm learning about how to open a dialog box in assembler.

    My original intention was to trace the target application, understand the encryption methodology, and then write the decryption logic in a higher level language that I am comfortable with.

    However I made the assumption that the encryption algorithm would be pretty simple and that I would be able to understand it by tracing the app. It may well be that the algorithm is not that advanced for someone with experience in this area but as it turns out it is more than a match for me - so I can't go with my original approach (it's roughly 450 steps per 8 bytes processed).

    I appreciate your offer to help, but short of writing the program for me (which you're welcome to do!), at the moment I don't think there is much more advice that anyone can provide.

    I may have some more questions if I master the basics, and get to the point where I am trying to integrate the app's code into my own program - if I get that far!

    Thanks

    VW
    I promise that I have read the FAQ and tried to use the Search to answer my question.

  14. #14
    heya,

    so when it doesnt ask for a key, thats very good. so i think a shortcut could be:
    check your encrypted file, ie note the 1st 16 bytes from a hex editor.
    break around the encryption function -> see where the encrypted block is accessed in memory - so scan the mem for those bytes at runtime.
    or .... other approach to get it: check for file access functions, and see where the encrypted file gets read and then stored in mem.

    once found, check where those functions decrypt the block to - or where the actual key gets read. from there you can trace backwards to see where that
    key comes from. that should not be too difficult i think?

    regards,

    0xf001

  15. #15
    Registered User
    Join Date
    Aug 2005
    Location
    Greece
    Posts
    157
    hi,
    if you dont want to understand how the encryption/decryption routine works and you only want to make the application read/write its configuration files in plain text you can disable these routines and fix the variables for the read/write buffers (and buffer sizes if needed).

    i suppose the application reads the data for the file in a buffer, then decrypts the data in that buffer and very possibly copy them to another buffer for parsing. what you need to do in situation like this is to make the program skip the decryption routine and give for parsing the first buffer instead of the second.

    ofcourse you should copy/dump from memory the plain text configuration data after the application decrypted them before using it for the above.

    if the application write back into the configuration files you need to skip encryption and swap the buffers again.

    have fun
    A picture worth 1K words (or .5K DWORDS).

Similar Threads

  1. encrypted harddisk
    By OHPen in forum RCE Cryptographics
    Replies: 13
    Last Post: January 29th, 2009, 12:55
  2. old encrypted cd
    By kqt in forum Malware Analysis and Unpacking Forum
    Replies: 2
    Last Post: June 4th, 2002, 01:24
  3. Editing DOS encrypted files
    By Unregistered in forum Malware Analysis and Unpacking Forum
    Replies: 0
    Last Post: October 10th, 2001, 06:28
  4. encrypted .exe?
    By spamal in forum Malware Analysis and Unpacking Forum
    Replies: 8
    Last Post: January 4th, 2001, 15:48

Bookmarks

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •