Webpage :- http://www.mutek.com (Trial Version 8.44Mb's).
About 18 months ago I remember looking at BugTrapper as a potential tool of our trade, its pretty much an automatic combination of a disassembler and SoftICE. You can get the trial from MuTek's webpage, they change the version a lot and monitor cracking sites too, the last time I published a crack they changed the version within 2 days so watch out. In terms of protection little has changed, you are supposed to insert a license key based on a System ID, ordinarily I might have considered buying BugTrapper were it not for this very restrictive 7 day trial.
The first thing you'll notice is MuTek's awareness of cracking techniques. Bt.exe is packed with what looks like an older version of ASPack, this has none of the anti-tracing and debugging tricks implemented by more recent versions. MuTek also saw fit to name their ASPack section .fucku, thats pretty unprofessional even in todays software world :-). At the end of this tutorial we'll manually unpack ASPack or modify it so that BugTrapper runs under NT, in the interim we'll use a very quick fix approach. With TRW2000 or IceDump/Bhrama set a breakpoint for 4B2EFA and dump the .exe to disk with pagein b or makepe. Load your newly unpacked Bt.exe into IDA (W32Dasm just won't cut it).
Now for some analysis, the OK button only enables when you enter a valid code (or at least a code that passes some preliminary checks). BugTrapper has various options too, using a HEX editor just take a look for 'Lite', you'll find the other option names lying pretty nearby (VB, VC++, I-Net, Support, Pro, Never). If you paste the System ID into the Registration Key dialog the OK button should enable, of course its not valid, but this confirms our theory of a 2 stage checking mechanism. With bpx's on Hmemcpy or GetWindowTextA and bpm work you'll find the validator sub_458010.
I traced through sub_458010, this function returns 1 of 4 possible values, 1 or 2 if your length doesn't match and 5 if the code is invalid, 0 of course is good. sub_4B2946 is an uppercasing routine and sub_4B2760 is somewhat mysterious, although when one is cracking your observation is sometimes as useful as your ASM analysis skills, I saw that the function took 2 parameters, a char/digit from your Registration Key (via EAX) and a pointer to a structure '0123456789KNUTLF', I looked also at the System ID and saw it was made up exclusively of just these table entries, what significance this has is unclear and at this point I wanted to really avoid pouring over lots of code.
Another thing I quickly noticed was how to get EAX=0 easily, I used as my bogus Registration Key 1111 2222 3333 4444 5555 6666 which might have been fortunate, look at this code :-
:00458111 MOV EAX, EDI <-- In EAX sum of all char's
:00458114 AND EAX, 0Fh <-- Keep lowest nibble.
:00458118 SUB EAX, EBX <-- EBX is last char's value (-30h).
:0045811B NEG EAX <-- We need EAX=0 here.
:0045811D SBB EAX, EAX
:00458120 AND EAX, 5 <-- Set bad guy EAX=5 if EAX != 0 at 0045811B.
You can easily see that (1+1+1+1) + (2+2+2+2) + (3+3+3+3) + (4+4+4+4) + (5+5+5+5) + (6+6+6) = 78 dec. (0x4E). Therefore changing the last 4 digits of the key to 6608 sets EAX=0 as required thus enabling the OK button. Of course doing this doesn't make a valid key, only one that satisfies this function, without delving to deeply into the code I was guessing this would hold so long as I used only numbers in the Registration Key (introduce a letter and it certainly doesn't hold). Now that we have a key that allows us to press OK I thought I'd try and see what enabled the various options. We'll do this by using our HEX editor.
Inside Hiew we find the following options (please note here that I'm using the TRW 2000 unpacked Bt.exe) :-
.00529124 Maintenance Till (45864F).
.0052913A VC++ (4585C7).
.00529142 VB (4585B4).
.0052914A Trial (4585A1).
.00529150 Wrong Mode (458574, 45858E).
.0052915C I-Net (45854D, 458563, 45856D).
.00529164 Support (45852A, 458546).
.0052916C Pro (458503, 458519, 458523).
.00529170 Lite (4584D6, 4584EC, 4584F9).
.00529178 Soon (4588B6).
.005291A0 Never (458769, 458783, 4587A4).
I didn't know whether or not you could have a Lite & Pro license together, obviously we want to avoid the Trial license at all costs. What I've added above is the addresses where these areas are referenced in the code (perform a HEX search for 91 52 00). What becomes very apparent is that the entire BugTrapper option validation dwells within some 500 bytes, some of the dual references are merely simple string length routines. Still using Hiew I reckoned CALL 458470 was a good bet, just by scrolling upwards from the Lite reference at 4584D6. IDA doesn't disassemble this CALL for reasons unknown, select Edit Code and it will. The Never reference is reached via separate sub_458730, when sub_458470 returns you should be able to force a path too it (don't forget to soft patch past the expiry date calculations though).
Back in the real world, the reality of this is that we are jumping too many steps forward, we can see where verification ought to be performed but clicking OK with our Registration ID won't break there. Searching out the Invalid Key message in Hiew is easily the best way to locate sub_456B60. Here's the real deal reversers, I single stepped to 456B70 and at ESI saw my Registration Key in memory, tracing below sub_4594D0 gave me nightmares, on the first pass I noticed many of the functions modifying and returning 128-bit data, these results were all encrypted against the remainder of my registration key. My thoughts turned to what this might be, it didn't look like hand cranked assembler, I suspected a proprietary encryption, DES I ruled out as its based on 56-bit keys and usually 64-bit responses (see an addition at the end, I make my mistakes and I even admit them :-) ), in my mind it had to be either a PGP variant (MD5) or RSA, 128-bit data looked like some sort of footprint, but I've seen hashing functions at work before, check out Bentley MicroStation /J or egis's CORE CrackMe if you want some mindless fun for a while :-), both of these push distinctive memory patterns to the hash function and I didn't see any evidence of those here. Essentially then we are left with a proprietary encryption which might be unfeasible to break.
The question for me now was could I patch the code in-memory (after all you've got to know what checks to pass before you work out how to pass them). sub_4594D0 calls a total of 7 functions, of these 4 calls are made to sub_459540, 2 to sub_4C42B0 and 1 to sub_4C4380, the latter 2 subs are glorified looping functions (don't single-step them too much). sub_459540 has several layers and does the majority of the encryption, this seems to be based on creating several boolean tables (sub_458C60) and shifting/ORing loops below other subs, check out below sub_458E10. What you've realised is that producing a key generator for BugTrapper is basically for sadists only, if you have a few days to spare then by all means go ahead and do it :-).
When you encounter situations such as this your best bet is to keep focused on the higher level application, see all high level programmers are indoctrinated with the 'function building block mentality', this isn't bad programming practise by any means, it actually makes source code a whole lot easier to read and maintain, yet this does allow reverse engineers to effectively patch large portions of code without needing to completely understand the internal details. Lets look again :-
:00456B60 SUB ESP, 1A0h <-- Make lots of stack space.
:00456C28 PUSH ECX
:00456C29 PUSH EDX <-- Parameters.
:00456C2A CALL sub_456950 <-- Amazing protection function.
:00456C39 TEST EAX, EAX <-- Is EAX=0?.
:00456C3B SETZ AL <-- AL=1 if it is.
:00456C43 MOV [EDI], AL <-- Flag it.
:00456C77 CMP EDX, ECX
:00456C82 JZ 456C9B <-- Good jump.
:00456C8E MOV [ESP+1B8+var_1A0], 11h <-- Else move error code 17.
:00456C96 CALL sub_4B1AAA <-- Display error.
:00456CFC CMP AX, 0D928 <-- Check.
:00456D00 JZ 456DC2 <-- Good jump.
Patching at this level is easily the best option, the ECX/EDX pushes are locations of results generated from all of the previous functions that we didn't want to reverse. Ultimately though it still comes down to code like TEST EAX, EAX / SETZ AL 1 for good, 0 for bad. Patching here ensures our key is no longer greeted with the invalid key message box. Lets continue our analysis and try now to enable our options (bpx sub_458470). Selecting Register or About from the menu should ensure a break here, now you can literally work your way through the checks by looking to see what is pushed in each instance, note if you don't want BugTrapper to expire (i.e. the Never reference) you'll need to bpx sub_458730 too :-).
The problem of course making all these changes is that when we come to run the original Bt.exe it doesn't work, we'll use instead some of our own self-modifying code, take a look at this code from the ASPack section :-
:005A54ED MOV [EBP+00442E7B],EAX <-- Write O.E.P.
:005A54F3 POPAD <-- Restore all registers.
:005A54F4 JNZ 005A54FE <-- Jump.
:005A54F6 MOV EAX,00000001
:005A54FB RET 000C
:005A54FE PUSH 00000000 <-- Original O.E.P.
:005A5503 RET <-- O.E.P.
005A54ED is not decrypted by ASPack and can be clearly viewed inside Hiew, all we therefore need to do is change this into a CALL or JMP to our patching code, (we can probably use the spare space in ASPack's .rsrc section). :-
:005A54ED PUSH EBP <-- Save EBP.
:005A54EE CALL 005B3B40 <-- CALL our code.
:005B3B40 MOV [EBP+00442E7B], EAX <-- Write O.E.P.
:005B3B45 POP EBP <-- Get return address in EBP (for base).
:005B3B46 SUB ESP, 4 <-- Restore stack pointer so we can RET.
:005B3BC2 RET <-- Return from patching.
I've deliberately not filled in the patching code for you, there is plenty of space to make the required changes (you can even remove the Eval string from the About menu too :-) ), use EBP as your relative base pointer, so MOV [EBP-offset], new_opcodes will be the form of your code.
NB - BugTrapper I-Net is actually a fuller featured option than the Pro version. Note that I cracked only Bt.exe here and virtually every other BugTrapper .exe/dll is packed as well, whether or not they actually house any meaningful protection is for you to investigate, BugTrapper Pro certainly worked OK for me beyond the expiry date. All the files I used here (including the original trial version are available if you ask me).
A note to MuTek should you happen to be reading this. I would really like to use legitimately your product as I'm sure its very useful for programmers and reverse engineers alike, somewhat reluctantly (because your product costs £2000) I will probably make do with my cracked version.
Its not often I acknowledge the use of warez groups releasing key generators for the losers, however CORE's familiar cryptoanalyst team has made a key generator for BugTrapper which you can get here (11.5k). BugTrapper is based upon DES, a brute-force attack against the key would require 256 attempts, evidently standard DES is no longer secure, I recommend you read Applied Cryptography, Second Edition, the assessment of DES here I have reproduced (and this was 5 years ago).
"The answer is both easy and hard. The easy answer just looks at key length. A brute-force DES-cracking machine that can find a key in an average of 3.5 hours cost only $1 million in 1993. DES is so widespread that it is naive to pretend that the NSA and its counterparts havent built such a machine. And remember that cost will drop by a factor of 5 every 10 years. DES will only become less secure as time goes on.
The hard answer tries to estimate cryptanalytic techniques. Differential cryptanalysis was known by the NSA long before the mid-1970s, when DES first became a standard. It is naive to pretend that the NSA theoreticians have been idle since then; almost certainly they have developed newer cryptanalytic techniques that can be applied against DES. But there are no facts, only rumors. Winn Schwartau writes that the NSA had built a massively parallel DES-cracking machine as early as the mid-1980s. At least one such machine was built by Harris Corp. with a Cray Y-MP as a front end. Supposedly there are a series of algorithms that can reduce the complexity of a DES brute-force search by several orders of magnitude. Contextual algorithms, based on the inner workings of DES, can scrap sets of possible keys based on partial solutions. Statistical algorithms reduce the effective key size even further. And other algorithms choose likely keyswords, printable ASCII, and so on to test. The rumour is that the NSA can crack DES in 3 to 15 minutes, depending on how much preprocessing they can do. And these machines cost only $50,000 each, in quantity."
So you see, DES just won't cut it today, go and use 1024-bit RSA or triple-DES (maybe).