As a fairly established product and one which it now seems is no longer supported, CASMate makes an ideal target. As you'll soon see there is also a pretty good protection to deal with, including a legacy example of the HASP envelope. The first thing you can try is running the program (cas_win.exe), for these tougher targets don't even think of using W32Dasm, whip out IDA and disassemble. You'll pretty soon get a lot of errors, "can't disassemble" and "execution flows beyond limits". You also won't find the error string that you see at first because its actually decrypted on-the-fly.
As Cas_win.exe is 16-bit we'll use Wldr.exe to gain an entry point, lets start tracing, but first a word about breakpoints. CASMate uses anti-debugging code that detects SoftICE bpx's (remember these actually change the code) and you'll end up with GPF's in certain places, the solution is pretty easy though, just use 'bpmb address x' which is functionally equivalent and already described elsewhere.
Starting off the program calls AllocCStoDSAlias which takes a code-segment selector and returns a data-segment selector that can be used to execute code in the code segment. Next INT 21h/AX=3306h gets the real OS version & INT 2Fh/AX=1600h is then called to check whether we have enhanced mode Windows. Soon we reach some action:-
3127:065E E8130B CALL 1174 <-- Trace to here.
3127:0661 E8D81A CALL 213C <-- Decrypt & Re-Encrypt.
3127:0664 E8BE00 CALL 0725 <-- Call decrypted code.
Before you trace over CALL 213C take a look at cs:0725, its currently encrypted and if you care to do a search in Hiew you'll find it starts at offset 3AB25. Tracing 213C is where the HASP envelope starts its tricks, firstly we do a little preparation before CALLing this decryption routine which should frustrate most crackers tools.
75B7:208C 83C0BF ADD AX,-41 75B7:208F FFE0 JMP AX 75B7:2091 83C33C ADD BX,3C 75B7:2094 FFE3 JMP BX 75B7:2096 83C10F ADD CX,0F 75B7:2099 FFE1 JMP CX ..... ad nauseam (balance that heavy object on F10).
This wonderful code structure accomplishes 2 main aims, firstly it decrypts the real decryption code and secondly it really frustrates debuggers, of course you don't need to trace this as you know where the decrypted code will be written so a bpr should do the trick :-), if you do trace balance something on the F10 key and make a nice drink while you wait, cs:2150 is where it really resumes and next up is the decryption.
2AB7:2154 STI <-- Re-enable all interrupts.
2AB7:2181 MOV CX,088A <-- Number of bytes to decrypt.
2AB7:2184 SUB CX,06 <-- Decrement 3 words.
2AB7:2187 ADD SI,CX <-- Retrieve end.
2AB7:2189 MOV DI,06D3 <-- Start from here.
2AB7:2193 MOV AX,CS:[DI] <-- Get word.
2AB7:2196 ADD AX,BX <-- .....
2AB7:2198 MOV BX,AX <-- .....
2AB7:219A ROR AX,1 <-- .....
2AB7:219C AND AX,8000 <-- Decrypt.
2AB7:219F ADD AX,BX <-- .....
2AB7:21A1 ADD AH,AL <-- .....
2AB7:21A3 MOV BX,AX <-- .....
2AB7:21A5 MOV [DI],AX <-- Write 1st decrypted word.
2AB7:21A7 ADD [DI+02],BX <-- Write 2nd decrypted word.
2AB7:21AA ADD [DI+04],BX <-- Write 3rd.
2AB7:21AD ADD DI,06 <-- Ready for next round.
2AB7:21B0 CMP DI,SI <-- Check if end reached.
2AB7:21B2 JB 2193 <-- Loop decryption.
This is the code we are interested in, because to crack the envelope we need to change the decryption bytes used here. Lets just move forward from here and view what we need to change.
60C7:072F CMP WORD PTR [C7C3],00 <-- Deciding flag.
60C7:0734 JNZ 0739 <-- Jump and its goodbye envelope.
60C7:0736 CALL 06D3 <-- Don't CALL this.
60C7:0739 MOV WORD PTR [C7C3],0001 <-- Move good flag conveniently for us.
Obviously we want to make as few byte changes as possible here because our decryption above has to be fixed, patching the JNZ 75 03 to JMP EB 03 seems like the best and easiest option, but we need our decryption to produce this. Tracing the code after this will find you a further 5 or 6 rounds of decryption (all identical to the above). After you force the envelope check, place a bpm on cs:0725 and see how the envelope actually re-encrypts this code.
The problem here looks to be that changing 1 byte will affect
subsequent decrypts, in fact this is true only to a point, consider:-
39 74 03 73 9A 8A C7 4A <-- Current bytes fed through encryption loop.
00 75 03 E8 9A FF C7 06 <-- Current decrypted result.
00 EB 03 E8 9A FF C7 06 <-- Desired result.
^^ ^^ ^^ ^^
If you start tracing the decryption slowly when DI points at cs:0733 you'll be able to observe how the 75 is generated, see how you'd like to change that to EB :-). Now follow the code slowly (I can't stress this enough), see how ADD is used with our byte stream to generate the next 2 words and then a final patch will be required to get everything back on course with the next 3 word round, bye bye envelope.
What's now left is the API, I assume you have your haspman.pdf. In older 16-bit HASP's I've seen, PrestoChangoSelector was a good bpx for locating the code just after the HASP API entry point, CASMate as you might have guessed from the very early direct INT calls is using an older alternative (FreeSelector). Finding the location is also pretty easy with a HEX editor, just find 'HASPDOSDRV' and change the next JMP E9 to E8 (its always the API entry point). Lets log the services as they occur.
First up is IsHasp (Service 1), no problem patch in AX=1 and continue, then we have 6 HaspCode (Service 2) possibilities, recall that CX/DX are the passwords for specific dongles.
239F:C863 PUSH DS
239F:C864 PUSH ES
239F:C865 PUSH SI
239F:C866 PUSH DI
239F:C867 PUSH BP <-- Parameters should be noted here.
239F:C868 JMP 0D2D <-- HASP entry point.
239F:C86B POP BP
239F:C86C POP DI
239F:C86D POP SI
239F:C86E POP ES
239F:C86F POP DS
239F:C870 POP BP
Password Pairs : 5804/30B4 3EB5/5C72 EE5A/C764 3EBD/5A5B
At the very end of this CASMate makes a 7th Service 2 check duplicating the second set of passwords (CX=3EB5, DX=5C72). My best theory is that each set of seed codes controls a specific flavour of dongle so I had to resort to confirming with a registered user that the 4rth pair is the Pro flavour of dongle although I see no reason why you wouldn't be able to force the others to work.
Seed F519h - Return codes 4232, D2B3, F168 & 665A.
Seed BF21h - Return codes D65F, 6AED, 1EDC & 9D0E.
Firstly change the JMP HASP entry point at offset 2E668 to CALL (E9 --> E8), this gives us an easy RET from our emulator. As I'm pretty certain CASMate uses a MemoHASP we'll emulate all relevant services (1, 2, 3, 4, 5, 6, 32 & 33). I present the code below:-
:120D2D CALL $+5 (E8 00 00)
:120D30 POP BP (5D)
:120D31 OR BH, BH (0A FF)
:120D33 JZ 120D6D (74 38) <-- Always fake successful service.
:120D35 CMP BH, 1 (80 FF 01) <-- IsHasp().
:120D38 JNZ 120D3E (75 04) <-- Check next service.
:120D3A MOV AX, 1 (B8 01 00) <-- HASP found.
:120D3D RET (C3) <-- Return from emulator.
:120D3E CMP BH, 2 (80 FF 02) <-- HaspCode().
:120D41 JNZ 120D76 (75 33) <-- Check next service.
:120D43 CMP CX, 03EBD (81 F9 BD 3E) <-- Check for correct password 1.
:120D47 JNZ 120D6D (75 24) <-- Wrong password.
:120D49 CMP AX, 0F519 (3D 19 F5) <-- Check which seed code.
:120D4C JNZ 120D5B (75 0D)
:120D4E MOV AX, 04232 (B8 32 42) <-- Return 1.
:120D51 MOV BX, 0D2B3 (BB B3 D2) <-- Return 2.
:120D54 MOV CX, 0F168 (B9 68 F1) <-- Return 3.
:120D57 MOV DX, 0665A (BA 5A 66) <-- Return 4.
:120D5A RET (C3)
:120D5B CMP AX, 0BF21 (3D 21 BF) <-- Is it this seed.
:120D5E JNZ 120D6D (75 0D)
:120D60 MOV AX, 0D65F (B8 5F D6) <-- Return 1.
:120D63 MOV BX, 06AED (BB ED 6A) <-- Return 2.
:120D66 MOV CX, 01EDC (B9 DC 1E) <-- Return 3.
:120D69 MOV DX, 09D0E (BA 0E 9D) <-- Return 4.
:120D6C RET (C3)
:120D6D XOR AX, AX (33 C0) <-- Clear AX.
:120D6F XOR BX, BX (33 DB) <-- Clear BX.
:120D71 XOR CX, CX (33 C9) <-- Clear CX.
:120D73 XOR DX, DX (33 D2) <-- Clear DX.
:120D75 RET (C3) <-- Service 4 also returns here.
:120D76 CMP BH, 3 (80 FF 03) <-- Read Word.
:120D79 JNZ 120D8A (75 0F)
:120D7B LEA BX, [BP+009B] (8D 9E 9B 00) <-- Start of simulated dongle memory.
:120D7F SHL DI, 1 (D1 E7)
:120D81 MOV BX, [BX+DI] (8B 19) <-- Word read.
:120D83 SHR DI, 1 (D1 EF) <-- Restore DI.
:120D85 MOV AX, DI (8B C7) <-- Place word read in AX.
:120D87 XOR CX, CX (33 C9) <-- Clear CX for success.
:120D89 RET (C3)
:120D8A CMP BH, 4 (80 FF 04) <-- Write Word.
:120D8D JZ 120D6D (74 DE) <-- Always successful (clear).
:120D8F CMP BH, 5 (80 FF 05) <-- HASP Status.
:120D92 JNZ 120D9C (75 08) <-- Check next service.
:120D94 MOV AX, 1 (B8 01 00)
:120D97 MOV BX, AX (8B D8)
:120D99 MOV CX, AX (8B C8) <-- MemoHASP connected.
:120D9B RET (C3)
:120D9C CMP BH, 6 (80 FF 06) <-- HASPId.
:120D9F JNZ 120DAA (75 09)
:120DA1 MOV AX, 0 (B8 00 00) <-- ID Low.
:120DA4 MOV BX, 0 (BB 00 00) <-- ID High.
:120DA7 XOR CX, CX (33 C9) <-- Success CX=0.
:120DA9 RET (C3)
:120DAA CMP BH, 33 (80 FF 33) <-- Write Block.
:120DAD JNZ 120DB2 (75 03)
:120DAF XOR CX, CX (33 C9) <-- Success CX=0.
:120DB1 RET (C3)
:120DB2 CMP BH, 32 (80 FF 32) <-- Read Block.
:120DB5 JNZ 120D6D (75 B6) <-- Fake universal success.
:120DB7 LEA BX, [BP+009B] (8D 9E 9B 00) <-- Get start of dongle memory.
:120DBB MOV CX, SI (8B CE) <-- Number of words to move.
:120DBD SHL DI, 1 (D1 E7)
:120DBF LEA SI, [BX+DI] (8D 31) <-- Read from here.
:120DC1 SHR DI, 1 (D1 EF)
:120DC3 XCHG DI, AX (97) <-- Move to here.
:120DC4 REPZ MOVSW (F3 A5) <-- Move words.
:120DC6 XCHG DI, AX (97) <-- Restore AX.
:120DC7 XOR CX, CX (33 C9) <-- Success CX=0.
:120DC9 RET (C3) <-- End of emulation routine.
:120DCB DB dup 112(0) <-- Start of simulated memory.
The key thing to watch here is service 32h (50) as it reads 38h words from the dongle (i.e. the entire 112 byte contents). After the first read Service 6 reads the dongles ID and after a further call to service 32h this is used to checksum areas of the dongles memory (in fact the HASPId is stored at words 3 & 4 in our simulated memory), so make sure this corresponds with the above returns you patch into the service 6 emulation above.
Options are controlled using various words, my advice is to set a bpr on the entire area, a good tip is to label your emulated memory consecutively i.e. 00, 01, 02, 03, this should make it pretty easy to see which positions are being used. There are also 3 DWORD checksums which must be corrected.
cs:0B48 CMP EAX,ES:[BX+02] <-- First checksum (bytes
cs:0B4D JZ 0B53 <-- Good jump.
2 further checksums follow this very shortly (just trace with F10).
I know that Fravia's site already hosts an essay on CASMate yet you should be able to see that there are just a few aesthetic mistakes, for example Shaman doesn't actually return all of the correct return codes with Service 2, he also mistakenly sets Service 5's 3rd return parameter, should be 60 for a MemoHASP-1 (there are several other niggles I might also have). If you own CASMate, you should be able to literally take out your HEX editor patch in the bytes I show above and then dump your existing dongle before pasting its contents over the space I've allocated for the simulated memory.