View Full Version : How to know which algo is used ?

November 4th, 2001, 04:08
You can find on the net how to resolve many algo (cryptographs works)

But how to know which algo is used in software or crackme (TMG crackmes) ? Do there exit tutorials ?



November 4th, 2001, 07:05
First of all, when you know it uses some kind of encryption, see if you can detect the encryption type, is it hashing, secret key, or public key alike. Then you can look at the encryption initialization constants, like SHA and MD5 is easy to detect as they have some constants like :

T1 0xd76aa478
T2 0xe8c7b756
T3 0x242070db

etc. Many other algorithm has similar tags known as initialization constants. If you cant find any of those, try looking at the source code of the most likely to be used algos. I have a small archive os assembly listenings of encryption algos, this way i can compare the assembly code pretty quick to determine the encryption used.

But the question is not always to detect the encryption, its mainly a question about howto break it. You wont want to be factoring a large RSA, ElGamal or ECC. But for the hashing, small rsa's ( N <= 256bit max..) and secret key algos its definitely possible.
But be aware that some companies has simple proprietary encryption algoes designed inhouse, these are useually simple xor encryptions. Dont let that fool you, and happy cracking

November 4th, 2001, 11:12
I think i should post an example too, this is from MD5 used in a very popular CAD program.
Analysing the licensing system will lead you to the suspicion it uses some kind of encryption to generate and validate the keyfiles, actually it was simple since it contained names like NewUserHashFingerprint etc. So knowing it was a hashing algo was rather obvious, but determining the algo was done pretty quick too. Observe the assembly listenings below:

.text:5388C590 mov eax, [esp+arg_0]
.text:5388C594 xor ecx, ecx
.text:5388C596 mov [eax+14h], ecx
.text:5388C599 mov [eax+10h], ecx
.text:5388C59C mov dword ptr [eax], 67452301h
.text:5388C5A2 mov dword ptr [eax+4], 0EFCDAB89h
.text:5388C5A9 mov dword ptr [eax+8], 98BADCFEh
.text:5388C5B0 mov dword ptr [eax+0Ch], 10325476h
.text:5388C5B7 retn
.text:5388C5B7 sub_5388C590 endp

Now looking in md5.c we see this:
md5_init(md5_state_t *pms)
pms->count[0] = pms->count[1] = 0;
pms->abcd[0] = 0x67452301;
pms->abcd[1] = 0xefcdab89;
pms->abcd[2] = 0x98badcfe;
pms->abcd[3] = 0x10325476;

Pretty obvious huh?
Traceing further will also reveal the Tx constants.

When you see a call with a lot of ror, shr, mov, add, lea etc. juggeling with alot of numbers, you might want to look at it in ida and check out what it might be.

November 5th, 2001, 09:09
Hey MarcElBichon, the toughest part is identifying what algo is used. You have to know the algorithms especialy how they check valid signatures or how they decrypt. I wrote a paper on keygening tE!'s keygenme #2 which used RipeMD-160 hashing and RSA signature. When I started looking at it I had no idea what I was looking at, so I looked around and I found places where error messages are generated and I saw some string refs. for MIRACLE so I started looking into crypto algorithms, checked out tE! site, got some source codes looked them over and looked over the disassembly of the keygenme and started labeling founctions until I isolated the checking routines, then identifying them was a bit easier.

You just have to know how signature checks, initialization routines, decryption routines, etc, for different algorithms look like in assembly and that will make life much simpler.


November 16th, 2001, 09:50
hello all,

i know that something is using MD5 but how de hell do you start with reversing this??
thanks to this thread i could indentify the algo used but after this what do you do next?

i really could use some suggestions
I started searching for all info i can find about md5.

Thx Sphinx

November 16th, 2001, 10:46
Hello Sphinx !

Reversing MD5??? Well, do it and you'll be a very rich man.

If you know that MD5 is used, you must find out how.

It could be used like "Serial = MD5(UserName)+MD5(EMail)" or "If MD5(LicenseFile) = RSADecryptedValue then FileInfoOK else RegBadFile".

I suggest to set a breakpoint in the MD5 proc, when you reach it, do a "P RET" and set a breakpoint on the call to the procedure. So the next time you'll know which data is hashed by MD5. After that you can trace further and se which value is compared to the hash value. Few patches for the beginning or keygen to go further and your problem is solved.

Easy, isn't it?

November 17th, 2001, 07:13
Most MD5 implementations have 3 functions:

MD5Init - Initialize the MD5 hash variables
MD5Update - Update the MD5 hash against the data
MD5Finish - Translate the hash variables to a string (or was it 4 longs)

So you can rename the function names in IDA or something and it will make your job a whole lot easier.

November 20th, 2001, 03:56
hello again,

thanks for the replies now i know how to start I`m on my way now.