View Full Version : problem with rsa

February 11th, 2004, 09:55
hello all,

im just having a closer look at a program thats using a 64-bit rsa protection. working with some tutorials and other docs i got to this point:

.text:00409BEC push esi ; empty miracle variable
.text:00409BED mov ecx, [eax]
.text:00409BEF mov eax, [edx]
.text:00409BF1 push ecx ; push n
.text:00409BF2 mov ecx, [esp+18h+arg_4]
.text:00409BF6 push eax ; push e
.text:00409BF7 mov edx, [ecx]
.text:00409BF9 push edx ; push my entered serial
.text:00409BFA call _powmod ; serial^e mod n

as far as i understand its encrypting(decrypting?) my serial with the powmod function from the miracl library.for e,n,d,p,q i got the following numbers:


after it has encrypted my serial it uses

.text:00409C47 call _cotstr

to make the result to string(im not really sure if i understood the miracl documentation right)
after its done with converting there is a compare routine that checks the result from encryption(decryption?) to the following string: B60FE67B
my problem is that im getting pretty confused how to calculate my serial to be B60FE67B after it got encrypted(decrypted?)

should i encrypt it before with: B60FE67B^e mod n
or : B60FE67B^d mod n
(i think i tried both but no luck seems im not getting something right here)

regards xoder

February 11th, 2004, 12:07
Hello xoder,
and hello to everyone in this forum.This seems to be a very nice rce place,which for some weird reason i found out only recently.

as far as i understand its encrypting(decrypting?) my serial with the powmod function from the miracl library.for e,n,d,p,q i got the following numbers:

It's really encrypting your serial since the powmod function has a parameter for the public key(e).So if you have a comparison to a static value that has to be fulfilled,you have to perform RSA decryption:

Static_Value ^ d mod N = Serial

This should be your correct serial i guess...


February 11th, 2004, 15:16
well i tried but it still doesnt work, seems there is something i havent seen in this protection, gonna have a closer look.

could someone possibly post a small visual studio .net project here that uses the miracl library? im trying to get it running since 3 hours no luck. maybe someone knows how to get it running, it would help me alot on getting some feeling for rsa.

February 11th, 2004, 15:29
Hi xoder,
im trying to get it running since 3 hours no luck
Which is exactly your problem between .net and miracl library?


February 11th, 2004, 15:52
i found some project on the web, lemme see if i can get it working it seems i included the wrong headers.

February 11th, 2004, 16:34
Ok but also take a look at the MSVISUAL file located inside miracl directory (if I am not wrong...), it explains all the tasks you need to do in order to set-up a working project.
Personally, I use to include 'miracl.h' and add ms32.lib to the project... it works :-)

Good luck,

February 11th, 2004, 16:59
well yeah that was the first thing i was stuck, i didnt find a way to insert ms32.lib into the project. at least it didnt work the way it was described in the msvisual.txt since i use visual studio .net 2003. do you have a clue how it works in vs .net 2k3?

February 11th, 2004, 18:06
it seems i got them added, but it still refuses to run. i did the following steps to get them into the project:
1. open project properties tab
2. go to linker
3. add ms32.lib and bc32.lib to additional dependencies
4. include "big.h"

result: fatal error LNK1136: invalid or corrupt file

any ideas?

February 11th, 2004, 18:28
Have you also include miracl.h?


February 11th, 2004, 18:43
the method you used to add the lib file is correct.
bc32.lib is designed for borland c++ compiler, if you use .net you don't need to add it to your project.
LNK1136: The input file either has a corrupt header or is zero size or abnormally small.
I never seen this error, maybe one of the file is really corrupted... is this error referred to a particular file?


February 12th, 2004, 04:58
human_thought, neither including miracl.h nor big.h worked, i just tried big.h cause i saw it in a project i found on the web and i thought its worth a try...

zai: no its a general error i just get the project name as reference. while reading through msvisual.txt i came across a point that tells to recompile the miracl lib when using it with visual studio mfc, is that really necessary?

February 12th, 2004, 05:45
I don't use mfc and I don't know if it's really necessary but, it's often better to follow these 'hints'

February 12th, 2004, 11:21
ok, im gonna try my luck with a (hopefully successful) recompile. i gonna let you know if it helped.

February 12th, 2004, 12:31
i actually managed to rebuild the miracl library with tons of warnings during build process. now i have another problem when trying to use it. i include "big.h" into my project also put "big.cpp" into it. but when trying to compile my program i get tge following errors:
1. error LNK2005: _multiply already defined in miracl2.lib(mrmodmul.obj)
2. error LNK2005: _redc already defined in miracl2.lib (mrmodmul.obj)

any ideas how to fix this?

February 12th, 2004, 15:10
ok i got one step further again... i build the library a second time and now the redefinition error from my previous post is gone. in my project i include the "big.h" and "big.cpp" and i can at least compile it without errors. the problem now is that when i try to create an instance of big the program crashes with an unhandeled exception that comes from on of the Big() constructors....since i dont seem to get this stuff running, does anyone know how to fix this or if there are other libraries to use for bignum calculations?

February 12th, 2004, 21:09
If you want a bignum calculator, I think there are some online; it isn't hard to write your own, either. You could get Perl and use the BigInt package. Or get MuPad.

Or get Crypto++ from Wei Dai's site; it has good bignum support.

February 13th, 2004, 08:43
mike, are you familiar with crypto++? i could need some help with it to set up an function with rsa encrypt. i downloaded the cryptosample from sourceforge but i cant get it running with visual studio 2003. even setting up my own rsa encrypt function doesnt work could you possibly give me some help with it?

February 13th, 2004, 14:22
I'm sure someone could help you, if you were only a little bit more specific than that...

February 15th, 2004, 13:36
i had a more detailed look on this program and the encrytion routine but i still dont get it. i tried alot but it seems nothing im doing leads to a solution, im totally stuck, maybe someone can help.

.text:00409BD0 push ecx ; save register
.text:00409BD1 push esi ; save register
.text:00409BD2 push edi ; save register
.text:00409BD3 push 0 ; init value for the new mirvar
.text:00409BD5 mov [esp+10h+var_4], 0 ; ?
.text:00409BDD call _mirvar ; create new mirvar
.text:00409BE2 mov edx, [esp+10h+arg_8] ; adr of exponent -> edx
.text:00409BE6 mov esi, eax ; adr of new mirvar -> esi
.text:00409BE8 mov eax, [esp+10h+arg_C] ; adr of modul -> eax
.text:00409BEC push esi ; empty mirvar for result of powmod
.text:00409BED mov ecx, [eax] ; modul -> ecx
.text:00409BEF mov eax, [edx] ; exponent -> eax
.text:00409BF1 push ecx ; modul n ( 8D4861EE1D906A2D )
.text:00409BF2 mov ecx, [esp+18h+arg_4] ; adr of entered serial -> ecx
.text:00409BF6 push eax ; exponent e ( 101B445 )
.text:00409BF7 mov edx, [ecx] ; load adr of entered serial in edx
.text:00409BF9 push edx ; entered serial
.text:00409BFA call _powmod ; serial^e mod n (if i got it right==RSA), result in esi
.text:00409BFF push 0 ; init value for mirvar
.text:00409C01 call _mirvar ; create a new mirvar
.text:00409C06 mov edi, [esp+24h+arg_0] ;
.text:00409C0A push eax ; new mirvar
.text:00409C0B push esi ; mirvar with result
.text:00409C0C mov [edi], eax ; save adr of mirvar containing the result in [edi]
.text:00409C0E call _copy ; copy result of powmod to new mirvar
.text:00409C13 push esi ; push adr of mirvar to kill
.text:00409C14 call _mirkill ; kill old result
.text:00409C19 add esp, 24h ; ?
.text:00409C1C mov eax, edi ; mov return value to eax (mirvar with result)
.text:00409C1E pop edi ; restore register
.text:00409C1F pop esi ; restore register
.text:00409C20 pop ecx ; restore register
.text:00409C21 retn

after this call has been executed my fake serial "4343434345454545" ("EEEECCCC" in ascii) was encrypted to "5D1EF5CFD6D4B1AF" (unprintable ascii).
the program now compares the encrypted serial to "4236304645363742" ("B60FE67B" in ascii). from the above call i took n and the exponent e and
inserted them into te's rsatool, which gave me the following two primes p=30B9E91, q=2E647197DD and d=3AE0DDA74A7F480D. my conclusion was, since
the program encrypts the serial c=m^e mod n i have to decrypt the valid serial with d before entering it into the program. m=c^d mod n is the
formula i thought that is right for this purpose. i tried a few different attemps to calculate the right serial but something doesnt seem to work.
i tried to calculate:

4236304645363742^d mod n which seemed to be the most logical to me but te's rsatool told me something about M>N.

i decided to code a little tool with the miracl library just to see what B60FE67B^d mod n is. i succeeded after some days of trying and my program
gave me "406A95C49EADCF02". i tried it in the program but it turned out that its a wrong serial, it was encrypted to some unreadable ascii crap.
so i thought i could patch the program at runtime just to be sure they didnt mess something up with the powmod function so i patched the exponent e
in memory to the private key d "3AE0DDA74A7F480D" right before the programs uses _cinstr to make e a mirvar. following this idea the program should
calculate the m=c^d mod n for me. as result i got "1228394F88E9A157". but as already expected this also did not work...

currently im running out of ideas....a patch would be pretty simple there is just one flag i needed to patch but since its just rsa-64 it should be
possible to somehow calculate a proper serial. maybe someone can give me a hint about how to get "4236304645363742" after the encrypt function.
does it maybe have something to do with the mip->IOBASE or something else specific to miracl maybe reverse byte order?

February 15th, 2004, 16:57
First do the exponentiation & stuff on a bignum calculator so you don't have to worry about things like endianness, implementations, etc. If some tool tells you p and q, multiply them together to make sure you get n back. Do the exponentiation in a controlled system where you know what's going on, and then see where things differ in this system.

If you want more explicit help, post n,p,q,d,e,m,c and I'll doublecheck your answers.

February 16th, 2004, 04:47
mike, do you have the url to such a bignum calculator? i just found an applet but since i work on my notebook i'd prefer to have it as an executable.

February 16th, 2004, 11:44
Thats what my uni teacher used when he got to RSA.


Its easy to use, has alot nice functions implemented.. Like, IsPrime() IsCarmichaelNumber(), Factor(), Mod()..... and its fast


February 16th, 2004, 14:30
thx donan will check it out. i hope it runs better than this calculator i have right now, it makes my processor usage go up to 100% but gives me no result i guess it just hangs

February 16th, 2004, 14:45
can someone tell me how many hours/days it take to calculate :


on a 2,2ghz cpu? im sitting here waiting and i dunno if the program just crashed or if its still working. how can miracl solve m^e mod n in less than 1 second but m^e takes so long?

February 16th, 2004, 20:25
It has most likely crashed or malfunctioned in some other way. That calculation should be done in notime...

February 17th, 2004, 05:47
does anyone know/have a good calculator for bignum operations? the only two i've found and tried are YACAS and BigInt Calculator but both seem to crash on the above math operation...

February 17th, 2004, 21:47
To be able to handle such large numbers you usually mod them with something during the same operation. the result would become all too large otherwise, I was assuming you already did that when I replied to your last post.

February 18th, 2004, 18:44
xoder, if 10^100 has 100 digits, how many digits are in

4236304645363742^8D4861EE1D906A2D ?

The way you calculate that thing mod n is NOT by doing the exponentiation first and then reducing modulo n. You first express the exponent as a bit string and then square and multiply, reducing mod n at every step.

For example, to raise m to the 11th (=1011 in binary) power, you calculate
1*m=m^1 (1=1 in binary)
((1*m)^2 mod n)=m^2 (2=10 in binary)
(((1*m)m^2 mod n)^2 mod n) = m^4 (4=100 in binary)
((((1*m)m^2 mod n)^2 mod n)*m mod n) = m^5 (5=101 in binary)
(((((1*m)m^2 mod n)^2 mod n)*m mod n)^2 mod n) = m^10 (10=1010 in binary)
((((((1*m)m^2 mod n)^2 mod n)*m mod n)^2 mod n)*m mod n) = m^11 (11=1011 in binary)

February 19th, 2004, 17:40
i finally managed to get a valid serial. it turned out that i had missed some part of the protection, so sorry guys for making this big confusion. i also wanna thank a nice chinese guy for showing me what i did wrong. im gonna code a keygen for it and also write a small tutorial, so if anyone knows a site that publishes tutorials or wanna have it when its done just let me know.

thank you all


February 21st, 2004, 01:11
Post the tutorial here =D We all would like to read something about RSA...