First we must take a look at the following piece of code that will be presented in the code segment 5:

004017FC |. B8 40000000 mov eax,40
00401801 |. 33C9 xor ecx,ecx
00401803 |> 8B940C C000000>/mov edx,dword ptr ss:[esp+ecx+C0]
0040180A |. 3B540C 70 |cmp edx,dword ptr ss:[esp+ecx+70]
0040180E 0F85 53010000 jnz main.00401967
00401814 |. 83E8 04 |sub eax,4
00401817 |. 83C1 04 |add ecx,4
0040181A |. 83F8 04 |cmp eax,4
0040181D |.^73 E4 jnb short main.00401803

We can see that it's comparing the stack memory at addresses from [esp+C0]-[esp+100] to [esp+70]-[esp+B0]. This is why from now on, we'll refer to these addresses as 0xC0 and 0x70 for clarity; but remember we're actually talking about the stack addresses being compared in the above piece of code.

The code in the logical segment 1 changes the stack memory at address 0x70, therefore actually changing the stack addresses from [esp+70] to [esp+B0]. The actual code is as follows:

004017AF |. 8D7C24 70 lea edi,dword ptr ss:[esp+70]
004017B3 |. F3:A5 rep movs dword ptr es:[edi],dword ptr ds:[esi]

We're loading the stack address [esp+70] into the register edi and then using the rep instruction to copy a string from register esi to edi until a null byte is encountered. The esi register contains the following value:

ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32
@ESETNOD32@ESETNOD32  @ESETNOD32@

The length of the above string is 80 bytes (0x50), which is how many bytes will be copied from one location to the other. Let's take a look at the stack before this function changes it:

0012EAB8 77D48808 [esp+70]
0012EABC FFFFFFFF [esp+74]
0012EAC0 77D487FF [esp+78]
0012EAC4 77D4B743 [esp+7C]
0012EAC8 00000000 [esp+80]
0012EACC 0040B13F [esp+84]
0012EAD0 0004018A [esp+88]
0012EAD4 0000000D [esp+8C]
0012EAD8 00000071 [esp+90]
0012EADC 00B98670 [esp+94]
0012EAE0 0082BAE4 [esp+98]
0012EAE4 00000001 [esp+9C]
0012EAE8 00B98670 [esp+A0]
0012EAEC 0082BAD0 [esp+A4]
0012EAF0 00000000 [esp+A8]
0012EAF4 0004018A [esp+AC]
0012EAF8 0082BAD0 [esp+B0]
0012EAFC 0012EB40 [esp+B4]
0012EB00 77D588A8 [esp+B8]
0012EB04 00000000 [esp+BC]
0012EB08 00000000 [esp+C0]

And the stack after the function changes it:

0012EAB8 54455345 [esp+70]
0012EABC 33444F4E [esp+74]
0012EAC0 53454032 [esp+78]
0012EAC4 4F4E5445 [esp+7C]
0012EAC8 40323344 [esp+80]
0012EACC 54455345 [esp+84]
0012EAD0 33444F4E [esp+88]
0012EAD4 53454032 [esp+8C]
0012EAD8 4F4E5445 [esp+90]
0012EADC 40323344 [esp+94]
0012EAE0 54455345 [esp+98]
0012EAE4 33444F4E [esp+9C]
0012EAE8 53454032 [esp+A0]
0012EAEC 4F4E5445 [esp+A4]
0012EAF0 40323344 [esp+A8]
0012EAF4 54455345 [esp+AC]
0012EAF8 33444F4E [esp+B0]
0012EAFC 53454032 [esp+B4]
0012EB00 4F4E5445 [esp+B8]
0012EB04 40323344 [esp+BC]
0012EB08 00000000 [esp+C0]

We can see that the ESETNOD32@ string is being copied into the interesting part of stack that we're later comparing. We can see the stack in the picture below:

Code Segment 2

The next piece of code also changes the stack memory on the address [esp+70]. The code is as follows:

004017B5 |. 83F8 50 cmp eax,50
004017B8 |. 72 05 jb short main.004017BF
004017BA |. B8 50000000 mov eax,50
004017BF |> 50 push eax
004017C0 |. 8D4C24 74 lea ecx,dword ptr ss:[esp+74]
004017C4 |. 53 push ebx
004017C5 |. 51 push ecx
004017C6 |. E8 455D0F00 call main.004F7510
004017CB |. 83C4 0C add esp,0C

This code copies the first 0x50 bytes of Name into the stack address [esp+0x70]. Note that the piece of code uses [esp+74] to refer to the stack address that was previously at [esp+70]; the change comes from the push eax instruction that decreases the top of the stack where esp points. But we'll still refer to the stack addresses as being stored at the [esp+70], so you have to keep that in mind when reading the rest of the section. If we enter 80 bytes of B's into the Name field, the stack would be as follows:

0012EAB8 42424242 [esp+70]
0012EABC 42424242 [esp+74]
0012EAC0 42424242 [esp+78]
0012EAC4 42424242 [esp+7C]
0012EAC8 42424242 [esp+80]
0012EACC 42424242 [esp+84]
0012EAD0 42424242 [esp+88]
0012EAD4 42424242 [esp+8C]
0012EAD8 42424242 [esp+90]
0012EADC 42424242 [esp+94]
0012EAE0 42424242 [esp+98]
0012EAE4 42424242 [esp+9C]
0012EAE8 42424242 [esp+A0]
0012EAEC 42424242 [esp+A4]
0012EAF0 42424242 [esp+A8]
0012EAF4 42424242 [esp+AC]
0012EAF8 42424242 [esp+B0]
0012EAFC 42424242 [esp+B4]
0012EB00 42424242 [esp+B8]
0012EB04 42424242 [esp+BC]

If the Name field contains less characters, the 80 bytes of string copied to the start are constructed by the value in the Name field, which is added the remainder of the string

***ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32***
***@ESETNOD32@ESETNOD32@ESETNOD32@***

If we put six A's into the Name field, the string copied to the stack would be as follows:

::

AAAAAAD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32 @ESETNOD32@ESETNOD32@ESETNOD32@

The stack is presented in the picture below:

Logical Code Segment 3

The next piece of code changes the stack memory residing at address [esp+0xC0]. The following code again uses the [esp+74] to actually refer to [esp+70] (in our notation), which happens because of the push instruction.

004017CE |. 8D9424 C000000>lea edx,dword ptr ss:[esp+C0]
004017D5 |. 52 push edx
004017D6 |. 8D4424 74 lea eax,dword ptr ss:[esp+74]
004017DA |. 50 push eax
004017DB |. B9 50000000 mov ecx,50
004017E0 |. E8 3B180000 call main.00403020

At the end of the code, we can see that a call to the function at address 0x00403020 is made, which takes two arguments:

• First arg: pointer to stack [esp+C0]
• Second arg: pointer to stack [esp+74] (actually [esp+70])

Also the ECX register is set to predefined constant value 0x50, which is used in the function. In order to figure out what this segment does, we must try to determine what the called function does. While debugging we can observe how the stack is changed after that function is called.

Since this piece of code changes the stack values on address [esp+0xC0], it's wise to observe what happens to memory addresses between [esp+0xC0]-[esp+0x100], which is later compared to another piece of stack memory. After a few different but very similar input values we can gather that the first 64 bytes are computed from the Name input field, and the next 4 bytes are actually the length of the value in field Key 1. The last 12 bytes contain gibberish.

0012EB08 183E5D47
0012EB0C 41E766F8
0012EB10 C2369D15
0012EB14 CEBA933D
0012EB18 A0E5B3AF
0012EB1C E24D9316
0012EB20 51A0D990
0012EB24 323FDE64
0012EB28 41955B79
0012EB2C 33E4CAFD
0012EB30 33071BEE
0012EB34 6A042F9F
0012EB38 1E1BC874
0012EB3C 0E06F1E6
0012EB40 170904C2
0012EB44 19428587
0012EB48 000000A2
0012EB4C ...
0012EB50 ...
0012EB54 ...

If we enter the same value in the Name and different values in Key 1, only the bytes at address 0x0012*EB*48 change, because the length of the Key 1 is changed. The first 64 bytes are the same. But if we change only one letter in the Name input field, the 128 bytes on the stack change completely. This rules out that the values are generated randomly and suggests that our Name input field is somehow represented by some kind of hash.

Function at the Address 0x00403020

We can figure out what's going on by looking what the function at address 0x00403020 does. The first part of the function initializes the stack in the first three lines. The important observation is the jb call at the end of the section. We can see that the jump is taken if the value in the edi register equals to 0x40. But how is the value in edi register put in there?

On the address 0x00403041, the value is taken from register ecx and stored in edi. So, we must follow the ecx register, which isn't changed in the rest of the function. This is why the value in register ecx is passed into the function from the called function. We can immediately remember that the value of 0x50 is being passed into the function in register ecx. So the jb jump is not taken, because the 0x40 doesn't equal 0x50.

The code of the function residing at address 0x00403020 is presented here:

00403020 /$ 55 push ebp
00403021 |. 8BEC mov ebp,esp
00403023 |. 83EC 5C sub esp,5C
00403026 |. A1 D0E45500 mov eax,dword ptr ds:[55E4D0]
0040302B |. 33C5 xor eax,ebp
0040302D |. 8945 FC mov dword ptr ss:[ebp-4],eax
00403030 |. 8B45 08 mov eax,dword ptr ss:[ebp+8]
00403033 |. 53 push ebx
00403034 |. 56 push esi
00403035 |. 57 push edi
00403036 |. 6A 40 push 40
00403038 |. 8945 AC mov dword ptr ss:[ebp-54],eax
0040303B |. 8B45 0C mov eax,dword ptr ss:[ebp+C]
0040303E |. 6A 00 push 0
00403040 |. 50 push eax
00403041 |. 8BF9 mov edi,ecx
00403043 |. 8945 B4 mov dword ptr ss:[ebp-4C],eax
00403046 |. E8 55420F00 call main.004F72A0
0040304B |. 83C4 0C add esp,0C
0040304E |. 33DB xor ebx,ebx
00403050 |. 83FF 40 cmp edi,40
00403053 |. 72 25 jb short main.0040307A

Right now, it's important to check whether the value 0x50 passed into the function in ecx register can somehow be changed to equal 0x40. In the code before the function call we can see that the value is statically put into the ecx register and we can't influence it with input arguments:

004017DB |. B9 50000000 mov ecx,50
004017E0 |. E8 3B180000 call main.00403020

Ok, so the jb jump at the address 0x00403053 is never evaluated true and the jump is never taken. This means that we can't jump over the next part of the code and need to study it further. The next piece of code is presented here. This code initialized 64 bytes to a value of zero on the stack address 0x70.

00403055 |. BE 40000000 mov esi,40
0040305A |. 8D9B 00000000 lea ebx,dword ptr ds:[ebx]
00403060 |> 8B4D AC /mov ecx,dword ptr ss:[ebp-54]
00403063 |. 8D5431 C0 |lea edx,dword ptr ds:[ecx+esi-40]
00403067 |. 52 |push edx
00403068 |. 8B55 B4 |mov edx,dword ptr ss:[ebp-4C]
0040306B |. E8 A0F8FFFF |call main.00402910
00403070 |. 83C6 40 |add esi,40
00403073 |. 83C3 40 |add ebx,40
00403076 |. 3BF7 |cmp esi,edi
00403078 |.^76 E6 jbe short main.00403060

When stepping through the debugger instruction by instruction, we can observe that the interesting values on the stack are changed when the function at address 00402910 is called. The loop is not repeating the code, so that function is called only once.

Afterwards we need to step through the rest of the function to diagnose another piece of code that might change the observed values on the stack. There are quite some instructions left to step through, but are not that important since none of them seem to change the values on the stack. The only exception is the following line of code:

0040313A |. E8 D1F7FFFF call main.00402910

We can see that we're again calling the function at address 00402910, which additionally changes the stack values. Soon after this call, our function returns.

We gathered enough information to say that the function first initialized the stack to all zeros, then calls the function at address 00402910 for the first time. At the end, the function is called again for the second time on the already changed values. Because of the initialization to all zeros and two subsequent function calls we can immediately say that the function depends on the previous values being set correctly. In order to figure out what the function is doing with the stack, we must take a look at the function at 0x00402910.

Controling the Stack Memory

The function at the address 0x00402910 is quite long, but we don't need to walk through it step by step. We can quickly figure out that the function takes the input argument and calculates some kind of hash and stores the value on the stack. The used algorithm can be any type: an already established known algorithm, custom made algorithm, etc. We have to figure out which algorithm it is.

After a while, we can figure out that the algorithm used is actually a whirpool cryptographic hash function. That is why we tried to compute the whirlpool hash function on the following two strings:

ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@
AAAAAAD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@

We've already seen that the piece of memory we're looking at depends only on the value present in the Name input field. This is why I've inputted six A's into the Name input field, which should give me the second string presented above (the one that starts with six A's).

I've used an online whirlpool implementation that I've found here: http://hash.online-convert.com/whirlpool-generator. The computed hashes were:

::

3fedda80d3dc0b5b37cdba4b5753473fb6f6a11cde91e1b5fdeb997d4579191619a1c580905 c0caa9eddeb0335c11cf4f7bbdbcfcc6babe3c56d679f9ad65258* fda92a50fff4aa0ae96decd685f6643526a3e090f15c374adb1e55aba917bc938e0d3af2938 d36db345c2ef284f814eeae4a58****d2ce246d0a21045a5f680a5812

If we break the execution of the program right after the above code segment is executed, that is at address 0x004017*E*5, we can see that the changed stack memory contains the following values:

0012EB08 502AA9FD
0012EB0C 0AAAF4FF
0012EB10 D6EC6DE9
0012EB14 3564F685
0012EB18 90E0A326
0012EB1C 4A375CF1
0012EB20 AB551EDB
0012EB24 93BC17A9
0012EB28 F23A0D8E
0012EB2C DB368D93
0012EB30 F22E5C34
0012EB34 EE14F884
0012EB38 D2584AAE
0012EB3C 0A6D24CE
0012EB40 5F5A0421
0012EB44 12580A68
0012EB48 000000B6

If we look at the values on the stack closely, we can see that the first 64 bytes are exactly the whirlpool hash followed by 4 bytes that represents the length of the Key 1 value. Let's present the code segment again:

004017CE |. 8D9424 C000000>lea edx,dword ptr ss:[esp+C0]
004017D5 |. 52 push edx
004017D6 |. 8D4424 74 lea eax,dword ptr ss:[esp+74]
004017DA |. 50 push eax
004017DB |. B9 50000000 mov ecx,50
004017E0 |. E8 3B180000 call main.00403020

The above ramblings give us enough information to conclude that the function at address 0x00403020 actually computes a whirlpool cryptographic hash of the value stored at the address of the second argument [esp+74] and stores the hash on the address of the first argument [esp+C0]. The address [esp+74] contains the input value to be hashed before the function at the address 0x00403020 is called:

AAAAAAD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@ESETNOD32@

After the function is called the value at the address [esp+C0] contains the computed hash:

fda92a50fff4aa0ae96decd685f6643526a3e090f15c374adb1e55aba917bc938e0d3af2938d36db345c2ef284f814eeae4a58d2ce246d0a21045a5f680a5812

This can be seen in the picture below:

The first 64 bytes contain the whirlpool hash, the length of the Key 1 field follows and then there are also 12 bytes of gibberish that are not important

Conclusion

We've looked at the logical code segments of the function that is responsible for the logic of accepting or denying the Name and Key 1 input values. The code presented in this article actually takes the input argument Name and adds the string ESETNOD32@ to it, making 64 bytes, then calculating the whirlpool hash function from that and storing it on the stack. We will continue our work in part 5 next.