CrackMe Challenge Part 3: The Logic Behind the First Challenge

If we take our predicate that we've seen in the end of part 2 into account and input at least 64 bytes (0x40) into the Key 1 field and leave the Name field at a value AAAAAAAA, a second message box is displayed as can be seen in the picture below:

image0

The message box says that we entered an Incorrect Name/Key pair and we must try again. Again, we must first look at the referenced strings and determine where the string comes from. This can be seen in the picture below:

image1

The picture deliberately contains two functions that display certain values in a message box. The first one start at address 0x00401720 and displays Congratulations !!! Go On !!! and the second function starts at address 0x004017240 and displays Incorrect Name/Key pair. Try again.

We can gather two things from this. First, we must backtrace our program from the address 0x004017240, which displays exactly the message box that's being shown. Secondly, we must tell the program to execute the function at address 0x00401720, which clearly displays the congratulations message box. This is a very good proof that when the program execution will be directed at address 0x00401720 rather than 0x00401740, our Stage 1 challenge will be complete and we will be able to begin solving the Stage 2 challenge.

Again, we must backtrace our program. The message box function was called from an address 0x00401967. It's always a good idea to try to deal with the whole function frame when trying to determine what the function does. The whole function in our case is presented here - again the code is quite daunting, but let's not despair:

00401760 /$ 55 push ebp
00401761 |. 8BEC mov ebp,esp
00401763 |. 83E4 F8 and esp,FFFFFFF8
00401766 |. 81EC FC000000 sub esp,0FC
0040176C |. A1 D0E45500 mov eax,dword ptr ds:[55E4D0]
00401771 |. 33C4 xor eax,esp
00401773 |. 898424 F800000>mov dword ptr ss:[esp+F8],eax
0040177A |. 53 push ebx
0040177B |. 56 push esi
0040177C |. 57 push edi
0040177D |. 6A 4C push 4C
0040177F |. 8D4424 14 lea eax,dword ptr ss:[esp+14]
00401783 |. 6A 00 push 0
00401785 |. 50 push eax
00401786 |. 8BD9 mov ebx,ecx
00401788 |. E8 135B0F00 call main.004F72A0
0040178D |. 83C4 0C add esp,0C
00401790 |. 837D 10 40 cmp dword ptr ss:[ebp+10],40
00401794 |. C74424 5C 4017>mov dword ptr ss:[esp+5C],main.00401740
0040179C |. 0F8E C5010000 jle main.00401967
004017A2 |. 8B45 08 mov eax,dword ptr ss:[ebp+8]
004017A5 |. B9 14000000 mov ecx,14
004017AA |. BE B8195400 mov esi,main.005419B8
004017AF |. 8D7C24 70 lea edi,dword ptr ss:[esp+70]
004017B3 |. F3:A5 rep movs dword ptr es:[edi],dword ptr ds:[esi]
004017B5 |. 83F8 50 cmp eax,50
004017B8 |. 72 05 jb short main.004017BF
004017BA |. B8 50000000 mov eax,50
004017BF |> 50 push eax
004017C0 |. 8D4C24 74 lea ecx,dword ptr ss:[esp+74]
004017C4 |. 53 push ebx
004017C5 |. 51 push ecx
004017C6 |. E8 455D0F00 call main.004F7510
004017CB |. 83C4 0C add esp,0C
004017CE |. 8D9424 C000000>lea edx,dword ptr ss:[esp+C0]
004017D5 |. 52 push edx
004017D6 |. 8D4424 74 lea eax,dword ptr ss:[esp+74]
004017DA |. 50 push eax
004017DB |. B9 50000000 mov ecx,50
004017E0 |. E8 3B180000 call main.00403020
004017E5 |. B8 3F000000 mov eax,3F
004017EA |. 8D4C24 70 lea ecx,dword ptr ss:[esp+70]
004017EE |. 8BFF mov edi,edi
004017F0 |> 8B55 0C /mov edx,dword ptr ss:[ebp+C]
004017F3 |. 8A1410 |mov dl,byte ptr ds:[eax+edx]
004017F6 |. 8811 |mov byte ptr ds:[ecx],dl
004017F8 |. 41 |inc ecx
004017F9 |. 48 |dec eax
004017FA |.^79 F4 jns short main.004017F0
004017FC |. B8 40000000 mov eax,40
00401801 |. 33C9 xor ecx,ecx
00401803 |> 8B940C C000000>/mov edx,dword ptr ss:[esp+ecx+C0]
0040180A |. 3B540C 70 |cmp edx,dword ptr ss:[esp+ecx+70]
0040180E |. 0F85 53010000 |jnz main.00401967
00401814 |. 83E8 04 |sub eax,4
00401817 |. 83C1 04 |add ecx,4
0040181A |. 83F8 04 |cmp eax,4
0040181D |.^73 E4 jnb short main.00401803
0040181F |. B9 10000000 mov ecx,10
00401824 |. 8DB424 C000000>lea esi,dword ptr ss:[esp+C0]
0040182B |. 8D7C24 10 lea edi,dword ptr ss:[esp+10]
0040182F |. F3:A5 rep movs dword ptr es:[edi],dword ptr ds>
00401831 |. 8B75 10 mov esi,dword ptr ss:[ebp+10]
00401834 |. 33C0 xor eax,eax
00401836 |. 85F6 test esi,esi
00401838 |. 7E 1E jle short main.00401858
0040183A |. 8D9B 00000000 lea ebx,dword ptr ds:[ebx]
00401840 |> 8B55 0C /mov edx,dword ptr ss:[ebp+C]
00401843 |. 8A1410 |mov dl,byte ptr ds:[eax+edx]
00401846 |. 8BC8 |mov ecx,eax
00401848 |. 83E1 3F |and ecx,3F
0040184B |. 30540C 10 |xor byte ptr ss:[esp+ecx+10],dl
0040184F |. 8D4C0C 10 |lea ecx,dword ptr ss:[esp+ecx+10]
00401853 |. 40 |inc eax
00401854 |. 3BC6 |cmp eax,esi
00401856 |.^7C E8 jl short main.00401840
00401858 |> 33F6 xor esi,esi
0040185A |. 33C9 xor ecx,ecx
0040185C |. 8D6424 00 lea esp,dword ptr ss:[esp]
00401860 |> 8A5C34 10 /mov bl,byte ptr ss:[esp+esi+10]
00401864 |. 0FB6C3 |movzx eax,bl
00401867 |. 99 |cdq
00401868 |. BF 50000000 |mov edi,50
0040186D |. F7FF |idiv edi
0040186F |. 83C6 04 |add esi,4
00401872 |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]
00401877 |. 03C1 |add eax,ecx
00401879 |. 99 |cdq
0040187A |. 8BCF |mov ecx,edi
0040187C |. F7F9 |idiv ecx
0040187E |. 0FB6CA |movzx ecx,dl
00401881 |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]
00401885 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl
00401889 |. 8A5C34 0D |mov bl,byte ptr ss:[esp+esi+D]
0040188D |. 884434 0C |mov byte ptr ss:[esp+esi+C],al
00401891 |. 0FB6C3 |movzx eax,bl
00401894 |. 99 |cdq
00401895 |. F7FF |idiv edi
00401897 |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]
0040189C |. 03C1 |add eax,ecx
0040189E |. 99 |cdq
0040189F |. 8BCF |mov ecx,edi
004018A1 |. F7F9 |idiv ecx
004018A3 |. 0FB6CA |movzx ecx,dl
004018A6 |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]
004018AA |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl
004018AE |. 8A5C34 0E |mov bl,byte ptr ss:[esp+esi+E]
004018B2 |. 884434 0D |mov byte ptr ss:[esp+esi+D],al
004018B6 |. 0FB6C3 |movzx eax,bl
004018B9 |. 99 |cdq
004018BA |. F7FF |idiv edi
004018BC |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]
004018C1 |. 03C1 |add eax,ecx
004018C3 |. 99 |cdq
004018C4 |. 8BCF |mov ecx,edi
004018C6 |. F7F9 |idiv ecx
004018C8 |. 0FB6CA |movzx ecx,dl
004018CB |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]
004018CF |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl
004018D3 |. 8A5C34 0F |mov bl,byte ptr ss:[esp+esi+F]
004018D7 |. 884434 0E |mov byte ptr ss:[esp+esi+E],al
004018DB |. 0FB6C3 |movzx eax,bl
004018DE |. 99 |cdq
004018DF |. F7FF |idiv edi
004018E1 |. 0FB64414 10 |movzx eax,byte ptr ss:[esp+edx+10]
004018E6 |. 03C1 |add eax,ecx
004018E8 |. 99 |cdq
004018E9 |. 8BCF |mov ecx,edi
004018EB |. F7F9 |idiv ecx
004018ED |. 0FB6CA |movzx ecx,dl
004018F0 |. 8A440C 10 |mov al,byte ptr ss:[esp+ecx+10]
004018F4 |. 885C0C 10 |mov byte ptr ss:[esp+ecx+10],bl
004018F8 |. 884434 0F |mov byte ptr ss:[esp+esi+F],al
004018FC |. 83FE 40 |cmp esi,40
004018FF |.^0F8C 5BFFFFFF jl main.00401860
00401905 |. 817C24 54 0A0B>cmp dword ptr ss:[esp+54],0D0C0B0A
0040190D |. 75 50 jnz short main.0040195F
0040190F |. 8D5424 70 lea edx,dword ptr ss:[esp+70]
00401913 |. 52 push edx
00401914 |. 8D4424 14 lea eax,dword ptr ss:[esp+14]
00401918 |. 50 push eax
00401919 |. 8D4F FC lea ecx,dword ptr ds:[edi-4]
0040191C |. C74424 6C CA8A>mov dword ptr ss:[esp+6C],12578ACA
00401924 |. C74424 70 78B6>mov dword ptr ss:[esp+70],EFCAB678
0040192C |. C74424 74 7856>mov dword ptr ss:[esp+74],12345678
00401934 |. E8 E7160000 call main.00403020
00401939 |. 8D47 BC lea eax,dword ptr ds:[edi-44]
0040193C |. 33C9 xor ecx,ecx
0040193E |. 8BFF mov edi,edi
00401940 |> 8B540C 70 /mov edx,dword ptr ss:[esp+ecx+70]
00401944 |. 3B540C 64 |cmp edx,dword ptr ss:[esp+ecx+64]
00401948 |. 75 1D |jnz short main.00401967
0040194A |. 83E8 04 |sub eax,4
0040194D |. 83C1 04 |add ecx,4
00401950 |. 83F8 04 |cmp eax,4
00401953 |.^73 EB jnb short main.00401940
00401955 |. C74424 5C 2017>mov dword ptr ss:[esp+5C],main.00401720
0040195D |. EB 08 jmp short main.00401967
0040195F |> C74424 5C 4017>mov dword ptr ss:[esp+5C],main.00401740
00401967 |> FF5424 5C call dword ptr ss:[esp+5C]
0040196B |. 8B8C24 0401000>mov ecx,dword ptr ss:[esp+104]
00401972 |. 5F pop edi
00401973 |. 5E pop esi
00401974 |. 5B pop ebx
00401975 |. 33CC xor ecx,esp
00401977 |. E8 3D410F00 call main.004F5AB9
0040197C |. 8BE5 mov esp,ebp
0040197E |. 5D pop ebp
0040197F . C2 0C00 retn 0C

Our message box function is called at the address 0x00401967, which defines the following instruction (from the above function code):

00401967 |> FF5424 5C call dword ptr ss:[esp+5C]

From the code around this function call, we can see instructions that are used to call either a function at address 0x00401740, which is our error message box that we're trying to avoid, or a function at address 0x00401720 which is the success message box that we're trying to call. The following instructions are the ones that determine which function is called:

00401955 |. C74424 5C 2017>mov dword ptr ss:[esp+5C],main.00401720
0040195D |. EB 08 jmp short main.00401967
0040195F |> C74424 5C 4017>mov dword ptr ss:[esp+5C],main.00401740
00401967 |> FF5424 5C call dword ptr ss:[esp+5C]

These instructions save a pointer to the address of either 0x00401720 or 0x00401740 function on the stack at address [esp+0x5C], which is then called by the call instruction. This effectively calls either the first or the second function. We can also see that if the program flow gets to the first instruction presented in the previous code output, the success message box is called. The failure message box must be called explicitly by jumping directly to the call assembler instruction.

Now we must decipher the above function to determine what the function does and how to convince it to execute the instruction at address 0x00401955, which would in turn execute the successful message box, letting us proceed to the next level.

If we look at the function, we can quickly determine that it uses various checks before displaying the success message box. This is why it's best to split the function down into logical segments and determine what each segment does.

The code of the first code segment is presented here:

00401760 /$ 55 push ebp
00401761 |. 8BEC mov ebp,esp
00401763 |. 83E4 F8 and esp,FFFFFFF8
00401766 |. 81EC FC000000 sub esp,0FC
0040176C |. A1 D0E45500 mov eax,dword ptr ds:[55E4D0]
00401771 |. 33C4 xor eax,esp
00401773 |. 898424 F800000>mov dword ptr ss:[esp+F8],eax
0040177A |. 53 push ebx
0040177B |. 56 push esi
0040177C |. 57 push edi
0040177D |. 6A 4C push 4C
0040177F |. 8D4424 14 lea eax,dword ptr ss:[esp+14]
00401783 |. 6A 00 push 0
00401785 |. 50 push eax
00401786 |. 8BD9 mov ebx,ecx
00401788 |. E8 135B0F00 call main.004F72A0
0040178D |. 83C4 0C add esp,0C
00401790 |. 837D 10 40 cmp dword ptr ss:[ebp+10],40
00401794 |. C74424 5C 4017>mov dword ptr ss:[esp+5C],main.00401740
0040179C |. 0F8E C5010000 jle main.00401967
004017A2 |. 8B45 08 mov eax,dword ptr ss:[ebp+8]
004017A5 |. B9 14000000 mov ecx,14
004017AA |. BE B8195400 mov esi,main.005419B8

This piece of code can be assumed to be an initialization code that doesn't really matter at this point. We can see that the function takes care of its own stack frame. It uses 0 xFC local arguments, which is quite something, so we can immediately infer that the function's logic will be complex, challenging and interesting.

The code above doesn't change the stack memory we're interested in, so we don't need to follow it in complete details. The only thing the code introduces and we've already looked at is the following instructions:

00401790 |. 837D 10 40 cmp dword ptr ss:[ebp+10],40
00401794 |. C74424 5C 4017>mov dword ptr ss:[esp+5C],main.00401740
0040179C |. 0F8E C5010000 jle main.00401967

The [ebp+10] input argument contains the length computed as 6/8*(length of Key 1), which is compared to a number 0x40. If the length of the input value is less than the specified number of bytes, the error message is returned.

Conclusion

In the next part we'll take a look at the other logical code segments that will eventually describe the whole function that is used for checking the correct Name and Key 1 input fields.

Comments