x86 Disassembly Tutorial 1 - The Basics of OllyDBG

Watch the video or follow the tutorial.

This is the first tutorial in my disassembly series exploring how to disassemble, analyse and modify exe files using OllyDBG. We do this because we have an innate desire to understand how things work and to see the inner workings and processes involved in the programs that we create. In this tutorial we will be using Visual MASM to code and build a windows exe in assembler, so if you haven't installed Visual MASM yet you can learn how to do it here. Once installed open it and learn how to create the default code template needed for all my tutorials here. We will be creating a small program in VisualMASM shortly but first we need to install OllyDBG. So open a web page and go to www.ollydbg.de once there click on the Odbg110.zip link on the left hand side under Files to download Olly.

Save it to wherever you want and once complete unzip the files to any folder.

Double-click on OLLYDBG.EXE

If Olly complains about not having Administrative Rights then you can either press Ok and ignore it if the program you are about to disassemble doesn't need it or restart Olly with Admin rights if it does. The program we are going to create does not need admin rights so in this instance you can just press OK.

We will now create a simple MessageBox application in VisualMASM. Open Visual MASM and create a new project choosing a standard MessaegBox application. If you want to actually understand the code we are using here then please go through my assembler MessageBox tutorial, otherwise just delete all code in the code window and replace it with the following:

; *************************************************************************
; 32-bit Windows Program
; *************************************************************************

.686                                      ; Enable 80686+ instruction set
.model flat, stdcall                ; Flat, 32-bit memory model (not used in 64-bit)
option casemap: none         ; Case sensitive syntax

; *************************************************************************
; MASM32 proto types for Win32 functions and structures
; *************************************************************************
include c:\masm32\include\windows.inc
include c:\masm32\include\user32.inc
include c:\masm32\include\kernel32.inc
include c:\masm32\include\masm32rt.inc     ; for using ustr$() and such like

; *************************************************************************
; MASM32 object libraries
; *************************************************************************
includelib c:\masm32\lib\user32.lib
includelib c:\masm32\lib\kernel32.lib

; *************************************************************************
; Our data section.
; *************************************************************************

strTitle DB "Message", 0
strMessage DB "Hello World", 0

; *************************************************************************
; Our unintialised data section.
; *************************************************************************

; *************************************************************************
; Our constant section.
; *************************************************************************

; *************************************************************************
; Macros
; *************************************************************************

; *************************************************************************
; Our executable assembly code starts here in the .code section
; *************************************************************************


     Push MB_OK
     Push Offset strTitle
     Push Offset strMessage
     Push 0
     Call MessageBox

     ; Exit app
     Push 0
     Call ExitProcess

end start

A text file with the above source code can be found: here

This code will, once compiled display a MessageBox to the user with the message "Hello World". So, save all files and press F9 or the green play button on the menu bar. When the MessageBox displays just hit Ok to close the app. The exe we want will be located in the Release folder of the folder where you saved the Visual MASM project files. If you haven't changed the filename it will be called Win32App.exe. You can leave it where it is or you can copy it to somewhere else it is up to you as long as you know where it is.

Go to Olly and open the exe file you just created by pressing the open folder icon and locating the file and selecting it.

While Olly loads the exe it may pause to report errors, when it says Paused in yellow at the bottom right press the blue play arrow on the menu bar. After a couple of goes it should say running. If the MessageBox appears don't close it just ignore it and bring focus back to Olly.

I will now give a brief explanation of what you see in Olly when the program is being debugged. In the above image the orange rectangle represents the different CPU registers and control flags, their current values and if they are in red the fact that they have been modified by the most recent program operation. It is important that you get to know this section of Olly's interface because this is where you will be looking to see what data is going through the CPU.

The green rectangle is the main window and shows the disassembled code from the executable file. It is currently at the point that the program is at. In our case it is waiting for a user action to be performed on the MessageBox and it will wait there indefinitely. The code window is probably going to be the part of Olly that you look at the most. The first column from the left is the memory address of each line of code, the next column to the right is the code operations in hexadecimal format and the next column is the same code operations but in assembler. The last column is whatever other information is available and can be shown, this is usually function and variables names and stuff like that.

The blue rectangle shows the memory being used by the program and the location in memory of all variables, strings etc, this is useful for locating specific stuff in memory that we might want to have a more detailed look at, more on that in a later tutorial.

The purple rectangle is the stack which is a temporary memory location where all active variables and stuff are stored. The stack is a last on first off storage location where anything that we need to store temporarily can be put. Function parameters are stored on the stack right before a function is called. If you don't know have a look at the Push and Pop assembler functions to get an idea of what the stack is used for.

Lastly the red rectangle shows a part of Olly that displays helpful messages to you about the current code operation, for example, if the current code operation was a part of the code that is the end point for a jump command the red rectangle would show you where in the code the associated jump command exists and how many there are if there are multiple instances. This little section can be quite useful and you should get to know it as you move forward.

So there you go, that is the basics of using OllyDBG to disassemble a windows exe, before I go a quick note about the MessageBox app we disassembled.

As you can see our program doesn't have too much code, remember though that there is a lot of header information not shown here and the code from the include files that is also not shown. The code shown here is only the MessageBox function and the ExitProcess function that was in our main code section of VisualMASM. Notice the strings that are Pushed onto the stack just before the MessageBox call just as we wrote and notice in the assembler window the associated Push command pushes not the variable but the location in memory of the string constant, ie, for the title string the command is Push Win32App.00403000 which if you locate in the memory window at the bottom left of the Olly interface you will see holds the Message string.

We will have a more in-depth look at these ideas in the next tutorial where we will re-examine the MessageBox app but this time make a few changes to the binary values and see what happens. So thank you for taking the time to go through my tutorial and I will see you at the next one so, until then, enjoy.