Reverse Engineering Through Inline Hooking

Reverse Engineering techniques are generally divided into two broad categories:
1. Static Analysis
2. Dynamic Analysis

Static Analysis
-Techniques which do not involve running the code
-Disassembly, file structure analysis, strings, etc.

Dynamic Analysis
-Techniques which involve running the code
-Behavioral analysis

Approaches to Dynamic analysis involve:
-Network Monitoring
      Isolated Physical Networks
      Virtual Networks
-Hardware Emulation
      Norman Sandbox, etc.
-Kernel-Level Monitoring (SSDT hooks)
      Sysinternal Process Monitor
-Debuggers

Kernel-Level Monitoring

Advantages
      Captures every system call
      Can’t be avoided from userland
Disadvantages
      Only captures functions implemented as system calls
      Not every important function call in the Win32 API is implemented as a system call
      Tools don’t differentiate between process housekeeping and calls from usercode
      Calls to internal DLL’s cannot be observed

Process Monitoring via Debugging

Advantages
      Debugger can trap any function call, not just system calls
      Trapped calls are more likely to be highly relevant to the program’s operation
Disadvantages
      Have to act as a debugger
      Susceptible to countless anti-debugging techniques

Inline Hooks

Advantages
      Can trap any function call, not just system calls
      Trapped calls are more likely to be highly relevant to the program’s operation
      Not operating as a debugger
      No device driver required

Disadvantages
      Hard to implement

Implementing Inline Hooks

1. Find a function of interest
2. Disassemble the beginning of the function
3. If possible, overwrite the beginning bytes of the function with a jump or call instruction
4. Implement a handler for the hooked function

What to do with hooked functions?

Observe and Report
      Collect data about the current function call by gathering data from stack and report to console
      Execute any instructions overwritten from the hook
      Jump back to the next instruction in the hooked function
Intercept and Emulate
      Perform a specified action instead of calling the intended function

Running your own Sandbox

-Trap gethostbyname() to always return a fixed IP address.
-A pseudo-handle interface to allow fake reads and writes to files and netwok sockets. Trap connect() to connection to a pseudo-socket. CreateFile(), ReadFile(), WriteFile(), etc.

API Thief Tool (by mandiant.com)

-Launches target process in a suspended state
-Injects a DLL into the process.
-The Injected DLL hooks all Win32 API functions before the target process is resumed
-API Call monitoring can be used simply with a process monitor-style console
-Embedded python can be used to write custom handlers for specific hooked functions

Practical Toolkit for Reverse Engineering

Many people has been involved in Reversing Engineering area for years. It is still considered attractive for many hackers and crackers to breakthrough and discover unknown possibilities exist to reverse the system objectives. Today in our article we will represent some of the core practical explanations on reverse engineering tactics.

Reverse Engineering is basically described as a way to generate high-level architectural view of piece of software from the given source. Several applications involved within RE scope are vulnerability analysis, malware analysis and breaking copy-protection schemes. One can start learning the basics of RE either through 'crack-me' approach or the real-life approach (take the real-world problem, break it and attack it).


Tools of The Trade

1. Debuggers
WinDbg - Rich features, Extensive C++ support, Poor interface.
Visual Studio Debugger - Not suitable for reversing, Good interface for development.
OllyDbg - Excellent interface, easy to use, wide range of plugins.
Immunity Debugger - Extends OllyDbg features, supports Python interpreter, command-line support with windbg commands, wide range of plugins.
GDb - Standard debugger for *NIX systems, not a complete RE debugger.

2. Disassemblers
Objdump - The standard tool for disassembley in Linux.
IDA Pro - Supports various binaries and architectures, Enhance Visualization, Advanced features.

3. System Monitoring Tools
Sysinternals Suite - Process Exlorer, RegMon, FileMon, TCPView

4. Binary Differential Tool
BinDiff

5. Decompilers
Hex-Rays

6. Reverse Engineering Frameworks
PaiMei
ERESI

7. Dedicated Exploitation/Reverse Engineering Environment
DVL (Damn Vulnerable Linux)

Cutting Edge Steps on Advanced Reverse Engineering

-Automation is one of the major tasks in advancing the RE process.
-Most of the tools are scriptable, extensible and programmable.
-Defeating a new anti-debugging solutions
-Develop new RE environment, such as, Virtualization and Sandboxing.
-Joining one on another tool can make a powerful toolset for RE.