API Hooking with IDA Pro

Introduction

Monitoring functions with API hooking is very useful when you're doing reverse engineering, especially malware or firmware analysis. It is especially convenient to do it directly inside your debugger. In this article, we will explain how to perform API hooking in IDA Pro.

Here are some situations I found API hooking especially helpful :

• Hooks on ssl_write() and ssl_read() to capture packets before they are respectively encrypted and after they are decrypted by SSL/TLS. This way, you don't have to use tools like SSLsplit to monitor the communications.
• Hooking string encryption or obfuscation routines that you often find in malwares.
• fprintf() or similar functions to dump debug messages for instance when you're reversing firmwares with no output.
• ...

The first step is to identify and name the functions we want to hook. If you want to hook functions from known libraries, IDA is able to load FLIRT signatures to identify functions from known libraries.

You can skip the next part if you already know which functions you want to hook.

Identifying key functions with FLIRT signatures

FLIRT signatures are a useful and convenient way to identify functions in IDA Pro. OpenSSL (and other) FLIRT signatures can be found here : https://github.com/push0ebp/sig-database.

Now that we identified the functions we are interested in monitoring, we will show how to hook them using conditional breakpoints in IDA Pro.

API Hooking with conditional breakpoints

There is no clean way to perform API hooking with IDA Pro. Of course, you could use hooking library like Detours, Deviare, or make your own, but that would be overkill and it's often very convenient to have it directly in your debugger for malware / firmware analysis.

The idea is to use the fact that IDAPython executes condition of conditional breakpoints as Python code. All you have to do is set a conditional breakpoint on the functions you want to monitor and put your hooking script inside the 'condition' field.

Here is an example, that attaches to a process and hooks _ssl3_read() and _ssl3_write() functions (you have to identify them first) :

Make sure to type the following command in the IDApython console before executing the script :

idaapi.enable_extlang_python(True)

This showcases the two types of API hooks you can use : pre-Call, where a buffer is already set and passed to the function (case of a write) and post-Call, where the buffer is filled by the function (case of a read).