Author Remio
Source https://xss.pro
REMY Q | 17/06/2025 | RecycledGate Technique and Windows API Unhooking overview
In this here post I shall demonstrate and explain some classic anti-hooking techniques for WindowsAPI. In order to understand this concept we must first familiarize ourselves with API hooking.
When we make a program it is that case that it will need to interact with the OS, and will inevitably be calling some functions. It is also the case that some of these functions may be ‘hooked’, meaning they will be intercepted or redirected by AV/ EDRs for analysis at runtime. These hooks are used to analyse and monitor your programs behaviour.
Anti-hooking techniques aims to detect or bypass these hooks, allowing your program to interact directly with the intended system functions. Each type of hook has the purpose of intercepting or redirecting the flow of execution for your program, all for the undesirable purpose of debugging, analyziing and monotoring its operation.
I will go into details about the workings of each type of hook in future posts, for I shall keep it simple and briefly describe the most common types of hook you will encounter being placed on functions by EDR:
IAT Hooking (Import Address Table hooking): Modifies the addresses in a program’s Import Address Table to point to custom functions. Often used by injected DLLs to hijack API calls.
Export Address Table (EAT) Hooking: Alters the export table of a DLL (like ntdll.dll) so that functions resolve to alternate addresses. Less common, but still used in advanced hooking frameworks.
Syscall Hooking: Patches syscall stubs in user-mode DLLs (like ntdll) or modifies the kernel’s syscall table. Often used by rootkits or advanced EDRs. Can be bypassed with direct syscalls.
VEH/SEH Hooking: Uses exception handlers to control execution flow without modifying code directly. More stealthy, but detectable by checking for registered handlers in the process.
Inline Hooking: It overwrites the first bytes of a function with a JMP instruction. This allows execution of our program to be redirected before the function does any processing. Most hooking methods use a 32bit relative jump (opcode 0xE9), which takes up 5 bytes of space. Again, very commonly used by most standard EDRs and debuggers. We can be detected this hooks by comparing in memory bytes to a clean copy on disk, we will explore this later on in this post.
MessageBoxA inline hooking example
Standard (Hooked) API Call:
RecycledGate Flow for MessageBoxA:
”All hope abandon, ye who enter here.” Over the gate to Hell, they should be made to crawl on their bellies to enter the kingdom of darkness…
Now we have an idea of what hooks are and how they work, we can start to think about methods & techniques to counter them. For this write up, I shall focus on the RecycledGate technique. To understand the RecycledGate technique , we must first understand the evolution of syscall based evasion methods, starting with Hell’s Gate.
Hell’s Gate is essentially a technique for dynamically resolving syscall IDs by parsing the in-memory export table of NTDLL, and constructing a clean syscall stub (a small piece of code that loads the syscall number and executes the syscall instruction to transition from user mode to kernel mode) in runtime memory. This bypassed inline hooks placed by usermode EDRs on functions like NtAllocateVirtualMemory or NtOpenProcess.
But as detection improved, security products started flagging manually crafted syscall stubs and memory with RWX protections. That led to the evolution of Halo’s Gate. Rather than crafting a syscall stub manually, Halo’s Gate searched for an unhooked syscall instruction (usually from a neighboring function), walked backwards from that point, and used that instead. This eliminated the need for shellcode-style allocations and reduced the footprint.
But Halo’s Gate still relies on assumptions, like static distances between syscalls, which made it susceptible to detection via behavioural patterns analysis. This lead to the development of Tartarus’ Gate. Tartarus’ Gate aims to be a more aggressive/flexible implementation, that scans the entire .text section of NTDLL for valid syscall instruction sequences that have not been overwritten by inline hooks. It introduces a generic mechanism to dynamically locate valid syscall instructions even when the target functions are hooked.
This leads us onto the technique we shall be examining in this paper: RecycledGate.
Essentially we can think of RecycledGate as a clean, reusable implementation of the Tartarus’ Gate concept. Rather than relying on offset assumptions or static signatures, it performs discovery of unhooked syscall instructions and maps them to specific API calls. What does this mean? think of it like this, instead of relying on hardcoded offsets or trying to guess where there is a a clean syscall, RecycledGate actively locates untouched syscall stubs: code sequences inside ntdll.dll that still contain the original mov r10, rcx; mov eax, SYSID; syscall pattern. These are usually from less commonly hooked functions that EDRs ignore. Once found, the stub is copied and the syscall number is patched to match the desired API (NtAllocateVirtualMemory, NtOpenProcess, etc.).
This allows you to bypass the inline hook entirely by using a ‘clean’ syscall gateway for a different syscall. It’s like rerouting the wires behind a access control panel, the button says “Open door” or something but behind the scenes, it can be rewired to unlock a vault. The watchers see nothing unusual, but the outcome is completely different.
The main takeaway here is that RecycledGate does not rely on shellcode or any fixed offsets. Instead it’s dynamically locating clean syscalls by walking ntdll in memory and searching for specific instruction patterns. It then recycles syscall stubs from unrelated or unhooked functions to better evade EDR, hence the name RecycledGate.
In the next part we shall explore an implimentation of this technique.
RecycledGate x64 demonstration Program
This demonstration tests 26 Windows APIs by detecting inline hooking and rerouting calls through recycled syscall stubs. It performs basic syscall chains across memory operations, file I/O, and handle management, logging which APIs succeed or fail when bypassing hooks. It provides you with a quick indication / validation of RecycledGate’s effectiveness in console log.
Github Link to Demo Program: https://github.com/RemyQue/RecycledGate-Demo
RecycledGate Project Source Files:
Assembly Engine (recycledgate.asm)
The syscall execution mechanism uses x64 AMD assembler to manage clean gate discovery and execution:
Data Architecture (RecycledGate.c)
We track syscall info through structured tables:
Hook Detection Engine
See below the multiple detection mechanisms:
Intelligent Gate Recycling
When our target functions are hooked, the system searches for alternative clean gates:
Stealth PE Parsing
Function resolution avoids monitored APIs through manual export table parsing:
Simple API Wrappers
RecycledGate provides dropin replacements for standard Windows APIs:
Demo Console Interface
The demo runs in as a x64 console app that provides feedback during hook bypass testing on execution of the program.
Final Overview of method & Demo
The RecycledGate method represents a substantial evolution in syscall ‘gate’ evasion techniques, addressing the prior weaknesses that earlier methods like Hell’s Gate and Halo’s Gate had rather effectively. By dynamically locating and repurposing clean syscall instructions from unhooked functions, it eliminates the usual signatures that modern EDRs detect.
This is indeed a favourable technique to this day due to its elegance and simplicity, working within the security implementation itself. Every syscall executed properly through RecycledGate uses legitimate ntdll.dll code paths, making it nearly indistinguishable from normal applications, therefore (in theory, more often than not we hope) EDR sees what appears to be a standard function call, while your code achieves direct kernel access through an entirely different route.
As EDRs adapt their detection strategies techniques like RecycledGate will keep on evolving. The fundamental principle however, that is to leverage legitimate system components against themselves, will remain a a powerful approach that’s difficult to defend against without breaking normal application functionality for the foreseeable future in my opinion at least.
The source code and demo provide everything needed to understand and test this technique for yourself. Remember to use these methods responsibly and only in authorized testing environments.
If this demonstrations is well recieved I shall mabe expand on this more and provide examples for hells gate and more explanations. Thanks for reading.
- Remy
Github Link to project: https://github.com/RemyQue/RecycledGate-Demo
”As above, so below. As I believe the world to be, so it is.”
Source https://xss.pro
REMY Q | 17/06/2025 | RecycledGate Technique and Windows API Unhooking overview
In this here post I shall demonstrate and explain some classic anti-hooking techniques for WindowsAPI. In order to understand this concept we must first familiarize ourselves with API hooking.
When we make a program it is that case that it will need to interact with the OS, and will inevitably be calling some functions. It is also the case that some of these functions may be ‘hooked’, meaning they will be intercepted or redirected by AV/ EDRs for analysis at runtime. These hooks are used to analyse and monitor your programs behaviour.
Anti-hooking techniques aims to detect or bypass these hooks, allowing your program to interact directly with the intended system functions. Each type of hook has the purpose of intercepting or redirecting the flow of execution for your program, all for the undesirable purpose of debugging, analyziing and monotoring its operation.
I will go into details about the workings of each type of hook in future posts, for I shall keep it simple and briefly describe the most common types of hook you will encounter being placed on functions by EDR:
IAT Hooking (Import Address Table hooking): Modifies the addresses in a program’s Import Address Table to point to custom functions. Often used by injected DLLs to hijack API calls.
Export Address Table (EAT) Hooking: Alters the export table of a DLL (like ntdll.dll) so that functions resolve to alternate addresses. Less common, but still used in advanced hooking frameworks.
Syscall Hooking: Patches syscall stubs in user-mode DLLs (like ntdll) or modifies the kernel’s syscall table. Often used by rootkits or advanced EDRs. Can be bypassed with direct syscalls.
VEH/SEH Hooking: Uses exception handlers to control execution flow without modifying code directly. More stealthy, but detectable by checking for registered handlers in the process.
Inline Hooking: It overwrites the first bytes of a function with a JMP instruction. This allows execution of our program to be redirected before the function does any processing. Most hooking methods use a 32bit relative jump (opcode 0xE9), which takes up 5 bytes of space. Again, very commonly used by most standard EDRs and debuggers. We can be detected this hooks by comparing in memory bytes to a clean copy on disk, we will explore this later on in this post.
MessageBoxA inline hooking example
Standard (Hooked) API Call:
Код:
Your Code → MessageBoxA → JMP to EDR Hook in user32.dll → Monitored or BlockedRecycledGate Flow for MessageBoxA:
Код:
Your Code → RG_MessageBoxA → RecycledGate → RecycledGateDescent → Clean syscall stub (e.g., NtUserMessageCall via unused function) → Kernel → MessageBox window appears”All hope abandon, ye who enter here.” Over the gate to Hell, they should be made to crawl on their bellies to enter the kingdom of darkness…
Now we have an idea of what hooks are and how they work, we can start to think about methods & techniques to counter them. For this write up, I shall focus on the RecycledGate technique. To understand the RecycledGate technique , we must first understand the evolution of syscall based evasion methods, starting with Hell’s Gate.
Hell’s Gate is essentially a technique for dynamically resolving syscall IDs by parsing the in-memory export table of NTDLL, and constructing a clean syscall stub (a small piece of code that loads the syscall number and executes the syscall instruction to transition from user mode to kernel mode) in runtime memory. This bypassed inline hooks placed by usermode EDRs on functions like NtAllocateVirtualMemory or NtOpenProcess.
But as detection improved, security products started flagging manually crafted syscall stubs and memory with RWX protections. That led to the evolution of Halo’s Gate. Rather than crafting a syscall stub manually, Halo’s Gate searched for an unhooked syscall instruction (usually from a neighboring function), walked backwards from that point, and used that instead. This eliminated the need for shellcode-style allocations and reduced the footprint.
But Halo’s Gate still relies on assumptions, like static distances between syscalls, which made it susceptible to detection via behavioural patterns analysis. This lead to the development of Tartarus’ Gate. Tartarus’ Gate aims to be a more aggressive/flexible implementation, that scans the entire .text section of NTDLL for valid syscall instruction sequences that have not been overwritten by inline hooks. It introduces a generic mechanism to dynamically locate valid syscall instructions even when the target functions are hooked.
This leads us onto the technique we shall be examining in this paper: RecycledGate.
Essentially we can think of RecycledGate as a clean, reusable implementation of the Tartarus’ Gate concept. Rather than relying on offset assumptions or static signatures, it performs discovery of unhooked syscall instructions and maps them to specific API calls. What does this mean? think of it like this, instead of relying on hardcoded offsets or trying to guess where there is a a clean syscall, RecycledGate actively locates untouched syscall stubs: code sequences inside ntdll.dll that still contain the original mov r10, rcx; mov eax, SYSID; syscall pattern. These are usually from less commonly hooked functions that EDRs ignore. Once found, the stub is copied and the syscall number is patched to match the desired API (NtAllocateVirtualMemory, NtOpenProcess, etc.).
This allows you to bypass the inline hook entirely by using a ‘clean’ syscall gateway for a different syscall. It’s like rerouting the wires behind a access control panel, the button says “Open door” or something but behind the scenes, it can be rewired to unlock a vault. The watchers see nothing unusual, but the outcome is completely different.
The main takeaway here is that RecycledGate does not rely on shellcode or any fixed offsets. Instead it’s dynamically locating clean syscalls by walking ntdll in memory and searching for specific instruction patterns. It then recycles syscall stubs from unrelated or unhooked functions to better evade EDR, hence the name RecycledGate.
In the next part we shall explore an implimentation of this technique.
RecycledGate x64 demonstration Program
This demonstration tests 26 Windows APIs by detecting inline hooking and rerouting calls through recycled syscall stubs. It performs basic syscall chains across memory operations, file I/O, and handle management, logging which APIs succeed or fail when bypassing hooks. It provides you with a quick indication / validation of RecycledGate’s effectiveness in console log.
Github Link to Demo Program: https://github.com/RemyQue/RecycledGate-Demo
RecycledGate Project Source Files:
Код:
|—RecycledGate/
|- main.c: our main entry point
|- RecycledGate.h:function prototypes & constants
|- RecycledGate.c: Core implementation logic
|- recycledgate.asm: Assembly trampoline functions
|- structs.h: Our Windows internal structuresAssembly Engine (recycledgate.asm)
The syscall execution mechanism uses x64 AMD assembler to manage clean gate discovery and execution:
Код:
; Global syscall state storage
wSystemCall DWORD 000h ; Current syscall number
qRecycledGate QWORD 0000000000000000h ; Address of clean syscall instructions
RecycledGate PROC
mov wSystemCall, 000h ; Clear previous state
mov wSystemCall, ecx ; Store syscall number (param 1)
mov qRecycledGate, rdx ; Store clean gate address (param 2)
ret
RecycledGate ENDP
RecycledGateDescent PROC
mov r10, rcx ; Follow x64 syscall convention
mov eax, wSystemCall ; Load our syscall number
jmp qword ptr [qRecycledGate] ; Execute through clean gate
RecycledGateDescent ENDPData Architecture (RecycledGate.c)
We track syscall info through structured tables:
Код:
typedef struct _RECYCLED_TABLE_ENTRY {
PVOID pAddress; // Original function address
DWORD64 dwHash; // Function name hash
WORD wSystemCall; // Syscall number
PVOID pRecycledGate; // Clean syscall gate address
BOOL bHooked; // Hook detection flag
} RECYCLED_TABLE_ENTRY;
typedef struct _RECYCLED_TABLE {
RECYCLED_TABLE_ENTRY NtAllocateVirtualMemory;
RECYCLED_TABLE_ENTRY NtProtectVirtualMemory;
RECYCLED_TABLE_ENTRY NtCreateThreadEx;
// … 23 additional APIs
} RECYCLED_TABLE;Hook Detection Engine
See below the multiple detection mechanisms:
Код:
// Inline hook detection via JMP instruction scanning
for (INT j = 0; j < SYS_STUB_SIZE; j++) {
if (pFunctionAddress[j] == 0xE9) { // JMP rel32
pTableEntry->bHooked = TRUE;
break;
}
}
// Syscall number extraction from mov eax, imm32
if (pFunctionAddress[j] == 0xB8) {
WORD wSystemCall = *((PWORD)(pFunctionAddress + j + 1));
pTableEntry->wSystemCall = wSystemCall;
}
// Clean gate discovery via pattern matching
if (pFunctionAddress[k] == 0x0F && // syscall
pFunctionAddress[k + 1] == 0x05 &&
pFunctionAddress[k + 2] == 0xC3) { // ret
pGate = (PVOID)(pFunctionAddress + k);
}Intelligent Gate Recycling
When our target functions are hooked, the system searches for alternative clean gates:
Код:
// Bidirectional neighbor search for clean gates
if (pTableEntry->bHooked) {
// Search forward through higher syscall numbers
for (INT offset = 1; offset <= 10; offset++) {
PBYTE pAltFunc = pFunctionAddress + (offset * 32);
// Pattern match for clean syscall; ret sequence
}
// If unsuccessful, search backward through lower syscall numbers
for (INT offset = 1; offset <= 10; offset++) {
PBYTE pAltFunc = pFunctionAddress - (offset * 32);
// Pattern match for clean syscall; ret sequence
}
}Stealth PE Parsing
Function resolution avoids monitored APIs through manual export table parsing:
Код:
// Direct export directory access
PIMAGE_EXPORT_DIRECTORY pImageExportDirectory =
(PIMAGE_EXPORT_DIRECTORY)((PBYTE)pModuleBase +
pImageNtHeaders->OptionalHeader.DataDirectory[IMAGE_DIRECTORY_ENTRY_EXPORT]. VirtualAddress);
// Hash-based function identification
DWORD64 HashFunctionName(BYTE* str) {
DWORD64 hash = 5381;
INT c;
while ((c = *str++))
hash = ((hash << 5) + hash) + c; // djb2 algorithm
return hash;
}Simple API Wrappers
RecycledGate provides dropin replacements for standard Windows APIs:
Код:
NTSTATUS RG_NtAllocateVirtualMemory(
HANDLE ProcessHandle,
PVOID* BaseAddress,
ULONG_PTR ZeroBits,
PSIZE_T RegionSize,
ULONG AllocationType,
ULONG Protect
) {
// Configure syscall parameters
RecycledGate(g_RecycledTable.NtAllocateVirtualMemory.wSystemCall,
g_RecycledTable.NtAllocateVirtualMemory.pRecycledGate);
// Execute through clean gate
return RecycledGateDescent(ProcessHandle, BaseAddress, ZeroBits,
RegionSize, AllocationType, Protect);
}Demo Console Interface
The demo runs in as a x64 console app that provides feedback during hook bypass testing on execution of the program.
- Syscall Info Table: Lists each API with its syscall number, gate address, and hook status (YES/NO).
- Hook Summary: Displays total, hooked, and clean function counts.
- Summarizes all test outcomes.
Final Overview of method & Demo
The RecycledGate method represents a substantial evolution in syscall ‘gate’ evasion techniques, addressing the prior weaknesses that earlier methods like Hell’s Gate and Halo’s Gate had rather effectively. By dynamically locating and repurposing clean syscall instructions from unhooked functions, it eliminates the usual signatures that modern EDRs detect.
This is indeed a favourable technique to this day due to its elegance and simplicity, working within the security implementation itself. Every syscall executed properly through RecycledGate uses legitimate ntdll.dll code paths, making it nearly indistinguishable from normal applications, therefore (in theory, more often than not we hope) EDR sees what appears to be a standard function call, while your code achieves direct kernel access through an entirely different route.
As EDRs adapt their detection strategies techniques like RecycledGate will keep on evolving. The fundamental principle however, that is to leverage legitimate system components against themselves, will remain a a powerful approach that’s difficult to defend against without breaking normal application functionality for the foreseeable future in my opinion at least.
The source code and demo provide everything needed to understand and test this technique for yourself. Remember to use these methods responsibly and only in authorized testing environments.
If this demonstrations is well recieved I shall mabe expand on this more and provide examples for hells gate and more explanations. Thanks for reading.
- Remy
Github Link to project: https://github.com/RemyQue/RecycledGate-Demo
”As above, so below. As I believe the world to be, so it is.”
Последнее редактирование модератором:
