Linux Kernel runtime unpacker and binary signature

by @Jonathan Salwan - 2013-03-25

Mach-OS and iOS have implemented a runtime unpacker and signature verification when binaries are exectued. This allows to check if the binary is approved by Apple and make it more complicated to reverse (We need to dump the memory). In this short note, we will see what can be done on Linux with this same implementation. I have implemented a runtime unpacker for the ndh2k13 CTF (crackme300) when a binary is loaded in memory. When reversing the binary, you can see that the TEXT section is packed, as shown below:

  ; [12] va=0x00400610 pa=0x00000610 sz=1048 vsz=1048 rwx=-r-x .text
/ function: section..text (155)
| ; -------- section..text:
| 0x00400610      23ae7deca991     and ebp, [rsi-0x6e561383]
| 0x00400616      94               xchg esp, eax
| 0x00400617      437ad8           jp loc.004005f2
| 0x0040061a      e6d5             out 0xd5, al
| 0x0040061c      d106             rol dword [rsi], 1
| 0x0040061e      d7               xlatb
| 0x0040061f      b30a             mov bl, 0xa
| 0x00400621      f01d1bc498ff     lock sbb eax, 0xff98c41b
| 0x00400627      93               xchg ebx, eax
| 0x00400628      643268b9         xor ch, [fs:rax-0x47]
| 0x0040062c      e341             jrcxz 0x40066f
| 0x0040062e      0f53e6           rcpps xmm4, xmm6
| 0x00400631      e0c7             loopnz 0x4005fa
| 0x00400633      12ab939a8730     adc ch, [rbx+0x30879a93]
| 0x00400639      285a08           sub [rdx+0x8], bl
| 0x0040063c      d8e6             fsub st0, st6
| 0x0040063e      dd               invalid
| 0x0040063f      291e             sub [rsi], ebx
| 0x00400641      08ff             or bh, bh
| 0x00400643      54               push rsp
| 0x00400644      39dd             cmp ebp, ebx
| 0x00400646      12cc             adc cl, ah
| 0x00400648      1d7720a79d       sbb eax, 0x9da72077
| 0x0040064d      b1b1             mov cl, 0xb1
| 0x0040064f      60               invalid
| 0x00400650      cdc0             int 0xc0
| 0x00400652      57               push rdi
| 0x00400653      82               invalid
| 0x00400654      7617             jbe 0x40066d
| 0x00400656      82               invalid
| 0x00400657      d3a4f5e85f4c5a   shl dword [rbp+rsi*8+0x5a4c5fe8], cl
| 0x0040065e      0800             or [rax], al

And the entrypoint is not modified.

$ readelf -h ./crackme.packed | grep "Entry point address"
  Entry point address:               0x400610

Now, to unpack the binary when it is executed on the Operating System, we need to patch the kernel source. When a binary is executed, the load_elf_binary function is called. This function setups all VMA and is the one that needs to be changed. First of all, we need to know if the binary is packed or not. To do that I decided to put a flag in the ELF header.

$ readelf -h ./crackme.packed | grep "Flags"
  Flags:                             0x20

When the ELFHeader.flags values 0x20 the binary is packed, otherwise it is not. In the load_elf_binary function we check if the binary has this flag.

#define PACKED_MASK 0x20


static int load_elf_binary(struct linux_binprm *bprm, struct pt_regs *regs)

    if (loc->elf_ex.e_flags & PACKED_MASK){
        packer_flag = 1;


Currently, only the TEXT section is packed, therefore we need to know what is the size of TEXT section.

elf_ppnt = elf_phdata;
for (i = 0; i < loc->elf_ex.e_phnum; i++, elf_ppnt++){
    if (elf_ppnt->p_type == PT_LOAD){
        if (packer_flag && (elf_ppnt->p_flags & 0x2))
            upk_binary(start_code, loc->elf_ex.e_entry, elf_ppnt->p_filesz);

The upk_binary is the unpack function, for the context of the NDH2k13 this function uses a simple xor-based algorithm.

static int upk_binary(long unsigned int start_code, unsigned long e_entry,
                      unsigned long p_filesz)
    unsigned long base, size, x, i;
    unsigned char *p;

    unsigned char key[] = {
                            0x12, 0x43, 0x34, 0x65, 0x78, 0xcf, 0xdc,
                            0xca, 0x98, 0x90, 0x65, 0x31, 0x21, 0x56,
                            0x83, 0xfa, 0xcd, 0x30, 0xfd, 0x12, 0x84,
                            0x98, 0xb7, 0x54, 0xa5, 0x62, 0x61, 0xf9,
                            0xe3, 0x09, 0xc8, 0x94, 0x12, 0xe6, 0x87

    base = e_entry - start_code;   /* base phy */
    size = p_filesz - base;

    p = (unsigned char *)e_entry;
    for (i = 0, x = 0 ; i < size ; i++, x++){
        if (x == 35)
            x = 0;
        p[i] ^= key[x];
    return 0;

You can found the complete patch to Linux 3.7.10 here. Currently, this implementation is not really ideal because the packing algorithm is not satisfying. And we cannot verify that the binary is signed. Recently RedHat commited some tools in Linux crypto API to manipulate RSA (patch) and their works on modsign feature. You can found his work on his repository.

Let's take a look at that.

$ cp /lib64/modules/3.6.11-gentoo/misc/vmmon.ko .
$ ./ ./vmmon.ko 2048R/328EABC4
$ ls -l ./vmmon.ko*
-rw-r--r-- 1 jonathan users 98K Mar 25 15:47 ./vmmon.ko
-rw-r--r-- 1 jonathan users 98K Mar 25 15:47 ./vmmon.ko.signed

2048R/328EABC4 is my gpg key.

$ gpg --list-key
pub   2048R/328EABC4 2013-02-01
uid                  Jonathan Salwan (shell-storm key) <jonathan.salwan at shell storm org>
sub   2048R/604B9EDF 2013-02-01

The script adds a section which contains the binary signature.

[+] section 28: .comment
[+] section 29: .note.GNU-stack
[+] section 30: .module_sig
      name string index         0000013f
      type                      00000001 (progbits)
      flags                     0000000000000003 details
      address                   0000000000000000
      offset                    000000000000b8dc
      size                      0000000000000048
      link                      0000000000000000
      info                      0000000000000001
      alignment                 00000000
      entsize                   00000000
[+] section 31: .shstrtab
[+] section 32: .symtab

Currently, this patch just signs Linux modules but that would be nice if the system could check the signature and decrypt the binary. In this context, the Kernel embeds the public key and when the load_elf_binary function is called, it check the signature and decrpyts the binary. This mechanism prevents anyone, except the developpers, from compiling and signing the binary. Maybe it will be useful in embedded system. I will try to implement it.