Today I tried testing my system to see whether the system call table switching is actually working. I failed of course.

I tested my system via these steps :

1. Create a replica of elf binary loader with slightly different elf signature (from ELF to JUHA or something)
2. Wire the custom system call table to the replica binary loader
3. Create a test program that executes some systems calls
4. Change the program's signature so that the program is loaded by the replica loader
5. Run the program and check whether the result was as desired

And during this process lots of bugs were found! Really lots of bugs..

Step 1-2. Fake ELF Binary Loader

First I made a complete replica of linux's elf loader. At first, I was planning to make a replica in a module I previously made, but because of some reference issues, I was not able to copy the elf loader to a module. So, I gave up with the module, just made a file inside the kernel and copied everything. I only changed the part where it checks the signature so that the kernel can have two loaders that work the same but handles two different program with different signature(and different system call table.)

I just made a file named "binfmt_my_elf.c" in the same directory as "binfmt_elf.c"

/* binfmt_my_elf.c */
/* everything is same except for these two parts : */
...
static int load_elf_binary(struct linux_binprm *bprm)
{
    ...
	/* First of all, some simple consistency checks */
	if (memcmp(elf_ex->e_ident, "JUHA", SELFMAG) != 0)
		goto out;
    ...
}
...
#include <linux/dynamic_sys_call_table.h>

static int __init init_elf_binfmt(void)
{
	register_binfmt(&elf_format);
	printk("myelf : elf_format : 0x%lx\n" , (unsigned long)&elf_format);
	sys_tbl_set_binary_handler(&elf_format , "myelf");
	return 0;
}

Testing & Debugging

While I was testing the loader to see if it was linked with the proper system call table, I found out that the registration of binary handlers occurs prior to the initialization of dynamic system call list. So, I just moved it to right before the task management initialization codes.

void start_kernel(void)
{
     ...
	// initialize the list and register all the system calls to the list
	sys_tbl_init_list();
	
	thread_stack_cache_init();
	cred_init();
	fork_init();
	proc_caches_init();
	...
}

To see whether the system call table is stored in the task_struct, I added some debugging printk codes. This code just prints out the lists of registered system call tables.

void __init sys_tbl_init_list(void) {
    INIT_LIST_HEAD(&sys_call_list);
    
    // register system call tables (architecture-specific function)
    sys_tbl_register_systbls();

    int i = 0;
    struct sys_call_array_container *cur;
    list_for_each_entry(cur , &sys_call_list , lst) {
        printk("dsct : (item %d) \"%s\" bin_handler:0x%lx tbl_ptr:0x%lx\n" , i , cur->platform , (unsigned long)cur->bin_handler , (unsigned long)cur->tbl_ptr);
        i++;
    }
}

To see whether the bprm contains the proper syscall ptr, I dumped the system call pointer via printk, and surprisingly the table pointer was 0x00!

Turns out the code below that I made previously was effectively useless. That part of code was where you handle the error/failure to find the bprm handler.

static int search_binary_handler(struct linux_binprm *bprm)
{
    bool need_retry = IS_ENABLED(CONFIG_MODULES);
    struct linux_binfmt *fmt;
    int retval;

...
    
    bprm->sys_call_tbl_ptr = sys_tbl_search_by_bin_handler(fmt);

    return retval;
}

Instead, the proper location to put the code was inside the loop, right after calling the load_binary() and checking whether it succeed or failed.

static int search_binary_handler(struct linux_binprm *bprm)
{
	...
	list_for_each_entry(fmt, &formats, lh) {
		if (!try_module_get(fmt->module))
			continue;
		read_unlock(&binfmt_lock);

		retval = fmt->load_binary(bprm);

		read_lock(&binfmt_lock);
		put_binfmt(fmt);
		if (bprm->point_of_no_return || (retval != -ENOEXEC)) {
			// success
			
			// set system call handler
			bprm->sys_call_tbl_ptr = sys_tbl_search_by_bin_handler(fmt);
			
			// if binary format is not registered, just use default table
			if(bprm->sys_call_tbl_ptr == 0x00) {
				printk("binary handler not registered to dynamic syscall list! using default table\n");
				bprm->sys_call_tbl_ptr = sys_tbl_get_default_table();
			}
			else {
				printk("using registered dynamic syscall, table : 0x%lx , fmt : 0x%lx\n" , (unsigned long)bprm->sys_call_tbl_ptr , (unsigned long)fmt);
			}
			
			read_unlock(&binfmt_lock);
			return retval;
		}
	}
	...
	return retval;
}

(Also added some additional codes for debugging)

It works fine now!

Step 3-4. Creating test program and modifying

I just wrote this rudimentary assembly program and compiled it with NASM. In theory, since the system call table for the program is set to have #69 to write, this program will print out "Hello world!".

[BITS 64]

SECTION .text

global _start

_start:
	push rbp
	mov rbp , rsp
	
	mov rax , 69 ; call number
	mov rdi , 1 ; stdout
	mov rsi , string_hello
	mov rdx , 14
	syscall
	
	mov rax , 0x3C
	mov edi , 0x00
	syscall

SECTION .data

string_hello: db "Hello world!" , 0x0D , 0x0A , 0x00

The system call table :

sys_call_ptr_t sys_call_table_myelf[] = {
__SYSCALL(69, sys_write)
__SYSCALL(60, sys_exit)
};

And finally, to make the program handled by our new binary handler, we need to change the signature to JUHA.

ELF file with signature changed to JUHA

Because I'm lazy, I just made some bash script that automatically compiles and changes the signature.

#!/bin/bash
nasm hello.asm -f elf64 -o hello.obj
ld hello.obj -o hello

cp new_signature.bin hello-modified
chmod +x hello-modified

file_size=$(stat -c %s hello)
file_size=$((file_size-4))

dd if=hello of=hello-modified seek=4 skip=4 count=${file_size} bs=1

Step 6. Testing the system & Troubleshooting

Obviously, the result was not as what I expected.

hmmm...

First, the system call was not switched as expected. Debugging the system call table shows that although the system call table in the task_struct is properly set, but the global system call table is not properly set.

I don't know what is wrong but I'm pretty sure it's related to the context switching issue.. I don't think that context switching of the system call table is not properly happening.