Finally, I made the integral parts of file system. I fixed all the bugs. I fixed all the issues. I integrated fat12 and fat16 file system. Now, I will explain what I did over the course of two weeks.

FAT12 / FAT16

I added the FAT12 driver. At first, I just thought the cluster area of FAT12 works completely identical to the FAT16 file system, so I just used all the FAT16 functions to the FAT12. That caused all sorts of problem. Turns out the entry size of FAT table of FAT12 is 1.5 bytes(12 bits), not 2 bytes(16 bits). I had no choice but to make individual functions for each file systems.

...

dword fat::find_next_cluster(blockdev::block_device *device , dword cluster , fat::general_fat_info_t &ginfo) {
    switch(ginfo.fat_type) {
        case GINFO_FAT_TYPE_12:
            return fat12_find_next_cluster(device , cluster , ginfo);
        case GINFO_FAT_TYPE_16:
            return fat16_find_next_cluster(device , cluster , ginfo);
        case GINFO_FAT_TYPE_32:
            return fat32_find_next_cluster(device , cluster , ginfo);
    }
    return 0x00;
}

static max_t get_total_cluster_count(fat::general_fat_info_t &ginfo , max_t bytes_per_sector) {
    switch(ginfo.fat_type) {
        case GINFO_FAT_TYPE_12:
            return (max_t)((ginfo.fat_size*bytes_per_sector/2)*3);
        case GINFO_FAT_TYPE_16:
            return (max_t)(ginfo.fat_size*bytes_per_sector/2);
        case GINFO_FAT_TYPE_32:
            return (max_t)(ginfo.fat_size*bytes_per_sector/4);
    }
    return 0x00;
}

...

The code for FAT16 and FAT32 is quite similar, as the calculation of entry location in FAT table is very simple, unlike FAT12.

my... brief explanation!

Basically the entry size is 12 bits, there needs some complicated operation to read the 12-bit data from 8-bit unit data.

Now we just implement this... there we go!

static dword fat12_find_next_cluster(blockdev::block_device *device , dword cluster , fat::general_fat_info_t &ginfo) {
    max_t index = ((cluster/2)*3)+(cluster%2);
    max_t sector_address = ((max_t)(index/((common_vbr_t *)ginfo.vbr)->bytes_per_sector))+ginfo.fat_area_loc;
    byte fat_area[512];
    word ret_cluster;
    device->device_driver->read(device , sector_address , 1 , fat_area);
    max_t b_index = (max_t)(index%((common_vbr_t *)ginfo.vbr)->bytes_per_sector);
    
    ret_cluster = fat_area[b_index]|(fat_area[b_index+1] << 8);
    
    if(cluster%2) ret_cluster >>= 4; // odd
    else ret_cluster &= 0x0FFF;      // even
    
    return ret_cluster;
}

Cache System

I integrated lots of frequently used features into one function.

get_file_by_cache_and_phys() function searches the required file_info handler in the cache. If handler is not found, the handler relating to the file is newly created using file system driver. If file does not exist at all, it returns error.

static file_info *get_file_by_cache_and_phys(const general_file_name file_path , int levels_to_exclude) {
    vfs::VirtualFileSystemManager *vfs_mgr = GLOBAL_OBJECT(vfs::VirtualFileSystemManager);
    char **file_list;
    int level_count;
    int last_hit_loc = 0;

    // failed to initialize the vfs manager
    if(!vfs_mgr->is_initialized_properly) return 0x00;

    level_count = get_file_name_list(file_path , file_list);

    file_info *file = vfs_mgr->search_object_last(level_count , file_path.root_directory , file_list , last_hit_loc);
    if(file != 0x00 && strcmp(file->file_name , file_list[level_count-1]) == 0) { // cash hit
        memory::pmem_free(file_list);

        return file;
    }
    debug::out::printf_function(DEBUG_TEXT , "get_file_by_cache" , "last cash hit : %s(file=0x%lx), hit_loc : %d\n" , file->file_name , file , last_hit_loc);
    
    file_info *tree_file = file;
    for(int i = last_hit_loc; i < level_count-levels_to_exclude; i++) {
        physical_file_location *pfileloc = &tree_file->file_loc_info;
        if(tree_file->is_mounted == true) pfileloc = &tree_file->mount_loc_info;

        file_info *new_file_handle = pfileloc->fs_driver->get_file_handle({file_list[i] , tree_file});
        if(new_file_handle == 0x00) {
            memory::pmem_free(file_list);
            return 0x00;
        }

        vfs_mgr->add_object(new_file_handle , tree_file);
        tree_file = new_file_handle;
    }

    memory::pmem_free(file_list);
    return tree_file;
}

This part of code searches for the file handler in the cache. The cache exists on the vfs_mgr structure, in a form of tree structure. When the file handler does not exist, it returns the last valid entry from the cache tree.

The cache diagram of get_file_by_cache_and_phys() function

get_buffer_by_cache_and_phys() function basically does the similar thing but with file buffer. When the cache entry does not exist, the cache entry is created and returned.

static block_cache_t *get_buffer_by_cache_and_phys(file_info *file , max_t linear_block_addr) {
    block_cache_t *cache;

    physical_file_location *file_loc = fsdev::get_physical_loc_info(file);
    max_t block_size = file_loc->block_device->geometry.block_size;
    max_t phys_block_loc = file_loc->fs_driver->get_phys_block_address(file , linear_block_addr);
    if(phys_block_loc == INVALID) { // does not exist, need to create new one
        return 0x00;
    }

    // cache does not exist, create new cache
    debug::out::printf("phys_block_loc : %d\n" , phys_block_loc);
    if((cache = file->disk_buffer_hash_table->search(phys_block_loc)) == 0x00) {
        cache = (block_cache_t *)memory::pmem_alloc(sizeof(block_cache_t));
        cache->block = (void *)memory::pmem_alloc(block_size);
        cache->block_size = block_size;
        cache->flushed = true; // newest version.

        file->disk_buffer_hash_table->add(phys_block_loc , cache);
        
        // block device as a unit
        file_loc->block_device->device_driver->read(file_loc->block_device , phys_block_loc , 1 , cache->block);
    }
    return cache;
}

The linear address of the file is converted to physical block address, and the function searches the buffer entry corresponding to the block address from the hash table of cache entries. If the entry does not exist in the hash table, the entry corresponding to the disk buffer is added. (The unit of one cache entry is the sector size of block device.)

When we need to create new block to the rear of file, we allocate new space in the disk via the file system driver functions.

long vfs::write(file_info *file , max_t size , const void *buffer) {
    ...
    max_t required_block_count = (create_size/block_size)+(create_size%block_size != 0);
    if(create_size != 0) {
        max_t block_created = 0;
        debug::out::printf("required_bs = %d\n" , required_block_count);
        while(block_created < required_block_count) {
            max_t block_loc , unit_size = 1;
            block_loc = file_loc->fs_driver->allocate_new_block(file , unit_size);
            block_created += unit_size;
            debug::out::printf("new_block_location : %d\n" , block_loc);
        }
    }
    ...
}

The vfs::write() function calculates the amount of new physical block required for the file operation. The physical block is directly allocated to the disk, and the change is instantly applied. However, the contents in the disk is still cached. Only the allocation is applied directly to the physical disk.

Directly applying the allocation of block could cause a potential conflict when multiple process opened one file.

This works completely fine for now, as there is only one process in the kernel. I will definitely fail to work properly when multiple processor are introduced.

To solve this synchronization problem, we need a separate queue to contain newly created block, so that the newly created block is processed different from already-existing cache entries when flushing..

Now I have to consider about the synchronization.. which I greatly hate.