Struct FlatChainStore

pub struct FlatChainStore {
    headers: MmapMut,
    metadata: MmapMut,
    block_index: BlockIndex,
    fork_headers: MmapMut,
    accumulator_file: File,
    cache: Mutex<LruCache<BlockHash, DiskBlockHeader>>,
}

The main struct that holds all the context for our flat chain store

This struct is kept in memory, and it holds multiple memory maps that may or may not be in RAM right now. All functions in the impl block are inherently unsafe, since we’re dealing with raw pointers and memory maps. We need to be very careful with them. All methods should be carefully tested and reviewed. This struct is not thread-safe, and it’s not meant to be used in multi-threaded environments without proper synchronization.

We only ever expect one chainstate to hold a FlatChainStore at a time. You can then use that chainstate to interact with the chainstore, even in a multi-threaded environment.

Fields

headers: MmapMut

The memory map for our headers

metadata: MmapMut

The memory map for our metadata

block_index: BlockIndex

The memory map for our block index

fork_headers: MmapMut

The memory map for our fork files

accumulator_file: File

The file containing the accumulators for each blocks

cache: Mutex<LruCache<BlockHash, DiskBlockHeader>>

A LRU cache for the last n blocks we’ve touched

Implementations

impl FlatChainStore

fn create_chain_store( config: FlatChainStoreConfig, ) -> Result<Self, FlatChainstoreError>

Creates a new storage, given a configuration

If any of the I/O operations fail, this function should return an error

pub fn new(config: FlatChainStoreConfig) -> Result<Self, FlatChainstoreError>

Opens a new storage. If it already exists, just load. If not, create a new one

unsafe fn add_index_entry( &mut self, hash: BlockHash, index: Index, ) -> Result<(), FlatChainstoreError>

Adds a new entry into the block index, given a block hash and its Index

This is the only place where we should call BlockIndex.set_index_for_hash. Increments occupancy only if the entry is new; errors if the index map is full.

fn check_integrity(&self) -> Result<(), FlatChainstoreError>

Checks the integrity of our database

This function will check the integrity of our database by comparing the checksum of the headers file, index map, and fork headers file with the checksum stored in the metadata.

As checksum, the xxHash of the memory-mapped region is used. This is a fast hash function that is very good at detecting errors in memory. It is not cryptographically secure, but it is enough for random errors in a file.

pub fn compute_checksum(&self) -> DbCheckSum

Computes the XXH3-64 checksum for our database

fn truncate_to_pow2(n: usize) -> usize

Truncates a number to the nearest power of 2

unsafe fn init_file( path: &str, size: usize, _mode: u32, ) -> Result<MmapMut, FlatChainstoreError>

Initializes a memory-mapped file with the specified byte size and permissions (mode). If the underlying file does not exist, it will be created.

unsafe fn get_disk_header( &self, index: Index, ) -> Result<&HashedDiskHeader, FlatChainstoreError>

Returns a reference to the respective disk header from the file. Errors if nothing is found.

unsafe fn get_disk_header_mut( &mut self, index: Index, ) -> Result<&mut HashedDiskHeader, FlatChainstoreError>

Returns a mutable reference to the respective disk header from the file, which may be uninitialized. This method must only be used for recording a new header or mutating one.

unsafe fn do_save_height( &mut self, best_block: &BestChain, ) -> Result<(), FlatChainstoreError>

unsafe fn get_best_chain(&self) -> Result<BestChain, FlatChainstoreError>

unsafe fn get_header_by_hash( &self, hash: BlockHash, ) -> Result<Option<DiskBlockHeader>, FlatChainstoreError>

Returns the block header, given a block hash

If the header doesn’t exist in our index, it’ll return an error

unsafe fn get_metadata(&self) -> Result<&Metadata, FlatChainstoreError>

unsafe fn get_metadata_mut( &mut self, ) -> Result<&mut Metadata, FlatChainstoreError>

unsafe fn write_header_to_storage( &mut self, header: DiskBlockHeader, ) -> Result<(), FlatChainstoreError>

Writes a block header in our storage

This function will allocate size_of(DiskBlockHeader) bytes in our file and write the raw header there. May return an error if we can’t grow the file

unsafe fn save_fork_block( &mut self, header: DiskBlockHeader, ) -> Result<(), FlatChainstoreError>

Saves a block that is not in our main chain

If called for a reorg, we must make sure that the chain is marked as inactive before marking the new chain as active. This happens because we’ll write over the old chain inside the headers file.

If we mark a chain as Inactive, when we call get_header_by_index it will return the main chain index. If we override this position in the headers file, we get a different hash. The algorithm will think that position is occupied with a different header and therefore keep looking for a vacant position. Therefore, we’ll have one stale index that will never be used.

When marking a chain active, because we don’t overwrite the fork block (we actually call update_index before saving the actual header), even if we get an Index to the fork block, it’ll return the same hash. Therefore, our find method will return the right entry that will be overwritten with the new position.

Here’s an example:

Say we have the following chain:

1 -> 2 -> 3 -> 4 -> 5
     \ -> 3' -> 4'

If we want to reorg to the fork chain, we must:

1. Mark the chain [3, 4, 5] as inactive
2. Mark the chain [3’, 4’] as active

If we do this in the wrong order, when we try to save, e.g. 3. The index will find a position for 3 in the main chain, and return the position of 3’. Since 3’ is different, the find algorithm will think this position doesn’t exist, returning the next vacant position.

We will write 3 in a new position, and it should work fine. However, we now have a stale 3 that points to the main chain position where it originally was. This will never be used again, but will occupy a position in the index. Increasing the load factor for no reason.

unsafe fn do_flush(&mut self) -> Result<(), FlatChainstoreError>

Trait Implementations

impl ChainStore for FlatChainStore

type Error = FlatChainstoreError

fn check_integrity(&self) -> Result<(), Self::Error>

Checks if our database didn’t get corrupted, and if it has, it returns an error.

fn flush(&mut self) -> Result<(), Self::Error>

Flushes write buffers to disk, this is called periodically by the ChainState, so in case of a crash, we don’t lose too much data. If the database doesn’t support write buffers, this method can be a no-op.

fn save_roots_for_block( &mut self, roots: Vec<u8>, height: u32, ) -> Result<(), Self::Error>

Saves the accumulator state for a given block height.

fn load_roots_for_block( &mut self, height: u32, ) -> Result<Option<Vec<u8>>, Self::Error>

Loads the state of our accumulator for a given block height.

fn get_header( &self, block_hash: &BlockHash, ) -> Result<Option<DiskBlockHeader>, Self::Error>

Get a block header from our database. See DiskBlockHeader for more info about the data we save.

fn get_header_by_height( &self, height: u32, ) -> Result<Option<DiskBlockHeader>, Self::Error>

Get a block header by its height in our database.

fn load_height(&self) -> Result<Option<BestChain>, Self::Error>

Loads the blockchain height

fn save_height(&mut self, height: &BestChain) -> Result<(), Self::Error>

Saves the blockchain height.

fn save_header(&mut self, header: &DiskBlockHeader) -> Result<(), Self::Error>

Saves a block header to our database. See DiskBlockHeader for more info about the data we save.

fn get_block_hash(&self, height: u32) -> Result<Option<BlockHash>, Self::Error>

Returns the block hash for a given height.

fn update_block_index( &mut self, height: u32, hash: BlockHash, ) -> Result<(), Self::Error>

Associates a block hash with a given height, so we can retrieve it later.