kvdb

kvdb

A simple, fast persistent key-value store based on Bitcask, and a TCP server compatible with the Redis protocol (RESP)

How to run

Using docker (recommended)

$ docker run -p 6379:6379 -t ananthvk0/kvdb:0.0.1 -db <databasename>

To ensure that data does not get lost on container restart,

$ docker run \
  -p 6379:6379 \
  -v kv-volume:/data \
  ananthvk0/kvdb:0.0.1 \
  -db /data/mydb

Install

$ go install github.com/ananthvk/kvdb/cmd/kvserver@latest

Then use it as,

$ kvserver -db mydb

Or, clone this repository, and then run the following commands

Standalone CLI

$ go run ./cmd/kvcli <path to database directory>

To run the redis compatible server

$ go run ./cmd/kvserver -db <path to db directory> -host 0.0.0.0 -port 6379

For example,

$ go run ./cmd/kvserver -db mydb

Access the server through redis-cli

Supported commands: GET, SET, ECHO, PING, KEYS *, DEL

To create dummy data,

$ go run ./cmd/kvjson -n 100000 -size 1024 -db testdb

or

$ go run ./cmd/kvmake -n 5000_000 5mdb

Features

• Merge and compaction to remove stale keys, and merge datafiles
• Hint files to improve startup time
• Redis (RESP) compatible TCP server
• Supports multiple readers, and a single writer (same process)
• Log rotation determined by log file size

File format specification for datafile

All integer values are stored in Little Endian format

The file header is 19 bytes in size

Log Format

Each log entry in the data file contains a log header (20 bytes) followed by variable length key and value, followed by the CRC of the header + key + value

Log record:

Note: Timestamps are unix timestamps, in microsecond format

Record type

0x50 ('P') - PUT record
0x44 ('D') - DELETE (Tombstone), Value Size is set to 0, and no value bytes are present

Directory structure

testdb/
	kvdb_store.meta
	data/
		0000000001.dat
		0000000002.dat
		...

The file with the largest numerical value is considered the active file when opening the datastore.

The file name is zero padded to length of 10 characters

A file kvdb_store.meta will indicate that the directory is a valid store, it also holds configuration of the datastore

It follows a simple INI like key=value structure

Example structure

name = testdb
version = 1.0.0
created = 2026-02-10 11:32:00.118157 +0000 UTC
max_datafile_size=5000000

Key & Value size limits

Keys have a maximum size of 1000 bytes (1 KB)

And values have a maximum size of 1000000 bytes (1 MB)

Default value of max data file size is 12800000 bytes (128MB) but it's configurable through kvdb_store.meta file

TODO

• Handling of corrupted records & hint file
• Crash recovery
• Add support for multiple processes
• Implement TTL
• Graceful shutdown