FerroDB
A relational database built from scratch in Rust — now AI-native with an HNSW vector index
"Can you build the machine, not just glue one together?" Most application developers never touch the layer beneath the ORM — storage engines, query planners, concurrency control, and index structures stay black boxes. FerroDB is the answer to that question: a database with no shortcuts, where every subsystem a production DB hides is instead built from first principles, tested, and documented in the open — including the modern piece everyone now needs, a vector index.
I built FerroDB bottom-up in Rust, one subsystem at a time: a page-based pager with checksums and a buffer pool, B+-tree indexes, write-ahead logging validated against crash-injection tests, MVCC snapshot isolation, a System-R-style cost-based optimizer with EXPLAIN, and a hand-rolled slice of the PostgreSQL wire protocol that a real psql client connects to. The ninth milestone makes it AI-native: an HNSW approximate-nearest-neighbor index — the algorithm behind pgvector, Qdrant, and FAISS — implemented from the paper as a secondary index. Vector search returns row keys, the B+-tree resolves rows, and MVCC filters deleted ghosts at fetch. The hot distance loop is hand-written AVX2 with runtime CPU dispatch, and filtered vector search threads WHERE predicates into the graph traversal to dodge the post-filter recall cliff.
HNSW Vector Index
An approximate-nearest-neighbor graph index implemented from the paper as a secondary index — the algorithm behind pgvector, Qdrant, and FAISS. Vector search returns row keys; the B+-tree resolves rows; MVCC filters deleted ghosts at fetch.
Filtered Vector Search
WHERE + k-NN resolved in one engine. Predicates are threaded into the graph traversal instead of applied afterward, dodging the recall cliff that post-filtering causes at high selectivity.
Hand-Written AVX2 SIMD
The hot distance loop (L2, dot, cosine) is hand-vectorized with runtime CPU dispatch — measured at 7–9× over scalar, property-tested to match the scalar reference exactly in ordering.
Measured Recall, Not Asserted
A recall harness benchmarks the index against brute-force ground truth across ef_search settings; recall@10 reaches ~0.99. The debugging trail is documented — including the time the benchmark itself was lying.
PostgreSQL Wire Protocol
A hand-rolled implementation of enough of the Postgres frontend/backend protocol that the real psql client connects, runs queries, and gets results back — no libpq shim, just the bytes on the wire.
WAL, MVCC & Crash Recovery
Write-ahead logging with a recovery path proven by crash-injection tests, plus MVCC snapshot isolation so readers never block writers and every transaction sees a stable view.
FerroDB is public on GitHub with green CI, a 122-test suite, a published architecture book, and a live WebAssembly playground with a B+-tree visualizer. Correctness is measured, not asserted: a recall harness benchmarks the vector index against brute-force ground truth (recall@10 reaches ~0.99), and the SIMD kernels are property-tested at 7–9× over scalar — with the debugging trail documented, including the time the benchmark itself was lying. It's the portfolio piece that preempts "did you actually build this?" before anyone asks.
122 tests · 9 milestones · recall@10 ~0.99 · 7–9× SIMD over scalar
Want something like this?
Let's build it together. I take on a limited number of full-stack and AI engineering engagements each quarter.
Start a Project