FerroDB
A relational database built from scratch in Rust, now AI-native with an HNSW vector index
"Can you build the machine, not just glue one together?" Most application developers never touch the layer beneath the ORM, storage engines, query planners, concurrency control, and index structures stay black boxes. FerroDB is the answer to that question: a database with no shortcuts, where every subsystem a production DB hides is instead built from first principles, tested, and documented in the open, including the modern piece everyone now needs, a vector index.
I built FerroDB bottom-up in Rust, one subsystem at a time: a page-based pager with checksums and a buffer pool, B+-tree indexes, write-ahead logging validated against crash-injection tests, MVCC snapshot isolation, a System-R-style cost-based optimizer with EXPLAIN, and a hand-rolled slice of the PostgreSQL wire protocol that a real psql client connects to. The ninth milestone makes it AI-native: an HNSW approximate-nearest-neighbor index, the algorithm behind pgvector, Qdrant, and FAISS, implemented from the paper as a secondary index. Vector search returns row keys, the B+-tree resolves rows, and MVCC filters deleted ghosts at fetch. The hot distance loop is hand-written AVX2 with runtime CPU dispatch, and filtered vector search threads WHERE predicates into the graph traversal to dodge the post-filter recall cliff.
HNSW Vector Index
An approximate-nearest-neighbor graph index implemented from the paper as a secondary index, the algorithm behind pgvector, Qdrant, and FAISS. Vector search returns row keys; the B+-tree resolves rows; MVCC filters deleted ghosts at fetch.
Filtered Vector Search
WHERE + k-NN resolved in one engine. Predicates are threaded into the graph traversal instead of applied afterward, dodging the recall cliff that post-filtering causes at high selectivity.
Hand-Written AVX2 SIMD
The hot distance loop (L2, dot, cosine) is hand-vectorized with runtime CPU dispatch, measured at 7–9× over scalar, property-tested to match the scalar reference exactly in ordering.
Measured Recall, Not Asserted
A recall harness benchmarks the index against brute-force ground truth across ef_search settings; recall@10 reaches ~0.99. The debugging trail is documented, including the time the benchmark itself was lying.
PostgreSQL Wire Protocol
A hand-rolled implementation of enough of the Postgres frontend/backend protocol that the real psql client connects, runs queries, and gets results back, no libpq shim, just the bytes on the wire.
WAL, MVCC & Crash Recovery
Write-ahead logging with a recovery path proven by crash-injection tests, plus MVCC snapshot isolation so readers never block writers and every transaction sees a stable view.
FerroDB is public on GitHub with green CI, a 122-test suite, a published architecture book, and a live WebAssembly playground with a B+-tree visualizer. Correctness is measured, not asserted: a recall harness benchmarks the vector index against brute-force ground truth (recall@10 reaches ~0.99), and the SIMD kernels are property-tested at 7–9× over scalar, with the debugging trail documented, including the time the benchmark itself was lying. It's the portfolio piece that preempts "did you actually build this?" before anyone asks.
122 tests · 9 milestones · recall@10 ~0.99 · 7–9× SIMD over scalar
Want something like this?
Let's build it together. I take on a limited number of full-stack and AI engineering engagements each quarter.
Get in touch