SochDB Performance

This page covers two performance areas of the SochDB engine (core v2.0.3):

1. Storage write throughput — the WAL/MVCC kernel optimizations that drive embedded insert performance.
2. Vector search (HNSW) — recall, the brute-force speedup claim, and the dimension-aware flat-scan crossover.

It closes with the Prometheus metric names exported by the server so you can wire up observability.

Where these numbers come from

Throughput figures are from the sochdb-bench crate and may vary by hardware, kernel, and filesystem. Recall figures are taken from in-code measurements and comments in sochdb-index/src/hnsw.rs. Treat them as representative, not guaranteed SLAs — re-run the benchmarks on your own hardware (see Reproducing Benchmarks).

Storage Write Throughput

The storage-layer work below targets sequential single-threaded inserts, taking the embedded path from roughly ~55k ops/sec to ~800k-1.3M ops/sec (an order-of-magnitude improvement). On the same machine SQLite (file, WAL mode) measured around ~1.16M ops/sec, so the embedded path lands in the same ballpark. These are point measurements on one machine, not a portable benchmark result.

Benchmark Results

Benchmark	Before	After	Improvement
SochDB Embedded (WAL)	~55k ops/sec	~761k ops/sec	13.8×
SochDB put_raw	N/A	~1.30M ops/sec	Direct storage
SochDB insert_row_slice	N/A	~1.29M ops/sec	Zero-alloc API
SQLite File (WAL)	~1.16M ops/sec	—	Baseline

---

Root Cause Analysis

Discovery: 10× Overhead in Database Layer

Initial profiling revealed that raw DurableStorage::put() achieved 1.5M ops/sec, but the Database layer only achieved ~150k ops/sec. This pointed to overhead in the transaction/database abstraction layer rather than the storage engine itself.

Key Bottleneck: Group Commit

The group commit mechanism, designed to batch multiple transactions for better fsync amortization, was counterproductive for sequential single-threaded inserts:

Group Commit Flow (BEFORE):
┌─────────────────────────────────────────────────┐
│  1. Acquire mutex                               │
│  2. Add to pending batch                        │
│  3. Wait on condvar for batch fill OR timeout   │  ← BLOCKING!
│  4. Leader does fsync                           │
│  5. Wake all waiters                            │
└─────────────────────────────────────────────────┘

For sequential inserts, each operation was waiting for either:

• Other transactions to join the batch (none coming)
• A timeout (adding latency)

Solution: Disabled group commit for the embedded connection benchmark, allowing direct WAL writes without the coordination overhead.

Architecture Context

SochDB has a properly layered architecture:

┌─────────────────────────────────────────────────────────────┐
│  Client Layer                                               │
│  ├── Connection (alias for DurableConnection) ← DEFAULT     │
│  ├── InMemoryConnection (SochConnection) ← Testing only     │
│  └── EmbeddedConnection (optional, wraps Database)          │
├─────────────────────────────────────────────────────────────┤
│  Storage Layer                                              │
│  ├── Database (tables, schemas, MVCC coordination)          │
│  ├── DurableStorage (WAL, transactions, sync modes)         │
│  └── TxnWal (transaction-aware WAL with CRC32)              │
└─────────────────────────────────────────────────────────────┘

The DurableConnection properly routes through the WAL/MVCC kernel, making all optimizations in the storage layer visible to the client API.

---

Optimizations Applied

1. Single-Pass CRC32 Calculation

File: sochdb-storage/src/txn_wal.rs

Before: CRC was calculated in a separate pass over the data.

After: CRC32 is calculated incrementally as data is written to the buffer.

pub fn write_no_flush_refs(&self, txn_id: TxnId, key: &[u8], value: &[u8]) -> Result<()> {
    let mut writer = self.writer.lock();
    
    // Single-pass: write and compute CRC simultaneously
    let mut crc = 0u32;
    
    // Write and accumulate CRC for each field
    writer.write_all(&txn_id.to_le_bytes())?;
    crc = crc32c::crc32c_append(crc, &txn_id.to_le_bytes());
    
    // ... continues for all fields
}

Impact: Eliminated redundant memory traversal, ~5-10% improvement.

---

2. Zero-Allocation WAL Writes

File: sochdb-storage/src/txn_wal.rs

Before: Each WAL write allocated a Vec<u8> buffer.

After: New write_no_flush_refs() method writes directly to BufWriter using references.

// OLD: Allocates Vec for each write
fn write(&self, key: Vec<u8>, value: Vec<u8>)

// NEW: Zero allocation, uses references
fn write_no_flush_refs(&self, txn_id: TxnId, key: &[u8], value: &[u8])

Impact: Eliminated per-write heap allocations.

---

3. DashMap for Lock-Free Concurrent Access

Files:

• sochdb-storage/src/durable_storage.rs
• sochdb-storage/src/database.rs

Before: RwLock<HashMap> for MVCC tracking and table metadata.

After: DashMap for lock-free concurrent reads.

// OLD: Lock contention on every access
tables: RwLock<HashMap<String, TableMetadata>>
active_txns: RwLock<HashMap<TxnId, TxnState>>

// NEW: Lock-free concurrent access
tables: DashMap<String, TableMetadata>
active_txns: DashMap<TxnId, TxnState>

Impact: Reduced lock contention, especially beneficial for concurrent workloads.

---

4. Zero-Allocation Row Insertion API

Files:

• sochdb-storage/src/packed_row.rs
• sochdb-storage/src/database.rs

Before: insert_row() required a HashMap<String, SochValue> allocation per row.

After: New insert_row_slice() accepts &[(&str, SochValue)] directly.

// OLD: Allocates HashMap per row
pub fn insert_row(&self, txn: TxnHandle, table: &str, 
                  row: HashMap<String, SochValue>) -> Result<u64>

// NEW: Zero allocation, uses slice
pub fn insert_row_slice(&self, txn: TxnHandle, table: &str,
                        row: &[(&str, SochValue)]) -> Result<u64>

PackedRow Enhancement:

// NEW: Pack from slice of references
pub fn pack_slice(values: &[Option<&SochValue>]) -> Vec<u8>

// NEW: Unpack to pre-sized Vec
pub fn unpack_to_vec(&self) -> Vec<Option<SochValue>>

Impact: Achieved 1.29M ops/sec, matching raw storage performance.

---

5. Cached Schema Lookup

File: sochdb-storage/src/database.rs

Before: Schema was fetched and parsed on every insert.

After: Schema is cached in DashMap<String, Vec<String>> after first access.

packed_schemas: DashMap<String, Vec<String>>  // table_name -> column_names

Impact: Eliminated repeated schema parsing overhead.

---

6. Lazy Query Iterator

File: sochdb-storage/src/database.rs

Before: execute() collected all results into a Vec before returning.

After: execute_iter() returns a lazy iterator that fetches rows on demand.

pub fn execute_iter(self) -> QueryRowIterator<'a> {
    QueryRowIterator {
        inner: self.storage.scan_prefix(&self.path_prefix).into_iter(),
        // ...
    }
}

Impact: Reduced memory usage for large result sets, enables streaming.

---

7. Group Commit Bypass for Sequential Workloads

File: sochdb-storage/src/durable_storage.rs

For benchmarks and sequential workloads, group commit can be disabled:

let config = DurableStorageConfig {
    group_commit: false,  // Disable for sequential inserts
    sync_mode: 1,         // NORMAL sync every 100 commits
    ..Default::default()
};

Impact: Eliminated condvar wait overhead, ~5× improvement for sequential inserts.

---

Architecture Insight

Performance Stack (ops/sec):
┌─────────────────────────────────────────┐
│  insert_row (HashMap)     ~150k         │  ← HashMap allocation overhead
├─────────────────────────────────────────┤
│  insert_row_slice         ~1.29M        │  ← Zero-allocation path
├─────────────────────────────────────────┤
│  put_raw                  ~1.30M        │  ← Direct storage bypass
├─────────────────────────────────────────┤
│  DurableStorage::put      ~1.50M        │  ← Raw storage layer
└─────────────────────────────────────────┘

Bottleneck removed: Group commit coordination

---

Key Takeaways

1. Group commit is workload-dependent: Great for concurrent multi-tenant, counterproductive for sequential single-threaded.

2. Allocation matters at scale: HashMap per row adds up to significant overhead at 1M+ ops/sec.

3. Lock-free structures help: DashMap eliminates lock contention for concurrent access patterns.

4. Layer overhead accumulates: Raw storage was fast; overhead was in the abstraction layers.

5. Measure before optimizing: Profiling revealed the true bottleneck was group commit, not I/O.

---

Vector Search (HNSW)

SochDB's approximate nearest-neighbor index lives in sochdb-index/src/hnsw.rs. The module header states a ~250x speedup over brute force for large collections, achieved by the standard HNSW graph navigation plus SIMD distance kernels.

Recall

Recall depends on the graph connectivity (M / M0) and the construction effort (ef_construction). The in-code measurements on the deep-1M dataset (recall@10) are:

M (max connections)	M0 (layer-0)	recall@10 (deep-1M)
16	32	~0.967
32 (default)	64 (default)	~0.988
48	96	~0.990

The defaults (M=32, M0=64) are the documented sweet spot — they "clear 95% out of the box" and give roughly recall@10 ~0.97-0.99 across typical embedding workloads. A full 1M-vector build reached recall@10 ~0.968 in ~195s (skipping the optional exact layer-0 rebuild, which does not scale to 1M). For hard, high-dimensional real embeddings (for example Cohere) raising ef_construction to 256 lifts recall that would otherwise sit near ~0.90.

Default HnswConfig

These are the actual defaults applied when a field is left unset (sochdb-index/src/hnsw.rs, also used by the gRPC CreateIndex fallback):

Parameter	Default	Notes
max_connections (M)	32	Graph degree above layer 0
max_connections_layer0 (M0)	64	Standard M0 = 2 * M
level_multiplier (mL)	1 / ln(32)	Layer assignment
ef_construction	256	Build-time candidate list
ef_search	500	Query-time candidate list
metric	Cosine	Also Euclidean, DotProduct
quantization_precision	F32	Also F16, BF16

ef_search is a single fixed default

The query-time ef_search default is a single value of 500. There is no dimension-aware ef_search split in the core engine. The dimension-aware logic that does exist is the flat-scan crossover described below.

Flat-scan crossover (small collections)

For small collections, an exact parallel SIMD brute-force scan is faster (and exact) than HNSW graph traversal. The crossover threshold is dimension-aware (hnsw.rs):

flat_scan_threshold = if dimension <= 128   { 10_000 }
                      else if dimension <= 384 { 4_000 }
                      else                      { 1_000 }   // 768D+

When the collection size is at or below the threshold, search runs the flat scan; above it, the HNSW graph is used. A higher-level search_smart(query, k, exact_threshold) helper also routes to search_exact when the dataset is at or below a small threshold (default 1000), otherwise to the adaptive HNSW search.

Adaptive ef (not adaptive RRF-k)

AdaptiveSearchConfig can binary-search the minimum ef that hits a target recall (default target_recall = 0.95). This is HNSW ef adaptivity. The RRF fusion constant k is fixed at 60.0 in the core engine — there is no adaptive RRF-k in the Rust core. (A separate adaptive_rrf_k option exists only in the Python HybridSearchIndex.)

---

Observability (Prometheus Metrics)

When you run the gRPC server (sochdb-grpc-server), Prometheus metrics are exported over HTTP. The endpoint is controlled by --metrics-port (default 9090; set to 0 to disable):

• GET /metrics — Prometheus text exposition format
• GET /health — returns 200 OK

The metrics HTTP server binds 0.0.0.0:<metrics-port> on a dedicated thread.

gRPC / SQL / storage metrics

Exported from sochdb-grpc/src/metrics_server.rs:

Metric	Type	Labels
sochdb_grpc_requests_total	counter	service, method
sochdb_grpc_errors_total	counter	service, method, code
sochdb_grpc_request_duration_seconds	histogram	—
sochdb_grpc_active_connections	gauge	—
sochdb_sql_queries_total	counter	statement_type
sochdb_sql_query_duration_seconds	histogram	—
sochdb_transactions_total	counter	outcome
sochdb_tables_count	gauge	—
sochdb_storage_bytes	gauge	—
sochdb_wal_bytes	gauge	—
sochdb_wal_writes_total	counter	—
sochdb_wal_fsync_total	counter	—
sochdb_cache_operations_total	counter	result
sochdb_uptime_seconds	gauge	—
sochdb_build_info	gauge	version, rustc

HNSW index metrics

The HNSW index registers its own metrics (sochdb-index/src/metrics.rs), which are auto-included in the default registry and therefore appear on the same /metrics endpoint:

Metric	Type	Description
hnsw_insert_total	counter	Total vector insertions
hnsw_search_total	counter	Total searches performed
hnsw_errors_total	counter	Total errors encountered
hnsw_insert_duration_seconds	histogram	Insert latency
hnsw_search_duration_seconds	histogram	Search latency
hnsw_operation_duration_seconds	histogram	Generic operation latency
hnsw_search_results_returned	histogram	Results returned per search
hnsw_distance_calculations_per_search	histogram	Distance computations per search
hnsw_nodes_total	gauge	Nodes in the index
hnsw_max_layer	gauge	Highest graph layer
hnsw_avg_neighbors	gauge	Average neighbor count
hnsw_memory_bytes	gauge	Index memory usage
hnsw_vector_dimension	gauge	Vector dimensionality

A separate gRPC health service (tonic_health) is also mounted on the gRPC port (not behind auth) for Kubernetes liveness/readiness probes.

---

Future Optimizations

See Architecture for remaining optimization areas:

• Memtable size limits: Flush to disk when threshold exceeded
• WAL compaction: Checkpoint/compaction to reclaim disk space
• Adaptive group commit: Switch modes based on workload pattern
• Connection pooling: For multi-tenant scenarios

---

Remaining Architectural Gaps

Based on the comprehensive architecture analysis in task.md, the following optimizations are planned:

High Priority (P0-P1)

Gap	Current State	Target	Impact
Epoch-Based GC	O(n) scan all keys	O(expired_versions) per cycle	10-100× GC reduction
Client GroupCommit	Duplicate implementation in client/storage	Single source in storage layer	Eliminates confusion
Adaptive Group Commit	Fixed thresholds (sync every 100 commits)	Little's Law: W* = √(τ/λ)	Better latency tuning

Medium Priority (P2)

Gap	Current State	Target	Impact
SSI Validation	read_set/write_set tracked but not validated	Full dangerous structure detection	Serializability
Vector Index WAL	HNSW uses periodic snapshots	WAL integration for crash recovery	ACID for embeddings
Cardinality Estimation	Static column_cardinalities	HyperLogLog++ streaming updates	5× plan quality

Lower Priority (P3)

Gap	Current State	Target	Impact
Clock-Pro Buffer	TinyLFU via moka	Clock-Pro + FIFO ghost cache	2× hit rate for scans
io_uring Integration	Sync fallback on non-Linux	SQ polling + batched submission	3× IOPS on Linux
Tiered Compaction	Implicit leveled strategy	Hybrid tiered/leveled	70% less write amp

Formulas Reference

Optimal Group Commit Wait Time (Little's Law):

W* = √(τ/λ)

Where:
- τ = fsync latency (~5ms on NVMe)
- λ = arrival rate (ops/sec)

For λ = 10,000 TPS, τ = 5ms:
  W* = √(0.005/10000) = 0.707ms
  Expected batch size = λ × W* = 7 transactions

MVCC GC with Epoch-Based Reclamation:

Memory bound = 3 × epoch_duration × write_rate

For epoch_duration = 100ms, write_rate = 100K/s:
  Max retained = 30,000 versions = ~1.9MB

---

Reproducing Benchmarks

The Rust benchmark crate is sochdb-bench:

cargo run -p sochdb-bench --release

This runs comparisons against SQLite and exercises the various SochDB storage APIs. Numbers vary by hardware, kernel, and filesystem, so always re-measure on your target environment before quoting figures.