SochDB End-to-End Profiling

Name: SochDB
Author: SochDB

Overview

This document describes the end-to-end profiling infrastructure for SochDB's HNSW vector index, covering the entire pipeline from the Python SDK through FFI to the Rust core.

Versions

This page reflects the pure-Python ctypes SDK (sochdb 0.5.9) and the SochDB core engine (2.0.3). The profiling helpers (enable_profiling(), disable_profiling(), dump_profiling()) live in the 0.5.9 SDK's sochdb.vector module; the native PyO3 package (also named sochdb, version 2.0.3) does not expose them.

Architecture

┌─────────────────────────────────────────────────────────────────────────────┐
│                           Python SDK Layer                                   │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ 10_e2e_profiling.py                                                  │   │
│  │  - PrecisionTimer (nanosecond resolution)                            │   │
│  │  - ProfiledVectorIndex wrapper                                       │   │
│  │  - Memory tracking via tracemalloc                                   │   │
│  │  - Per-operation timing: numpy, dtype, validation, FFI ptr creation  │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
│                                    │                                         │
│                                    ▼                                         │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ sochdb.vector (FFI bindings, 0.5.9 SDK)                              │   │
│  │  - ctypes bindings to libsochdb_index.dylib                          │   │
│  │  - insert_batch → hnsw_insert_batch                                  │   │
│  │  - Profiling control: enable_profiling() / disable_profiling()       │   │
│  │                        / dump_profiling()                            │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘
                                     │
                                     │ FFI Boundary
                                     ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                           Rust FFI Layer (ffi.rs)                            │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ hnsw_insert_batch()                                                  │   │
│  │  - Profiled: ffi.slice_from_raw, ffi.id_conversion                   │   │
│  │  - Calls: index.insert_batch_contiguous()                            │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘
                                     │
                                     ▼
┌─────────────────────────────────────────────────────────────────────────────┐
│                           HNSW Core (hnsw.rs)                                │
│  ┌─────────────────────────────────────────────────────────────────────┐   │
│  │ insert_batch_contiguous_bulk()                                       │   │
│  │                                                                       │   │
│  │ Phase 1: Parallel Quantization (hnsw.phase1.quantize_parallel)       │   │
│  │   - rayon parallel iterator                                          │   │
│  │   - Creates HnswNode with QuantizedVector                            │   │
│  │                                                                       │   │
│  │ Phase 2: Map Insert (hnsw.phase2.map_insert)                         │   │
│  │   - Insert nodes into DashMap                                        │   │
│  │   - Update entry point                                               │   │
│  │                                                                       │   │
│  │ Phase 3: Connection Building (hnsw.phase3.*)                         │   │
│  │   - connect_total: Overall connection time                           │   │
│  │   - search_layer: Graph navigation                                   │   │
│  │   - neighbor_select: RNG heuristic ← MAIN BOTTLENECK                 │   │
│  │   - add_connections: Bidirectional edge creation                     │   │
│  └─────────────────────────────────────────────────────────────────────┘   │
└─────────────────────────────────────────────────────────────────────────────┘

Profiling Infrastructure

Python-Side Profiling

File: sochdb-python-sdk/examples/10_e2e_profiling.py

# Key components
class PrecisionTimer:
    """Nanosecond-precision timer using time.perf_counter_ns()"""
    
class TimingStats:
    """Statistical aggregation: count, total, mean, std, min, max, p50, p95, p99"""

class ProfiledVectorIndex:
    """Wrapper that instruments all VectorIndex operations"""
    
    def insert_batch_profiled(vectors, ids):
        # Times: numpy_ascontiguous, dtype_conversion, data_validation, 
        #        ffi_ptr_creation, ffi_call_overhead, batch_total

Output: profiling_results.json

Profiling control hooks

The sochdb.vector module (pure-Python sochdb 0.5.9 SDK) exposes three no-argument helpers that toggle the Rust-side profiling collector across the FFI boundary:

from sochdb.vector import enable_profiling, disable_profiling, dump_profiling

enable_profiling()      # turn on the Rust-side profiling collector
# ... run your inserts / searches ...
dump_profiling()        # write /tmp/sochdb_profile.json and print a summary
disable_profiling()     # turn the collector back off

These call the FFI exports sochdb_profiling_enable(), sochdb_profiling_dump(), and sochdb_profiling_disable() respectively. Enabling at runtime is equivalent to setting SOCHDB_PROFILING=1 before launching the process.

Rust-Side Profiling

File: sochdb-index/src/profiling.rs

// Enabling profiling
SOCHDB_PROFILING=1  // Environment variable

// Key components
pub struct Timer { /* start, stop, record */ }
pub struct OpStats { count, total_ns, min_ns, max_ns, item_count }
pub struct ProfileCollector { /* global singleton with thread-safe stats */ }

// FFI exports
pub extern "C" fn sochdb_profiling_enable();
pub extern "C" fn sochdb_profiling_disable();
pub extern "C" fn sochdb_profiling_dump();

Output: /tmp/sochdb_profile.json

Running Profiling

Basic Usage

cd sochdb-python-sdk

# Enable profiling and run
export SOCHDB_PROFILING=1
export SOCHDB_LIB_PATH=/path/to/target/release/libsochdb_index.dylib
python3 examples/10_e2e_profiling.py

# Analyze results
python3 examples/11_profiling_analysis.py

Customizable Parameters

Edit 10_e2e_profiling.py:

# Configuration
NUM_VECTORS = 1000        # Number of vectors to insert
DIMENSION = 768           # Vector dimension (e.g., OpenAI embeddings)
EF_CONSTRUCTION = 200     # HNSW ef parameter (quality vs speed)
MAX_CONNECTIONS = 16      # HNSW M parameter

Profiling Results (1000 vectors, 768d)

Summary Metrics

Metric	Value
Total Insert Time	13,533 ms (13.5s)
Throughput	74 vectors/sec
Latency	13.5 ms/vector
Peak Memory	9.45 MB
Search Time	1.7 ms

Time Breakdown

Phase	Time (ms)	Percentage	Per-Vector
Neighbor Selection	10,365	76.6%	11.07 ms
Add Connections	1,908	14.1%	2.04 ms
Search Layer	1,076	7.9%	1.15 ms
Quantization	0.99	`<0.01%`	1.05 µs
Map Insert	0.11	`<0.01%`	0.12 µs
FFI Overhead	0.003	`<0.01%`	0.003 µs

Bottleneck Analysis

CRITICAL BOTTLENECK: Neighbor Selection (RNG Heuristic) - 77%

The select_neighbors_heuristic function implements the Relative Neighborhood 
Graph (RNG) pruning which is essential for HNSW quality but computationally 
expensive.

For each of N vectors inserted:
- Search returns ~ef candidates (200)
- RNG checks each candidate against all previously selected neighbors
- Each check requires a full 768-dimensional distance calculation
- Complexity: O(N × ef × m × D) where D is dimension

With N=1000, ef=200, m=16, D=768:
- Approximately 1000 × 200 × 16 = 3.2M distance calculations
- Each 768d cosine distance = 3 SIMD operations (dot, norm_a, norm_b)

Optimization Recommendations

1. Reduce ef_construction

# Current: ef_construction = 200
# Recommended: ef_construction = 100-128
# Trade-off: ~5% recall reduction, 2x faster insert

index = VectorIndex(
    dimension=768,
    ef_construction=100,  # Reduced from 200
    max_connections=16,
)

2. Use Batch Distance Computation

The RNG heuristic currently does scalar distance calculations. Batching candidates and using SIMD would provide 4-8x speedup on distance computation.

3. Pre-compute Distance Cache

Cache pairwise distances between candidates during search to avoid recomputation in neighbor selection.

4. Approximate RNG for Early Inserts

Use simpler heuristics (top-k by distance) when graph is sparse, switch to full RNG when graph is denser.

5. Parallel Neighbor Selection

The RNG check for each candidate can be parallelized since it's independent.

Files Created

File	Purpose
`sochdb-python-sdk/examples/10_e2e_profiling.py`	Main profiling script
`sochdb-python-sdk/examples/11_profiling_analysis.py`	Analysis and visualization
`sochdb-python-sdk/src/sochdb/vector.py`	`enable_profiling` / `disable_profiling` / `dump_profiling` FFI hooks
`sochdb-index/src/profiling.rs`	Rust profiling infrastructure
`profiling_results.json`	Python-side profiling output
`/tmp/sochdb_profile.json`	Rust-side profiling output

JSON Output Format

Python Profile (profiling_results.json)

{
  "timestamp": "2025-12-29T00:33:55",
  "config": {
    "num_vectors": 1000,
    "dimension": 768,
    "ef_construction": 200,
    "max_connections": 16
  },
  "summary": {
    "total_insert_time_ms": 13533.17,
    "vectors_per_second": 73.89,
    "peak_memory_mb": 9.45
  },
  "python_layer": {
    "timings": {
      "numpy_ascontiguous": { "total_ms": 0.008 },
      "dtype_conversion": { "total_ms": 0.013 },
      "ffi_ptr_creation": { "total_ms": 0.032 },
      "batch_total": { "total_ms": 13533.05 }
    }
  }
}

Rust Profile (/tmp/sochdb_profile.json)

{
  "total_elapsed_ms": 13537.17,
  "operations": {
    "hnsw.phase3.neighbor_select": {
      "count": 1,
      "total_ms": 10365.00,
      "mean_us": 10365003.96,
      "item_count": 936,
      "per_item_us": 11073.72
    },
    "hnsw.phase3.add_connections": { ... },
    "hnsw.phase3.search_layer": { ... },
    "hnsw.phase1.quantize_parallel": { ... },
    "hnsw.phase2.map_insert": { ... }
  }
}

Conclusion

The profiling reveals that 77% of insertion time is spent in the RNG neighbor selection heuristic, which is the core HNSW algorithm for maintaining graph quality. This is expected behavior for high-dimensional vectors.

Key insights:

Python/FFI overhead is negligible (<0.001%)
Quantization and map operations are very fast (<0.01%)
The bottleneck is algorithmic, not I/O or memory
Optimization should focus on the neighbor selection loop

For faster inserts at the cost of some recall:

Reduce ef_construction from 200 to 100
Use lower precision (F16) for distance calculations
Implement batched SIMD distance in the RNG loop

Overview​

Architecture​

Profiling Infrastructure​

Python-Side Profiling​

Profiling control hooks​

Rust-Side Profiling​

Running Profiling​

Basic Usage​

Customizable Parameters​

Profiling Results (1000 vectors, 768d)​

Summary Metrics​

Time Breakdown​

Bottleneck Analysis​

Optimization Recommendations​

1. Reduce ef_construction​

2. Use Batch Distance Computation​

3. Pre-compute Distance Cache​

4. Approximate RNG for Early Inserts​

5. Parallel Neighbor Selection​

Files Created​

JSON Output Format​

Python Profile (profiling_results.json)​

Rust Profile (/tmp/sochdb_profile.json)​

Conclusion​