Skip to main content

Tutorial: Vector Search with SochDB

🔧 Skill Level: Intermediate
⏱️ Time Required: 20 minutes
📦 Requirements: Python 3.9+, numpy, sentence-transformers
Versions: Core engine 2.0.3 · Python SDK 0.5.9

Learn how to build a semantic search system using SochDB's HNSW vector index.

Two Python packages named sochdb

There are two importable packages, both named sochdb:

  • The pure-Python ctypes SDK (v0.5.9) — the broad embedded + server SDK (Database, Namespace, Collection, VectorIndex, AgentMemory, …). This tutorial uses this package for general usage.
  • The native PyO3 engine (v2.0.3) — a focused HNSW/BM25/RRF engine exposing HnswIndex, build_index_from_numpy, recommended_hnsw_params, MultiShardHnswIndex, etc.

Both publish under pip install sochdb. Examples below note which class comes from which package.

🎯 What You'll Build​

A document search system that:

  • ✅ Stores documents with vector embeddings
  • ✅ Performs semantic similarity search
  • ✅ Returns relevant results based on meaning, not keywords

Step 1: Setup​

# Create project
mkdir semantic-search && cd semantic-search
python -m venv venv
source venv/bin/activate

# Install dependencies
pip install sochdb numpy sentence-transformers

Note: sentence-transformers provides real embedding models. For production, you might also use OpenAI, Cohere, or other embedding APIs.

What are Embeddings?​

Embeddings convert text into numerical vectors that capture semantic meaning:

"The cat sat on the mat"  ->  [0.12, -0.34, 0.56, ...]  (384 dimensions)
"A feline rested on a rug" ->  [0.11, -0.32, 0.55, ...]  (similar vector!)
"Python programming"       ->  [-0.45, 0.78, -0.23, ...] (different vector)

Similar meanings = similar vectors = small distance

How HNSW Works​

HNSW (Hierarchical Navigable Small World) is a graph-based index:

Layer 2:       ●───────────────────────●
               │                       │
Layer 1:       ●───●───────●───────────●───────●
               │   │       │           │       │
Layer 0:   ●───●───●───●───●───●───●───●───●───●───●
  • Search: Start at top layer, greedily descend
  • Complexity: O(log N) average
  • Recall: 95%+ at the engine's default settings (deep-1M recall@10 = 0.988 at M=32)

Step 3: Create the Embedding Service​

Create embeddings.py:

"""Embedding service using sentence-transformers."""

from sentence_transformers import SentenceTransformer
import numpy as np

class EmbeddingService:
    def __init__(self, model_name: str = "all-MiniLM-L6-v2"):
        """Initialize with a sentence-transformer model.

        Models:
        - all-MiniLM-L6-v2: Fast, good quality, 384 dims
        - all-mpnet-base-v2: Better quality, 768 dims, slower
        - all-distilroberta-v1: Balanced, 768 dims
        """
        print(f"Loading model: {model_name}...")
        self.model = SentenceTransformer(model_name)
        self.dimension = self.model.get_sentence_embedding_dimension()
        print(f"Model loaded. Embedding dimension: {self.dimension}")

    def embed(self, texts: list[str]) -> np.ndarray:
        """Convert texts to embeddings."""
        embeddings = self.model.encode(
            texts,
            convert_to_numpy=True,
            normalize_embeddings=True  # L2 normalize for cosine similarity
        )
        return embeddings.astype(np.float32)

    def embed_single(self, text: str) -> np.ndarray:
        """Convert a single text to embedding."""
        return self.embed([text])[0]


# Test it
if __name__ == "__main__":
    service = EmbeddingService()

    texts = [
        "The quick brown fox jumps over the lazy dog",
        "A fast auburn canine leaps above a sleepy hound",
        "Python programming is fun",
    ]

    embeddings = service.embed(texts)

    # Check similarity (dot product of normalized vectors = cosine similarity)
    sim_01 = np.dot(embeddings[0], embeddings[1])
    sim_02 = np.dot(embeddings[0], embeddings[2])

    print(f"Similarity (fox/canine): {sim_01:.3f}")  # Should be high (~0.7+)
    print(f"Similarity (fox/python): {sim_02:.3f}")  # Should be low (~0.2)

Run it:

python embeddings.py
# Loading model: all-MiniLM-L6-v2...
# Model loaded. Embedding dimension: 384
# Similarity (fox/canine): 0.763
# Similarity (fox/python): 0.186

Step 4: Build the Search System​

We use VectorIndex from the 0.5.9 ctypes SDK — an embedded HNSW index whose defaults (max_connections=32, ef_construction=256) mirror the engine's HnswConfig::default().

Create search.py:

"""Semantic search system with SochDB."""

from sochdb import Database, VectorIndex
from embeddings import EmbeddingService
import numpy as np
import json
import os

class SemanticSearch:
    def __init__(self, db_path: str = "./search_db"):
        self.db = Database.open(db_path)        # embedded FFI database
        self.embeddings = EmbeddingService()
        self.index = VectorIndex(dimension=self.embeddings.dimension)
        # ef_search controls recall vs. latency. Default is engine-side; raise
        # it for higher recall. See "Search-time tuning" below.
        self.index.ef_search = 100
        self._documents = []
        self._load_documents()

    def _load_documents(self):
        """Load existing documents from the database and rebuild the index."""
        for key, value in self.db.scan(b"docs/"):
            doc = json.loads(value.decode())
            self._documents.append(doc)
        if self._documents:
            self._rebuild_index()
        print(f"Loaded {len(self._documents)} existing documents")

    def add_documents(self, documents: list[dict]):
        """Add documents with automatic embedding generation.

        Each document should have at least a 'content' field.
        Additional fields (title, metadata) are preserved.
        """
        print(f"Adding {len(documents)} documents...")

        start_id = len(self._documents)
        with self.db.transaction() as txn:
            for i, doc in enumerate(documents):
                doc_id = start_id + i
                doc["id"] = doc_id
                txn.put(f"docs/{doc_id}".encode(), json.dumps(doc).encode())
                self._documents.append(doc)

        # Rebuild the index with all embeddings
        self._rebuild_index()
        print(f"✅ Added {len(documents)} documents (total: {len(self._documents)})")

    def _rebuild_index(self):
        """(Re)build the HNSW index from all documents."""
        if not self._documents:
            return

        contents = [doc["content"] for doc in self._documents]
        embeddings = self.embeddings.embed(contents)

        # Fresh index, then a single zero-copy batch insert
        self.index = VectorIndex(dimension=self.embeddings.dimension)
        self.index.ef_search = 100
        ids = np.arange(len(self._documents), dtype=np.uint64)
        count = self.index.insert_batch(ids, embeddings)
        print(f"Index built: {count} vectors")

    def search(self, query: str, k: int = 5) -> list[dict]:
        """Search for documents similar to the query."""
        if not self._documents:
            return []

        query_embedding = self.embeddings.embed_single(query)

        # search() returns list[(id, distance)]
        results = self.index.search(query_embedding, k=k)

        search_results = []
        for doc_id, distance in results:
            if doc_id < len(self._documents):
                doc = self._documents[doc_id].copy()
                doc["score"] = 1.0 - distance  # cosine distance -> similarity
                doc["distance"] = distance
                search_results.append(doc)

        return search_results


def main():
    search = SemanticSearch()

    documents = [
        {
            "title": "SochDB Overview",
            "content": "SochDB is an LLM-native database designed for AI applications. It provides token-efficient storage and vector search capabilities."
        },
        {
            "title": "Vector Search Basics",
            "content": "Vector search uses embeddings to find semantically similar documents. HNSW is a popular algorithm for approximate nearest neighbor search."
        },
        {
            "title": "Python Development",
            "content": "Python is a versatile programming language popular for data science, web development, and AI applications."
        },
        {
            "title": "Database Transactions",
            "content": "ACID transactions ensure data integrity. SochDB supports MVCC with serializable snapshot isolation for concurrent access."
        },
        {
            "title": "Machine Learning Models",
            "content": "Embedding models convert text to vectors. Popular models include sentence-transformers, OpenAI embeddings, and Cohere embeddings."
        },
    ]

    search.add_documents(documents)

    print("\n" + "=" * 60)

    queries = [
        "How does SochDB handle AI workloads?",
        "What is HNSW algorithm?",
        "How to ensure data consistency?",
    ]

    for query in queries:
        print(f"\n🔍 Query: {query}")
        print("-" * 40)
        for i, result in enumerate(search.search(query, k=3), 1):
            print(f"{i}. [{result['score']:.3f}] {result['title']}")
            print(f"   {result['content'][:80]}...")


if __name__ == "__main__":
    main()

Run it:

python search.py

Expected output:

Loading model: all-MiniLM-L6-v2...
Model loaded. Embedding dimension: 384
Loaded 0 existing documents
Adding 5 documents...
Index built: 5 vectors
✅ Added 5 documents (total: 5)

============================================================

🔍 Query: How does SochDB handle AI workloads?
----------------------------------------
1. [0.842] SochDB Overview
   SochDB is an LLM-native database designed for AI applications...
2. [0.534] Machine Learning Models
   Embedding models convert text to vectors...
3. [0.423] Vector Search Basics
   Vector search uses embeddings to find semantically similar documents...

🔍 Query: What is HNSW algorithm?
----------------------------------------
1. [0.756] Vector Search Basics
   Vector search uses embeddings to find semantically similar documents...
2. [0.412] SochDB Overview
   SochDB is an LLM-native database designed for AI applications...
3. [0.389] Machine Learning Models
   Embedding models convert text to vectors...

🔍 Query: How to ensure data consistency?
----------------------------------------
1. [0.698] Database Transactions
   ACID transactions ensure data integrity...
2. [0.312] SochDB Overview
   SochDB is an LLM-native database designed for AI applications...
3. [0.287] Vector Search Basics
   Vector search uses embeddings to find semantically similar documents...
Tiny datasets search exactly

With only 5 vectors, the engine does not build an HNSW graph at query time — it uses an exact parallel SIMD flat scan (see Flat-scan below). HNSW only matters once you cross the per-dimension flat-scan threshold.

Step 5: HNSW Configuration and Defaults​

The core engine's HnswConfig (Rust, sochdb-index/src/hnsw.rs) ships these defaults — they are tuned to clear 95% recall@10 out of the box:

ParameterDefaultMeaning
max_connections (M)32Neighbors per node above layer 0
max_connections_layer0 (M0)64Neighbors at layer 0 (= 2·M)
level_multiplier (mL)1 / ln(32)Layer-assignment probability
ef_construction256Build-time search breadth (richer graph)
ef_search500Query-time search breadth (recall vs. latency)
metricCosineDistance metric
quantization_precisionF32Vector storage precision
enable_product_quantizationfalseOptional PQ for very large/high-dim sets

Reported recall@10 on deep-1M: 0.967 at M=16/M0=32, 0.988 at M=32 (the default), 0.990 at M=48. ef_construction=256 particularly helps hard real embeddings (e.g. Cohere) where M alone left recall near 0.90.

No dimension-aware ef_search split in the core

ef_search defaults to a single fixed value of 500; there is no 500/1500-by-dimension branching in the core engine. The dimension-aware logic that does exist is the flat-scan threshold. The Python recommended_hnsw_params() helper (native package) does derive an ef_search from dimension + target recall — that is an SDK convenience, not a core default.

Distance metrics​

The DistanceMetric enum offers three options:

MetricUse whenNotes
Cosine (default)Normalized text/embedding similarityVectors are unit-normalized at ingest when enabled
Euclidean (L2)Geometric / magnitude-aware distanceStraight-line distance
DotProductInner-product / un-normalized modelsLarger = more similar

Precision (quantization)​

Vectors can be stored at reduced precision to cut memory (via the half crate):

PrecisionBytes/elementUse when
F32 (default)4Maximum recall, baseline
F162~2× memory savings, slight recall loss
BF162Wider dynamic range than F16

Product Quantization (enable_product_quantization) is an additional, optional codec that is off by default, intended for very large (> 100k vectors) and high-dimensional (> 128D) collections.

Step 6: Flat-Scan, search_smart, and search_exact​

Flat-scan (exact) for small datasets​

For small indexes, an exact parallel SIMD flat scan is both correct and faster than walking an HNSW graph. The engine chooses flat scan when the vector count is at or below a dimension-aware threshold:

DimensionFlat-scan threshold (vectors)
≤ 12810,000
≤ 3844,000
768+ (else)1,000

At or below the threshold, search runs a rayon-parallel SIMD brute-force scan and returns exact results. Above it, the approximate HNSW path takes over.

search_exact vs. search_smart​

The engine exposes explicit exact search and an automatic chooser:

  • search_exact(query, k) — always brute-force exact k-NN (also search_exact_f64 for f64 inputs). Use this for ground-truth recall checks.
  • search_smart(query, k, exact_threshold) — if dataset_size <= exact_threshold (default 1000) it calls search_exact; otherwise it runs the adaptive HNSW search. A convenient default for mixed-size workloads.

In the 0.5.9 ctypes SDK these surface on VectorIndex as search_exact(...) / search_exact_f64(...) alongside the regular search(...), plus build_flat_cache() to warm the flat-scan path.

Step 7: Graph Maintenance and Diagnostics​

After heavy inserts (especially incremental ones), you can repair and tighten the graph for better recall. These operations exist in the native 2.0.3 package on HnswIndex:

MethodWhat it does
refine_graph_additive() -> intParallel additive neighbor refinement (fills empty slots, never removes edges); returns count improved
refine_graph() -> intGeneral neighbor refinement pass
optimize() -> intExact brute-force layer-0 rebuild (recall booster for small indexes; ~0.3s per 10K)
repair() -> intReconnect orphaned nodes via BFS
diagnose() -> dictConnectivity/degree report: reachable, total, orphan_count, avg_degree, zero_degree_nodes, target_degree
optimize() is a small-index booster, not a large-index step

The exact layer-0 rebuild is O(N²·D). At ~1M vectors it can OOM a large box and run for hours, so the core guards it as a no-op above a size cap — the same 1M build skipping it succeeded in ~195s at recall@10 = 0.968. Treat optimize() as a recall booster for small-to-medium indexes only.

# Native package (sochdb 2.0.3): build, refine, inspect
import numpy as np
from sochdb import HnswIndex

index = HnswIndex(dimension=768, m=32, ef_construction=256, metric="cosine")
index.insert_batch(np.random.randn(50_000, 768).astype(np.float32))

improved = index.refine_graph_additive()   # tighten the graph
report = index.diagnose()                    # connectivity / degree stats
print(f"refined={improved} orphans={report['orphan_count']} "
      f"avg_degree={report['avg_degree']:.1f}")

Step 8: Tune Performance​

Build-time parameters​

ParameterEffectTrade-off
m=16Faster build, less memoryLower recall on high-dim data
m=32Engine default95+ recall@10 out of the box
m=48Best recall (0.990 @ deep-1M)More memory, slower build
ef_construction=200Faster buildSlightly lower quality
ef_construction=256Engine defaultStrong quality on hard embeddings
ef_construction=400Highest qualitySlow build

Search-time tuning​

ef_search (the query-time beam width) trades recall for latency. The 0.5.9 VectorIndex exposes it as a settable property:

index.ef_search = 50    # fast, ~0.9 recall
index.ef_search = 100   # balanced, ~0.95 recall
index.ef_search = 200   # high recall, ~0.99

The native recommended_hnsw_params(dimension, target_recall=0.95) helper returns dimension-aware m, ef_construction, and ef_search suggestions:

  • M: dim ≤ 128 → 16; 129–512 → 24; 513+ → 32
  • ef_construction: max(200, M*8)
  • ef_search: recall ≥ 0.99 → 40·M; ≥ 0.95 → 20·M; ≥ 0.90 → 10·M; else 6·M
Synthetic vs. real data

The helper's note warns that synthetic uniform-random vectors need roughly 2–3× higher ef_search than real embedding data to hit the same recall.

Memory estimation​

Memory ≈ vectors × dimensions × bytes/element × ~1.5 overhead

F32 example: 100,000 × 384 × 4 × 1.5 ≈ 230 MB
F16 example: 100,000 × 384 × 2 × 1.5 ≈ 115 MB

Per-SDK Quick Reference​

The same HNSW engine is reachable from every SDK. Constructor parameter names and defaults differ per language; the table-defining values above are the core engine's.

# Embedded VectorIndex (defaults mirror the engine: M=32, efc=256)
import numpy as np
from sochdb import VectorIndex

index = VectorIndex(dimension=768, max_connections=32, ef_construction=256)
index.ef_search = 100

ids = np.arange(1000, dtype=np.uint64)
index.insert_batch(ids, np.random.randn(1000, 768).astype(np.float32))

query = np.random.randn(768).astype(np.float32)
for doc_id, distance in index.search(query, k=10):
    print(doc_id, distance)

# Exact ground-truth search for recall checks
exact = index.search_exact(query, k=10)

Step 9: Production Considerations​

Use persistent embeddings​

Don't regenerate embeddings on every load — store them alongside the document and reload them on startup:

def add_documents_with_cache(self, documents):
    """Store embeddings alongside documents."""
    contents = [doc["content"] for doc in documents]
    embeddings = self.embeddings.embed(contents)

    with self.db.transaction() as txn:
        for i, (doc, emb) in enumerate(zip(documents, embeddings)):
            doc_id = len(self._documents) + i
            txn.put(f"docs/{doc_id}".encode(), json.dumps(doc).encode())
            txn.put(f"embeddings/{doc_id}".encode(), emb.tobytes())  # binary

Batch processing​

For large datasets, insert in batches to manage memory:

def add_documents_batched(self, documents, batch_size=1000):
    """Add documents in batches to manage memory."""
    for i in range(0, len(documents), batch_size):
        batch = documents[i:i + batch_size]
        self.add_documents(batch)
        print(f"Processed {min(i + batch_size, len(documents))}/{len(documents)}")

For the fastest bulk path that bypasses Python FFI overhead, see Bulk Operations.

What You Learned​

ConceptWhat You Did
EmbeddingsConverted text to vectors using sentence-transformers
HNSW indexingBuilt an approximate nearest neighbor index with engine defaults (M=32, ef_construction=256)
Flat-scan & exact searchUnderstood when SochDB searches exactly vs. approximately
TuningUsed ef_search, metrics, precision, and graph maintenance
SochDB integrationStored documents and vectors together

Next Steps​

GoalResource
Filtered / session-scoped searchHNSW Vector Search with Filtering
Fastest bulk index buildsBulk Operations
Build a RAG systemMCP Integration
Optimize performancePerformance Guide
Production deploymentDeployment Guide

Tutorial completed! You've built a working semantic search system with SochDB. 🎉