Go SDK Guide

Skill Level: Beginner Time Required: 30 minutes Requirements: Go 1.24+ SDK Version: 0.4.5 (Apache-2.0)

Guide to SochDB's Go SDK: a remote-first gRPC/IPC client with an optional in-process embedded engine, plus namespaces, collections, priority queues, an LLM-native memory system, semantic cache, context builder, and graph APIs.

License

The Go SDK is licensed Apache-2.0. The SochDB core engine (the Rust workspace, the sochdb crate, server, and MCP) is AGPL-3.0-or-later with commercial licensing available. These are separate components.

---

Installation

go get github.com/sochdb/sochdb-go

The module path is github.com/sochdb/sochdb-go and the current version is 0.4.5 (const Version = "0.4.5").

Stale package doc string

The package-level doc comment in sochdb.go still reads "v0.4.3". The authoritative version is the Version constant (0.4.5); the doc comment is a known inconsistency.

---

Remote-first by default

This is the most important thing to understand about the Go SDK:

A plain go build / go get (with no build tags) compiles only the remote/gRPC and pure-Go portions of the SDK. The in-process embedded FFI engine is excluded unless you build with -tags sochdb_embedded.

# Default build: gRPC + IPC client + pure-Go subsystems only
go build ./...

# Opt in to the embedded FFI engine (requires CGO + native libsochdb_storage + pkg-config)
go build -tags sochdb_embedded ./...

Always compiled (no build tag needed): the gRPC client (grpc_client.go), the IPC client (client.go), namespaces/collections (namespace.go), priority queue (queue.go), context builder (context_builder.go), memory types (memory_types.go), errors (errors.go), and data formats (format.go).

Behind -tags sochdb_embedded (requires CGO, the native libsochdb_storage library, and pkg-config): the embedded FFI database (embedded/database.go), the embedded semantic cache, and the entire embedded memory system (extraction, consolidation, retrieval).

"Remote-first" is build-tag gating, not transport auto-selection

There is no unified Connect() overload or Config struct that auto-selects remote vs. embedded transport. "Remote-first" means the default compilation target is remote. The connection entrypoints are distinct, explicitly named functions (below).

---

Connection entrypoints

There is no single unified Client type. The SDK exposes three distinct clients.

gRPC (remote, default build)

package main

import (
	"context"
	"log"

	sochdb "github.com/sochdb/sochdb-go"
)

func main() {
	// Convenience: dial a gRPC server.
	client, err := sochdb.GrpcConnect("localhost:50051")
	if err != nil {
		log.Fatal(err)
	}
	defer client.Close()

	// Remote KV.
	err = client.PutKv(context.Background(), "default", "key", []byte("value"))
	if err != nil {
		log.Fatal(err)
	}
}

For more control, build options explicitly:

client, err := sochdb.NewGrpcClient(sochdb.GrpcClientOptions{
	Address: "localhost:50051",
	Timeout: 30 * time.Second,
	Secure:  false,
})

GrpcClientOptions defaults:

Field	Type	Default	Notes
Address	string	localhost:50051	Used when empty.
Timeout	time.Duration	30 * time.Second	Used when zero.
Secure	bool	false	true returns an error.

TLS is not implemented

Setting Secure: true returns "secure connections not yet implemented". The gRPC client uses insecure credentials only. Max receive message size is 100 MiB.

IPC (Unix socket)

package main

import (
	"log"

	sochdb "github.com/sochdb/sochdb-go"
)

func main() {
	// Dial a server's Unix socket directly.
	client, err := sochdb.Connect("/path/to/sochdb.sock")
	if err != nil {
		log.Fatal(err)
	}
	defer client.Close()

	// Or join "<dbPath>/sochdb.sock" for you.
	client2, err := sochdb.ConnectToDatabase("/path/to/db")
	if err != nil {
		log.Fatal(err)
	}
	defer client2.Close()
}

Both return an *IPCClient.

Embedded FFI (build tag)

Requires -tags sochdb_embedded, CGO, and the native libsochdb_storage library.

package main

import (
	"log"

	"github.com/sochdb/sochdb-go/embedded"
)

func main() {
	db, err := embedded.Open("./mydb")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	// Multi-reader / single-writer mode (lock-free ~100ns reads).
	cdb, err := embedded.OpenConcurrent("./mydb")
	if err != nil {
		log.Fatal(err)
	}
	defer cdb.Close()

	_ = cdb.IsConcurrent()
}

---

CLI tools

The SDK ships Go-native command wrappers installable via go install:

go install github.com/sochdb/sochdb-go/cmd/sochdb-server@latest
go install github.com/sochdb/sochdb-go/cmd/sochdb-bulk@latest
go install github.com/sochdb/sochdb-go/cmd/sochdb-grpc-server@latest

---

Key-value operations

The KV surface differs by client.

gRPC

ctx := context.Background()

// Convenience helpers.
err := client.PutKv(ctx, "default", "key", []byte("value"))
value, err := client.GetKv(ctx, "default", "key")

// Low-level (namespace + optional TTL).
err = client.GrpcPut([]byte("key"), []byte("value"), "default", 0 /* ttlSeconds */)
v, found, err := client.GrpcGet([]byte("key"), "default")
err = client.GrpcDelete([]byte("key"), "default")

IPC

err := client.Put([]byte("user:123"), []byte(`{"name":"Alice"}`))
value, err := client.Get([]byte("user:123"))
err = client.Delete([]byte("user:123"))

// Hierarchical paths.
err = client.PutPath("users/alice/email", []byte("alice@example.com"))
email, err := client.GetPath("users/alice/email")

Embedded FFI

// Auto-transaction helpers on *embedded.Database.
err := db.Put([]byte("key"), []byte("value"))
value, err := db.Get([]byte("key"))
err = db.Delete([]byte("key"))
err = db.PutPath("users/alice/email", []byte("alice@example.com"))
v, err := db.GetPath("users/alice/email")

---

Prefix scanning and ScanPrefix safety

The IPC client offers a materialized prefix scan that returns a slice of KeyValue{ Key, Value []byte }:

results, err := client.Scan("users/")
if err != nil {
	log.Fatal(err)
}
for _, kv := range results {
	fmt.Printf("%s: %s\n", kv.Key, kv.Value)
}

// Paginated prefix query (path + limit + offset). This is NOT SQL.
page, err := client.Query("users/", 100 /* limit */, 0 /* offset */)

The embedded engine exposes a streaming iterator instead:

it := db.ScanPrefix([]byte("users/"))
defer it.Close() // REQUIRED — the iterator holds a transaction open.

for {
	key, val, ok := it.Next()
	if !ok {
		break
	}
	fmt.Printf("%s: %s\n", key, val)
}
if err := it.Err(); err != nil {
	log.Fatal(err)
}

Always Close() a *ScanIterator

The embedded ScanPrefix iterator holds an open transaction for the lifetime of iteration. You must call Close() (defer it) or you will leak the transaction.

Do not split queue keys on /

Queue keys contain binary big-endian fields that may include the / byte (0x2F). Splitting a queue key on / is unsafe — use the SDK's positional DecodeQueueKey (used internally by the queue API) instead of string splitting.

---

Transactions

Commit returns an error only in the Go SDK — it does not return a commit timestamp. (The uint64 you may have seen elsewhere is the transaction id returned by BeginTransaction, and the embedded FFI Checkpoint() (uint64, error).)

Embedded FFI

// Closure form: commit on nil return, abort on error.
err := db.WithTransaction(func(txn *embedded.Transaction) error {
	if err := txn.Put([]byte("account:1"), []byte("1000")); err != nil {
		return err
	}
	return txn.Put([]byte("account:2"), []byte("500"))
})

// Manual form.
txn := db.Begin()
defer txn.Abort() // safety net; no-op after a successful commit

txn.Put([]byte("key1"), []byte("value1"))
if err := txn.Commit(); err != nil {
	log.Printf("commit failed: %v", err)
}

_ = txn.ID()         // uint64 transaction id
_ = txn.SnapshotTS() // uint64 snapshot timestamp

Serialization conflicts

A commit may fail with "SSI conflict: transaction aborted due to serialization failure" (FFI error code -2). Retry the transaction in that case.

IPC

txnID, err := client.BeginTransaction() // returns the uint64 txn id
if err != nil {
	log.Fatal(err)
}

// ... perform operations referencing txnID ...

if err := client.CommitTransaction(txnID); err != nil {
	// On failure you can abort:
	_ = client.AbortTransaction(txnID)
	log.Printf("commit failed: %v", err)
}

---

SQL

No SQL execution surface in the Go SDK

The Go SDK has no exported SQL method. The OpExecuteSQL opcode is defined internally but is not wired to any Go method, and there are no CREATE TABLE / INSERT / DDL / DML helpers. IPCClient.Query(prefix, limit, offset) is a prefix query, not SQL. (The only Execute() in the SDK is ContextQueryBuilder.Execute(), which is unrelated.)

To run SQL against SochDB, use the server's SQL/pg-wire interface or another SDK.

---

Namespaces and collections

Namespaces provide multi-tenant isolation; collections hold vectors.

Embedded / pure-Go handles

// Create a collection on a namespace handle.
collection, err := ns.CreateCollection(sochdb.CollectionConfig{
	Name:               "documents",
	Dimension:          384,
	Metric:             sochdb.DistanceMetricCosine,
	Indexed:            true,
	HNSWM:              16,
	HNSWEfConstruction: 100,
})
if err != nil {
	log.Fatal(err)
}

// Insert a vector (empty id auto-generates one).
id, err := collection.Insert(
	[]float32{0.1, 0.2, 0.3 /* ... */},
	map[string]interface{}{"source": "web"},
	"",
)

// Search.
results, err := collection.Search(sochdb.SearchRequest{
	QueryVector:     queryEmbedding,
	K:               10,
	IncludeMetadata: true,
})
for _, r := range results {
	fmt.Printf("ID: %s, Score: %.4f\n", r.ID, r.Score)
}

Namespace handle methods include Collection(name), GetOrCreateCollection, DeleteCollection(name), and ListCollections().

DistanceMetric string constants: DistanceMetricCosine ("cosine"), DistanceMetricEuclidean ("euclidean"), DistanceMetricDotProduct ("dot").

Embedded vector search is brute-force

On the embedded path, Collection.Search performs brute-force cosine similarity over a prefix scan (sorted descending, truncated to K). The HNSWM / HNSWEfConstruction fields are stored as JSON metadata only — there is no real HNSW index on the embedded handle.

gRPC

ctx := context.Background()

err := client.CreateCollection("documents", 384 /* dimension */, "default" /* namespace */)

ids, err := client.AddDocuments("documents", []sochdb.GrpcDocument{
	{ID: "d1", Content: "hello", Embedding: vec, Metadata: map[string]string{"k": "v"}},
}, "default")

docs, err := client.SearchCollection("documents", queryEmbedding, 10 /* k */, "default")

Low-level vector index operations are also available: CreateIndex(name, dimension, metric), InsertVectors(indexName, ids, vectors), and GrpcSearch(indexName, query, k) returning []GrpcSearchResult{ ID uint64; Distance float32 }.

gRPC CreateIndex HNSW config is fixed

CreateIndex hard-codes the HNSW configuration (it is not parameterized in Go): MaxConnections=16, MaxConnectionsLayer0=32, EfConstruction=100, EfSearch=64. The metric accepts l2, dot/dot_product, and falls back to cosine. There is no DropIndex/DeleteIndex.

CreateNamespace not in the public-facing files surveyed

The doc comments reference db.CreateNamespace(...), but that method is not defined in the surveyed namespace/SDK files. Treat its exact signature as unverified pending the embedded/FFI source; do not rely on the older db.CreateNamespace("tenant_123") form without checking your build.

---

Priority queue

The priority queue is pure-Go and always compiled.

import sochdb "github.com/sochdb/sochdb-go"

// nil config uses defaults: VisibilityTimeout=30000ms, MaxRetries=3.
queue := sochdb.NewPriorityQueue(db, "tasks", nil)

taskID, err := queue.Enqueue(1 /* priority */, []byte("high priority task"), nil /* metadata */)
if err != nil {
	log.Fatal(err)
}

task, err := queue.Dequeue("worker-1")
if err != nil {
	log.Fatal(err)
}
if task != nil {
	fmt.Printf("Processing: %s\n", task.Payload)
	if err := queue.Ack(task.TaskID); err != nil {
		// On failure, return to the queue:
		_ = queue.Nack(task.TaskID)
	}
}

stats, err := queue.Stats()
purged, err := queue.Purge()

QueueConfig fields: Name, VisibilityTimeout int (default 30000ms), MaxRetries int (default 3), DeadLetterQueue string. Task states: pending, claimed, completed, dead_lettered. CreateQueue is an alias for NewPriorityQueue.

---

Memory system

Embedded only

The memory system requires -tags sochdb_embedded and an *embedded.Database. The pure-Go types (Entity, Relation, Assertion, RawAssertion, CanonicalFact, ExtractionResult, ExtractionSchema, ConsolidationConfig, RetrievalConfig, etc.) are always compiled, but the pipelines below are not.

// Extraction: compile LLM output into typed entities/relations/assertions.
pipeline := sochdb.NewExtractionPipeline(db, "memory", schema)
result, err := pipeline.ExtractAndCommit(ctx, text)
entities := pipeline.GetEntities()

// Consolidation: merge multi-source raw assertions into canonical facts.
consolidator := sochdb.NewConsolidator(db, "memory", &sochdb.ConsolidationConfig{ /* ... */ })
consolidator.Add(raw)
n, err := consolidator.Consolidate()
facts := consolidator.GetCanonicalFacts()

// Retrieval: BM25 + semantic hybrid.
retriever := sochdb.NewHybridRetriever(db, "memory", &sochdb.RetrievalConfig{ /* ... */ })
retriever.IndexDocuments(docs)
resp, err := retriever.Retrieve("What did Alice buy?", allowedSet)

ExtractorFunction is func(text string) (map[string]interface{}, error). A standalone scorer is available via NewBM25Scorer(k1, b float64).

Allowed sets and policy

Pre-filtering for retrieval uses the AllowedSet interface with constructors NewIdsAllowedSet, NewNamespaceAllowedSet, NewFilterAllowedSet, and NewAllAllowedSet (these types are pure-Go and always compiled).

Tenant-isolation policy types are also pure-Go: NamespacePolicy string constants NamespacePolicyStrict, NamespacePolicyExplicit, NamespacePolicyPermissive, plus NamespaceGrant for cross-namespace access.

---

Semantic cache

There are three implementations depending on transport.

gRPC

err := client.CachePut("llm_cache", "key", "value", keyEmbedding, 3600 /* ttlSeconds */)
value, hit, err := client.CacheGet("llm_cache", queryEmbedding, 0.95 /* threshold */)
if hit {
	fmt.Println("cache hit:", value)
}

IPC

// IPCClient also exposes CachePut / CacheGet.
err := client.CachePut(/* ... */)
value, hit, err := client.CacheGet(/* ... */)

Embedded FFI

Requires -tags sochdb_embedded.

cache := sochdb.NewSemanticCache(db, "llm_cache")
err := cache.Put(/* ... */)
value, err := cache.Get(/* ... */)
_ = cache.Clear()
stats, _ := cache.Stats()
removed, _ := cache.PurgeExpired()

---

Context builder

Pure-Go and always compiled. A fluent builder for token-budgeted context assembly.

result, err := sochdb.NewContextQueryBuilder().
	ForSession("session-123").
	WithBudget(4096).                       // default 4096
	SetFormat(sochdb.FormatTOON).           // default FormatTOON
	SetTruncation(sochdb.TailDrop).         // default TailDrop
	Literal("system", 100, "You are a helpful assistant.").
	Execute()
if err != nil {
	log.Fatal(err)
}
// Execute returns *ContextResult{ Text, TokenCount, Sections, Truncated }.
fmt.Printf("context (%d tokens): %s\n", result.TokenCount, result.Text)

Output formats: FormatTOON (default), FormatJSON, FormatMarkdown. Truncation strategies: TailDrop (default), HeadDrop, Proportional.

TOON is the default wire format

SochDB defaults to TOON for wire and context formats (documented as "40-66% fewer tokens than JSON"). The format.go package exposes WireFormat, ContextFormat, and CanonicalFormat with parsers and FormatCapabilities for round-trip checks.

---

Graph

Graph APIs are available over both transports.

IPC

err := client.AddNode("default", "alice", "Person", map[string]string{"role": "admin"})
err = client.AddEdge(/* ... */)
res, err := client.Traverse("default", "alice", 3 /* maxDepth */, "bfs" /* order */)

Types: GraphNode, GraphEdge, TraverseResult.

gRPC

err := client.AddGraphNode(/* ... */)
err = client.AddGraphEdge(/* ... */)
nodes, edges, err := client.TraverseGraph("alice", 3, "bfs", "default")

// Convenience helpers.
err = client.AddEdge(ctx, "default", edge)
res, err := client.QueryGraph(ctx, "default", "alice", "knows", 10)

Types: GrpcGraphNode, GrpcGraphEdge.

---

Tracing

Observability spans are available over both transports.

// IPC: StartTrace / StartSpan / EndSpan (types TraceInfo, SpanData).
// gRPC: equivalent StartTrace / StartSpan / EndSpan on *GrpcClient.
trace, err := client.StartTrace(/* ... */)
span, err := client.StartSpan(/* ... */)
err = client.EndSpan(/* ... */)

---

No routing module

No routing/router subsystem in Go

Unlike some other SochDB SDKs, the Go SDK has no routing or router subsystem. (IndexPolicy and NamespacePolicy are the only "policy" types; neither performs request routing.)

---

Error handling

The Go SDK exposes sentinel errors (compatible with errors.Is) and typed error structs.

import "errors"

value, found, err := client.GrpcGet([]byte("key"), "default")
if err != nil {
	switch {
	case errors.Is(err, sochdb.ErrClosed):
		log.Println("client closed")
	case errors.Is(err, sochdb.ErrNotFound):
		log.Println("key not found")
	case errors.Is(err, sochdb.ErrDatabaseLocked):
		log.Println("database is locked")
	default:
		log.Printf("error: %v", err)
	}
}
_ = found

Sentinel errors

Sentinel	Meaning
ErrClosed	Client/connection closed.
ErrNotFound	Key/resource not found.
ErrInvalidResponse	Malformed server response.
ErrDatabaseLocked	Database lock held by another process.
ErrLockTimeout	Timed out acquiring a lock.
ErrEpochMismatch	Epoch/generation mismatch.
ErrSplitBrain	Split-brain condition detected.

Typed error structs

• Connection/protocol/server: ConnectionError{Address, Err} (implements Unwrap), ProtocolError{Message}, ServerError{Message}, TransactionError{Message}, SochDBError{Op, Message}.
• Lock/concurrency (each implements Is() matching to the sentinel above): LockError{Path, Message, Remediation}, DatabaseLockedError{Path, HolderPID}, LockTimeoutError{Path, TimeoutSecs}, EpochMismatchError{Expected, Actual}, SplitBrainError{Message}.
• Namespace/collection: NamespaceNotFoundError, NamespaceExistsError, CollectionNotFoundError, CollectionExistsError.
• Format: FormatConversionError{FromFormat, ToFormat, Reason}.

---

Best practices

Always close clients and iterators

client, err := sochdb.GrpcConnect("localhost:50051")
if err != nil {
	log.Fatal(err)
}
defer client.Close()

it := db.ScanPrefix([]byte("users/"))
defer it.Close() // iterator holds a transaction open

Use transactions for atomic, multi-key writes

// Atomic on the embedded engine.
db.WithTransaction(func(txn *embedded.Transaction) error {
	txn.Put([]byte("balance:1"), []byte("900"))
	txn.Put([]byte("balance:2"), []byte("1100"))
	return nil
})

Retry on serialization conflict

for attempts := 0; attempts < 3; attempts++ {
	err := db.WithTransaction(doWork)
	if err == nil {
		break
	}
	// "SSI conflict: ... serialization failure" — safe to retry.
}

Prefer Scan / ScanPrefix for prefix iteration and multi-tenancy

prefix := fmt.Sprintf("tenants/%s/", tenantID)
data, _ := client.Scan(prefix)

---

Resources

• Go SDK on GitHub
• pkg.go.dev
• Python SDK
• Node.js SDK
• Rust SDK

---

Last updated: June 2026 (SDK v0.4.5)