gRPC / Protobuf API Reference
The SochDB server (sochdb-grpc-server, engine 2.0.3) exposes a network-first,
"thick server / thin client" gRPC surface. All business logic lives in the Rust server;
the language SDKs are thin RPC wrappers. This page is a machine-readable catalog of the
services, RPCs, and message shapes so that agents and code generators have an exact surface
to target.
- Default gRPC port:
50051(--port/-p). - Default bind host:
127.0.0.1(--host). Loopback-only unless you change it. - Proto package:
sochdb.v1(syntax = "proto3"). - Go package option:
github.com/sochdb/sochdb/proto/v1;sochdbv1. - Java package option:
com.sochdb.v1.
The canonical proto for the running server is sochdb-grpc/proto/sochdb.proto. A second,
slightly smaller copy at proto/sochdb.proto defines the same services but omits
SubscriptionService. Generate clients from sochdb-grpc/proto/sochdb.proto to get the full
surface, including CDC subscriptions.
This is a deliberately literal reference: exact RPC names, message field numbers, enum values, and metadata headers. Use it when generating a client, wiring an agent, or verifying a request shape against the server.
Service catalog
The server registers 12 application gRPC services plus a standard gRPC health service.
All RPC and message names below are taken directly from sochdb-grpc/proto/sochdb.proto.
| Service | RPCs | Purpose | Auth interceptor |
|---|---|---|---|
VectorIndexService | CreateIndex, DropIndex, InsertBatch, InsertStream, Search, SearchBatch, GetStats, HealthCheck | HNSW vector index ops | No (see note) |
GraphService | AddNode, GetNode, DeleteNode, AddEdge, GetEdges, DeleteEdge, Traverse, ShortestPath, GetNeighbors, AddTemporalEdge, QueryTemporalGraph | Graph overlay + temporal graph | Yes |
PolicyService | RegisterPolicy, Evaluate, ListPolicies, DeletePolicy | Policy evaluation/enforcement | Yes |
ContextService | Query, WriteEpisode, EstimateTokens, FormatContext | LLM context assembly + episode ingest | Yes |
CollectionService | CreateCollection, GetCollection, ListCollections, DeleteCollection, AddDocuments, SearchCollection, GetDocument, DeleteDocument | Document collections | Yes |
NamespaceService | CreateNamespace, GetNamespace, ListNamespaces, DeleteNamespace, SetQuota | Multi-tenant namespaces + quotas | Yes |
SemanticCacheService | Get, Put, Invalidate, GetStats | Semantic LLM cache | Yes |
TraceService | StartTrace, StartSpan, EndSpan, AddEvent, GetTrace, ListTraces | Trace/span observability | Yes |
CheckpointService | CreateCheckpoint, RestoreCheckpoint, ListCheckpoints, DeleteCheckpoint, ExportCheckpoint, ImportCheckpoint | State checkpoint/restore | Yes |
McpService | RegisterTool, ExecuteTool, ListTools, UnregisterTool, GetToolSchema | MCP tool routing | Yes |
KvService | Get, Put, Delete, Scan, BatchGet, BatchPut | Key-value ops | Yes |
SubscriptionService | Subscribe, WatchKey, ListSubscriptions, CancelSubscription | CDC change-data-capture streams | Yes |
A standard grpc.health.v1.Health service (via tonic_health) is also mounted on the same
port. It is not behind the auth interceptor so Kubernetes probes can reach it; the empty
service name "" is reported as Serving.
VectorIndexService is the one application service registered without the auth interceptor,
and it is configured with a 64 MB max message size (both decode and encode) so large vector
batches fit. Every other service uses tonic defaults (4 MB decode) and runs behind the auth
interceptor. See Max message size.
Connecting
The server speaks plain gRPC over HTTP/2. By default it binds 127.0.0.1:50051 with no TLS and
no auth (anonymous principal). The following examples target a default local server.
- grpcurl
- Python (grpcio)
- Go
# List services via server reflection is NOT enabled; pass the proto explicitly.
grpcurl -plaintext \
-import-path sochdb/sochdb-grpc/proto \
-proto sochdb.proto \
127.0.0.1:50051 \
sochdb.v1.VectorIndexService/HealthCheck
# Search a vector index
grpcurl -plaintext \
-import-path sochdb/sochdb-grpc/proto -proto sochdb.proto \
-d '{"index_name":"docs","query":[0.1,0.2,0.3],"k":5}' \
127.0.0.1:50051 \
sochdb.v1.VectorIndexService/Search
import grpc
import sochdb_pb2 as pb
import sochdb_pb2_grpc as rpc
channel = grpc.insecure_channel("127.0.0.1:50051")
vec = rpc.VectorIndexServiceStub(channel)
# Health check (no auth required on VectorIndexService)
health = vec.HealthCheck(pb.HealthCheckRequest())
print(health.version, health.status)
# k-NN search
resp = vec.Search(pb.SearchRequest(index_name="docs", query=[0.1, 0.2, 0.3], k=5))
for r in resp.results:
print(r.id, r.distance, r.metric)
package main
import (
"context"
"log"
"google.golang.org/grpc"
"google.golang.org/grpc/credentials/insecure"
pb "github.com/sochdb/sochdb/proto/v1"
)
func main() {
conn, err := grpc.NewClient("127.0.0.1:50051",
grpc.WithTransportCredentials(insecure.NewCredentials()))
if err != nil {
log.Fatal(err)
}
defer conn.Close()
vec := pb.NewVectorIndexServiceClient(conn)
resp, err := vec.Search(context.Background(), &pb.SearchRequest{
IndexName: "docs",
Query: []float32{0.1, 0.2, 0.3},
K: 5,
})
if err != nil {
log.Fatal(err)
}
for _, r := range resp.Results {
log.Printf("%d %f %s", r.Id, r.Distance, r.Metric)
}
}
Authentication
Auth is off by default. When the server is started with --auth, the auth interceptor runs
on every service except VectorIndexService and the health service.
Metadata headers
Clients present credentials via gRPC request metadata:
| Header | Format | Notes |
|---|---|---|
authorization | Bearer <token> | Preferred. <token> is a JWT when JWT is enabled, otherwise an API key. |
x-api-key | <key> | Fallback. Internally rewritten to Bearer <key>. |
If no credential header is present (and --auth is on), the server returns
UNAUTHENTICATED. The interceptor pipeline is: authenticate → rate-limit check → inject the
resolved Principal into request extensions for downstream handlers.
When --auth is not passed, every request resolves to an anonymous principal with
Read + Write + ManageCollections capabilities — convenient for local development, unsafe for
shared deployments.
- grpcurl
- Python (grpcio)
# Bearer token (JWT or API key)
grpcurl -plaintext \
-H "authorization: Bearer ${SOCHDB_TOKEN}" \
-import-path sochdb/sochdb-grpc/proto -proto sochdb.proto \
-d '{"namespace":"default","key":"dXNlcjox"}' \
127.0.0.1:50051 \
sochdb.v1.KvService/Get
# API key via x-api-key
grpcurl -plaintext \
-H "x-api-key: ${SOCHDB_API_KEY}" \
-import-path sochdb/sochdb-grpc/proto -proto sochdb.proto \
-d '{"namespace":"default","key":"dXNlcjox"}' \
127.0.0.1:50051 \
sochdb.v1.KvService/Get
import grpc
import sochdb_pb2 as pb
import sochdb_pb2_grpc as rpc
channel = grpc.insecure_channel("127.0.0.1:50051")
kv = rpc.KvServiceStub(channel)
# Pass auth as call metadata. KvService runs behind the interceptor.
md = [("authorization", f"Bearer {token}")]
resp = kv.Get(pb.KvGetRequest(namespace="default", key=b"user:1"), metadata=md)
print(resp.found, resp.value)
Tokens, keys, and roles
- JWT is validation-only (HS256). The server validates
exp, optional issuer, and optional audience using a shared secret (SOCHDB_JWT_SECRETor thejwt-secretmounted secret). There is no token-issuance API — tokens are minted by an external IdP or by the calling service. JWT claims carrysub,tenant_id, arolestring, and an explicitcapabilitieslist. - API keys are hashed. Stored as
SHA-256(key), orHMAC-SHA256(pepper, key)whenSOCHDB_API_KEY_PEPPERis set. Keys are never stored in plaintext. (Argon2 is used only for user passwords, not API keys.) - RBAC roles are
Owner,Editor,Viewer(plus aCustomrole with an explicit capability set). Capabilities:Owner→ all:Admin, Read, Write, ManageCollections, ManageIndexes, ViewMetrics, ManageBackups, ManageUsersEditor→Read, Write, ManageCollections, ManageIndexesViewer→Read, ViewMetrics
- Roles bind globally or per-namespace/per-collection (
RoleScope); the server resolves effective capabilities by unioning all bindings that apply to the target namespace.
When --auth is passed, the server enables both JWT and API-key validation, but the
authenticate() path routes every Bearer token to JWT verification and never falls through to
the API-key hash lookup. A bare --api-key is registered, but it is only reachable via the
x-api-key path when JWT is disabled. If you intend to use raw API keys, do not enable JWT.
TLS / mTLS
TLS is enabled when both --tls-cert and --tls-key (or the SOCHDB_TLS_CERT /
SOCHDB_TLS_KEY env vars) are set. Adding --tls-ca enables mutual TLS (client-certificate
verification). Certificates support hot-reload on file change.
sochdb-grpc-server \
--host 0.0.0.0 --port 50051 \
--auth \
--tls-cert /etc/sochdb/tls/cert.pem \
--tls-key /etc/sochdb/tls/key.pem \
--tls-ca /etc/sochdb/tls/ca.pem
Max message size
Only VectorIndexService raises the gRPC message limit:
- Max decoding message size: 64 MB
- Max encoding message size: 64 MB
This lets large InsertBatch / SearchBatch payloads (flat row-major float vectors) flow
without manual chunking. All other services use tonic defaults (4 MB decode). For very large
inserts that still exceed 64 MB, use the client-streaming InsertStream RPC.
Rate limiting
When auth is enabled, the interceptor enforces a per-tenant token bucket. Defaults: 1000
requests/second with a burst of 100. Exceeding the limit returns RESOURCE_EXHAUSTED.
Audit logging is on by default.
VectorIndexService
The most-used service. Vectors are passed as flat, row-major repeated float arrays.
Key RPCs
CreateIndex(CreateIndexRequest) → CreateIndexResponseInsertBatch(InsertBatchRequest) → InsertBatchResponseInsertStream(stream InsertStreamRequest) → InsertStreamResponse— client-streaming, for batches over the 64 MB unary limit.Search(SearchRequest) → SearchResponseSearchBatch(SearchBatchRequest) → SearchBatchResponseGetStats(GetStatsRequest) → GetStatsResponseHealthCheck(HealthCheckRequest) → HealthCheckResponse
HNSW defaults
The proto field comments on HnswConfig are stale. When a CreateIndex request leaves an
HnswConfig field at 0, the server fills it from the engine's HnswConfig::default():
| Field | Server default |
|---|---|
max_connections (M) | 32 |
max_connections_layer0 (M0) | 64 |
ef_construction | 256 |
ef_search | 500 |
| metric | Cosine |
SearchRequest.ef overrides ef_search per query; leaving it 0 uses the index default.
Reading SearchResult.distance
SearchResult carries both distance and the index's metric string
("cosine" | "euclidean" | "dot_product" | "unspecified") so callers can interpret the value
without external knowledge:
- For
cosine/euclidean(L2): lowerdistancemeans more similar. - For
dot_product: the raw inner product is negated before return, so lower (more negative) also means more similar; recover the raw dot product as-distance.
Proto excerpt
syntax = "proto3";
package sochdb.v1;
service VectorIndexService {
rpc CreateIndex(CreateIndexRequest) returns (CreateIndexResponse);
rpc DropIndex(DropIndexRequest) returns (DropIndexResponse);
rpc InsertBatch(InsertBatchRequest) returns (InsertBatchResponse);
rpc InsertStream(stream InsertStreamRequest) returns (InsertStreamResponse);
rpc Search(SearchRequest) returns (SearchResponse);
rpc SearchBatch(SearchBatchRequest) returns (SearchBatchResponse);
rpc GetStats(GetStatsRequest) returns (GetStatsResponse);
rpc HealthCheck(HealthCheckRequest) returns (HealthCheckResponse);
}
message CreateIndexRequest {
string name = 1;
uint32 dimension = 2; // e.g. 768 (BERT), 1536 (OpenAI)
HnswConfig config = 3;
DistanceMetric metric = 4;
}
message SearchRequest {
string index_name = 1;
repeated float query = 2;
uint32 k = 3;
uint32 ef = 4; // 0 = use index ef_search default (500)
}
message SearchResponse {
repeated SearchResult results = 1; // ordered by distance ascending
uint64 duration_us = 2;
string error = 3;
}
message SearchResult {
uint64 id = 1;
float distance = 2;
string metric = 3; // "cosine" | "euclidean" | "dot_product" | "unspecified"
}
message HnswConfig {
uint32 max_connections = 1; // server default 32 when 0
uint32 max_connections_layer0 = 2; // server default 64 when 0
uint32 ef_construction = 3; // server default 256 when 0
uint32 ef_search = 4; // server default 500 when 0
}
enum DistanceMetric {
DISTANCE_METRIC_UNSPECIFIED = 0;
DISTANCE_METRIC_L2 = 1;
DISTANCE_METRIC_COSINE = 2;
DISTANCE_METRIC_DOT_PRODUCT = 3;
}
KvService
Basic namespaced key-value operations. Keys and values are bytes.
service KvService {
rpc Get(KvGetRequest) returns (KvGetResponse);
rpc Put(KvPutRequest) returns (KvPutResponse);
rpc Delete(KvDeleteRequest) returns (KvDeleteResponse);
rpc Scan(KvScanRequest) returns (stream KvScanResponse); // server-streaming
rpc BatchGet(KvBatchGetRequest) returns (KvBatchGetResponse);
rpc BatchPut(KvBatchPutRequest) returns (KvBatchPutResponse);
}
message KvGetRequest {
string namespace = 1;
bytes key = 2;
}
message KvGetResponse {
bytes value = 1;
bool found = 2;
string error = 3;
}
message KvPutRequest {
string namespace = 1;
bytes key = 2;
bytes value = 3;
uint64 ttl_seconds = 4; // optional TTL
}
SubscriptionService (CDC)
SubscriptionService streams change-data-capture events derived from the WAL commit path.
Events are delivered in monotonically increasing sequence order. This service is defined only
in sochdb-grpc/proto/sochdb.proto.
RPCs
Subscribe(SubscribeRequest) → stream SubscribeEvent— server-streaming change feed.WatchKey(WatchKeyRequest) → stream WatchKeyEvent— stream changes for one key.ListSubscriptions(ListSubscriptionsRequest) → ListSubscriptionsResponseCancelSubscription(CancelSubscriptionRequest) → CancelSubscriptionResponse
Resuming and filtering
start_sequence = 0streams from the latest event;start_sequence > 0resumes from that sequence number. If the requested position has already been evicted from the in-memory ring buffer (default capacity 65,536), the stream errors with an overrun.tablesfilters to specific table names. Enforced.operationsfilters byOperationType. Enforced.batch_sizecaps events per stream message (0= unbounded; server default 64).- You can cancel either via
CancelSubscriptionor by simply dropping the stream.
where_predicate is accepted but not yet enforcedSubscribeRequest.where_predicate (a SQL WHERE clause for row-level delta filtering) is part
of the proto and is accepted by the server, but the streaming handler does not currently read
or apply it. Only table-name and operation-type filtering are enforced today. Do not rely on
where_predicate for security or correctness.
SubscribeEvent.before_value is currently always empty — the CDC engine emits after-images
only. The field exists for forward compatibility.
Proto excerpt
service SubscriptionService {
rpc Subscribe(SubscribeRequest) returns (stream SubscribeEvent);
rpc WatchKey(WatchKeyRequest) returns (stream WatchKeyEvent);
rpc ListSubscriptions(ListSubscriptionsRequest) returns (ListSubscriptionsResponse);
rpc CancelSubscription(CancelSubscriptionRequest) returns (CancelSubscriptionResponse);
}
message SubscribeRequest {
string namespace = 1;
repeated string tables = 2; // enforced filter
repeated OperationType operations = 3; // enforced filter
uint64 start_sequence = 4; // 0 = latest, >0 = resume
string where_predicate = 5; // accepted but NOT yet enforced
uint32 batch_size = 6; // 0 = unbounded; default 64
}
enum OperationType {
OPERATION_UNSPECIFIED = 0;
OPERATION_INSERT = 1;
OPERATION_UPDATE = 2;
OPERATION_DELETE = 3;
OPERATION_SCHEMA_CHANGE = 4;
}
message SubscribeEvent {
uint64 sequence = 1;
uint64 timestamp_us = 2;
uint64 txn_id = 3;
string table = 4;
bytes key = 5;
OperationType operation = 6;
bytes after_value = 7;
bytes before_value = 8; // currently always empty (after-image only)
string ddl = 9; // populated for OPERATION_SCHEMA_CHANGE
}
message WatchKeyRequest {
string namespace = 1;
string table = 2;
bytes key = 3;
}
message ListSubscriptionsResponse {
repeated SubscriptionInfo subscriptions = 1;
}
message SubscriptionInfo {
string subscription_id = 1; // "sub-<n>"
string namespace = 2;
repeated string tables = 3;
uint64 start_sequence = 4;
uint64 current_sequence = 5;
uint64 created_at_us = 6;
}
message CancelSubscriptionRequest {
string subscription_id = 1;
}
Example: subscribe to inserts and updates
- Python (grpcio)
- Go
import grpc
import sochdb_pb2 as pb
import sochdb_pb2_grpc as rpc
channel = grpc.insecure_channel("127.0.0.1:50051")
sub = rpc.SubscriptionServiceStub(channel)
req = pb.SubscribeRequest(
namespace="default",
tables=["users"],
operations=[pb.OPERATION_INSERT, pb.OPERATION_UPDATE],
start_sequence=0, # from latest
batch_size=64,
)
# Server-streaming: iterate as events arrive.
for event in sub.Subscribe(req):
print(event.sequence, event.table, event.operation, event.after_value)
sub := pb.NewSubscriptionServiceClient(conn)
stream, err := sub.Subscribe(context.Background(), &pb.SubscribeRequest{
Namespace: "default",
Tables: []string{"users"},
Operations: []pb.OperationType{pb.OperationType_OPERATION_INSERT, pb.OperationType_OPERATION_UPDATE},
})
if err != nil {
log.Fatal(err)
}
for {
ev, err := stream.Recv()
if err != nil {
break
}
log.Printf("seq=%d table=%s op=%v", ev.Sequence, ev.Table, ev.Operation)
}
Other services (proto excerpts)
GraphService
Graph overlay with BFS/DFS traversal, shortest path, and a temporal-graph mode (point-in-time / range / current). All requests are namespaced.
service GraphService {
rpc AddNode(AddNodeRequest) returns (AddNodeResponse);
rpc GetNode(GetNodeRequest) returns (GetNodeResponse);
rpc DeleteNode(DeleteNodeRequest) returns (DeleteNodeResponse);
rpc AddEdge(AddEdgeRequest) returns (AddEdgeResponse);
rpc GetEdges(GetEdgesRequest) returns (GetEdgesResponse);
rpc DeleteEdge(DeleteEdgeRequest) returns (DeleteEdgeResponse);
rpc Traverse(TraverseRequest) returns (TraverseResponse);
rpc ShortestPath(ShortestPathRequest) returns (ShortestPathResponse);
rpc GetNeighbors(GetNeighborsRequest) returns (GetNeighborsResponse);
rpc AddTemporalEdge(AddTemporalEdgeRequest) returns (AddTemporalEdgeResponse);
rpc QueryTemporalGraph(QueryTemporalGraphRequest) returns (QueryTemporalGraphResponse);
}
enum EdgeDirection {
EDGE_DIRECTION_OUTGOING = 0;
EDGE_DIRECTION_INCOMING = 1;
EDGE_DIRECTION_BOTH = 2;
}
enum TemporalQueryMode {
TEMPORAL_QUERY_MODE_POINT_IN_TIME = 0;
TEMPORAL_QUERY_MODE_RANGE = 1;
TEMPORAL_QUERY_MODE_CURRENT = 2;
}
ContextService
LLM context assembly under a token budget, plus episode ingest for agent memory.
service ContextService {
rpc Query(ContextQueryRequest) returns (ContextQueryResponse);
rpc WriteEpisode(WriteEpisodeRequest) returns (WriteEpisodeResponse);
rpc EstimateTokens(EstimateTokensRequest) returns (EstimateTokensResponse);
rpc FormatContext(FormatContextRequest) returns (FormatContextResponse);
}
enum OutputFormat {
OUTPUT_FORMAT_TOON = 0;
OUTPUT_FORMAT_JSON = 1;
OUTPUT_FORMAT_MARKDOWN = 2;
OUTPUT_FORMAT_TEXT = 3;
}
McpService
Model Context Protocol tool routing: register, execute, list, unregister, and inspect tools. Tool input/output schemas are passed as JSON Schema strings.
service McpService {
rpc RegisterTool(RegisterToolRequest) returns (RegisterToolResponse);
rpc ExecuteTool(ExecuteToolRequest) returns (ExecuteToolResponse);
rpc ListTools(ListToolsRequest) returns (ListToolsResponse);
rpc UnregisterTool(UnregisterToolRequest) returns (UnregisterToolResponse);
rpc GetToolSchema(GetToolSchemaRequest) returns (GetToolSchemaResponse);
}
The native MCP tool catalog uses underscore-delimited tool names, for example
sochdb_query, sochdb_get, sochdb_put, sochdb_context_query,
memory_search_episodes, and sochdb_grep. Older dot-delimited names in stale mcp.json /
README listings are out of date.
The remaining services — PolicyService, CollectionService, NamespaceService,
SemanticCacheService, TraceService, and CheckpointService — follow the same request/response
pattern: a typed request message, a response carrying the result plus a string error field that
is empty on success. See the full proto at sochdb-grpc/proto/sochdb.proto for every message.
Non-gRPC surfaces on the same binary
The server binary also exposes several non-gRPC ports. These are documented in detail on the server pages; summarized here for completeness:
| Surface | Flag | Default port | Notes |
|---|---|---|---|
| Prometheus metrics | --metrics-port | 9090 | GET /metrics, GET /health; 0 disables. |
| WebSocket gateway | --ws-port | 8080 | JSON protocol at ws://host:8080/; 0 disables. |
| PostgreSQL wire | --pg-port | 5433 | Simple-query only; 0 disables. |
The pg-wire endpoint is simple-query only, cleartext (no SSL/TLS), and uses trust auth (no
password). It is intended for loopback use. Real SQL (SELECT/INSERT/UPDATE/DELETE/DDL, including
joins) requires --pg-data-dir; without it, queries are echoed by a placeholder executor. Do not
expose this port on a non-loopback address.
SochDB ships an EncryptionEngine (AES-256-GCM-SIV, 32-byte key) for encrypting data blocks,
WAL entries, and checkpoint files. It is a tested library capability but is not yet wired to a
sochdb-grpc-server CLI flag — the server binary does not construct or install it. Treat
at-rest encryption as available-as-API / planned wiring, not as a runtime toggle.
Server flags reference
sochdb-grpc-server \
--host 127.0.0.1 \
--port 50051 \
--metrics-port 9090 \
--ws-port 8080 \
--pg-port 5433 \
--auth \
--api-key "$SOCHDB_API_KEY" \
--tls-cert /path/cert.pem --tls-key /path/key.pem --tls-ca /path/ca.pem \
--pg-data-dir /var/lib/sochdb/sql \
--secrets-path /var/run/secrets/sochdb
| Flag | Default | Env var | Purpose |
|---|---|---|---|
--host | 127.0.0.1 | — | Bind address |
-p, --port | 50051 | — | gRPC port |
-d, --debug | off | — | Debug logging |
--metrics-port | 9090 | — | Prometheus port (0 disables) |
--ws-port | 8080 | — | WebSocket port (0 disables) |
--pg-port | 5433 | — | PostgreSQL wire port (0 disables) |
--auth | off | — | Enable gRPC auth |
--api-key | none | SOCHDB_API_KEY | Register an API key (requires --auth) |
--tls-cert | none | SOCHDB_TLS_CERT | TLS cert PEM (enables TLS) |
--tls-key | none | SOCHDB_TLS_KEY | TLS private key PEM |
--tls-ca | none | SOCHDB_TLS_CA | CA cert for mTLS |
--secrets-path | none | SOCHDB_SECRETS_PATH | K8s Secrets mount path |
--pg-data-dir | none | SOCHDB_PG_DATA_DIR | Persistent dir for real SQL via pg-wire |
Additional env vars read directly (not flags): SOCHDB_API_KEY_PEPPER, SOCHDB_JWT_SECRET,
SOCHDB_ENCRYPTION_KEY, SOCHDB_API_KEYS (comma-separated).