Skip to main content

gRPC / Protobuf API Reference

The SochDB server (sochdb-grpc-server, engine 2.0.3) exposes a network-first, "thick server / thin client" gRPC surface. All business logic lives in the Rust server; the language SDKs are thin RPC wrappers. This page is a machine-readable catalog of the services, RPCs, and message shapes so that agents and code generators have an exact surface to target.

  • Default gRPC port: 50051 (--port / -p).
  • Default bind host: 127.0.0.1 (--host). Loopback-only unless you change it.
  • Proto package: sochdb.v1 (syntax = "proto3").
  • Go package option: github.com/sochdb/sochdb/proto/v1;sochdbv1.
  • Java package option: com.sochdb.v1.

The canonical proto for the running server is sochdb-grpc/proto/sochdb.proto. A second, slightly smaller copy at proto/sochdb.proto defines the same services but omits SubscriptionService. Generate clients from sochdb-grpc/proto/sochdb.proto to get the full surface, including CDC subscriptions.

Why this page exists

This is a deliberately literal reference: exact RPC names, message field numbers, enum values, and metadata headers. Use it when generating a client, wiring an agent, or verifying a request shape against the server.

Service catalog

The server registers 12 application gRPC services plus a standard gRPC health service. All RPC and message names below are taken directly from sochdb-grpc/proto/sochdb.proto.

ServiceRPCsPurposeAuth interceptor
VectorIndexServiceCreateIndex, DropIndex, InsertBatch, InsertStream, Search, SearchBatch, GetStats, HealthCheckHNSW vector index opsNo (see note)
GraphServiceAddNode, GetNode, DeleteNode, AddEdge, GetEdges, DeleteEdge, Traverse, ShortestPath, GetNeighbors, AddTemporalEdge, QueryTemporalGraphGraph overlay + temporal graphYes
PolicyServiceRegisterPolicy, Evaluate, ListPolicies, DeletePolicyPolicy evaluation/enforcementYes
ContextServiceQuery, WriteEpisode, EstimateTokens, FormatContextLLM context assembly + episode ingestYes
CollectionServiceCreateCollection, GetCollection, ListCollections, DeleteCollection, AddDocuments, SearchCollection, GetDocument, DeleteDocumentDocument collectionsYes
NamespaceServiceCreateNamespace, GetNamespace, ListNamespaces, DeleteNamespace, SetQuotaMulti-tenant namespaces + quotasYes
SemanticCacheServiceGet, Put, Invalidate, GetStatsSemantic LLM cacheYes
TraceServiceStartTrace, StartSpan, EndSpan, AddEvent, GetTrace, ListTracesTrace/span observabilityYes
CheckpointServiceCreateCheckpoint, RestoreCheckpoint, ListCheckpoints, DeleteCheckpoint, ExportCheckpoint, ImportCheckpointState checkpoint/restoreYes
McpServiceRegisterTool, ExecuteTool, ListTools, UnregisterTool, GetToolSchemaMCP tool routingYes
KvServiceGet, Put, Delete, Scan, BatchGet, BatchPutKey-value opsYes
SubscriptionServiceSubscribe, WatchKey, ListSubscriptions, CancelSubscriptionCDC change-data-capture streamsYes

A standard grpc.health.v1.Health service (via tonic_health) is also mounted on the same port. It is not behind the auth interceptor so Kubernetes probes can reach it; the empty service name "" is reported as Serving.

VectorIndexService is special

VectorIndexService is the one application service registered without the auth interceptor, and it is configured with a 64 MB max message size (both decode and encode) so large vector batches fit. Every other service uses tonic defaults (4 MB decode) and runs behind the auth interceptor. See Max message size.

Connecting

The server speaks plain gRPC over HTTP/2. By default it binds 127.0.0.1:50051 with no TLS and no auth (anonymous principal). The following examples target a default local server.

# List services via server reflection is NOT enabled; pass the proto explicitly.
grpcurl -plaintext \
-import-path sochdb/sochdb-grpc/proto \
-proto sochdb.proto \
127.0.0.1:50051 \
sochdb.v1.VectorIndexService/HealthCheck

# Search a vector index
grpcurl -plaintext \
-import-path sochdb/sochdb-grpc/proto -proto sochdb.proto \
-d '{"index_name":"docs","query":[0.1,0.2,0.3],"k":5}' \
127.0.0.1:50051 \
sochdb.v1.VectorIndexService/Search
Use the SDKs for normal apps

For day-to-day application code, prefer the language SDKs (Python, Node.js, Go) over hand-written gRPC stubs. This page is for code generation, agents, and low-level debugging.

Authentication

Auth is off by default. When the server is started with --auth, the auth interceptor runs on every service except VectorIndexService and the health service.

Metadata headers

Clients present credentials via gRPC request metadata:

HeaderFormatNotes
authorizationBearer <token>Preferred. <token> is a JWT when JWT is enabled, otherwise an API key.
x-api-key<key>Fallback. Internally rewritten to Bearer <key>.

If no credential header is present (and --auth is on), the server returns UNAUTHENTICATED. The interceptor pipeline is: authenticate → rate-limit check → inject the resolved Principal into request extensions for downstream handlers.

When --auth is not passed, every request resolves to an anonymous principal with Read + Write + ManageCollections capabilities — convenient for local development, unsafe for shared deployments.

# Bearer token (JWT or API key)
grpcurl -plaintext \
-H "authorization: Bearer ${SOCHDB_TOKEN}" \
-import-path sochdb/sochdb-grpc/proto -proto sochdb.proto \
-d '{"namespace":"default","key":"dXNlcjox"}' \
127.0.0.1:50051 \
sochdb.v1.KvService/Get

# API key via x-api-key
grpcurl -plaintext \
-H "x-api-key: ${SOCHDB_API_KEY}" \
-import-path sochdb/sochdb-grpc/proto -proto sochdb.proto \
-d '{"namespace":"default","key":"dXNlcjox"}' \
127.0.0.1:50051 \
sochdb.v1.KvService/Get

Tokens, keys, and roles

  • JWT is validation-only (HS256). The server validates exp, optional issuer, and optional audience using a shared secret (SOCHDB_JWT_SECRET or the jwt-secret mounted secret). There is no token-issuance API — tokens are minted by an external IdP or by the calling service. JWT claims carry sub, tenant_id, a role string, and an explicit capabilities list.
  • API keys are hashed. Stored as SHA-256(key), or HMAC-SHA256(pepper, key) when SOCHDB_API_KEY_PEPPER is set. Keys are never stored in plaintext. (Argon2 is used only for user passwords, not API keys.)
  • RBAC roles are Owner, Editor, Viewer (plus a Custom role with an explicit capability set). Capabilities:
    • Owner → all: Admin, Read, Write, ManageCollections, ManageIndexes, ViewMetrics, ManageBackups, ManageUsers
    • EditorRead, Write, ManageCollections, ManageIndexes
    • ViewerRead, ViewMetrics
  • Roles bind globally or per-namespace/per-collection (RoleScope); the server resolves effective capabilities by unioning all bindings that apply to the target namespace.
JWT and API keys are mutually exclusive at runtime

When --auth is passed, the server enables both JWT and API-key validation, but the authenticate() path routes every Bearer token to JWT verification and never falls through to the API-key hash lookup. A bare --api-key is registered, but it is only reachable via the x-api-key path when JWT is disabled. If you intend to use raw API keys, do not enable JWT.

TLS / mTLS

TLS is enabled when both --tls-cert and --tls-key (or the SOCHDB_TLS_CERT / SOCHDB_TLS_KEY env vars) are set. Adding --tls-ca enables mutual TLS (client-certificate verification). Certificates support hot-reload on file change.

sochdb-grpc-server \
--host 0.0.0.0 --port 50051 \
--auth \
--tls-cert /etc/sochdb/tls/cert.pem \
--tls-key /etc/sochdb/tls/key.pem \
--tls-ca /etc/sochdb/tls/ca.pem

Max message size

Only VectorIndexService raises the gRPC message limit:

  • Max decoding message size: 64 MB
  • Max encoding message size: 64 MB

This lets large InsertBatch / SearchBatch payloads (flat row-major float vectors) flow without manual chunking. All other services use tonic defaults (4 MB decode). For very large inserts that still exceed 64 MB, use the client-streaming InsertStream RPC.

Rate limiting

When auth is enabled, the interceptor enforces a per-tenant token bucket. Defaults: 1000 requests/second with a burst of 100. Exceeding the limit returns RESOURCE_EXHAUSTED. Audit logging is on by default.

VectorIndexService

The most-used service. Vectors are passed as flat, row-major repeated float arrays.

Key RPCs

  • CreateIndex(CreateIndexRequest) → CreateIndexResponse
  • InsertBatch(InsertBatchRequest) → InsertBatchResponse
  • InsertStream(stream InsertStreamRequest) → InsertStreamResponse — client-streaming, for batches over the 64 MB unary limit.
  • Search(SearchRequest) → SearchResponse
  • SearchBatch(SearchBatchRequest) → SearchBatchResponse
  • GetStats(GetStatsRequest) → GetStatsResponse
  • HealthCheck(HealthCheckRequest) → HealthCheckResponse

HNSW defaults

The proto field comments on HnswConfig are stale. When a CreateIndex request leaves an HnswConfig field at 0, the server fills it from the engine's HnswConfig::default():

FieldServer default
max_connections (M)32
max_connections_layer0 (M0)64
ef_construction256
ef_search500
metricCosine

SearchRequest.ef overrides ef_search per query; leaving it 0 uses the index default.

Reading SearchResult.distance

SearchResult carries both distance and the index's metric string ("cosine" | "euclidean" | "dot_product" | "unspecified") so callers can interpret the value without external knowledge:

  • For cosine / euclidean (L2): lower distance means more similar.
  • For dot_product: the raw inner product is negated before return, so lower (more negative) also means more similar; recover the raw dot product as -distance.

Proto excerpt

syntax = "proto3";
package sochdb.v1;

service VectorIndexService {
rpc CreateIndex(CreateIndexRequest) returns (CreateIndexResponse);
rpc DropIndex(DropIndexRequest) returns (DropIndexResponse);
rpc InsertBatch(InsertBatchRequest) returns (InsertBatchResponse);
rpc InsertStream(stream InsertStreamRequest) returns (InsertStreamResponse);
rpc Search(SearchRequest) returns (SearchResponse);
rpc SearchBatch(SearchBatchRequest) returns (SearchBatchResponse);
rpc GetStats(GetStatsRequest) returns (GetStatsResponse);
rpc HealthCheck(HealthCheckRequest) returns (HealthCheckResponse);
}

message CreateIndexRequest {
string name = 1;
uint32 dimension = 2; // e.g. 768 (BERT), 1536 (OpenAI)
HnswConfig config = 3;
DistanceMetric metric = 4;
}

message SearchRequest {
string index_name = 1;
repeated float query = 2;
uint32 k = 3;
uint32 ef = 4; // 0 = use index ef_search default (500)
}

message SearchResponse {
repeated SearchResult results = 1; // ordered by distance ascending
uint64 duration_us = 2;
string error = 3;
}

message SearchResult {
uint64 id = 1;
float distance = 2;
string metric = 3; // "cosine" | "euclidean" | "dot_product" | "unspecified"
}

message HnswConfig {
uint32 max_connections = 1; // server default 32 when 0
uint32 max_connections_layer0 = 2; // server default 64 when 0
uint32 ef_construction = 3; // server default 256 when 0
uint32 ef_search = 4; // server default 500 when 0
}

enum DistanceMetric {
DISTANCE_METRIC_UNSPECIFIED = 0;
DISTANCE_METRIC_L2 = 1;
DISTANCE_METRIC_COSINE = 2;
DISTANCE_METRIC_DOT_PRODUCT = 3;
}

KvService

Basic namespaced key-value operations. Keys and values are bytes.

service KvService {
rpc Get(KvGetRequest) returns (KvGetResponse);
rpc Put(KvPutRequest) returns (KvPutResponse);
rpc Delete(KvDeleteRequest) returns (KvDeleteResponse);
rpc Scan(KvScanRequest) returns (stream KvScanResponse); // server-streaming
rpc BatchGet(KvBatchGetRequest) returns (KvBatchGetResponse);
rpc BatchPut(KvBatchPutRequest) returns (KvBatchPutResponse);
}

message KvGetRequest {
string namespace = 1;
bytes key = 2;
}

message KvGetResponse {
bytes value = 1;
bool found = 2;
string error = 3;
}

message KvPutRequest {
string namespace = 1;
bytes key = 2;
bytes value = 3;
uint64 ttl_seconds = 4; // optional TTL
}

SubscriptionService (CDC)

SubscriptionService streams change-data-capture events derived from the WAL commit path. Events are delivered in monotonically increasing sequence order. This service is defined only in sochdb-grpc/proto/sochdb.proto.

RPCs

  • Subscribe(SubscribeRequest) → stream SubscribeEvent — server-streaming change feed.
  • WatchKey(WatchKeyRequest) → stream WatchKeyEvent — stream changes for one key.
  • ListSubscriptions(ListSubscriptionsRequest) → ListSubscriptionsResponse
  • CancelSubscription(CancelSubscriptionRequest) → CancelSubscriptionResponse

Resuming and filtering

  • start_sequence = 0 streams from the latest event; start_sequence > 0 resumes from that sequence number. If the requested position has already been evicted from the in-memory ring buffer (default capacity 65,536), the stream errors with an overrun.
  • tables filters to specific table names. Enforced.
  • operations filters by OperationType. Enforced.
  • batch_size caps events per stream message (0 = unbounded; server default 64).
  • You can cancel either via CancelSubscription or by simply dropping the stream.
where_predicate is accepted but not yet enforced

SubscribeRequest.where_predicate (a SQL WHERE clause for row-level delta filtering) is part of the proto and is accepted by the server, but the streaming handler does not currently read or apply it. Only table-name and operation-type filtering are enforced today. Do not rely on where_predicate for security or correctness.

Change events are after-image only

SubscribeEvent.before_value is currently always empty — the CDC engine emits after-images only. The field exists for forward compatibility.

Proto excerpt

service SubscriptionService {
rpc Subscribe(SubscribeRequest) returns (stream SubscribeEvent);
rpc WatchKey(WatchKeyRequest) returns (stream WatchKeyEvent);
rpc ListSubscriptions(ListSubscriptionsRequest) returns (ListSubscriptionsResponse);
rpc CancelSubscription(CancelSubscriptionRequest) returns (CancelSubscriptionResponse);
}

message SubscribeRequest {
string namespace = 1;
repeated string tables = 2; // enforced filter
repeated OperationType operations = 3; // enforced filter
uint64 start_sequence = 4; // 0 = latest, >0 = resume
string where_predicate = 5; // accepted but NOT yet enforced
uint32 batch_size = 6; // 0 = unbounded; default 64
}

enum OperationType {
OPERATION_UNSPECIFIED = 0;
OPERATION_INSERT = 1;
OPERATION_UPDATE = 2;
OPERATION_DELETE = 3;
OPERATION_SCHEMA_CHANGE = 4;
}

message SubscribeEvent {
uint64 sequence = 1;
uint64 timestamp_us = 2;
uint64 txn_id = 3;
string table = 4;
bytes key = 5;
OperationType operation = 6;
bytes after_value = 7;
bytes before_value = 8; // currently always empty (after-image only)
string ddl = 9; // populated for OPERATION_SCHEMA_CHANGE
}

message WatchKeyRequest {
string namespace = 1;
string table = 2;
bytes key = 3;
}

message ListSubscriptionsResponse {
repeated SubscriptionInfo subscriptions = 1;
}

message SubscriptionInfo {
string subscription_id = 1; // "sub-<n>"
string namespace = 2;
repeated string tables = 3;
uint64 start_sequence = 4;
uint64 current_sequence = 5;
uint64 created_at_us = 6;
}

message CancelSubscriptionRequest {
string subscription_id = 1;
}

Example: subscribe to inserts and updates

import grpc
import sochdb_pb2 as pb
import sochdb_pb2_grpc as rpc

channel = grpc.insecure_channel("127.0.0.1:50051")
sub = rpc.SubscriptionServiceStub(channel)

req = pb.SubscribeRequest(
namespace="default",
tables=["users"],
operations=[pb.OPERATION_INSERT, pb.OPERATION_UPDATE],
start_sequence=0, # from latest
batch_size=64,
)

# Server-streaming: iterate as events arrive.
for event in sub.Subscribe(req):
print(event.sequence, event.table, event.operation, event.after_value)

Other services (proto excerpts)

GraphService

Graph overlay with BFS/DFS traversal, shortest path, and a temporal-graph mode (point-in-time / range / current). All requests are namespaced.

service GraphService {
rpc AddNode(AddNodeRequest) returns (AddNodeResponse);
rpc GetNode(GetNodeRequest) returns (GetNodeResponse);
rpc DeleteNode(DeleteNodeRequest) returns (DeleteNodeResponse);
rpc AddEdge(AddEdgeRequest) returns (AddEdgeResponse);
rpc GetEdges(GetEdgesRequest) returns (GetEdgesResponse);
rpc DeleteEdge(DeleteEdgeRequest) returns (DeleteEdgeResponse);
rpc Traverse(TraverseRequest) returns (TraverseResponse);
rpc ShortestPath(ShortestPathRequest) returns (ShortestPathResponse);
rpc GetNeighbors(GetNeighborsRequest) returns (GetNeighborsResponse);
rpc AddTemporalEdge(AddTemporalEdgeRequest) returns (AddTemporalEdgeResponse);
rpc QueryTemporalGraph(QueryTemporalGraphRequest) returns (QueryTemporalGraphResponse);
}

enum EdgeDirection {
EDGE_DIRECTION_OUTGOING = 0;
EDGE_DIRECTION_INCOMING = 1;
EDGE_DIRECTION_BOTH = 2;
}

enum TemporalQueryMode {
TEMPORAL_QUERY_MODE_POINT_IN_TIME = 0;
TEMPORAL_QUERY_MODE_RANGE = 1;
TEMPORAL_QUERY_MODE_CURRENT = 2;
}

ContextService

LLM context assembly under a token budget, plus episode ingest for agent memory.

service ContextService {
rpc Query(ContextQueryRequest) returns (ContextQueryResponse);
rpc WriteEpisode(WriteEpisodeRequest) returns (WriteEpisodeResponse);
rpc EstimateTokens(EstimateTokensRequest) returns (EstimateTokensResponse);
rpc FormatContext(FormatContextRequest) returns (FormatContextResponse);
}

enum OutputFormat {
OUTPUT_FORMAT_TOON = 0;
OUTPUT_FORMAT_JSON = 1;
OUTPUT_FORMAT_MARKDOWN = 2;
OUTPUT_FORMAT_TEXT = 3;
}

McpService

Model Context Protocol tool routing: register, execute, list, unregister, and inspect tools. Tool input/output schemas are passed as JSON Schema strings.

service McpService {
rpc RegisterTool(RegisterToolRequest) returns (RegisterToolResponse);
rpc ExecuteTool(ExecuteToolRequest) returns (ExecuteToolResponse);
rpc ListTools(ListToolsRequest) returns (ListToolsResponse);
rpc UnregisterTool(UnregisterToolRequest) returns (UnregisterToolResponse);
rpc GetToolSchema(GetToolSchemaRequest) returns (GetToolSchemaResponse);
}
MCP tool names use underscores

The native MCP tool catalog uses underscore-delimited tool names, for example sochdb_query, sochdb_get, sochdb_put, sochdb_context_query, memory_search_episodes, and sochdb_grep. Older dot-delimited names in stale mcp.json / README listings are out of date.

The remaining services — PolicyService, CollectionService, NamespaceService, SemanticCacheService, TraceService, and CheckpointService — follow the same request/response pattern: a typed request message, a response carrying the result plus a string error field that is empty on success. See the full proto at sochdb-grpc/proto/sochdb.proto for every message.

Non-gRPC surfaces on the same binary

The server binary also exposes several non-gRPC ports. These are documented in detail on the server pages; summarized here for completeness:

SurfaceFlagDefault portNotes
Prometheus metrics--metrics-port9090GET /metrics, GET /health; 0 disables.
WebSocket gateway--ws-port8080JSON protocol at ws://host:8080/; 0 disables.
PostgreSQL wire--pg-port5433Simple-query only; 0 disables.
PostgreSQL wire protocol has no auth

The pg-wire endpoint is simple-query only, cleartext (no SSL/TLS), and uses trust auth (no password). It is intended for loopback use. Real SQL (SELECT/INSERT/UPDATE/DELETE/DDL, including joins) requires --pg-data-dir; without it, queries are echoed by a placeholder executor. Do not expose this port on a non-loopback address.

At-rest encryption is a library API, not a server flag

SochDB ships an EncryptionEngine (AES-256-GCM-SIV, 32-byte key) for encrypting data blocks, WAL entries, and checkpoint files. It is a tested library capability but is not yet wired to a sochdb-grpc-server CLI flag — the server binary does not construct or install it. Treat at-rest encryption as available-as-API / planned wiring, not as a runtime toggle.

Server flags reference

sochdb-grpc-server \
--host 127.0.0.1 \
--port 50051 \
--metrics-port 9090 \
--ws-port 8080 \
--pg-port 5433 \
--auth \
--api-key "$SOCHDB_API_KEY" \
--tls-cert /path/cert.pem --tls-key /path/key.pem --tls-ca /path/ca.pem \
--pg-data-dir /var/lib/sochdb/sql \
--secrets-path /var/run/secrets/sochdb
FlagDefaultEnv varPurpose
--host127.0.0.1Bind address
-p, --port50051gRPC port
-d, --debugoffDebug logging
--metrics-port9090Prometheus port (0 disables)
--ws-port8080WebSocket port (0 disables)
--pg-port5433PostgreSQL wire port (0 disables)
--authoffEnable gRPC auth
--api-keynoneSOCHDB_API_KEYRegister an API key (requires --auth)
--tls-certnoneSOCHDB_TLS_CERTTLS cert PEM (enables TLS)
--tls-keynoneSOCHDB_TLS_KEYTLS private key PEM
--tls-canoneSOCHDB_TLS_CACA cert for mTLS
--secrets-pathnoneSOCHDB_SECRETS_PATHK8s Secrets mount path
--pg-data-dirnoneSOCHDB_PG_DATA_DIRPersistent dir for real SQL via pg-wire

Additional env vars read directly (not flags): SOCHDB_API_KEY_PEPPER, SOCHDB_JWT_SECRET, SOCHDB_ENCRYPTION_KEY, SOCHDB_API_KEYS (comma-separated).

See also