Token-Aware Context Query

SochDB ships a context-query builder for assembling LLM prompt context that fits a token budget. You declare named sections with priorities, the builder fills them from the database (or from literal text), counts tokens, and drops or truncates the lowest-priority sections when the budget is exceeded. Output can be emitted as TOON, JSON, or Markdown.

Per-SDK availability

The context-query builder is a real, importable module in the Rust SDK (context_query), the Node.js SDK (context-builder), and the Go SDK (context_builder.go). In Python it is an example pattern, not an importable SDK class (see the Python SDK guide). The capabilities differ per SDK: the Rust and Node builders can pull live data from the database (get / last / search / whereEq / sql), while the Go builder is literal-text only.

What the builder does

1. Token budgeting — keep assembled context within a model's context window.
2. Priority ordering — sections are sorted by priority (lower number = higher priority).
3. Truncation — when over budget, drop or shrink sections by a chosen strategy.
4. Structured output — emit TOON (default), JSON, or Markdown ready for a prompt.

Token counting is a heuristic (roughly 4 characters per token) unless your SDK exposes a real tokenizer. Treat the budget as a guide, not a hard guarantee.

Quick start

Rust

The Rust builder lives in the sochdb crate's context_query module. Sections can be literal text or live queries; search references an embedding bound earlier with set_var.

use std::sync::Arc;
use sochdb::SochConnection;
use sochdb::context_query::{ContextQueryBuilder, ContextFormat, ContextValue};

let conn = Arc::new(SochConnection::open("./my_data")?);

let result = ContextQueryBuilder::with_connection(conn.clone())
    .for_session("session_123")
    .with_budget(4000)
    .format(ContextFormat::Soch) // Soch == TOON, the default
    .set_var("query_embedding", ContextValue::Embedding(query_embedding))
    .literal("SYSTEM", 0, "You are a helpful assistant.")
    .section("DOCS", 1)
        .search("documents", "query_embedding", 10)
        .done()
    .execute()?;

println!("context:\n{}", result.context);
println!("tokens: {}/{}", result.token_count, result.token_budget);
println!("utilization: {:.0}%", result.utilization() * 100.0);

ContextFormat is Soch (TOON, default), Json, Markdown, or Text. ContextValue covers String, Int, Float, Bool, Embedding(Vec<f32>), and Binary.

Node.js / TypeScript

The Node builder lives in @sochdb/sochdb (context-builder). It is synchronous — execute() returns a ContextResult, not a Promise.

import { ContextQueryBuilder, ContextOutputFormat } from '@sochdb/sochdb';

const builder = new ContextQueryBuilder()
  .forSession('session_123')
  .withBudget(4000)                       // default budget is 4096
  .setFormat(ContextOutputFormat.MARKDOWN);

builder
  .literal('SYSTEM', 0, 'You are a helpful assistant.')
  .section('DOCS', 1)
    .search('documents', queryEmbedding, 10)
    .done();

const result = builder.execute();

console.log(result.text);
console.log(`tokens: ${result.tokenCount}`);
console.log(`sections: ${result.sections.length}`);

ContextResult is { text, tokenCount, sections: Array<{ name, tokenCount, truncated }> }.

Go

The Go builder (NewContextQueryBuilder) is always compiled (no build tag needed). It assembles literal text sections only — it does not query the database.

import sochdb "github.com/sochdb/sochdb-go"

builder := sochdb.NewContextQueryBuilder().
    ForSession("session_123").
    WithBudget(4000).               // default budget is 4096
    SetFormat(sochdb.FormatMarkdown).
    SetTruncation(sochdb.TailDrop)

builder.
    Literal("SYSTEM", 0, "You are a helpful assistant.").
    Literal("USER", 1, "How do I configure the index?")

result, err := builder.Execute()
if err != nil {
    log.Fatal(err)
}

fmt.Println(result.Text)
fmt.Printf("tokens: %d, truncated: %v\n", result.TokenCount, result.Truncated)

ContextResult is { Text, TokenCount, Sections []ContextSection, Truncated }.

Python

Python: example pattern, not an SDK class

There is no importable ContextQueryBuilder in the Python SDK (sochdb 0.5.9). The fluent context-query builder shown for the other languages exists only as a demonstration in the separate sochdb-python-examples repository (context_builder/context_builder.py, plus graph_overlay_examples/context_query_example.py). Do not import it from sochdb.

For shipped, importable token-aware context assembly in Python, use AgentMemory from the 0.5.9 SDK. It writes episodes and returns budgeted, formatted context:

from sochdb import Database, AgentMemory

db = Database.open("./my_data")
memory = AgentMemory(db, namespace="default", token_limit=4096, output_format="markdown")

memory.write_episode("The user prefers concise answers.")

result = memory.search(
    "what does the user prefer?",
    token_limit=4000,
    lanes="hybrid",          # "lexical" | "three_lane" | "hybrid" | "bm25" | "trigram"
    format="markdown",       # "markdown" | "json" | "toon"
)

print(result.context)
print(f"tokens: {result.token_count}")

AgentMemory.search also accepts as_of (bi-temporal point-in-time, unix ms) and a per-call namespace. See the Python SDK guide for details.

note

The 2.0.3 native Python package (HnswIndex, BM25Index, RRFFusion, …) does not provide a context-query builder. AgentMemory is in the 0.5.9 ctypes SDK only.

Sections

A context is built from named, prioritized sections. Lower priority numbers are kept first when the budget is tight.

Literal sections

Available in every SDK. The text is inserted verbatim and counted against the budget.

Rust

let result = ContextQueryBuilder::with_connection(conn.clone())
    .with_budget(2000)
    .literal("SYSTEM", 0, "You are an expert data scientist.")
    .literal("USER", 1, "How do I filter rows in a DataFrame?")
    .execute()?;

Node.js / TypeScript

const result = new ContextQueryBuilder()
  .withBudget(2000)
  .literal('SYSTEM', 0, 'You are an expert data scientist.')
  .literal('USER', 1, 'How do I filter rows in a DataFrame?')
  .execute();

Go

result, err := sochdb.NewContextQueryBuilder().
    WithBudget(2000).
    Literal("SYSTEM", 0, "You are an expert data scientist.").
    Literal("USER", 1, "How do I filter rows in a DataFrame?").
    Execute()

Data-backed sections (Rust and Node only)

The Rust and Node builders can populate a section directly from the database. Open a section with section(name, priority), add one content directive, then done() (done is implicit on execute in Node).

Rust

let result = ContextQueryBuilder::with_connection(conn.clone())
    .with_budget(4000)
    .set_var("q", ContextValue::Embedding(query_embedding))
    // GET a path expression
    .section("PROFILE", 0).get("users/alice/profile").done()
    // LAST N rows from a table
    .section("HISTORY", 1).last(5, "messages").done()
    // Vector SEARCH against a collection using a bound variable
    .section("DOCS", 2).search("documents", "q", 10).done()
    .execute()?;

SectionBuilder also supports select(columns, table), where_eq, where_gt, where_lt, where_like, limit, min_score, project, and summarize.

Node.js / TypeScript

const builder = new ContextQueryBuilder().withBudget(4000);

builder
  .section('PROFILE', 0).get('users/alice/profile').done()
  .section('HISTORY', 1).last(5, 'messages').done()
  .section('DOCS', 2).search('documents', queryEmbedding, 10).done()
  .section('STATS', 3).sql('SELECT count(*) FROM events').done();

const result = builder.execute();

Node section directives: get(path), last(n, table), whereEq(field, value), search(collection, embedding, k), and sql(query).

Go

note

The Go builder does not support data-backed sections — it assembles literal text only. To pull data into context in Go, query the database yourself (for example with IPCClient.Scan or GrpcClient.SearchCollection) and pass the formatted result to Literal(...).

Output formats

All three builders support TOON (the default), JSON, and Markdown. TOON is SochDB's compact wire format and typically uses 40–66% fewer tokens than equivalent JSON.

Format	Rust (ContextFormat)	Node (ContextOutputFormat)	Go (ContextOutputFormat)
TOON (default)	Soch	TOON	FormatTOON
JSON	Json	JSON	FormatJSON
Markdown	Markdown	MARKDOWN	FormatMarkdown

Rust

use sochdb::context_query::ContextFormat;

let result = ContextQueryBuilder::with_connection(conn.clone())
    .format(ContextFormat::Markdown)
    .literal("DOCS", 0, "Deep learning models use neural networks...")
    .execute()?;

Node.js / TypeScript

import { ContextOutputFormat } from '@sochdb/sochdb';

const result = new ContextQueryBuilder()
  .setFormat(ContextOutputFormat.JSON)
  .literal('DOCS', 0, 'Deep learning models use neural networks...')
  .execute();

Go

result, err := sochdb.NewContextQueryBuilder().
    SetFormat(sochdb.FormatJSON).
    Literal("DOCS", 0, "Deep learning models use neural networks...").
    Execute()

Markdown output looks like:

## SYSTEM

You are a helpful assistant.

## DOCS

Deep learning models use neural networks...

Truncation strategies

When assembled sections exceed the budget, the builder applies a truncation strategy. The defaults differ slightly: Rust and Node keep highest-priority sections and drop the rest; the Go default is TailDrop.

Strategy	Rust / Node / Go	Behavior
Tail drop	TruncationStrategy::TailDrop / TruncationStrategy.TAIL_DROP / sochdb.TailDrop	Keep highest-priority sections; drop the lowest-priority ones that don't fit.
Head drop	HeadDrop / HEAD_DROP / sochdb.HeadDrop	Drop highest-priority sections first (keep the most recent / lowest-priority).
Proportional	Proportional / PROPORTIONAL / sochdb.Proportional	Shrink every section's text proportionally to fit the budget.

Rust

use sochdb::context_query::TruncationStrategy;

let result = ContextQueryBuilder::with_connection(conn.clone())
    .with_budget(100)
    .truncation(TruncationStrategy::Proportional)
    .literal("SYSTEM", 0, "You are a helpful assistant with broad expertise.")
    .literal("CONTEXT", 1, "The user is working on a Python data-processing project.")
    .execute()?;

Node.js / TypeScript

import { TruncationStrategy } from '@sochdb/sochdb';

const result = new ContextQueryBuilder()
  .withBudget(100)
  .setTruncation(TruncationStrategy.PROPORTIONAL)
  .literal('SYSTEM', 0, 'You are a helpful assistant with broad expertise.')
  .literal('CONTEXT', 1, 'The user is working on a Python data-processing project.')
  .execute();

Go

result, err := sochdb.NewContextQueryBuilder().
    WithBudget(100).
    SetTruncation(sochdb.Proportional).
    Literal("SYSTEM", 0, "You are a helpful assistant with broad expertise.").
    Literal("CONTEXT", 1, "The user is working on a Python data-processing project.").
    Execute()

Result metrics

Each builder returns the assembled context plus accounting fields.

Rust

let result = builder.execute()?;

println!("context:\n{}", result.context);
println!("tokens used: {}", result.token_count);
println!("token budget: {}", result.token_budget);
println!("utilization: {:.0}%", result.utilization() * 100.0);
println!("included: {:?}", result.included_sections());
println!("dropped:  {:?}", result.dropped_sections());
println!("truncated: {:?}", result.truncated_sections());

Node.js / TypeScript

const result = builder.execute();

console.log(result.text);
console.log(`tokens used: ${result.tokenCount}`);
for (const s of result.sections) {
  console.log(`  ${s.name}: ${s.tokenCount} tokens, truncated=${s.truncated}`);
}

Go

result, _ := builder.Execute()

fmt.Println(result.Text)
fmt.Printf("tokens used: %d\n", result.TokenCount)
fmt.Printf("truncated:   %v\n", result.Truncated)
for _, s := range result.Sections {
    fmt.Printf("  %s: %d tokens, truncated=%v\n", s.Name, s.TokenCount, s.Truncated)
}

Best practices

1. Leave headroom. Budget below the model's true window so there is room for the system prompt and the model's reply (for example, 4000 for an 8K window).
2. Order by importance. Give the system prompt and the user's latest question the lowest priority numbers so they survive truncation.
3. Prefer TOON for data-heavy context. It typically uses 40–66% fewer tokens than JSON for tabular results.
4. Monitor what got dropped. In Rust, check dropped_sections() / truncated_sections(); a high drop rate means the budget is too tight or sections are too large.
5. In Go, pre-query and pass literals. The Go builder only assembles literal text, so do retrieval first and feed the formatted output to Literal(...).

See also

• Graph Overlay — agent memory relationships
• Policy Hooks — safety policies
• Vector Search — HNSW indexing
• Python SDK — AgentMemory for token-aware context in Python