Multi-Agent Tool Routing

SochDB ships a tool-routing primitive for building multi-agent systems: register specialized agents with capabilities, register tools, and dispatch tool calls to the right agent using a pluggable selection strategy. Agent registrations are persisted to the database so they survive process restarts.

Availability by SDK

Tool routing is a real, importable module only in the Rust SDK (sochdb::routing).

• Rust SDK 2.0.3 — first-class module sochdb::routing (AgentRegistry, ToolRouter, ToolDispatcher). This is what the page documents.
• Python SDK 0.5.9 — no built-in router. There is only an example (tool_routing_example.py) that implements an app-level router in plain Python. See App-level routing in Python.
• Node.js SDK 0.5.3 — no routing module exists in the package. Implement routing at the application level.
• Go SDK 0.4.5 — no routing module.

If you previously saw ToolDispatcher examples for Python/Go/Node, those did not match the shipped SDKs and have been removed.

Overview

The Rust routing system has three components, all in sochdb::routing:

Component	Purpose
AgentRegistry<C>	Register agents and their tool capabilities; track per-agent call stats
ToolRouter<C>	Hold tool definitions and select an agent per call using a strategy
ToolDispatcher<C>	High-level facade that wires a registry + router over one connection

All three are generic over C: ConnectionTrait, so they work over any SochDB connection type (for example Connection / DurableConnection).

Architecture

┌─────────────────────────────────────────────────────────────┐
│                    Tool Dispatcher                          │
├─────────────────────────────────────────────────────────────┤
│ ┌─────────────────┐     ┌─────────────────────────────────┐ │
│ │ Agent Registry  │     │        Tool Router              │ │
│ │                 │     │                                 │ │
│ │ agent_001 ──────┼────►│ Strategy Selection:             │ │
│ │   - Code        │     │  • RoundRobin                   │ │
│ │   - Shell       │     │  • LeastLoaded                  │ │
│ │                 │     │  • Priority (default)           │ │
│ │ agent_002 ──────┼────►│  • Sticky (session affinity)    │ │
│ │   - Search      │     │  • Random                       │ │
│ │   - Web         │     │  • Fastest (latency-based)      │ │
│ └─────────────────┘     └─────────────────────────────────┘ │
└─────────────────────────────────────────────────────────────┘

Quick Start (Rust)

Add the SDK (crate name is sochdb):

[dependencies]
sochdb = "2.0.3"
serde_json = "1"

ToolDispatcher::new takes a connection by value and builds the registry and router for you. Invocation is synchronous — there is no async/await in the Rust SDK.

use sochdb::Connection;
use sochdb::routing::{ToolDispatcher, Tool, ToolCategory};
use serde_json::json;

let conn = Connection::open("./agent_data")?;
let dispatcher = ToolDispatcher::new(conn);

// Register a local (in-process) agent: id, capabilities, handler, priority.
dispatcher.register_local_agent(
    "code_agent",
    vec![ToolCategory::Code, ToolCategory::Shell],
    |tool, args| Ok(json!({ "output": format!("ran {tool} with {args}") })),
    10, // priority
);

// Register a tool the router can dispatch.
dispatcher.register_tool(Tool {
    name: "code_exec".to_string(),
    description: "Execute code".to_string(),
    category: ToolCategory::Code,
    ..Default::default()
});

// Invoke with automatic routing. Returns a RouteResult (not a Result/Promise).
let route = dispatcher.invoke("code_exec", json!({ "code": "print('hello')" }), None);
if route.success {
    println!("agent={} latency={:.1}ms", route.agent_id, route.latency_ms);
} else {
    eprintln!("routing failed: {:?}", route.error);
}

invoke does not return an error type

ToolDispatcher::invoke(tool_name, args, context) -> RouteResult always returns a RouteResult. Check route.success and route.error rather than using ?. If the tool name is unknown or no capable agent is registered, success is false and error is populated.

Registering agents

There are three registration entry points:

use sochdb::routing::{AgentConfig, ToolCategory};

// 1. Local handler (executes in-process):
dispatcher.register_local_agent(
    "code_agent",
    vec![ToolCategory::Code],
    |tool, args| Ok(serde_json::json!({ "ok": true, "tool": tool, "args": args })),
    10,
);

// 2. Remote HTTP endpoint (see caution below — not yet executed):
dispatcher.register_remote_agent(
    "search_agent",
    vec![ToolCategory::Search, ToolCategory::Web],
    "http://localhost:8002/invoke",
    5,
);

// 3. Full control via the registry + AgentConfig:
dispatcher.registry().register_agent(
    "graph_agent",
    vec![ToolCategory::Graph],
    AgentConfig::with_handler(|tool, _args| Ok(serde_json::json!({ "tool": tool })))
        .priority(20),
);

AgentConfig builders: AgentConfig::with_endpoint(url), AgentConfig::with_handler(fn), and .priority(i32). Defaults are priority = 100, max_concurrent = 10, status = AgentStatus::Available.

Remote (HTTP) invocation is not implemented yet

You can register a remote agent with register_remote_agent / AgentConfig::with_endpoint, but the dispatcher does not yet make the HTTP call. An endpoint-only agent returns the error "Remote invocation to <endpoint> not yet implemented in Rust SDK" when selected. Only agents with a local handler actually execute today. Plan your routing so that a capable handler-backed agent exists, or the call will fail over to the next capable agent and ultimately report success = false.

Tool Categories

ToolCategory (used for agent capabilities and tool classification):

Variant	Typical use
Code	Code execution / interpreters
Search	Vector or full-text search
Database	SQL queries, key-value ops
Web	HTTP requests, scraping
File	File system operations
Git	Git operations
Shell	Shell commands
Email	Email send/read
Calendar	Calendar operations
Memory	Agent memory read/write
Vector	Vector operations
Graph	Graph traversal
Custom	Anything else (default for Tool::default())

A Tool is matched to agents by its category, or by its explicit required_capabilities list if non-empty. An agent is capable when it holds all required categories.

Routing Strategies

RoutingStrategy is configured on the router (not passed per call). The default is Priority.

Strategy	Behavior
RoundRobin	Rotate through capable agents
Random	Pseudo-random selection
LeastLoaded	Pick the agent with the fewest in-flight calls (current_load)
Priority (default)	Highest priority, breaking ties by lower load
Sticky	Reuse the agent previously bound to the session id
Fastest	Lowest average recorded latency

Set a non-default strategy by building a custom router via ToolRouter::with_default_strategy:

use std::sync::Arc;
use sochdb::routing::{AgentRegistry, ToolRouter, RoutingStrategy};

let conn = Arc::new(sochdb::Connection::open("./agent_data")?);
let registry = Arc::new(AgentRegistry::new(Arc::clone(&conn)));
let router = ToolRouter::new(Arc::clone(&registry), Arc::clone(&conn))
    .with_default_strategy(RoutingStrategy::LeastLoaded);

Sticky sessions

Maintain session affinity for stateful interactions by passing a RoutingContext with a session id. With the Sticky strategy, repeated calls for the same session id are routed to the same capable agent:

use sochdb::routing::RoutingContext;
use serde_json::json;

let ctx = RoutingContext::new().with_session_id("session_abc123");

let first = dispatcher.invoke("chat", json!({ "message": "Write a function" }), Some(ctx.clone()));
let second = dispatcher.invoke("chat", json!({ "message": "Now test it" }), Some(ctx));
// `second` routes to the same agent as `first` (when using RoutingStrategy::Sticky).

RoutingContext also supports .with_preferred_agent(id) (force a specific agent when capable) and .exclude_agent(id) (skip an agent). A preferred agent overrides the strategy whenever it is capable.

Persistent Agent State

Agent registrations are written to the database under the /_routing/agents/ prefix, so they outlive the process:

// Registrations are persisted on register.
dispatcher.register_local_agent("agent_001", vec![ToolCategory::Code], handler, 10);

// In a later process, a fresh dispatcher can re-list the persisted records.
let dispatcher = ToolDispatcher::new(conn);
let agents = dispatcher.list_agents(); // Vec<serde_json::Value>

Handlers are not persisted

Only the agent metadata (id, capabilities, endpoint, priority, max_concurrent) is stored. Closures/handlers cannot be serialized, so after a restart you must re-register local handlers before those agents can execute. The persisted record is enough to rebuild remote/endpoint agents and to inspect the registry.

Observability

There is no separate metrics API. Use dispatcher.list_agents() to read per-agent runtime stats; each entry is a JSON object with agent_id, capabilities, status, priority, current_load, total_calls, avg_latency_ms, has_endpoint, and has_handler. The router records call latency and success/failure for every dispatch via AgentRegistry::record_call, which feeds the LeastLoaded and Fastest strategies.

for agent in dispatcher.list_agents() {
    println!(
        "{}: calls={} avg_latency_ms={}",
        agent["agent_id"], agent["total_calls"], agent["avg_latency_ms"]
    );
}

No automatic health checks

There is no health-check loop or auto-eviction of unhealthy agents in the current SDK. AgentRegistry::update_agent_status exists but is a documented placeholder (agent status is not mutable through the shared Arc today), so agents are not automatically removed from routing. To take an agent out of rotation, call dispatcher.registry().unregister_agent(id) or exclude it per call with RoutingContext::exclude_agent.

App-level routing in Python

The Python SDK (sochdb 0.5.9) does not expose a router. The repository ships a demonstration of how to build one in a few lines of plain Python at sochdb-python-examples/graph_overlay_examples/tool_routing_example.py. The pattern mirrors the Rust strategies (priority / round-robin / least-loaded) without any SDK dependency:

agents = [
    {"id": "code_analyzer", "capabilities": {"code", "search"}, "priority": 100},
    {"id": "db_agent",      "capabilities": {"database", "memory"}, "priority": 90},
    {"id": "vector_search", "capabilities": {"vector", "search"}, "priority": 110},
    {"id": "fallback",      "capabilities": {"code", "database", "vector", "search"}, "priority": 10},
]

tools = {
    "search_code":    {"required": {"code", "search"}},
    "vector_search":  {"required": {"vector"}},
    "query_database": {"required": {"database"}},
}

def route_priority(tool_name):
    req = tools[tool_name]["required"]
    candidates = [a for a in agents if req.issubset(a["capabilities"])]
    return sorted(candidates, key=lambda a: a["priority"], reverse=True)[0]["id"]

print(route_priority("vector_search"))  # -> vector_search
print(route_priority("search_code"))    # -> code_analyzer

You can persist this state in SochDB yourself (for example under a tools/agents/ path with db.put_path(...)), but there is no built-in ToolDispatcher class to import in Python.

Combining with policy hooks (Rust)

Tool routing pairs well with the policy engine to enforce which agents may handle sensitive tools. The PolicyEngine lives in sochdb::policy and is also generic over the connection:

use sochdb::policy::{PolicyEngine, PolicyAction};

let policy = PolicyEngine::new(conn);

// Deny writes under tools/sensitive/ unless a trusted agent is in context.
policy.before_write("tools/sensitive/*", |ctx| {
    match ctx.get("agent_id") {
        Some(id) if id == "trusted_agent_1" || id == "trusted_agent_2" => PolicyAction::Allow,
        _ => PolicyAction::Deny,
    }
});

In Python, policy hooks are example-only patterns (see sochdb-python-examples/graph_overlay_examples/policy_hooks_example.py); in the Node.js SDK the analogous primitive is the PolicyService namespace-ACL module.

Best Practices

1. Categorize tools accurately — capability matching is exact (all required categories must be present).
2. Always register at least one handler-backed agent — remote/endpoint invocation is not wired up yet.
3. Use Sticky + RoutingContext session ids for stateful, multi-turn work.
4. Set realistic priorities — Priority (the default) prefers higher priority, then lower load.
5. Re-register handlers after a restart — handlers are not persisted.
6. Inspect list_agents() for latency and load instead of relying on a metrics API.

See Also

• Policy & Safety Hooks - Enforce security policies
• Rust SDK Guide - Connections, transactions, and overlays
• Analytics Guide - Usage telemetry
• Deployment Guide - Production deployment patterns