Developer Documentation
Overview
MentisDB is a standalone Rust crate available on crates.io. Add it to your project:
mentisdb = "0.12.2"The minimum supported Rust version is 1.90 (rust-version in the crate manifest; CI checks it).
The crate provides the full memory engine, skill registry, storage adapter interface, agent registry, and optional HTTP server stack (MCP + REST + dashboard) behind the server feature flag.
Full API Reference
The complete, generated API documentation is hosted on docs.rs:
docs.rs/mentisdb
Full rustdoc for all public types, traits, functions, and modules. Includes MentisDb, Thought, ThoughtInput, ThoughtQuery, StorageAdapter, SkillRegistry, BinaryStorageAdapter, and the HTTP server.
MCP Server
MentisDB ships a built-in MCP (Model Context Protocol) HTTP server. Enable it with the server feature. All MentisDB operations are exposed as MCP tools over JSON-RPC 2.0 via HTTP, making it compatible with any MCP-capable AI tool. The canonical onboarding path is not a separate URL: it happens during the MCP initialize call via startup instructions plus the embedded resource mentisdb://skill/core exposed through resources/list and resources/read.
Default HTTP endpoint: http://127.0.0.1:9471 — HTTPS endpoint: https://127.0.0.1:9473.
Lean output on MCP
MCP tool results are read by a language model, so the streamable-HTTP MCP surface (JSON-RPC tools/call, and stdio mode, which proxies to it) shapes them for its context window. REST, POST /tools/execute and the CLI keep the full response. Since 0.12.1:
- Results are minified JSON text, with no second
structuredContentcopy. No tool declares anoutputSchema, so read the text content; it is one JSON document. mentisdb_ranked_search,mentisdb_federated_search,mentisdb_lexical_searchandmentisdb_context_bundlesdefault tocompact: trueand leave out defaults: noprovenanceis a public, unsigned thought (each response says so once indefault_provenance; a thought that arrives without provenance is marked{"unknown": true}), noroleisMemory, noimportanceis 0.5, nochain_keyis the chain searched. Floats are rounded, timestamps keep full precision.successor_ofis one older version plus asuperseded_versionscount (a lower bound), andsuccessor_kindkeeps the kind of the link it replaces (absent meansSupersedes).compact: falsereturns the full record. Acompactorinclude_entriesthat is not a boolean is refused with JSON-RPC-32602.mentisdb_standing_contextreturnspromptandduplicates_collapsed; theentriesarray needsinclude_entries: true.
On a generated 3,500-thought chain a 10-hit ranked search costs 5,948 characters instead of 23,657, and standing context 13,050 instead of 36,191 (cargo run --release --example token_diet). The rules are in the mentisdb::server_lean rustdoc.
All 45 MCP Tools
| Tool | Description | Key Parameters |
|---|---|---|
mentisdb_bootstrap | Create a chain if needed and write one bootstrap checkpoint when it is empty | chain_key,content,concepts,importance |
mentisdb_append | Append a durable semantic thought with optional tags, concepts, refs, scope, and signature metadata | chain_key,thought_type,content,agent_id,tags,concepts,scope,importance,confidence |
mentisdb_append_retrospective | Append a guided retrospective memory to prevent future agents from repeating a hard failure | chain_key,content,concepts,refs,importance |
mentisdb_search | Search thoughts by semantic filters, identity filters, time bounds, and scoring thresholds | chain_key,text,thought_types,roles,agent_ids,since,until,min_importance,min_confidence |
mentisdb_lexical_search | Return flat ranked lexical matches with explainable term and field provenance | chain_key,text,thought_types,limit,offset,compact |
mentisdb_ranked_search | Return flat ranked results with lexical, graph-aware, or heuristic score breakdowns. Supports point-in-time queries and memory scope filtering | chain_key,text,as_of,scope,enable_reranking,rerank_k,concepts_any,roles,limit,compact |
mentisdb_federated_search | Ranked retrieval over several chains at once, merged into one list. Each chain is authorized and filtered for the caller on its own | chain_keys,text,limit,graph,as_of,include_invalidated,compact |
mentisdb_context_bundles | Return seed-anchored grouped support context beneath the best lexical seeds | chain_key,text,as_of,scope,graph,limit,compact(MCP only) |
mentisdb_list_chains | List known chains with version, storage adapter, counts, and storage location | chain_key |
mentisdb_migration_status | Report the background storage-format migration state of every chain that is, was, or failed to be migrated: phase, percent done, attempts and error. Global token only | |
mentisdb_merge_chains | Merge all thoughts from a source chain into a target chain, then permanently delete the source | source_chain_key,target_chain_key |
mentisdb_branch_from | Create a new chain diverging from a thought on an existing chain. Searches on the branch transparently include ancestor results | source_chain_key,branch_chain_key,branch_thought_id |
mentisdb_list_agents | List the distinct agent identities participating in one chain | chain_key |
mentisdb_get_agent | Return one full agent registry record including status, aliases, description, keys, and per-chain activity metadata | agent_id,chain_key |
mentisdb_list_agent_registry | Return the full per-chain agent registry | chain_key |
mentisdb_upsert_agent | Create or update a registry record before or after an agent writes thoughts | agent_id,display_name,agent_owner,description,status,chain_key |
mentisdb_set_agent_description | Set or clear the description stored for one registered agent | agent_id,description,chain_key |
mentisdb_add_agent_alias | Add a historical or alternate alias to a registered agent | agent_id,alias,chain_key |
mentisdb_get_access_policy | Return the chain's access policy as the caller may see it: mode, sequence, key-set commitment and segment boundaries for everyone; the allowed keys and record hashes only for a reader who proved an authorized key | chain_key |
mentisdb_verify_signatures | Re-verify the signed thoughts and governance records the caller may see; per-thought status (valid, unsigned, invalid), signature version and totals | chain_key |
mentisdb_standing_context | Render durable, age-independent memory (constraints, preferences, user traits, goals, decisions, lessons, newest handoff and checkpoint) as a prompt block. Call at session start before mentisdb_recent_context | chain_key,agent_id,types,since,as_of,include_invalidated,per_type_limit,limit,include_entries(MCP only) |
mentisdb_recent_context | Render recent thoughts into a prompt snippet for session resumption | chain_key,last_n |
mentisdb_memory_markdown | Export a MEMORY.md-style Markdown view of the full chain or a filtered subset | chain_key,agent_ids,thought_types,roles,since,until,limit |
mentisdb_import_memory_markdown | Import a MEMORY.md-formatted Markdown document into a target chain | markdown,default_agent_id,chain_key |
mentisdb_get_thought | Return one stored thought by stable id, chain index, or content hash | thought_id,thought_index,thought_hash,chain_key |
mentisdb_get_genesis_thought | Return the first thought ever recorded in the chain, if any | chain_key |
mentisdb_traverse_thoughts | Traverse the chain forward or backward in append order from a chosen anchor, in chunks, with optional filters | chain_key,anchor_id,anchor_index,anchor_hash,direction,chunk_size,thought_types,roles,since,until |
mentisdb_skill_md | Return the official embedded MENTISDB_SKILL.md Markdown file | (none) |
mentisdb_list_skills | List versioned skill summaries from the skill registry | chain_key |
mentisdb_skill_manifest | Return the versioned skill-registry manifest including searchable fields and supported formats | (none) |
mentisdb_upload_skill | Upload a new immutable skill version from Markdown or JSON | agent_id,skill_id,content,format,signing_key_id,skill_signature,chain_key |
mentisdb_search_skill | Search skills by indexed metadata such as ids, names, tags, triggers, uploader identity, status, format, schema version, and time window | chain_key,text,skill_ids,names,statuses,since,until,limit |
mentisdb_read_skill | Read one stored skill as Markdown or JSON. Responses include trust warnings for untrusted or malicious skill content | skill_id,version_id,format,chain_key |
mentisdb_skill_versions | List immutable uploaded versions for one skill | skill_id,chain_key |
mentisdb_deprecate_skill | Mark a skill as deprecated while preserving all prior versions | skill_id,reason,chain_key |
mentisdb_revoke_skill | Mark a skill as revoked while preserving audit history | skill_id,reason,chain_key |
mentisdb_delete_skill | Permanently remove a skill and all of its versions. After delete the same skill_id can be uploaded again. Prefer revoke when you need an audit row. | skill_id,chain_key |
mentisdb_head | Return head metadata, the latest thought at the current chain tip, and integrity state | chain_key |
mentisdb_summary_candidates | Suggest groups of thoughts that are candidates for summarization | chain_key,text,limit |
mentisdb_register_webhook | Register a webhook to receive HTTP POST notifications when thoughts are appended. Delivery is fire-and-forget with exponential backoff (up to 3 attempts) | url,chain_key_filter,thought_type_filter |
mentisdb_list_webhooks | List all registered webhooks | chain_key |
mentisdb_delete_webhook | Remove a webhook registration by its UUID | id |
mentisdb_list_entity_types | List the entity types registered on a chain | chain_key |
mentisdb_upsert_entity_type | Create or update an entity type in the per-chain registry | chain_key,entity_type |
mentisdb_extract_memories | Turn free-form text into typed memory candidates with the configured LLM. Nothing is appended; needs the llm-extraction feature | text,chain_key,agent_id,prompt_template |
Since 0.11, registering or revoking agent keys, disabling agents and changing a chain's access policy are operator-only: mentisdb_add_agent_key, mentisdb_revoke_agent_key, mentisdb_disable_agent and mentisdb_set_access_policy are no longer offered, and a status change through mentisdb_upsert_agent is refused (restating the current status is still accepted). The operator uses the dashboard behind its PIN or the local mentisdb command line; see Public & Private Chains. A tool call against a chain that is still being migrated returns a tool error with the retry delay and percent done; retry after it.
JSON-RPC Request/Response Example
MCP uses JSON-RPC 2.0 over HTTP. Send a POST to the MCP endpoint:
POST / HTTP/1.1
Host: 127.0.0.1:9471
Content-Type: application/json
{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"name": "mentisdb_append",
"arguments": {
"chain_key": "default",
"agent_id": "assistant",
"thought_type": "Insight",
"content": "Memory deduplication triggers when Jaccard similarity exceeds threshold",
"importance": 0.7,
"confidence": 0.9,
"tags": ["feature:dedup"],
"concepts": ["memory-dedup", "jaccard-similarity"]
}
}
}
// Response:
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"content": [
{
"type": "text",
"text": "Thought appended: a1b2c3... (index 42)"
}
]
}
}The legacy CloudLLM endpoint POST /tools/execute takes tool + parameters; JSON-RPC tools/call takes name + arguments. Authorization checks exactly the fields the transport executes, and a payload that mixes both shapes is rejected with 400.
MCP HTTP Ports
The MCP server listens on two ports:
http://127.0.0.1:9471— HTTP (unencrypted), suitable for local developmenthttps://127.0.0.1:9473— HTTPS (TLS), recommended for production
The daemon binary (mentisdb) starts the MCP server automatically. For embedding in your own Axum app, see the docs.rs API reference for server module details.
REST API
The same daemon also serves a REST API alongside MCP. Key endpoints:
| Method | Path | Description |
|---|---|---|
POST | /v1/thoughts | Append a new thought to a chain. Supports scope parameter for memory scope tagging and source_episode (0.12.2) for provenance |
POST | /v1/search | Search/query thoughts. Supports as_of for point-in-time queries, scope for scope filtering and an exact-match source_episode filter (0.12.2) |
POST | /v1/lexical-search | Ranked lexical search with scores and matched-term diagnostics. Pass compact: true to drop match_sources and the thought's integrity fields |
POST | /v1/ranked-search | Canonical flat ranked retrieval with lexical + vector + graph-aware score breakdowns (hybrid when managed sidecars are available). Supports as_of, scope, enable_reranking, and rerank_k parameters, an exact-match source_episode filter (0.12.2), and the ranking knobs resolve_successors, collapse_duplicates, vector_fusion, synonym_weight, df_scope and df_cutoffs (bad values are a 400). Pass compact: true for smaller hits |
POST | /v1/context-bundles | Seed-anchored grouped support context for agent reasoning and dashboard inspection. Supports as_of and scope parameters, and an exact-match source_episode filter (0.12.2) |
GET | /v1/chains | List available chain keys; each chain summary carries derived_backlog (0.12) |
POST | /v1/chains/branch | Create a branch chain diverging from a thought on an existing chain |
POST | /v1/agents | List agents in a chain or inspect the registry |
GET | /mentisdb_skill_md | Compatibility fallback for clients that cannot use MCP resources; MCP-native clients should read mentisdb://skill/core after initialize |
POST | /v1/skills/upload | Upload a skill version |
POST | /v1/skills/read | Read a skill (latest or specific version) |
POST | /v1/recent-context | Render a compact resumption prompt for recent, important context |
POST | /v1/standing-context | Durable, age-independent memory as a prompt block plus typed entries (entries, duplicates_collapsed, prompt). Load at session start before recent context. Over MCP, entries need include_entries: true |
POST | /v1/memory-markdown | Export a chain as MentisDB-flavored MEMORY.md markdown |
POST | /v1/head | Return current chain head metadata and the latest thought |
POST | /v1/federated-search | Ranked search fused across several chains (chain_keys). Accepts compact: true and an exact-match source_episode filter (0.12.2) |
POST | /v1/summary-candidates | Suggest groups of thoughts that are candidates for summarization |
POST | /v1/extract-memories | Turn raw text into typed memory candidates with the configured LLM |
POST | /v1/entity-types | List entity types defined on a chain |
POST | /v1/entity-types/upsert | Create or update an entity type on a chain |
GET | /v1/webhooks | List registered webhooks |
POST | /v1/webhooks | Register a webhook (url, optional chain_key_filter, optional thought_type_filter) |
DELETE | /v1/webhooks/{id} | Delete a webhook by id |
GET | /health | Liveness health check |
POST | /v1/bootstrap | Bootstrap a chain with an initial memory |
POST | /v1/retrospectives | Append a guided retrospective thought |
POST | /v1/thought | Retrieve a single thought by ID, hash, or index |
POST | /v1/thoughts/genesis | Retrieve the first thought in a chain |
POST | /v1/thoughts/traverse | Traverse thoughts in append order with filtering and pagination |
POST | /v1/import-markdown | Import MEMORY.md-formatted markdown into a chain |
GET | /v1/skills | List uploaded skill summaries |
GET | /v1/skills/manifest | Return the skill registry manifest with searchable fields |
POST | /v1/skills/search | Search the versioned skill registry by indexed fields |
POST | /v1/skills/versions | List immutable versions for a stored skill |
POST | /v1/skills/deprecate | Mark a stored skill as deprecated |
POST | /v1/skills/revoke | Mark a stored skill as revoked |
POST | /v1/skills/delete | Permanently delete a skill and all versions so the skill_id can be reused |
POST | /v1/agent | Get a single agent record by ID |
POST | /v1/agent-registry | List the full agent registry for a chain |
POST | /v1/agents/upsert | Create or update an agent record |
POST | /v1/agents/description | Set or clear an agent's description |
POST | /v1/agents/aliases | Add an alias to an agent |
POST | /v1/agents/keys | Answers 403 (operator-only since 0.11): register keys from the dashboard (Generate key pair, Add Key) or with mentisdb agent key generate / agent key add |
POST | /v1/agents/keys/revoke | Answers 403 (operator-only since 0.11): mentisdb agent key revoke |
POST | /v1/agents/disable | Answers 403 (operator-only since 0.11): mentisdb agent disable / activate |
POST | /v1/vectors/rebuild | Rebuild vector sidecar indexes for a chain |
POST | /v1/chains/merge | Merge all thoughts from a source chain into a target, then delete the source. Refused for a source with private segments, governance records or signed thoughts |
POST | /v1/access/policy | A chain's access policy as the caller may see it: mode, sequence, key-set commitment and segment boundaries; keys and record hashes only for an authorized reader |
POST | /v1/access/verify | Re-verify the signatures and governance records the caller may see |
POST | /v1/access/policy/set | Answers 403 (operator-only since 0.11): mentisdb chain access set|allow|deny |
GET | /v1/migrations | Background migration state of every chain (global token) |
POST | /v1/migrations/retry | Clear a chain's migration quarantine so it is migrated again; body chain_key (global token) |
POST | /v1/migrations/skip | Leave a chain unmigrated and unavailable until retried; body chain_key (global token) |
Chain Resolution and Error Statuses
Only writes create a chain whose chain_key does not exist yet: /v1/bootstrap, /v1/thoughts, /v1/retrospectives, /v1/import-markdown, /v1/agents/upsert, /v1/entity-types/upsert and the target of /v1/chains/merge. Every other route returns 404 for an unknown chain and persists nothing; the configured default chain is the only one created on demand for reads. Chain keys must be non-empty, at most 256 bytes and free of control characters (400 otherwise). The source, target and branch keys of /v1/chains/merge and /v1/chains/branch are trimmed of surrounding whitespace before that check. The MCP tools behave the same way.
| Status | Meaning |
|---|---|
400 | Invalid input: malformed JSON, bad UUID, unknown thought type or role, invalid chain key, an unknown ranking-knob value, a v2-signed append built on an older head (fetch the head and sign again), a result window (offset + limit) above 10,000 on ranked, lexical or federated search or context bundles, or a standing-context per_type_limit above 100 |
401 | Missing or invalid bearer token, a token whose scope does not cover the operation, or an X-Mentis-Auth read proof that is present but invalid (bad signature, clock more than 60 s off, reused nonce) |
403 | Permission denied: missing or invalid signature on a signed skill upload, a write the chain's access policy does not accept (the body does not say whether the agent or key exists), or an operator-only route |
404 | Unknown chain or other missing resource |
408 | A request body stopped arriving for MENTISDB_BODY_IDLE_TIMEOUT_SECS, a small or anonymous body missed its total deadline (twice that), or any body fell below MENTISDB_MIN_BODY_BYTES_PER_SEC (64 KiB/s by default) in a window after its grace period (twice the idle timeout); on every route |
409 | Conflict, e.g. branching into a chain key that already exists |
413 | Request body over the route limit (2 MiB; MENTISDB_MAX_BODY_BYTES, default 64 MiB, on the import routes for an authenticated caller), or over 1 MiB for an X-Mentis-Auth read proof that does not also send X-Mentis-Content-Sha256 (the SHA-256 the proof signs; the CLI and pymentisdb send it, so a large memory import works) |
500 | Server fault; the body says only "internal server error" and details go to the daemon log |
501 | /v1/extract-memories on a build without the llm-extraction feature |
503 | The chain is being migrated (Retry-After header, retry_after_secs and percent done in the body) or failed to migrate (the body names the fix, e.g. mentisdb migrate retry --chain KEY); also, with Retry-After, when the in-flight body budget is exhausted |
With bearer-token access enabled, GET /v1/chains, GET /v1/webhooks, DELETE /v1/webhooks/{id}, /v1/skills/{upload,deprecate,revoke,delete} and /v1/admin/flush require a global token; POST /v1/webhooks with a chain-scoped token requires a chain_key_filter inside its scope.
For MCP-native agents, prefer the streamable HTTP root at POST / and let the agent bootstrap itself from the initialize instructions and resource catalog.
For direct crate users, 0.8.0 keeps vector sidecars additive and rebuildable, but ranked search now blends lexical, graph, and managed-sidecar vector signals automatically when sidecars are enabled. The append-only chain remains canonical; sidecars are derived acceleration state.
The daemon now applies persisted managed-vector settings whenever it opens a chain. By default each chain gets the built-in local-text provider (local-text-v1), and ranked search across REST, MCP, and dashboard surfaces upgrades to hybrid scoring when that sidecar is available.
REST Example: POST /v1/thoughts (Append Thought)
POST /v1/thoughts HTTP/1.1
Host: 127.0.0.1:9471
Content-Type: application/json
{
"chain_key": "default",
"agent_id": "assistant",
"thought_type": "Insight",
"content": "Memory deduplication triggers when Jaccard similarity exceeds threshold",
"importance": 0.7,
"confidence": 0.9,
"tags": ["feature:dedup"],
"concepts": ["memory-dedup", "jaccard-similarity"],
"scope": "user",
"relations": [
{ "kind": "ContinuesFrom", "target_id": "<uuid-of-prior-turn>" }
]
}
// Response:
{
"thought": {
"id": "a1b2c3d4-...",
"index": 42,
"hash": "9f14...",
"thought_type": "Insight",
"content": "Memory deduplication triggers when Jaccard similarity exceeds threshold",
"agent_id": "assistant",
"importance": 0.7,
"confidence": 0.9,
"tags": ["feature:dedup"],
"concepts": ["memory-dedup", "jaccard-similarity"],
"created_at": "2026-04-14T12:00:00Z"
}
}REST Example: POST /v1/ranked-search
POST /v1/ranked-search HTTP/1.1
Host: 127.0.0.1:9471
Content-Type: application/json
{
"chain_key": "default",
"text": "how does memory deduplication work",
"scope": "user",
"limit": 10,
"enable_reranking": true,
"rerank_k": 50
}
// Response:
{
"results": [
{
"thought": {
"id": "a1b2c3d4-...",
"thought_type": "Insight",
"content": "Memory deduplication triggers when Jaccard similarity exceeds threshold",
"importance": 0.7
},
"score": 4.82,
"lexical_score": 1.2,
"vector_score": 3.1,
"session_cohesion": 0.4,
"rank": 1
}
],
"total": 1,
"query_time_ms": 12
}REST Example: compact ranked search
Over REST, ranked, federated and lexical search accept an opt-in compact: true (/v1/ranked-search, /v1/federated-search, /v1/lexical-search; CLI --compact on ranked and federated search). Over MCP, compact is the default and goes further (see Lean output on MCP). A compact hit is chain_key, the total score as a number and the thought without hash, prev_hash, thought_signature, signing_key_id, schema_version, agent_name, agent_owner and top-level null or empty fields; matched_terms, match_sources and the score components are left out. The provenance object, the version fields (version_status, successors, current_version, successor_of, collapsed_duplicates) and graph provenance are always kept, and the public/private boundary is applied before compaction. Fields are dropped by name from a fixed list, so any field added to a thought later is kept by default. Skip it when you need hashes or signatures to verify a thought or per-component scores; POST /v1/thought returns the full record of any hit.
POST /v1/ranked-search
{"chain_key": "project", "text": "job queue storage", "limit": 5, "compact": true}
// Full response (default)
{
"backend": "hybrid",
"total": 1,
"results": [
{
"chain_key": "project",
"score": {
"confidence": 0.0, "graph": 0.0, "importance": 0.17541419, "lexical": 1.9490464,
"recency": 0.05, "relation": 0.0, "rrf": 0.0, "seed_support": 0.0,
"session_cohesion": 0.0, "total": 5.932654, "vector": 0.34079674
},
"matched_terms": ["job", "queue", "storag"],
"match_sources": ["content", "tags", "concepts"],
"graph_distance": null, "graph_path": null,
"graph_relation_kinds": [], "graph_seed_paths": 0,
"thought": {
"agent_id": "builder", "agent_name": "Builder", "agent_owner": "platform",
"concepts": ["job queue"], "confidence": null,
"content": "Use Postgres for the job queue; revisit if throughput exceeds 5k jobs per second.",
"entity_type": null,
"hash": "31de8f264b1588a29be30a278a24e8623cf51b8f3ce8483a7e52f69ce80e21e1",
"id": "1182d44f-3834-4ff0-9121-eab986b1a4f5",
"importance": 0.800000011920929, "index": 0, "prev_hash": "",
"provenance": {
"segment": "public", "signature": "unsigned", "signature_version": null,
"signed": false, "trust": "untrusted_data", "verified_agent": null
},
"refs": [], "relations": [], "role": "Memory", "schema_version": 3,
"session_id": null, "signing_key_id": null, "source_episode": null,
"tags": ["storage"], "thought_signature": null, "thought_type": "Decision",
"timestamp": "2026-11-01T23:03:19.669167Z"
}
}
]
}
// Same request with "compact": true
{
"backend": "hybrid",
"total": 1,
"results": [
{
"chain_key": "project",
"score": 5.932654,
"thought": {
"agent_id": "builder",
"concepts": ["job queue"],
"content": "Use Postgres for the job queue; revisit if throughput exceeds 5k jobs per second.",
"id": "1182d44f-3834-4ff0-9121-eab986b1a4f5",
"importance": 0.800000011920929, "index": 0,
"provenance": {
"segment": "public", "signature": "unsigned", "signature_version": null,
"signed": false, "trust": "untrusted_data", "verified_agent": null
},
"role": "Memory", "tags": ["storage"], "thought_type": "Decision",
"timestamp": "2026-11-01T23:03:19.669167Z"
}
}
]
}REST Example: POST /v1/context-bundles
POST /v1/context-bundles HTTP/1.1
Host: 127.0.0.1:9471
Content-Type: application/json
{
"chain_key": "default",
"text": "memory deduplication decision",
"scope": "user",
"limit": 5
}
// Response:
{
"bundles": [
{
"seed": {
"id": "a1b2c3d4-...",
"thought_type": "Decision",
"content": "Enable dedup at 0.85 Jaccard threshold",
"score": 5.1
},
"supporting": [
{
"id": "b2c3d4e5-...",
"thought_type": "LessonLearned",
"content": "Previous attempts at 0.95 threshold produced too many false negatives",
"chain_key": "default"
}
],
"children": []
}
]
}REST Example: POST /v1/webhooks
POST /v1/webhooks HTTP/1.1
Host: 127.0.0.1:9472
Content-Type: application/json
{
"url": "https://myapp.example.com/mentisdb-webhook",
"chain_key_filter": "default",
"thought_type_filter": ["Decision", "Constraint"]
}
// Response:
{
"webhook": {
"id": "6f1c2a9e-...",
"url": "https://myapp.example.com/mentisdb-webhook",
"chain_key_filter": "default",
"thought_type_filter": ["Decision", "Constraint"],
"created_at": "2026-04-14T12:00:00Z",
"active": true
}
}Targets must be public http or https endpoints: loopback, private, link-local, CGNAT, documentation, multicast and IPv4-embedding IPv6 addresses are rejected, also after DNS resolution, and redirects are not followed. After 5 consecutive failed deliveries a webhook is paused for 5 minutes.
Self-hosted operators can open specific private ranges with MENTISDB_WEBHOOK_ALLOWED_CIDRS (e.g. a Tailscale tailnet); see Private webhook targets.
0.8.0 Search & Storage Improvements
MentisDB 0.8.0 introduces five major improvements to the search and storage pipeline: Porter stemming, tiered vector-lexical fusion, importance weighting, bincode hashing, and managed sidecar entries with auto_sync.
Porter Stemming (Normalizer v2)
The lexical tokenizer now applies English Porter stemming. Words like preferences → prefer and running → run now match in lexical search. Existing chains auto-reindex on first open — no manual migration step required.
Tiered Vector-Lexical Fusion
Ranked search scoring replaces the flat addition model with a tiered boost that respects the relationship between lexical and vector signals:
- Lexical = 0 and vector > 0:
vector × 60— full boost for semantic-only hits - 0 < lexical < 1.0 and vector > 0:
vector × (1 + 20 × fraction)— partial boost proportional to lexical overlap - Lexical ≥ 1.0: vector as-is — no boost, lexical dominates
Failed logins are rate-limited per client (an IPv4 address or IPv6 /64) with a global budget and backoff to prevent brute-force enumeration; the dashboard is always served over TLS so the PIN is never transmitted in the clear on a local network.
Automatic Thesaurus in Ranked Search (0.9.9)
The static thesaurus (~900 headwords + 300+ lemmas) now applies <strong>automatically by default</strong> to every ranked search query (REST <code>POST /v1/ranked-search</code>, MCP <code>mentisdb_ranked_search</code>, dashboard, and CLI). No client changes or extra parameters are required. With full embeddings this delivered the target LoCoMo-10P $R@10 = 72.6%. The expansion happens server-side via <code>apply_thesaurus_if_text</code> before BM25 scoring.
Invalidation filtering (default retrieval)
Thoughts that are the target of a later Supersedes, Corrects, or Invalidates relation are recorded in invalidated_thought_ids and excluded by default from MentisDb::query, query_ranked, context bundles, recent context, and related REST/MCP search. Pass include_invalidated: true (crate builders: with_include_invalidated(true)) for audit. Point-in-time as_of still returns thoughts that were valid at that timestamp.
Auto-dedup is off unless MENTISDB_DEDUP_THRESHOLD is set (recommended 0.85 for long-lived agent chains). Without dedup or agent-written invalidation edges, the set stays empty and search cannot hide noise.
0.12 Library Changes
0.12.2: source_episode, size cap, MCP keepalive
0.12.2 changes no public signature and needs no data migration. What changed:
source_episodeis stored.POST /v1/thoughtsand the MCPmentisdb_appendtool accepted the field but dropped it, so the thought came back withsource_episode: null. The append now stores it. It is also an exact-match filter on/v1/search,/v1/ranked-search,/v1/federated-searchand/v1/context-bundlesand on their MCP tools./v1/lexical-search, thought traversal, summary candidates and the MEMORY.md export do not take the filter yet. Library:ThoughtInput::with_source_episodesets it.- CLI:
--source-episode IDonthought append(stored) and onsearch,search filter,search federatedandsearch bundles(filter).thought traverse,thought candidates,memory exportandsearch lexicalrefuse the flag instead of sending a field the server would drop. - A thought record over 10 MiB is refused at append. The cap was enforced only when a chain was opened, so an oversized append was written and the next open failed. The append now returns
InvalidInput(HTTP 400) before anything is written. Ordinary REST and MCP bodies are capped at 2 MiB, so the way in was the library or the large-body import routes (/v1/import-markdown,/v1/extract-memories,/v1/skills/upload). - MCP event stream keepalive.
GET /on the MCP port sends an SSE comment (: keepalive) after 15 s with nothing else to send, so clients with an idle-body timeout (Claude Code's HTTP client) stop dropping and reconnecting. It comes fromcloudllm_mcp0.4.1, which mentisdb now requires. Embedders that build their own server can tune it withStreamableHttpConfig::with_sse_keepalive(Some(duration))or turn it off withwith_sse_keepalive(None)(a zero duration also disables it). - Markdown skill import reads standard YAML frontmatter: block scalars (
>,|) fordescriptionand block lists (- item) fortriggers,tagsandwarnings. A multi-line description is exported as a literal block. - The dashboard Settings screen lists every daemon setting; one catalog drives which names it shows and accepts and whether a change needs a restart.
0.12.1: live memories, agent key pairs
0.12.1 changes no public signature; it adds mentisdb::server_lean and PROVENANCE_REMINDER. What changed is how MentisDb::query_ranked fills a page, and how agent keys are made:
- Only live memories compete. Every match is scored once, a superseded match nominates its live successor, and the walk stops at
limitdistinct live memories. It used to rank a fixed2 * limitwindow that included superseded thoughts, so one decision rewritten 30 times turned alimit = 5search into one hit.total_candidatescounts live matches. Context-bundle seeds use the same walk. - Score totals are non-increasing in page order: a stand-in or contester that would outscore the hit before it reports a total just below it, so offset paging tiles the ranking.
successor_oflists the matched older versions met before the page filled: a lower bound on the lineage, not all of it.- The refill after near-duplicate collapsing resumes instead of restarting and is capped at
max(16 * limit, 4096)ranked slots, so a page whose distinct matches lie deeper comes back short. - Agent keys:
mentisdb agent key generate AGENT --key-id ID --chain KEY [--out FILE] [--force]creates an Ed25519 pair, writes the 64-hex seed to a 0600 file and sends the operatorregister_keyoperation with only the public key (--json:registered,agent_id,key_id,chain_key,public_key_hex,key_file). The dashboard's Generate key pair builds the samementisdb/key-registration/v1proof of possession in the browser and posts only the public key toPOST /dashboard/api/agents/{chain}/{agent}/keys; Add Key now uses the operator route. Both dashboard key routes (add and delete) need the dashboard PIN and answer 403 without one. - Security:
MentisDb::add_agent_key(the proof-based path) no longer replaces a key filed under an existing key id or revives a revoked one. A different key under an existing id is refused; the same active key again is a no-op, as operator registration already behaved. Rotate with a new key id. MentisDb::query_federatedstill cuts each chain at its query'slimitbefore redirecting and removing duplicates; the server's federated search widens the fetch up tomax(16 * (offset + limit), 4096)per chain.
MentisDB 0.12.0 is a minor-version release because public library types changed. Sidecar files, their digests, persisted HNSW graphs, the wire protocol and the CLI are unchanged.
Breaking: one flat vector store (VectorRow)
Every vector of a managed sidecar now lives once, in 128-row, 64-byte-aligned chunks shared by the sidecar entries, the exact index, the HNSW graph and the implicit-edge scanner. Exact search runs a lane-parallel dot product over it (about 4.3x faster at 100,000 vectors; scores within 1e-5 of the old cosine, so near-ties may swap; the HNSW and implicit-edge paths keep the sequential cosine and can score a pair up to that much apart from exact search). The exact index reuses removed slots and compacts once dead rows outnumber live ones, so it does not grow under churn. Signature changes:
VectorSidecarEntry::vectoris amentisdb::search::flat_store::VectorRow(dereferences to&[f32]), not aVec<f32>. Use.to_vec()for aVecandVectorRow::standalone(&values)to make one.VectorSidecarEntry::newtakesimpl Into<VectorRow>:Vec<f32>and&[f32]convert, but a bare.collect()whose type is only inferred no longer compiles; annotate it.VectorSidecarhas a private field: build it withVectorSidecar::build, not a struct literal.- New:
search::flat_store(VectorStore,VectorRow,dot,DOT_EPSILON),VectorIndex::{from_rows, upsert_row},VectorSearchBackend::upsert_row(provided, implementors keep compiling) andHnswBackend::{from_rows, share_cached_rows}.
use mentisdb::search::VectorSidecarEntry;
use mentisdb::search::flat_store::VectorRow;
let entry = VectorSidecarEntry::new(id, index, hash, vec![0.1, 0.2, 0.3]); // unchanged
let owned: Vec<f32> = entry.vector.to_vec(); // 0.11: entry.vector.clone()
let slice: &[f32] = &entry.vector; // unchanged
let values: Vec<f32> = (0..3).map(|i| i as f32).collect(); // annotate inferred types
let entry2 = VectorSidecarEntry::new(id, index, hash, values);
let literal = VectorSidecarEntry {
thought_id: id, thought_index: index, thought_hash: hash.to_string(),
vector: VectorRow::standalone(&[0.1, 0.2, 0.3]), // 0.11: vec![0.1, 0.2, 0.3]
};entry.vector.clone() still compiles but now returns a VectorRow (a pointer copy), not a Vec<f32>.
Breaking: MentisDbServiceConfig is #[non_exhaustive]
It can no longer be built with a struct literal; the fields stay readable and assignable. Build it with MentisDbServiceConfig::new(..) and the with_* builders. The body-admission settings are with_max_body_bytes (import routes, clamped to 1 MiB..1 GiB), with_max_inflight_body_bytes and with_body_idle_timeout.
use mentisdb::server::MentisDbServiceConfig;
use mentisdb::StorageAdapterKind;
use std::time::Duration;
let config = MentisDbServiceConfig::new("/var/lib/mentisdb".into(), "project", StorageAdapterKind::Binary)
.with_verbose(true)
.with_max_body_bytes(128 * 1024 * 1024)
.with_max_inflight_body_bytes(512 * 1024 * 1024)
.with_body_idle_timeout(Duration::from_secs(60))
.with_min_body_bytes_per_sec(16 * 1024); // 0 turns the rate floor offRequest body admission
An outermost layer (server_body) decides before reading a body: the bearer token is checked from the headers first; ordinary routes and every MCP request take at most SMALL_BODY_BYTES (2 MiB); only the routes in LARGE_BODY_ROUTES take up to the configured cap, from an authenticated caller. Imports are reserved from their declared length against the in-flight budget (2x the cap and LARGE_BODIES_PER_PEER (4) bodies per peer). Other authenticated requests are charged per arriving byte from SMALL_POOL_BYTES (64 MiB, SMALL_PEER_BYTES = 32 MiB per peer); anonymous requests (bearer on, no valid token) from their own ANONYMOUS_POOL_BYTES (16 MiB, ANONYMOUS_PEER_BYTES = 8 MiB per peer). Small and anonymous bodies have a total deadline of twice the idle timeout, and every body must reach the configured minimum rate (with_min_body_bytes_per_sec, MENTISDB_MIN_BODY_BYTES_PER_SEC, default MIN_BODY_BYTES_PER_SEC = 64 KiB/s, 0 = off) in every window of one idle timeout after a grace period of twice the idle timeout; each window counts only its own bytes, so credit cannot be banked. Too slow or stalled is 408, an exhausted budget 503 with Retry-After, on every route (the layer replaces a handler's 400 when it was the one that cut the body off). Read proofs that send X-Mentis-Content-Sha256 are verified before the body is read and the body is hashed as it streams.
Derived-state backlog
MentisDb::derived_backlog returns a DerivedBacklog summing a chain's managed sidecars: embeddings_in_flight (requested, not committed), embeddings_parked (ready, waiting for an earlier thought's vector), embeddings_missing (waiting to be embedded again) and implicit_edges (applied vectors whose edges have not reached the overlay). All are zero when the chain is caught up. GET /v1/chains and the MCP chain listing report it per chain as derived_backlog, with zeros for a chain that is not open or whose private segments are hidden from the caller.
"derived_backlog": {
"embeddings_in_flight": 0, "embeddings_parked": 0,
"embeddings_missing": 0, "implicit_edges": 312
}The implicit-edge queue holds positions, not vectors, so a backlog never grows memory by more than a few bytes, never blocks an append and never drops an edge. The scan sustains about 155 appends per second at 100,000 thoughts and 720 at 20,000 (4 threads, 384 dimensions); faster appends make implicit_edges grow, and it drains when they slow down.
0.11 Library Changes
MentisDB 0.11.0 changes what default retrieval returns, adds public and private chains, and moves format upgrades into a background migration manager. Full rustdoc is on docs.rs; the design of the access model is in docs/CHAIN_ACCESS.md.
Valid-version retrieval (module lineage)
Lineage is derived when a chain opens; no migration is needed. A superseded thought that would have ranked is replaced by its live successor (RankedSearchHit::version.successor_of names what it stands in for). A thought that a newer Correction, AssumptionInvalidated or Decision only references, or that anything Contradicts, stays visible as VersionStatus::Contested with the contesting thought promoted above it. New: MentisDb::{version_status, current_version, version_info, hit_json, append_advice}, VersionStatus, AppendAdvice and lineage::cross_chain_successor for a relation that carries a chain_key (applied wherever one read path sees both chains: branch search and federated search). Every thought JSON carries version_status, successors and current_version when it is not plainly current.
use mentisdb::{RankedSearchQuery, VersionStatus};
let hits = chain.query_ranked(&RankedSearchQuery::new().with_text("release policy Fridays"));
// The live successor is returned in place of the superseded policy.
assert_eq!(hits.hits[0].version.successor_of, vec![old.id]);
assert_eq!(chain.version_status(old.id), VersionStatus::Superseded);RankedSearchQuery defaults changed
RankedSearchQuery::new() now applies relative vector fusion, near-duplicate collapse at Jaccard 0.9 (text queries), successor redirection and thesaurus expansion at weight 0.3 (the server default; the library used to apply none). Result lists and total_candidates differ from 0.10. Opt out per query:
use mentisdb::{RankedSearchQuery, VectorFusion};
let query = RankedSearchQuery::new()
.with_text("deploy window")
.with_vector_fusion(VectorFusion::Absolute) // the 0.10 blend
.with_collapse_duplicates(None) // keep near-duplicates
.with_resolve_successors(false) // plain hiding of superseded thoughts
.with_thesaurus(false); // literal terms onlyProcess-wide: MENTISDB_VECTOR_FUSION=absolute, MENTISDB_COLLAPSE_DUPLICATES=off. with_df_scope and with_df_cutoffs choose how the lexical stop-word gate measures document frequency (DfScope::Global stays the default). Over REST and MCP the same knobs are request fields: resolve_successors, collapse_duplicates (0 disables), vector_fusion, synonym_weight (0 disables), df_scope and df_cutoffs.
Standing context (module standing)
MentisDb::standing_context(&StandingContextQuery) returns the durable half of memory regardless of age: live Constraint, PreferenceUpdate, UserTrait, RelationshipUpdate, Goal, Decision and LessonLearned thoughts, ranked by importance then recency, restatements collapsed, capped at 8 per type and 40 in total, plus the newest Handoff and Checkpoint/Summary. The StandingContext result has entries, duplicates_collapsed and a ready-to-load prompt. A superseded constraint or decision is replaced by its successor.
Public and private chains
mentisdb::audience::{Audience, Segment, SegmentMap}: every read path takes an audience.ThoughtQuery::audience,StandingContextQuery::audienceand*_for(audience)variants (get_thought_for,thought_json_for,hit_json_for,append_advice_for, ...). The library default isAudience::Authorized: a program that holds the data directory holds every byte.mentisdb::read_auth::{sign_request, ReadSigner}build theX-Mentis-Authread-proof header for HTTP clients. To send a body over 1 MiB also sendX-Mentis-Content-Sha256(body_digest,content_digest_header_value): the server verifies the proof from the declared digest before reading the body and hashes the body as it streams, failing the request at its end on a mismatch, before any handler has run.mentisdb::operator::applyruns operator commands (mode changes, allowed keys, key registry, agent status) signed with<data dir>/operator/operator.key;MentisDb::add_agent_key_as_operatorregisters a key the operator vouches for.MentisDb::add_agent_keystill requires a&KeyRegistrationProof(proof of possession) and a valid curve point.MentisDb::verificationreports the open-time check of governed chains (quarantined thoughts, rejected policy records).AccessError::StaleHeadis returned for a v2-signed append built on an older head (fetch the head and sign again); a forged signature staysBadSignature.- A stored zero-length signature is a placeholder and counts as unsigned everywhere; new writes normalize an empty signature to none.
Thought::signature_bytesreturns the real signature or nothing.
Storage and validation
mentisdb::validate_chain_keyguards every entry point (library open, branch, restore): non-empty, at most 256 bytes, no control characters.MentisDb::defer_next_embeddinglets a caller embed outside the chain's write lock, as the daemon now does for every append (lock hold per append about 0.5 ms instead of about 9.5 ms with a 5 ms model). Library callers that do not opt in are unchanged.- The daemon migrates legacy chains in the background through a journaled, backed-up manager. A library user that opens a legacy chain directly keeps the in-place upgrade, now with a one-time copy of the original under
migrations/backup/in-place/. A chain or data directory written by a newer release is refused with a clear message. - Chains from March and April 2026 that stamp schema 4 on schema-3 records open again: schema 4 is an accepted historical alias, so the next thought schema is 5.
Deprecated
RankedSearchQuery::{llm_rerank, llm_model, llm_rerank_top_k, with_llm_reranking} and RankedSearchScore::llm_score are no-ops (LLM reranking was never implemented), and search::ppr is never called by ranked search. Both are marked #[deprecated] and will be removed in the next breaking release.
Benchmarks
Full LoCoMo-10P and LongMemEval protocol run on the 0.11 defaults (fresh ingest, fastembed-minilm): LoCoMo-10P R@10 76.7% (73.5% in 0.10.9.54), LongMemEval R@5 69.8% (68.0%). See results/benchmarks/2026-11-01-full-protocol.
Benchmarking
MentisDB ships with two benchmark styles:
- Criterion microbenchmarks for in-process append, query, traversal, import, and skill-registry hot paths
- HTTP concurrency benchmarks for live
mentisdbwrite/read waves under concurrent client load
Run the full benchmark suite with:
make benchThis currently runs cargo bench and saves stdout to /tmp/mentisdb_bench_results.txt.
Durable vs Buffered Write Mode
Benchmark write-heavy changes in both persistence modes. They now behave differently enough that one run is not representative of the other:
MENTISDB_BENCH_AUTO_FLUSH=true— durable mode. Appends are queued to the bounded background writer and the request only returns after the writer flushes them. Concurrent appends may share a short group-commit window; a lone append flushes immediately. Since 0.10.9 commits are fsynced only with MENTISDB_FSYNC=1, so benchmark both settings when touching the writer.MENTISDB_BENCH_AUTO_FLUSH=false— buffered mode. Appends are queued and acknowledged before the worker flushes them, trading durability for higher write throughput.
MENTISDB_BENCH_AUTO_FLUSH=true cargo bench --bench http_concurrency
MENTISDB_BENCH_AUTO_FLUSH=false cargo bench --bench http_concurrencyCriterion Baselines
Criterion benchmarks such as thought_chain and skill_registry automatically compare the current run against the saved baseline in target/criterion/. That is why the output includes messages like Performance has improved, Performance has regressed, or No change in performance detected.
time: [low mid high]— lower is betterthrpt: [low mid high]— higher is betterchange— percentage delta versus the saved baselinep < 0.05— the change is statistically meaningful
If you have changed the benchmark harness itself or want a fresh comparison, clear the old Criterion baseline before trusting the regression/improvement messages.
HTTP Concurrency Baselines
The custom http_concurrency benchmark now persists its own baselines under target/http_concurrency/ and prints delta tables on later runs. This means you no longer need to manually compare the Markdown output tables by eye.
MENTISDB_BENCH_AUTO_FLUSH=true MENTISDB_BENCH_BASELINE=durable cargo bench --bench http_concurrency
MENTISDB_BENCH_AUTO_FLUSH=false MENTISDB_BENCH_BASELINE=buffered cargo bench --bench http_concurrencyUseful environment variables:
MENTISDB_BENCH_CONCURRENCY— comma-separated client counts such as100,1000,10000MENTISDB_BENCH_AUTO_FLUSH—truefor durable group commit,falsefor buffered throughput modeMENTISDB_BENCH_BASELINE— names the saved baseline file so you can keep separate durable, buffered, nightly, or branch-specific histories
How to Read the HTTP Table
wall_ms— total time for the whole wavereq/s— throughputp50_ms,p95_ms,p99_ms— median and tail latencyerrors— non-2xx responses or transport failures
For write-path work, focus on the Write — POST /v1/thoughts table at high concurrency. Read-path changes should primarily move the Read — POST /v1/head table and the query/traversal Criterion benches.
What to Benchmark After Storage Changes
cargo bench --bench thought_chain append_single -- --noplot— strict append latencycargo bench --bench thought_chain query_by_tag -- --noplot— indexed query pathcargo bench --bench thought_chain traverse_filtered_miss_10 -- --noplot— full-scan traversal miss pathcargo bench --bench http_concurrency— live daemon concurrency behavior
A useful workflow is to measure a clean baseline on master, apply the storage change, rerun the same focused benches, and only then trust the wider make bench output.
Contributing
MentisDB is open source under the MIT license.
git clone https://github.com/cloudllm-ai/mentisdbRun the test suite:
cargo testRun benchmarks:
make bench