Developer Documentation

Overview

MentisDB is a standalone Rust crate available on crates.io. Add it to your project:

mentisdb = "0.12.2"

The minimum supported Rust version is 1.90 (rust-version in the crate manifest; CI checks it).

The crate provides the full memory engine, skill registry, storage adapter interface, agent registry, and optional HTTP server stack (MCP + REST + dashboard) behind the server feature flag.

Full API Reference

The complete, generated API documentation is hosted on docs.rs:

docs.rs/mentisdb

Full rustdoc for all public types, traits, functions, and modules. Includes MentisDb, Thought, ThoughtInput, ThoughtQuery, StorageAdapter, SkillRegistry, BinaryStorageAdapter, and the HTTP server.

View on docs.rs →

MCP Server

MentisDB ships a built-in MCP (Model Context Protocol) HTTP server. Enable it with the server feature. All MentisDB operations are exposed as MCP tools over JSON-RPC 2.0 via HTTP, making it compatible with any MCP-capable AI tool. The canonical onboarding path is not a separate URL: it happens during the MCP initialize call via startup instructions plus the embedded resource mentisdb://skill/core exposed through resources/list and resources/read.

Default HTTP endpoint: http://127.0.0.1:9471 — HTTPS endpoint: https://127.0.0.1:9473.

Lean output on MCP

MCP tool results are read by a language model, so the streamable-HTTP MCP surface (JSON-RPC tools/call, and stdio mode, which proxies to it) shapes them for its context window. REST, POST /tools/execute and the CLI keep the full response. Since 0.12.1:

  • Results are minified JSON text, with no second structuredContent copy. No tool declares an outputSchema, so read the text content; it is one JSON document.
  • mentisdb_ranked_search, mentisdb_federated_search, mentisdb_lexical_search and mentisdb_context_bundles default to compact: true and leave out defaults: no provenance is a public, unsigned thought (each response says so once in default_provenance; a thought that arrives without provenance is marked {"unknown": true}), no role is Memory, no importance is 0.5, no chain_key is the chain searched. Floats are rounded, timestamps keep full precision. successor_of is one older version plus a superseded_versions count (a lower bound), and successor_kind keeps the kind of the link it replaces (absent means Supersedes).
  • compact: false returns the full record. A compact or include_entries that is not a boolean is refused with JSON-RPC -32602.
  • mentisdb_standing_context returns prompt and duplicates_collapsed; the entries array needs include_entries: true.

On a generated 3,500-thought chain a 10-hit ranked search costs 5,948 characters instead of 23,657, and standing context 13,050 instead of 36,191 (cargo run --release --example token_diet). The rules are in the mentisdb::server_lean rustdoc.

All 45 MCP Tools

ToolDescriptionKey Parameters
mentisdb_bootstrapCreate a chain if needed and write one bootstrap checkpoint when it is emptychain_key,content,concepts,importance
mentisdb_appendAppend a durable semantic thought with optional tags, concepts, refs, scope, and signature metadatachain_key,thought_type,content,agent_id,tags,concepts,scope,importance,confidence
mentisdb_append_retrospectiveAppend a guided retrospective memory to prevent future agents from repeating a hard failurechain_key,content,concepts,refs,importance
mentisdb_searchSearch thoughts by semantic filters, identity filters, time bounds, and scoring thresholdschain_key,text,thought_types,roles,agent_ids,since,until,min_importance,min_confidence
mentisdb_lexical_searchReturn flat ranked lexical matches with explainable term and field provenancechain_key,text,thought_types,limit,offset,compact
mentisdb_ranked_searchReturn flat ranked results with lexical, graph-aware, or heuristic score breakdowns. Supports point-in-time queries and memory scope filteringchain_key,text,as_of,scope,enable_reranking,rerank_k,concepts_any,roles,limit,compact
mentisdb_federated_searchRanked retrieval over several chains at once, merged into one list. Each chain is authorized and filtered for the caller on its ownchain_keys,text,limit,graph,as_of,include_invalidated,compact
mentisdb_context_bundlesReturn seed-anchored grouped support context beneath the best lexical seedschain_key,text,as_of,scope,graph,limit,compact(MCP only)
mentisdb_list_chainsList known chains with version, storage adapter, counts, and storage locationchain_key
mentisdb_migration_statusReport the background storage-format migration state of every chain that is, was, or failed to be migrated: phase, percent done, attempts and error. Global token only
mentisdb_merge_chainsMerge all thoughts from a source chain into a target chain, then permanently delete the sourcesource_chain_key,target_chain_key
mentisdb_branch_fromCreate a new chain diverging from a thought on an existing chain. Searches on the branch transparently include ancestor resultssource_chain_key,branch_chain_key,branch_thought_id
mentisdb_list_agentsList the distinct agent identities participating in one chainchain_key
mentisdb_get_agentReturn one full agent registry record including status, aliases, description, keys, and per-chain activity metadataagent_id,chain_key
mentisdb_list_agent_registryReturn the full per-chain agent registrychain_key
mentisdb_upsert_agentCreate or update a registry record before or after an agent writes thoughtsagent_id,display_name,agent_owner,description,status,chain_key
mentisdb_set_agent_descriptionSet or clear the description stored for one registered agentagent_id,description,chain_key
mentisdb_add_agent_aliasAdd a historical or alternate alias to a registered agentagent_id,alias,chain_key
mentisdb_get_access_policyReturn the chain's access policy as the caller may see it: mode, sequence, key-set commitment and segment boundaries for everyone; the allowed keys and record hashes only for a reader who proved an authorized keychain_key
mentisdb_verify_signaturesRe-verify the signed thoughts and governance records the caller may see; per-thought status (valid, unsigned, invalid), signature version and totalschain_key
mentisdb_standing_contextRender durable, age-independent memory (constraints, preferences, user traits, goals, decisions, lessons, newest handoff and checkpoint) as a prompt block. Call at session start before mentisdb_recent_contextchain_key,agent_id,types,since,as_of,include_invalidated,per_type_limit,limit,include_entries(MCP only)
mentisdb_recent_contextRender recent thoughts into a prompt snippet for session resumptionchain_key,last_n
mentisdb_memory_markdownExport a MEMORY.md-style Markdown view of the full chain or a filtered subsetchain_key,agent_ids,thought_types,roles,since,until,limit
mentisdb_import_memory_markdownImport a MEMORY.md-formatted Markdown document into a target chainmarkdown,default_agent_id,chain_key
mentisdb_get_thoughtReturn one stored thought by stable id, chain index, or content hashthought_id,thought_index,thought_hash,chain_key
mentisdb_get_genesis_thoughtReturn the first thought ever recorded in the chain, if anychain_key
mentisdb_traverse_thoughtsTraverse the chain forward or backward in append order from a chosen anchor, in chunks, with optional filterschain_key,anchor_id,anchor_index,anchor_hash,direction,chunk_size,thought_types,roles,since,until
mentisdb_skill_mdReturn the official embedded MENTISDB_SKILL.md Markdown file(none)
mentisdb_list_skillsList versioned skill summaries from the skill registrychain_key
mentisdb_skill_manifestReturn the versioned skill-registry manifest including searchable fields and supported formats(none)
mentisdb_upload_skillUpload a new immutable skill version from Markdown or JSONagent_id,skill_id,content,format,signing_key_id,skill_signature,chain_key
mentisdb_search_skillSearch skills by indexed metadata such as ids, names, tags, triggers, uploader identity, status, format, schema version, and time windowchain_key,text,skill_ids,names,statuses,since,until,limit
mentisdb_read_skillRead one stored skill as Markdown or JSON. Responses include trust warnings for untrusted or malicious skill contentskill_id,version_id,format,chain_key
mentisdb_skill_versionsList immutable uploaded versions for one skillskill_id,chain_key
mentisdb_deprecate_skillMark a skill as deprecated while preserving all prior versionsskill_id,reason,chain_key
mentisdb_revoke_skillMark a skill as revoked while preserving audit historyskill_id,reason,chain_key
mentisdb_delete_skillPermanently remove a skill and all of its versions. After delete the same skill_id can be uploaded again. Prefer revoke when you need an audit row.skill_id,chain_key
mentisdb_headReturn head metadata, the latest thought at the current chain tip, and integrity statechain_key
mentisdb_summary_candidatesSuggest groups of thoughts that are candidates for summarizationchain_key,text,limit
mentisdb_register_webhookRegister a webhook to receive HTTP POST notifications when thoughts are appended. Delivery is fire-and-forget with exponential backoff (up to 3 attempts)url,chain_key_filter,thought_type_filter
mentisdb_list_webhooksList all registered webhookschain_key
mentisdb_delete_webhookRemove a webhook registration by its UUIDid
mentisdb_list_entity_typesList the entity types registered on a chainchain_key
mentisdb_upsert_entity_typeCreate or update an entity type in the per-chain registrychain_key,entity_type
mentisdb_extract_memoriesTurn free-form text into typed memory candidates with the configured LLM. Nothing is appended; needs the llm-extraction featuretext,chain_key,agent_id,prompt_template

Since 0.11, registering or revoking agent keys, disabling agents and changing a chain's access policy are operator-only: mentisdb_add_agent_key, mentisdb_revoke_agent_key, mentisdb_disable_agent and mentisdb_set_access_policy are no longer offered, and a status change through mentisdb_upsert_agent is refused (restating the current status is still accepted). The operator uses the dashboard behind its PIN or the local mentisdb command line; see Public & Private Chains. A tool call against a chain that is still being migrated returns a tool error with the retry delay and percent done; retry after it.

JSON-RPC Request/Response Example

MCP uses JSON-RPC 2.0 over HTTP. Send a POST to the MCP endpoint:

POST / HTTP/1.1
Host: 127.0.0.1:9471
Content-Type: application/json

{
"jsonrpc": "2.0",
"id": 1,
"method": "tools/call",
"params": {
"name": "mentisdb_append",
"arguments": {
"chain_key": "default",
"agent_id": "assistant",
"thought_type": "Insight",
"content": "Memory deduplication triggers when Jaccard similarity exceeds threshold",
"importance": 0.7,
"confidence": 0.9,
"tags": ["feature:dedup"],
"concepts": ["memory-dedup", "jaccard-similarity"]
}
}
}

// Response:
{
"jsonrpc": "2.0",
"id": 1,
"result": {
"content": [
{
"type": "text",
"text": "Thought appended: a1b2c3... (index 42)"
}
]
}
}

The legacy CloudLLM endpoint POST /tools/execute takes tool + parameters; JSON-RPC tools/call takes name + arguments. Authorization checks exactly the fields the transport executes, and a payload that mixes both shapes is rejected with 400.

MCP HTTP Ports

The MCP server listens on two ports:

  • http://127.0.0.1:9471 — HTTP (unencrypted), suitable for local development
  • https://127.0.0.1:9473 — HTTPS (TLS), recommended for production

The daemon binary (mentisdb) starts the MCP server automatically. For embedding in your own Axum app, see the docs.rs API reference for server module details.

REST API

The same daemon also serves a REST API alongside MCP. Key endpoints:

MethodPathDescription
POST/v1/thoughtsAppend a new thought to a chain. Supports scope parameter for memory scope tagging and source_episode (0.12.2) for provenance
POST/v1/searchSearch/query thoughts. Supports as_of for point-in-time queries, scope for scope filtering and an exact-match source_episode filter (0.12.2)
POST/v1/lexical-searchRanked lexical search with scores and matched-term diagnostics. Pass compact: true to drop match_sources and the thought's integrity fields
POST/v1/ranked-searchCanonical flat ranked retrieval with lexical + vector + graph-aware score breakdowns (hybrid when managed sidecars are available). Supports as_of, scope, enable_reranking, and rerank_k parameters, an exact-match source_episode filter (0.12.2), and the ranking knobs resolve_successors, collapse_duplicates, vector_fusion, synonym_weight, df_scope and df_cutoffs (bad values are a 400). Pass compact: true for smaller hits
POST/v1/context-bundlesSeed-anchored grouped support context for agent reasoning and dashboard inspection. Supports as_of and scope parameters, and an exact-match source_episode filter (0.12.2)
GET/v1/chainsList available chain keys; each chain summary carries derived_backlog (0.12)
POST/v1/chains/branchCreate a branch chain diverging from a thought on an existing chain
POST/v1/agentsList agents in a chain or inspect the registry
GET/mentisdb_skill_mdCompatibility fallback for clients that cannot use MCP resources; MCP-native clients should read mentisdb://skill/core after initialize
POST/v1/skills/uploadUpload a skill version
POST/v1/skills/readRead a skill (latest or specific version)
POST/v1/recent-contextRender a compact resumption prompt for recent, important context
POST/v1/standing-contextDurable, age-independent memory as a prompt block plus typed entries (entries, duplicates_collapsed, prompt). Load at session start before recent context. Over MCP, entries need include_entries: true
POST/v1/memory-markdownExport a chain as MentisDB-flavored MEMORY.md markdown
POST/v1/headReturn current chain head metadata and the latest thought
POST/v1/federated-searchRanked search fused across several chains (chain_keys). Accepts compact: true and an exact-match source_episode filter (0.12.2)
POST/v1/summary-candidatesSuggest groups of thoughts that are candidates for summarization
POST/v1/extract-memoriesTurn raw text into typed memory candidates with the configured LLM
POST/v1/entity-typesList entity types defined on a chain
POST/v1/entity-types/upsertCreate or update an entity type on a chain
GET/v1/webhooksList registered webhooks
POST/v1/webhooksRegister a webhook (url, optional chain_key_filter, optional thought_type_filter)
DELETE/v1/webhooks/{id}Delete a webhook by id
GET/healthLiveness health check
POST/v1/bootstrapBootstrap a chain with an initial memory
POST/v1/retrospectivesAppend a guided retrospective thought
POST/v1/thoughtRetrieve a single thought by ID, hash, or index
POST/v1/thoughts/genesisRetrieve the first thought in a chain
POST/v1/thoughts/traverseTraverse thoughts in append order with filtering and pagination
POST/v1/import-markdownImport MEMORY.md-formatted markdown into a chain
GET/v1/skillsList uploaded skill summaries
GET/v1/skills/manifestReturn the skill registry manifest with searchable fields
POST/v1/skills/searchSearch the versioned skill registry by indexed fields
POST/v1/skills/versionsList immutable versions for a stored skill
POST/v1/skills/deprecateMark a stored skill as deprecated
POST/v1/skills/revokeMark a stored skill as revoked
POST/v1/skills/deletePermanently delete a skill and all versions so the skill_id can be reused
POST/v1/agentGet a single agent record by ID
POST/v1/agent-registryList the full agent registry for a chain
POST/v1/agents/upsertCreate or update an agent record
POST/v1/agents/descriptionSet or clear an agent's description
POST/v1/agents/aliasesAdd an alias to an agent
POST/v1/agents/keysAnswers 403 (operator-only since 0.11): register keys from the dashboard (Generate key pair, Add Key) or with mentisdb agent key generate / agent key add
POST/v1/agents/keys/revokeAnswers 403 (operator-only since 0.11): mentisdb agent key revoke
POST/v1/agents/disableAnswers 403 (operator-only since 0.11): mentisdb agent disable / activate
POST/v1/vectors/rebuildRebuild vector sidecar indexes for a chain
POST/v1/chains/mergeMerge all thoughts from a source chain into a target, then delete the source. Refused for a source with private segments, governance records or signed thoughts
POST/v1/access/policyA chain's access policy as the caller may see it: mode, sequence, key-set commitment and segment boundaries; keys and record hashes only for an authorized reader
POST/v1/access/verifyRe-verify the signatures and governance records the caller may see
POST/v1/access/policy/setAnswers 403 (operator-only since 0.11): mentisdb chain access set|allow|deny
GET/v1/migrationsBackground migration state of every chain (global token)
POST/v1/migrations/retryClear a chain's migration quarantine so it is migrated again; body chain_key (global token)
POST/v1/migrations/skipLeave a chain unmigrated and unavailable until retried; body chain_key (global token)

Chain Resolution and Error Statuses

Only writes create a chain whose chain_key does not exist yet: /v1/bootstrap, /v1/thoughts, /v1/retrospectives, /v1/import-markdown, /v1/agents/upsert, /v1/entity-types/upsert and the target of /v1/chains/merge. Every other route returns 404 for an unknown chain and persists nothing; the configured default chain is the only one created on demand for reads. Chain keys must be non-empty, at most 256 bytes and free of control characters (400 otherwise). The source, target and branch keys of /v1/chains/merge and /v1/chains/branch are trimmed of surrounding whitespace before that check. The MCP tools behave the same way.

StatusMeaning
400Invalid input: malformed JSON, bad UUID, unknown thought type or role, invalid chain key, an unknown ranking-knob value, a v2-signed append built on an older head (fetch the head and sign again), a result window (offset + limit) above 10,000 on ranked, lexical or federated search or context bundles, or a standing-context per_type_limit above 100
401Missing or invalid bearer token, a token whose scope does not cover the operation, or an X-Mentis-Auth read proof that is present but invalid (bad signature, clock more than 60 s off, reused nonce)
403Permission denied: missing or invalid signature on a signed skill upload, a write the chain's access policy does not accept (the body does not say whether the agent or key exists), or an operator-only route
404Unknown chain or other missing resource
408A request body stopped arriving for MENTISDB_BODY_IDLE_TIMEOUT_SECS, a small or anonymous body missed its total deadline (twice that), or any body fell below MENTISDB_MIN_BODY_BYTES_PER_SEC (64 KiB/s by default) in a window after its grace period (twice the idle timeout); on every route
409Conflict, e.g. branching into a chain key that already exists
413Request body over the route limit (2 MiB; MENTISDB_MAX_BODY_BYTES, default 64 MiB, on the import routes for an authenticated caller), or over 1 MiB for an X-Mentis-Auth read proof that does not also send X-Mentis-Content-Sha256 (the SHA-256 the proof signs; the CLI and pymentisdb send it, so a large memory import works)
500Server fault; the body says only "internal server error" and details go to the daemon log
501/v1/extract-memories on a build without the llm-extraction feature
503The chain is being migrated (Retry-After header, retry_after_secs and percent done in the body) or failed to migrate (the body names the fix, e.g. mentisdb migrate retry --chain KEY); also, with Retry-After, when the in-flight body budget is exhausted

With bearer-token access enabled, GET /v1/chains, GET /v1/webhooks, DELETE /v1/webhooks/{id}, /v1/skills/{upload,deprecate,revoke,delete} and /v1/admin/flush require a global token; POST /v1/webhooks with a chain-scoped token requires a chain_key_filter inside its scope.

For MCP-native agents, prefer the streamable HTTP root at POST / and let the agent bootstrap itself from the initialize instructions and resource catalog.

For direct crate users, 0.8.0 keeps vector sidecars additive and rebuildable, but ranked search now blends lexical, graph, and managed-sidecar vector signals automatically when sidecars are enabled. The append-only chain remains canonical; sidecars are derived acceleration state.

The daemon now applies persisted managed-vector settings whenever it opens a chain. By default each chain gets the built-in local-text provider (local-text-v1), and ranked search across REST, MCP, and dashboard surfaces upgrades to hybrid scoring when that sidecar is available.

REST Example: POST /v1/thoughts (Append Thought)

POST /v1/thoughts HTTP/1.1
Host: 127.0.0.1:9471
Content-Type: application/json

{
"chain_key": "default",
"agent_id": "assistant",
"thought_type": "Insight",
"content": "Memory deduplication triggers when Jaccard similarity exceeds threshold",
"importance": 0.7,
"confidence": 0.9,
"tags": ["feature:dedup"],
"concepts": ["memory-dedup", "jaccard-similarity"],
"scope": "user",
"relations": [
{ "kind": "ContinuesFrom", "target_id": "<uuid-of-prior-turn>" }
]
}

// Response:
{
"thought": {
"id": "a1b2c3d4-...",
"index": 42,
"hash": "9f14...",
"thought_type": "Insight",
"content": "Memory deduplication triggers when Jaccard similarity exceeds threshold",
"agent_id": "assistant",
"importance": 0.7,
"confidence": 0.9,
"tags": ["feature:dedup"],
"concepts": ["memory-dedup", "jaccard-similarity"],
"created_at": "2026-04-14T12:00:00Z"
}
}

REST Example: POST /v1/ranked-search

POST /v1/ranked-search HTTP/1.1
Host: 127.0.0.1:9471
Content-Type: application/json

{
"chain_key": "default",
"text": "how does memory deduplication work",
"scope": "user",
"limit": 10,
"enable_reranking": true,
"rerank_k": 50
}

// Response:
{
"results": [
{
"thought": {
"id": "a1b2c3d4-...",
"thought_type": "Insight",
"content": "Memory deduplication triggers when Jaccard similarity exceeds threshold",
"importance": 0.7
},
"score": 4.82,
"lexical_score": 1.2,
"vector_score": 3.1,
"session_cohesion": 0.4,
"rank": 1
}
],
"total": 1,
"query_time_ms": 12
}

REST Example: compact ranked search

Over REST, ranked, federated and lexical search accept an opt-in compact: true (/v1/ranked-search, /v1/federated-search, /v1/lexical-search; CLI --compact on ranked and federated search). Over MCP, compact is the default and goes further (see Lean output on MCP). A compact hit is chain_key, the total score as a number and the thought without hash, prev_hash, thought_signature, signing_key_id, schema_version, agent_name, agent_owner and top-level null or empty fields; matched_terms, match_sources and the score components are left out. The provenance object, the version fields (version_status, successors, current_version, successor_of, collapsed_duplicates) and graph provenance are always kept, and the public/private boundary is applied before compaction. Fields are dropped by name from a fixed list, so any field added to a thought later is kept by default. Skip it when you need hashes or signatures to verify a thought or per-component scores; POST /v1/thought returns the full record of any hit.

POST /v1/ranked-search
{"chain_key": "project", "text": "job queue storage", "limit": 5, "compact": true}

// Full response (default)
{
  "backend": "hybrid",
  "total": 1,
  "results": [
    {
      "chain_key": "project",
      "score": {
        "confidence": 0.0, "graph": 0.0, "importance": 0.17541419, "lexical": 1.9490464,
        "recency": 0.05, "relation": 0.0, "rrf": 0.0, "seed_support": 0.0,
        "session_cohesion": 0.0, "total": 5.932654, "vector": 0.34079674
      },
      "matched_terms": ["job", "queue", "storag"],
      "match_sources": ["content", "tags", "concepts"],
      "graph_distance": null, "graph_path": null,
      "graph_relation_kinds": [], "graph_seed_paths": 0,
      "thought": {
        "agent_id": "builder", "agent_name": "Builder", "agent_owner": "platform",
        "concepts": ["job queue"], "confidence": null,
        "content": "Use Postgres for the job queue; revisit if throughput exceeds 5k jobs per second.",
        "entity_type": null,
        "hash": "31de8f264b1588a29be30a278a24e8623cf51b8f3ce8483a7e52f69ce80e21e1",
        "id": "1182d44f-3834-4ff0-9121-eab986b1a4f5",
        "importance": 0.800000011920929, "index": 0, "prev_hash": "",
        "provenance": {
          "segment": "public", "signature": "unsigned", "signature_version": null,
          "signed": false, "trust": "untrusted_data", "verified_agent": null
        },
        "refs": [], "relations": [], "role": "Memory", "schema_version": 3,
        "session_id": null, "signing_key_id": null, "source_episode": null,
        "tags": ["storage"], "thought_signature": null, "thought_type": "Decision",
        "timestamp": "2026-11-01T23:03:19.669167Z"
      }
    }
  ]
}

// Same request with "compact": true
{
  "backend": "hybrid",
  "total": 1,
  "results": [
    {
      "chain_key": "project",
      "score": 5.932654,
      "thought": {
        "agent_id": "builder",
        "concepts": ["job queue"],
        "content": "Use Postgres for the job queue; revisit if throughput exceeds 5k jobs per second.",
        "id": "1182d44f-3834-4ff0-9121-eab986b1a4f5",
        "importance": 0.800000011920929, "index": 0,
        "provenance": {
          "segment": "public", "signature": "unsigned", "signature_version": null,
          "signed": false, "trust": "untrusted_data", "verified_agent": null
        },
        "role": "Memory", "tags": ["storage"], "thought_type": "Decision",
        "timestamp": "2026-11-01T23:03:19.669167Z"
      }
    }
  ]
}

REST Example: POST /v1/context-bundles

POST /v1/context-bundles HTTP/1.1
Host: 127.0.0.1:9471
Content-Type: application/json

{
"chain_key": "default",
"text": "memory deduplication decision",
"scope": "user",
"limit": 5
}

// Response:
{
"bundles": [
{
"seed": {
"id": "a1b2c3d4-...",
"thought_type": "Decision",
"content": "Enable dedup at 0.85 Jaccard threshold",
"score": 5.1
},
"supporting": [
{
"id": "b2c3d4e5-...",
"thought_type": "LessonLearned",
"content": "Previous attempts at 0.95 threshold produced too many false negatives",
"chain_key": "default"
}
],
"children": []
}
]
}

REST Example: POST /v1/webhooks

POST /v1/webhooks HTTP/1.1
Host: 127.0.0.1:9472
Content-Type: application/json

{
"url": "https://myapp.example.com/mentisdb-webhook",
"chain_key_filter": "default",
"thought_type_filter": ["Decision", "Constraint"]
}

// Response:
{
"webhook": {
"id": "6f1c2a9e-...",
"url": "https://myapp.example.com/mentisdb-webhook",
"chain_key_filter": "default",
"thought_type_filter": ["Decision", "Constraint"],
"created_at": "2026-04-14T12:00:00Z",
"active": true
}
}

Targets must be public http or https endpoints: loopback, private, link-local, CGNAT, documentation, multicast and IPv4-embedding IPv6 addresses are rejected, also after DNS resolution, and redirects are not followed. After 5 consecutive failed deliveries a webhook is paused for 5 minutes.

Self-hosted operators can open specific private ranges with MENTISDB_WEBHOOK_ALLOWED_CIDRS (e.g. a Tailscale tailnet); see Private webhook targets.

0.8.0 Search &amp; Storage Improvements

MentisDB 0.8.0 introduces five major improvements to the search and storage pipeline: Porter stemming, tiered vector-lexical fusion, importance weighting, bincode hashing, and managed sidecar entries with auto_sync.

Porter Stemming (Normalizer v2)

The lexical tokenizer now applies English Porter stemming. Words like preferences → prefer and running → run now match in lexical search. Existing chains auto-reindex on first open — no manual migration step required.

Tiered Vector-Lexical Fusion

Ranked search scoring replaces the flat addition model with a tiered boost that respects the relationship between lexical and vector signals:

  • Lexical = 0 and vector &gt; 0: vector × 60 — full boost for semantic-only hits
  • 0 &lt; lexical &lt; 1.0 and vector &gt; 0: vector × (1 + 20 × fraction) — partial boost proportional to lexical overlap
  • Lexical ≥ 1.0: vector as-is — no boost, lexical dominates

Failed logins are rate-limited per client (an IPv4 address or IPv6 /64) with a global budget and backoff to prevent brute-force enumeration; the dashboard is always served over TLS so the PIN is never transmitted in the clear on a local network.

Automatic Thesaurus in Ranked Search (0.9.9)

The static thesaurus (~900 headwords + 300+ lemmas) now applies <strong>automatically by default</strong> to every ranked search query (REST <code>POST /v1/ranked-search</code>, MCP <code>mentisdb_ranked_search</code>, dashboard, and CLI). No client changes or extra parameters are required. With full embeddings this delivered the target LoCoMo-10P $R@10 = 72.6%. The expansion happens server-side via <code>apply_thesaurus_if_text</code> before BM25 scoring.

Invalidation filtering (default retrieval)

Thoughts that are the target of a later Supersedes, Corrects, or Invalidates relation are recorded in invalidated_thought_ids and excluded by default from MentisDb::query, query_ranked, context bundles, recent context, and related REST/MCP search. Pass include_invalidated: true (crate builders: with_include_invalidated(true)) for audit. Point-in-time as_of still returns thoughts that were valid at that timestamp.

Auto-dedup is off unless MENTISDB_DEDUP_THRESHOLD is set (recommended 0.85 for long-lived agent chains). Without dedup or agent-written invalidation edges, the set stays empty and search cannot hide noise.

0.12 Library Changes

0.12.2: source_episode, size cap, MCP keepalive

0.12.2 changes no public signature and needs no data migration. What changed:

  • source_episode is stored. POST /v1/thoughts and the MCP mentisdb_append tool accepted the field but dropped it, so the thought came back with source_episode: null. The append now stores it. It is also an exact-match filter on /v1/search, /v1/ranked-search, /v1/federated-search and /v1/context-bundles and on their MCP tools. /v1/lexical-search, thought traversal, summary candidates and the MEMORY.md export do not take the filter yet. Library: ThoughtInput::with_source_episode sets it.
  • CLI: --source-episode ID on thought append (stored) and on search, search filter, search federated and search bundles (filter). thought traverse, thought candidates, memory export and search lexical refuse the flag instead of sending a field the server would drop.
  • A thought record over 10 MiB is refused at append. The cap was enforced only when a chain was opened, so an oversized append was written and the next open failed. The append now returns InvalidInput (HTTP 400) before anything is written. Ordinary REST and MCP bodies are capped at 2 MiB, so the way in was the library or the large-body import routes (/v1/import-markdown, /v1/extract-memories, /v1/skills/upload).
  • MCP event stream keepalive. GET / on the MCP port sends an SSE comment (: keepalive) after 15 s with nothing else to send, so clients with an idle-body timeout (Claude Code's HTTP client) stop dropping and reconnecting. It comes from cloudllm_mcp 0.4.1, which mentisdb now requires. Embedders that build their own server can tune it with StreamableHttpConfig::with_sse_keepalive(Some(duration)) or turn it off with with_sse_keepalive(None) (a zero duration also disables it).
  • Markdown skill import reads standard YAML frontmatter: block scalars (>, |) for description and block lists (- item) for triggers, tags and warnings. A multi-line description is exported as a literal block.
  • The dashboard Settings screen lists every daemon setting; one catalog drives which names it shows and accepts and whether a change needs a restart.

0.12.1: live memories, agent key pairs

0.12.1 changes no public signature; it adds mentisdb::server_lean and PROVENANCE_REMINDER. What changed is how MentisDb::query_ranked fills a page, and how agent keys are made:

  • Only live memories compete. Every match is scored once, a superseded match nominates its live successor, and the walk stops at limit distinct live memories. It used to rank a fixed 2 * limit window that included superseded thoughts, so one decision rewritten 30 times turned a limit = 5 search into one hit. total_candidates counts live matches. Context-bundle seeds use the same walk.
  • Score totals are non-increasing in page order: a stand-in or contester that would outscore the hit before it reports a total just below it, so offset paging tiles the ranking.
  • successor_of lists the matched older versions met before the page filled: a lower bound on the lineage, not all of it.
  • The refill after near-duplicate collapsing resumes instead of restarting and is capped at max(16 * limit, 4096) ranked slots, so a page whose distinct matches lie deeper comes back short.
  • Agent keys: mentisdb agent key generate AGENT --key-id ID --chain KEY [--out FILE] [--force] creates an Ed25519 pair, writes the 64-hex seed to a 0600 file and sends the operator register_key operation with only the public key (--json: registered, agent_id, key_id, chain_key, public_key_hex, key_file). The dashboard's Generate key pair builds the same mentisdb/key-registration/v1 proof of possession in the browser and posts only the public key to POST /dashboard/api/agents/{chain}/{agent}/keys; Add Key now uses the operator route. Both dashboard key routes (add and delete) need the dashboard PIN and answer 403 without one.
  • Security: MentisDb::add_agent_key (the proof-based path) no longer replaces a key filed under an existing key id or revives a revoked one. A different key under an existing id is refused; the same active key again is a no-op, as operator registration already behaved. Rotate with a new key id.
  • MentisDb::query_federated still cuts each chain at its query's limit before redirecting and removing duplicates; the server's federated search widens the fetch up to max(16 * (offset + limit), 4096) per chain.

MentisDB 0.12.0 is a minor-version release because public library types changed. Sidecar files, their digests, persisted HNSW graphs, the wire protocol and the CLI are unchanged.

Breaking: one flat vector store (VectorRow)

Every vector of a managed sidecar now lives once, in 128-row, 64-byte-aligned chunks shared by the sidecar entries, the exact index, the HNSW graph and the implicit-edge scanner. Exact search runs a lane-parallel dot product over it (about 4.3x faster at 100,000 vectors; scores within 1e-5 of the old cosine, so near-ties may swap; the HNSW and implicit-edge paths keep the sequential cosine and can score a pair up to that much apart from exact search). The exact index reuses removed slots and compacts once dead rows outnumber live ones, so it does not grow under churn. Signature changes:

  • VectorSidecarEntry::vector is a mentisdb::search::flat_store::VectorRow (dereferences to &[f32]), not a Vec<f32>. Use .to_vec() for a Vec and VectorRow::standalone(&values) to make one.
  • VectorSidecarEntry::new takes impl Into<VectorRow>: Vec<f32> and &[f32] convert, but a bare .collect() whose type is only inferred no longer compiles; annotate it.
  • VectorSidecar has a private field: build it with VectorSidecar::build, not a struct literal.
  • New: search::flat_store (VectorStore, VectorRow, dot, DOT_EPSILON), VectorIndex::{from_rows, upsert_row}, VectorSearchBackend::upsert_row (provided, implementors keep compiling) and HnswBackend::{from_rows, share_cached_rows}.
use mentisdb::search::VectorSidecarEntry;
use mentisdb::search::flat_store::VectorRow;

let entry = VectorSidecarEntry::new(id, index, hash, vec![0.1, 0.2, 0.3]); // unchanged
let owned: Vec<f32> = entry.vector.to_vec();   // 0.11: entry.vector.clone()
let slice: &[f32] = &entry.vector;            // unchanged
let values: Vec<f32> = (0..3).map(|i| i as f32).collect(); // annotate inferred types
let entry2 = VectorSidecarEntry::new(id, index, hash, values);
let literal = VectorSidecarEntry {
    thought_id: id, thought_index: index, thought_hash: hash.to_string(),
    vector: VectorRow::standalone(&[0.1, 0.2, 0.3]),   // 0.11: vec![0.1, 0.2, 0.3]
};

entry.vector.clone() still compiles but now returns a VectorRow (a pointer copy), not a Vec<f32>.

Breaking: MentisDbServiceConfig is #[non_exhaustive]

It can no longer be built with a struct literal; the fields stay readable and assignable. Build it with MentisDbServiceConfig::new(..) and the with_* builders. The body-admission settings are with_max_body_bytes (import routes, clamped to 1 MiB..1 GiB), with_max_inflight_body_bytes and with_body_idle_timeout.

use mentisdb::server::MentisDbServiceConfig;
use mentisdb::StorageAdapterKind;
use std::time::Duration;

let config = MentisDbServiceConfig::new("/var/lib/mentisdb".into(), "project", StorageAdapterKind::Binary)
    .with_verbose(true)
    .with_max_body_bytes(128 * 1024 * 1024)
    .with_max_inflight_body_bytes(512 * 1024 * 1024)
    .with_body_idle_timeout(Duration::from_secs(60))
    .with_min_body_bytes_per_sec(16 * 1024);   // 0 turns the rate floor off

Request body admission

An outermost layer (server_body) decides before reading a body: the bearer token is checked from the headers first; ordinary routes and every MCP request take at most SMALL_BODY_BYTES (2 MiB); only the routes in LARGE_BODY_ROUTES take up to the configured cap, from an authenticated caller. Imports are reserved from their declared length against the in-flight budget (2x the cap and LARGE_BODIES_PER_PEER (4) bodies per peer). Other authenticated requests are charged per arriving byte from SMALL_POOL_BYTES (64 MiB, SMALL_PEER_BYTES = 32 MiB per peer); anonymous requests (bearer on, no valid token) from their own ANONYMOUS_POOL_BYTES (16 MiB, ANONYMOUS_PEER_BYTES = 8 MiB per peer). Small and anonymous bodies have a total deadline of twice the idle timeout, and every body must reach the configured minimum rate (with_min_body_bytes_per_sec, MENTISDB_MIN_BODY_BYTES_PER_SEC, default MIN_BODY_BYTES_PER_SEC = 64 KiB/s, 0 = off) in every window of one idle timeout after a grace period of twice the idle timeout; each window counts only its own bytes, so credit cannot be banked. Too slow or stalled is 408, an exhausted budget 503 with Retry-After, on every route (the layer replaces a handler's 400 when it was the one that cut the body off). Read proofs that send X-Mentis-Content-Sha256 are verified before the body is read and the body is hashed as it streams.

Derived-state backlog

MentisDb::derived_backlog returns a DerivedBacklog summing a chain's managed sidecars: embeddings_in_flight (requested, not committed), embeddings_parked (ready, waiting for an earlier thought's vector), embeddings_missing (waiting to be embedded again) and implicit_edges (applied vectors whose edges have not reached the overlay). All are zero when the chain is caught up. GET /v1/chains and the MCP chain listing report it per chain as derived_backlog, with zeros for a chain that is not open or whose private segments are hidden from the caller.

"derived_backlog": {
  "embeddings_in_flight": 0, "embeddings_parked": 0,
  "embeddings_missing": 0, "implicit_edges": 312
}

The implicit-edge queue holds positions, not vectors, so a backlog never grows memory by more than a few bytes, never blocks an append and never drops an edge. The scan sustains about 155 appends per second at 100,000 thoughts and 720 at 20,000 (4 threads, 384 dimensions); faster appends make implicit_edges grow, and it drains when they slow down.

0.11 Library Changes

MentisDB 0.11.0 changes what default retrieval returns, adds public and private chains, and moves format upgrades into a background migration manager. Full rustdoc is on docs.rs; the design of the access model is in docs/CHAIN_ACCESS.md.

Valid-version retrieval (module lineage)

Lineage is derived when a chain opens; no migration is needed. A superseded thought that would have ranked is replaced by its live successor (RankedSearchHit::version.successor_of names what it stands in for). A thought that a newer Correction, AssumptionInvalidated or Decision only references, or that anything Contradicts, stays visible as VersionStatus::Contested with the contesting thought promoted above it. New: MentisDb::{version_status, current_version, version_info, hit_json, append_advice}, VersionStatus, AppendAdvice and lineage::cross_chain_successor for a relation that carries a chain_key (applied wherever one read path sees both chains: branch search and federated search). Every thought JSON carries version_status, successors and current_version when it is not plainly current.

use mentisdb::{RankedSearchQuery, VersionStatus};

let hits = chain.query_ranked(&RankedSearchQuery::new().with_text("release policy Fridays"));
// The live successor is returned in place of the superseded policy.
assert_eq!(hits.hits[0].version.successor_of, vec![old.id]);
assert_eq!(chain.version_status(old.id), VersionStatus::Superseded);

RankedSearchQuery defaults changed

RankedSearchQuery::new() now applies relative vector fusion, near-duplicate collapse at Jaccard 0.9 (text queries), successor redirection and thesaurus expansion at weight 0.3 (the server default; the library used to apply none). Result lists and total_candidates differ from 0.10. Opt out per query:

use mentisdb::{RankedSearchQuery, VectorFusion};

let query = RankedSearchQuery::new()
    .with_text("deploy window")
    .with_vector_fusion(VectorFusion::Absolute) // the 0.10 blend
    .with_collapse_duplicates(None)             // keep near-duplicates
    .with_resolve_successors(false)             // plain hiding of superseded thoughts
    .with_thesaurus(false);                     // literal terms only

Process-wide: MENTISDB_VECTOR_FUSION=absolute, MENTISDB_COLLAPSE_DUPLICATES=off. with_df_scope and with_df_cutoffs choose how the lexical stop-word gate measures document frequency (DfScope::Global stays the default). Over REST and MCP the same knobs are request fields: resolve_successors, collapse_duplicates (0 disables), vector_fusion, synonym_weight (0 disables), df_scope and df_cutoffs.

Standing context (module standing)

MentisDb::standing_context(&StandingContextQuery) returns the durable half of memory regardless of age: live Constraint, PreferenceUpdate, UserTrait, RelationshipUpdate, Goal, Decision and LessonLearned thoughts, ranked by importance then recency, restatements collapsed, capped at 8 per type and 40 in total, plus the newest Handoff and Checkpoint/Summary. The StandingContext result has entries, duplicates_collapsed and a ready-to-load prompt. A superseded constraint or decision is replaced by its successor.

Public and private chains

  • mentisdb::audience::{Audience, Segment, SegmentMap}: every read path takes an audience. ThoughtQuery::audience, StandingContextQuery::audience and *_for(audience) variants (get_thought_for, thought_json_for, hit_json_for, append_advice_for, ...). The library default is Audience::Authorized: a program that holds the data directory holds every byte.
  • mentisdb::read_auth::{sign_request, ReadSigner} build the X-Mentis-Auth read-proof header for HTTP clients. To send a body over 1 MiB also send X-Mentis-Content-Sha256 (body_digest, content_digest_header_value): the server verifies the proof from the declared digest before reading the body and hashes the body as it streams, failing the request at its end on a mismatch, before any handler has run.
  • mentisdb::operator::apply runs operator commands (mode changes, allowed keys, key registry, agent status) signed with <data dir>/operator/operator.key; MentisDb::add_agent_key_as_operator registers a key the operator vouches for. MentisDb::add_agent_key still requires a &KeyRegistrationProof (proof of possession) and a valid curve point.
  • MentisDb::verification reports the open-time check of governed chains (quarantined thoughts, rejected policy records).
  • AccessError::StaleHead is returned for a v2-signed append built on an older head (fetch the head and sign again); a forged signature stays BadSignature.
  • A stored zero-length signature is a placeholder and counts as unsigned everywhere; new writes normalize an empty signature to none. Thought::signature_bytes returns the real signature or nothing.

Storage and validation

  • mentisdb::validate_chain_key guards every entry point (library open, branch, restore): non-empty, at most 256 bytes, no control characters.
  • MentisDb::defer_next_embedding lets a caller embed outside the chain's write lock, as the daemon now does for every append (lock hold per append about 0.5 ms instead of about 9.5 ms with a 5 ms model). Library callers that do not opt in are unchanged.
  • The daemon migrates legacy chains in the background through a journaled, backed-up manager. A library user that opens a legacy chain directly keeps the in-place upgrade, now with a one-time copy of the original under migrations/backup/in-place/. A chain or data directory written by a newer release is refused with a clear message.
  • Chains from March and April 2026 that stamp schema 4 on schema-3 records open again: schema 4 is an accepted historical alias, so the next thought schema is 5.

Deprecated

RankedSearchQuery::{llm_rerank, llm_model, llm_rerank_top_k, with_llm_reranking} and RankedSearchScore::llm_score are no-ops (LLM reranking was never implemented), and search::ppr is never called by ranked search. Both are marked #[deprecated] and will be removed in the next breaking release.

Benchmarks

Full LoCoMo-10P and LongMemEval protocol run on the 0.11 defaults (fresh ingest, fastembed-minilm): LoCoMo-10P R@10 76.7% (73.5% in 0.10.9.54), LongMemEval R@5 69.8% (68.0%). See results/benchmarks/2026-11-01-full-protocol.

Benchmarking

MentisDB ships with two benchmark styles:

  • Criterion microbenchmarks for in-process append, query, traversal, import, and skill-registry hot paths
  • HTTP concurrency benchmarks for live mentisdb write/read waves under concurrent client load

Run the full benchmark suite with:

make bench

This currently runs cargo bench and saves stdout to /tmp/mentisdb_bench_results.txt.

Durable vs Buffered Write Mode

Benchmark write-heavy changes in both persistence modes. They now behave differently enough that one run is not representative of the other:

  • MENTISDB_BENCH_AUTO_FLUSH=true — durable mode. Appends are queued to the bounded background writer and the request only returns after the writer flushes them. Concurrent appends may share a short group-commit window; a lone append flushes immediately. Since 0.10.9 commits are fsynced only with MENTISDB_FSYNC=1, so benchmark both settings when touching the writer.
  • MENTISDB_BENCH_AUTO_FLUSH=false — buffered mode. Appends are queued and acknowledged before the worker flushes them, trading durability for higher write throughput.
MENTISDB_BENCH_AUTO_FLUSH=true  cargo bench --bench http_concurrency
MENTISDB_BENCH_AUTO_FLUSH=false cargo bench --bench http_concurrency

Criterion Baselines

Criterion benchmarks such as thought_chain and skill_registry automatically compare the current run against the saved baseline in target/criterion/. That is why the output includes messages like Performance has improved, Performance has regressed, or No change in performance detected.

  • time: [low mid high] — lower is better
  • thrpt: [low mid high] — higher is better
  • change — percentage delta versus the saved baseline
  • p &lt; 0.05 — the change is statistically meaningful

If you have changed the benchmark harness itself or want a fresh comparison, clear the old Criterion baseline before trusting the regression/improvement messages.

HTTP Concurrency Baselines

The custom http_concurrency benchmark now persists its own baselines under target/http_concurrency/ and prints delta tables on later runs. This means you no longer need to manually compare the Markdown output tables by eye.

MENTISDB_BENCH_AUTO_FLUSH=true  MENTISDB_BENCH_BASELINE=durable  cargo bench --bench http_concurrency
MENTISDB_BENCH_AUTO_FLUSH=false MENTISDB_BENCH_BASELINE=buffered cargo bench --bench http_concurrency

Useful environment variables:

  • MENTISDB_BENCH_CONCURRENCY — comma-separated client counts such as 100,1000,10000
  • MENTISDB_BENCH_AUTO_FLUSH — true for durable group commit, false for buffered throughput mode
  • MENTISDB_BENCH_BASELINE — names the saved baseline file so you can keep separate durable, buffered, nightly, or branch-specific histories

How to Read the HTTP Table

  • wall_ms — total time for the whole wave
  • req/s — throughput
  • p50_ms, p95_ms, p99_ms — median and tail latency
  • errors — non-2xx responses or transport failures

For write-path work, focus on the Write — POST /v1/thoughts table at high concurrency. Read-path changes should primarily move the Read — POST /v1/head table and the query/traversal Criterion benches.

What to Benchmark After Storage Changes

  • cargo bench --bench thought_chain append_single -- --noplot — strict append latency
  • cargo bench --bench thought_chain query_by_tag -- --noplot — indexed query path
  • cargo bench --bench thought_chain traverse_filtered_miss_10 -- --noplot — full-scan traversal miss path
  • cargo bench --bench http_concurrency — live daemon concurrency behavior

A useful workflow is to measure a clean baseline on master, apply the storage change, rerun the same focused benches, and only then trust the wider make bench output.

Contributing

MentisDB is open source under the MIT license.

git clone https://github.com/cloudllm-ai/mentisdb

Run the test suite:

cargo test

Run benchmarks:

make bench
View on GitHub →