Noosphere Memory

Now with OpenAI Codex Support as well as optional vector retrieval and auto-capture

Noosphere is a self-hosted knowledge and memory layer for AI agents and humans.
Agents use it to recall, save, and organize durable and detailed project knowledge; humans use the same data as a browsable Markdown wiki with topics, revisions, scoped access, and Obsidian-friendly export/import.

All agent systems and humans access the same memory data. You can start in OpenClaw, continue in Codex CLI or OpenCode, and add new data yourself through the web UI. The system is database PostgreSQL based with Redis for fast recall. Markdown import and export is possible by the user and agents.

It sits between a chat transcript and a full documentation site:

• Agent memory: recall relevant project context, save draft memory candidates, and promote useful facts into curated articles.
• Human wiki: browse, edit, review, restore, and search Markdown articles.
• Scoped access: give agents or users narrow API keys for only the knowledge they should read or write.
• Integration-first design: OpenClaw, Codex CLI, Hermes Agent, OpenCode, Kilo Code, any MCP client, and any REST client can use the same Noosphere instance.

The old long-form README is preserved at README-legacy.md.

Quick Start

Prerequisites

Install Docker with Docker Compose v2, Node.js 22 or newer, and curl. The installer checks the remaining command-line prerequisites before changing the machine.

Install

This launcher is coupled to the coordinated v1.15.0 image, packages, and Hermes archive. Before running it, confirm that the v1.15.0 release exists with all six installer assets; a merged commit alone is not a release.

Run the guided installer:

curl -fsSL https://raw.githubusercontent.com/SweetSophia/noosphere/7ebffe1f21578cd6e2ad17a7f6fa5baddc88769e/install.sh | bash

The URL is pinned to an immutable Git commit—never master or main. The launcher then checksum-verifies its exact backend before execution.

The installer:

• shows its plan and detects fresh-install, interrupted-run, or upgrade state;
• offers OpenClaw, Hermes Agent, OpenCode, and Kilo Code integrations found on the machine;
• generates distinct runtime credentials without printing them;
• preserves the named PostgreSQL volume and uses the guarded stateful upgrade path when an existing installation is present;
• starts the services and verifies Noosphere health.

When it finishes, open http://127.0.0.1:6578/wiki. The actual URL is shown if you selected another bind address.

Generated administrator credentials are stored at ~/.noosphere/credentials.json in a mode-0700 directory with mode 0600. The normal output reports the path, not the password or API key.

For an independently verified download, automation flags, existing-installation warnings, manual Compose deployment, and bootstrap-secret path rules, read Installing Noosphere.

Existing or customized installations

Do not replace the guided upgrade with an unrestricted docker compose pull && docker compose up. First inspect the plan without changing the machine:

curl -fsSL https://raw.githubusercontent.com/SweetSophia/noosphere/7ebffe1f21578cd6e2ad17a7f6fa5baddc88769e/install.sh \
  | bash -s -- --dry-run --core-only

Preserve your configuration, database backup, and local repair records before upgrading. The installer keeps the noosphere_postgres_data volume and admits an image transition only through its offline, restore-tested controller. See PostgreSQL pgvector Compose upgrade for the full stateful recovery contract.

Optional pgvector hybrid storage remains a separate activation step. Operator how-to (llama.cpp local embeddings, no wiki toggle): docs/HYBRID-RETRIEVAL-ACTIVATION.md for a source Compose checkout, or docs/HERMES-INSTALLER-HYBRID.md for a guided-installer runtime plus Hermes. SQL/privilege contract: docker/hybrid-storage/README.md.

Choose an Integration

System	What it gets	Start here
OpenClaw	Explicit tools, optional prompt-time auto-recall, memory corpus supplement, CLI helpers	openclaw-noosphere-memory/README.md
Codex CLI / MCP	Five native MCP tools, a Codex skill, and a secret-safe guided installer	noosphere-mcp/README.md
Hermes Agent	First-class Hermes MemoryProvider, recall/get/topics/save tools, optional memory mirroring	hermes-noosphere-memory/README.md
Opencode	Prompt-time auto-recall, optional idle auto-save, manual memory tools	opencode-noosphere-memory/README.md
Kilo Code	Prompt-time auto-recall, optional idle auto-save, manual memory tools	kilocode-noosphere-memory/README.md
REST clients	Article CRUD, ingest, memory recall/get/save, export/import, graph, health	API Snapshot

Use integration-specific environment variables when multiple tools run on one machine, for example OPENCLAW_NOOSPHERE_API_KEY, CODEX_NOOSPHERE_API_KEY, HERMES_NOOSPHERE_API_KEY, OPENCODE_NOOSPHERE_API_KEY, or KILOCODE_NOOSPHERE_API_KEY. The generic NOOSPHERE_API_KEY fallback remains available for simple single-tool setups.

Install the Codex CLI integration independently of the core Docker installer:

npx -y @sweetsophia/noosphere-mcp@1.15.0 install-codex

The command stores the MCP launcher, skill, and approved environment-variable names; it never writes the API key or any other environment-variable value. See the integration guide for runtime environment and remote HTTPS-origin configuration.

Core Concepts

Topics and Articles

Topics form an unlimited-depth hierarchy. Articles live inside topics, render as GitHub-flavored Markdown, and can include tags, source metadata, images, confidence, status, revision history, and related-article edges.

Memory Recall

The memory layer normalizes results from providers, ranks them, deduplicates overlap, handles conflicts, and budgets the returned context for prompt use. Current providers include Noosphere articles and Hindsight; the provider contract is extensible.

See docs/NOOSPHERE-MEMORY-ARCHITECTURE.md for the implementation model.

Draft Saves and Curation

Agent saves are draft memory candidates by default. That keeps automatic memory capture inspectable before it becomes curated wiki knowledge.

Scopes

Restricted articles use restrictedTags; scoped API keys and scoped users can only read or write content allowed by their scopes. A wildcard * scope grants full restricted-content access and should be reserved for admin workflows.

Obsidian and Markdown

Noosphere can export/import Markdown vault archives and supports an Obsidian sync workflow through a versioned frontmatter codec. The sync design lives in docs/OBSIDIAN-SYNC-SPEC.md.

Core Memory Features

Feature	Noosphere	Hindsight	QMD	memU	mem0	LanceDB Pro
Auto-Capture	⚠️ Disabled-by-default private capture API, principal/lineage storage, and cleanup foundation; OpenClaw turn hook/extraction planned	✅ Every turn	❌ Manual indexing	✅ Continuous learning	✅ memory.add()	✅ Smart extraction
Auto-Recall	✅ Capture guidance on clean misses + recall results when available; provider errors fail open	✅ Before each turn	✅ Keyword search only	✅ Proactive context loading	✅ memory.search()	✅ Before prompt build
Manual Recall	✅ REST API + tools	✅ MCP tools	✅ CLI / tool query	✅ REST API	✅ SDK + REST	✅ CLI + MCP tools
Semantic Search	✅ PostgreSQL FTS (default) + implemented opt-in pgvector/RRF; rollout pending	✅ Vector + biomimetic	⚠️ Keyword + pending vector	✅ pgvector	✅ Semantic + BM25 + entity fusion	✅ Vector + BM25 hybrid
Keyword Search	✅ PostgreSQL full-text	✅	✅ Primary mode	✅	✅ BM25	✅ BM25
Cross-Encoder Rerank	❌ (planned)	❌	❌	❌	❌	✅ Cross-encoder
Memory Types	Articles (wiki)	world / experience / observation	Markdown files	Categories / Items / Resources	Facts (ADD-only v3)	6-category classification
Curation Levels	✅ ephemeral → managed → curated	❌	❌	❌	❌	❌
Confidence Scoring	✅ low / medium / high	❌	❌	❌	❌	❌ (decay model)
Status Lifecycle	✅ draft → reviewed → published	❌	❌	❌	❌	❌

---

Advanced Memory Features

Feature	Noosphere	Hindsight	QMD	memU	mem0	LanceDB Pro
Multi-Provider Recall	✅ Noosphere + Hindsight + extensible	❌ (single provider)	❌ (single store)	❌ (single provider)	❌ (single provider)	❌ (single store)
Recall Orchestration	✅ Concurrent fan-out + ranking	❌	❌	❌	❌	❌
Cross-Provider Dedup	✅	❌	❌	❌	❌	❌
Conflict Detection	✅ Configurable strategies	❌	❌	❌	❌	❌
Token Budget Manager	✅ Prompt-safe recall blocks	✅ recallMaxTokens	❌	❌	❌	❌
Promotion (ephemeral → curated)	⚠️ Pure threshold/review scaffolding; durable statistics and worker wiring planned	❌	❌	❌	❌	⚠️ Decay model (Weibull)
Backfill / Synthesis	⚠️ Pure job/content helpers; durable execution wiring planned	✅ Historical backfill CLI	❌	❌	❌	❌
Local Scheduler	⚠️ Health plus durable automatic-memory expiry/privacy cleanup; extraction/promotion workers planned	❌	❌	✅ Continuous sync loop	❌	❌
Revision History	✅ Per-article	❌	❌	❌	❌	❌
Topic Hierarchy	✅ Unlimited depth	❌	❌	✅ Category hierarchy	❌	❌
Tags / Relations	✅ Tags + article edges	❌	❌	✅ Cross-references	✅ Entity linking (v3)	❌
Soft Delete / Trash	✅	❌	❌	❌	✅	✅

---

API Snapshot

Base URL:

http://localhost:6578/api

Authentication (replace <api_key> with a scoped Noosphere API key):

Authorization: Bearer <api_key>

Common endpoints:

Method	Endpoint	Purpose
GET	/api/health	Service health check
GET	/api/topics	List the complete topic tree (maximum 500 topics; returns 409 above the limit)
GET	/api/articles	Search/list articles
POST	/api/articles	Create an article
PATCH	/api/articles/:id	Update an article
POST	/api/ingest	Ingest external material into articles
POST	/api/answer	Save a synthesized answer as an article
GET	/api/graph	Read the wiki graph
GET	/api/export	Export a Markdown vault ZIP
POST	/api/import	Import a Markdown vault ZIP
GET	/api/memory/status	Memory provider/settings overview
POST	/api/memory/recall	Recall ranked memory results
POST	/api/memory/get	Fetch one memory by canonical ref or ID
POST	/api/memory/save	Save a draft memory candidate
POST	/api/memory/captures	Accept one private automatic-memory observation when explicitly enabled
GET	/api/memory/captures	Admin capture inspection (raw text is detail-only)
GET	/api/memory/captures/:id	Read eligible capture status/raw detail as its bound creator, or inspect quarantined evidence as a scope-authorized administrator
GET/POST	/api/memory/principals	Admin principal inspection/provisioning
POST	/api/memory/revocations	Admin session-lineage revocation
GET	/api/memory/{candidates,jobs,tombstones,privacy-reviews}	Admin Phase A lifecycle inspection

JSON write endpoints reject malformed or excessively nested payloads and return 413 when their route-specific body-size limit is exceeded. Most routes allow 64 KiB; article writes allow approximately 1 MiB, and batch ingest allows 4 MiB. POST /api/memory/recall allows 120 requests per minute per client IP by default so several local coding CLIs can use prompt-time recall concurrently. Set NOOSPHERE_MEMORY_RECALL_RATE_LIMIT_PER_MINUTE to tune that read-only endpoint for your deployment.

Example recall request:

curl -s -X POST http://localhost:6578/api/memory/recall \
  -H "Authorization: Bearer ***" \
  -H "Content-Type: application/json" \
  -d '{"query":"deployment runbook","mode":"auto","resultCap":5}'

Local Development

Prerequisites:

• Node.js 22+
• Docker and Docker Compose

Setup:

npm install
cp .env.example .env
# Edit the bootstrap, migration, and application database credentials plus
# NEXTAUTH_SECRET, NEXTAUTH_URL, and APP_URL.
docker network create noosphere-net 2>/dev/null || true
mkdir -p .noosphere/postgres-pgvector-backups
chmod 700 .noosphere/postgres-pgvector-backups
guard=(./scripts/switch-pgvector-compose.sh --compose-file "$PWD/docker-compose.yml" \
  --env-file "$PWD/.env" --db-container noosphere-db --app-container noosphere-app \
  --backup-dir "$PWD/.noosphere/postgres-pgvector-backups")
"${guard[@]}" --prepare-new-install
docker compose up -d db redis
docker compose run --rm -T init
"${guard[@]}" --record-new-install
set -a
. ./.env
set +a
DATABASE_URL="$NOOSPHERE_APP_DATABASE_URL" PORT=6578 npm run dev

The external noosphere_postgres_authorization volume is created only by the guard, so an ordinary candidate Compose start fails closed. When the development Compose project reuses an existing noosphere_postgres_data volume created by the former source image, run the guarded PostgreSQL image transition before the first docker compose up from this revision.

Useful checks:

npm run lint
npm run typecheck
npm run test
npm run build

Package-specific plugin checks live in each plugin README.

Operations

Health and deployment checks:

curl http://127.0.0.1:6578/api/health
NOOSPHERE_POSTGRES_EVIDENCE=/absolute/private/path/postgres-pgvector/noosphere_postgres_data.phase-a2b.json \
  npm run deploy:verify
docker compose logs -f app

Production deploys should preserve the pinned Compose project and named volumes:

• Compose project: noosphere
• PostgreSQL volume: noosphere_postgres_data
• Redis volume: noosphere_redis_data

Candidate verification requires the active completed Phase A2b journal through NOOSPHERE_POSTGRES_EVIDENCE. npm run deploy:verify fails if that evidence does not bind the running database image, data volume, authorization volume, and template probe; if PostgreSQL exposes the wrong pgvector capability or has activated vector in any database/template; or if there are no topics, articles, or API keys. The guarded Phase A2b transition and recovery contract is documented in docs/POSTGRES-PGVECTOR-COMPOSE-UPGRADE.md.

Keep detailed recovery work in deployment/runbook docs rather than this README.

Documentation

Document	Use it for
README-legacy.md	Previous full README content kept for reference during the docs split
docs/MEMORY-REVAMP-STATUS.md	Authoritative implementation and rollout matrix for automatic capture and hybrid retrieval
docs/INSTALLATION.md	Guided installer, auditable download, upgrades, manual Compose, and credential handling
docs/HERMES-INSTALLER-HYBRID.md	Installer-runtime work-VM hybrid runbook (~/.noosphere, sidecar A/B/C, llama.cpp)
docs/OPENCLAW-OFFICIAL-PLUGIN-SETUP.md	OpenClaw install, operations, upgrade, troubleshooting, and uninstall
docs/POSTGRES-PGVECTOR-COMPOSE-UPGRADE.md	Guarded PostgreSQL image transition, proof, rollback, and recovery
docs/NOOSPHERE-MEMORY-ARCHITECTURE.md	Provider abstraction, recall orchestration, ranking, budgeting, and scheduler
docs/NOOSPHERE_MEMORY_COMPARISON.md	Comparison with Hindsight, QMD, memU, mem0, and LanceDB Pro
docs/NOOSPHERE-SKILL.md	Agent-facing wiki skill reference
docs/OBSIDIAN-SYNC-SPEC.md	Obsidian sync design and Markdown frontmatter contract
docs/OBSIDIAN-SYNC-REVIEW.md	Obsidian sync review notes
docs/SECURITY-AUDIT-2026-04-16.md	Security audit notes
openclaw-noosphere-memory/README.md	OpenClaw plugin configuration and tools
hermes-noosphere-memory/README.md	Hermes Agent provider install and verification
opencode-noosphere-memory/README.md	Opencode plugin install, configuration, and tools
kilocode-noosphere-memory/README.md	Kilo Code plugin install, configuration, and tools

License

Apache 2.0. See LICENSE and NOTICE.

The Apache License 2.0 applies to all source code in this repository and to the five integrations (openclaw-noosphere-memory, opencode-noosphere-memory, hermes-noosphere-memory, kilocode-noosphere-memory, noosphere-mcp). It does not govern the article content stored inside a Noosphere wiki instance — content licensing is a separate decision left to the wiki operator.

Redistributions must preserve the applicable attribution notices as described by Apache License 2.0 Section 4(d). The NOTICE file is informational and does not add a UI-credit or hosted-service condition.