## Efficient Lifelong Memory for LLM Agents — Text & Multimodal Store, compress, and retrieve long-term memories with semantic lossless compression. Now with multimodal support for text, image, audio & video.

Works with any AI platform that supports MCP (text memory) or Python integration (full multimodal)

_{Claude Desktop}

_Cursor

_{LM Studio}

_{Cherry Studio}

_{PyPI Package}

_{+ Any MCP
Client}

[🇨🇳 中文](./docs/i18n/README.zh-CN.md) • [🇯🇵 日本語](./docs/i18n/README.ja.md) • [🇰🇷 한국어](./docs/i18n/README.ko.md) • [🇪🇸 Español](./docs/i18n/README.es.md) • [🇫🇷 Français](./docs/i18n/README.fr.md) • [🇩🇪 Deutsch](./docs/i18n/README.de.md) • [🇧🇷 Português](./docs/i18n/README.pt-br.md)
[🇷🇺 Русский](./docs/i18n/README.ru.md) • [🇸🇦 العربية](./docs/i18n/README.ar.md) • [🇮🇹 Italiano](./docs/i18n/README.it.md) • [🇻🇳 Tiếng Việt](./docs/i18n/README.vi.md) • [🇹🇷 Türkçe](./docs/i18n/README.tr.md)
[![Project Page](https://img.shields.io/badge/🎬_INTERACTIVE_DEMO-Visit_Our_Website-FF6B6B?style=for-the-badge&labelColor=FF6B6B&color=4ECDC4&logoColor=white)](https://aiming-lab.github.io/SimpleMem-Page)

[🚀 Quick Start](#-quick-start) • [🌟 Overview](#-overview) • [📦 Installation](#-installation) • [🔌 MCP Server](#-mcp-server-text-memory) • [📊 Reproduce](#-reproduce-paper-results) • [📝 Citation](#-citation)

## 🔥 News - **[05/21/2026]** 📦 **Unified `simplemem` package — one import, auto-routing!** SimpleMem, Omni-SimpleMem, and EvolveMem now live in a single package. `from simplemem import SimpleMem` auto-selects the text or multimodal backend from the first method you call, and `simplemem.optimize(...)` taps EvolveMem's self-evolution loop. Install in one step with `pip install -e .`. - **[05/14/2026]** 🧬 **EvolveMem (v3.0) — Self-Evolving Memory via AutoResearch!** The retrieval infrastructure itself now self-evolves through LLM-driven closed-loop diagnosis. On LoCoMo, EvolveMem outperforms the strongest baseline by **+25.7% relative**; on MemBench, by **+18.9% relative**. The system discovers entirely new retrieval dimensions not present in the original design. [View EvolveMem →](EvolveMem/) - **[04/02/2026]** 🧠 **Omni-SimpleMem (v2.0) — Multimodal Memory is Here!** SimpleMem now supports **text, image, audio & video** memory. Achieving **new SOTA on LoCoMo (F1=0.613, +47%)** and **Mem-Gallery (F1=0.810, +51%)** over previous best. [View Omni-SimpleMem →](OmniSimpleMem/) - **[02/09/2026]** 🚀 **Cross-Session Memory — Outperforming Claude-Mem by 64%!** [View Cross-Session Documentation →](cross/README.md) - **[01/20/2026]** 📦 **SimpleMem is now available on PyPI!** Install via `pip install simplemem`. [View Package Usage Guide →](docs/PACKAGE_USAGE.md) - **[01/14/2026]** 🎉 **SimpleMem MCP Server is LIVE!** Cloud-hosted at [mcp.simplemem.cloud](https://mcp.simplemem.cloud). [View MCP Documentation →](MCP/README.md) - **[01/05/2026]** SimpleMem paper was released on [arXiv](https://arxiv.org/abs/2601.02553)! --- ## 📑 Table of Contents - [🚀 Quick Start](#-quick-start) - [🌟 Overview](#-overview) - [📦 Installation](#-installation) - [🐳 Docker](#-run-with-docker) - [🔌 MCP Server](#-mcp-server-text-memory) - [📊 Reproduce Paper Results](#-reproduce-paper-results) - [🗺️ Roadmap](#️-roadmap) - [📝 Citation](#-citation) --- ## 🚀 Quick Start ### 🧠 Understanding the Basic Workflow At a high level, SimpleMem works as a long-term memory system for LLM-based agents. The workflow consists of three simple steps: 1. **Store information** – Dialogues or facts are processed and converted into structured, atomic memories. 2. **Index memory** – Stored memories are organized using semantic embeddings and structured metadata. 3. **Retrieve relevant memory** – When a query is made, SimpleMem retrieves the most relevant stored information based on meaning rather than keywords. This design allows LLM agents to maintain context, recall past information efficiently, and avoid repeatedly processing redundant history. ### 🎓 Basic Usage SimpleMem ships as a single `simplemem` package. The default `mode="auto"` **automatically detects** which backend to use based on what you call — no manual configuration needed: ```python from simplemem import SimpleMem mem = SimpleMem() # mode="auto" — backend chosen by first call ``` The first method you call determines the backend: | First call | Backend selected | Why | |:--|:--|:--| | `add_dialogue()` | **Text** (SimpleMem) | Dialogue-based API → text mode | | `add_text()` / `add_image()` / `add_audio()` / `add_video()` | **Omni** (Omni-SimpleMem) | Multimodal API → omni mode |

**📝 Auto → Text** (pure text input) ```python from simplemem import SimpleMem mem = SimpleMem() # auto mode # add_dialogue() → text backend auto-selected mem.add_dialogue( "Alice", "Bob, let's meet at Starbucks tomorrow at 2pm", "2025-11-15T14:30:00", ) mem.add_dialogue( "Bob", "Sure, I'll bring the market analysis report", "2025-11-15T14:31:00", ) mem.finalize() answer = mem.ask("When and where will Alice and Bob meet?") # → "16 November 2025 at 2:00 PM at Starbucks" ```

**🧠 Auto → Omni** (multimodal input) ```python from simplemem import SimpleMem mem = SimpleMem() # auto mode # add_image() → omni backend auto-selected mem.add_text( "User loves hiking in the Rocky Mountains.", tags=["session_id:D1"], ) mem.add_image("photo.jpg", tags=["session_id:D1"]) mem.add_audio("voice_note.wav", tags=["session_id:D1"]) result = mem.query("What does the user enjoy?", top_k=5) for item in result.items: print(item["summary"]) mem.close() ```

> **💡 Tip**: Auto mode picks the lightest backend that fits your data. You can still use `mode="text"` or `mode="omni"` explicitly if you prefer. --- ### 🧬 Advanced: Optimize Retrieval Config Tune retrieval hyperparameters offline on your own dev set, then deploy the resulting `Config` for inference. This is a thin wrapper around EvolveMem's self-evolution loop: ```python import simplemem from simplemem import SimpleMem, load_config # mem is a finalized SimpleMem instance with memories already built dev_questions = [ ("When is the meeting?", "2pm tomorrow at Starbucks"), ("What should Bob prepare?", "market analysis report"), ] config = simplemem.optimize(mem, dev_questions, max_rounds=3) config.save("my_config.json") # Later, deploy with the optimized config config = load_config("my_config.json") mem = SimpleMem(config=config) ``` > EvolveMem runs an LLM-driven Evaluate → Diagnose → Propose → Guard cycle over your dev questions, adjusting global retrieval flags (top_k, fusion mode, answer verification, reflection rounds, ...). For the full standalone version with benchmark adapters and per-category overrides, see [`EvolveMem/`](EvolveMem/). --- ### 🚄 Advanced: Parallel Processing For large-scale dialogue processing, enable parallel mode: ```python from simplemem import create mem = create( mode="text", clear_db=True, enable_parallel_processing=True, # ⚡ Parallel memory building max_parallel_workers=8, enable_parallel_retrieval=True, # 🔍 Parallel query execution max_retrieval_workers=4 ) ``` > **💡 Pro Tip**: Parallel processing significantly reduces latency for batch operations! --- ## 🌟 Overview **SimpleMem** is a unified memory stack for LLM agents, built on one principle: store *semantically lossless* memory at high information density, so an agent recalls more while spending far fewer tokens. The package brings together three works that share this principle but attack different parts of the problem. ### 📝 SimpleMem: the efficiency core (text) Most memory systems force a bad trade-off. They either passively accumulate raw interaction history (redundant, token-hungry) or run expensive reasoning loops to filter noise (slow, costly). SimpleMem instead compresses interactions through a three-stage pipeline: | Stage | What it does | |:--|:--| | **1. Semantic Structured Compression** | Distills unstructured interactions into compact memory units (self-contained facts with resolved coreferences and absolute timestamps), each indexed through multiple complementary views for flexible retrieval. | | **2. Online Semantic Synthesis** | Merges related context within a session into unified abstract representations, removing redundancy as memory is built rather than at query time. | | **3. Intent-Aware Retrieval Planning** | Infers the search intent behind a query to decide *what* to retrieve and assemble a precise, compact context. | On the LoCoMo benchmark this delivers a 26.4% average F1 gain over prior systems while cutting inference-time token consumption by roughly 30x. Mechanism details (hybrid index layers, compression examples, retrieval planning): [**SimpleMem text memory →**](docs/text-memory.md). ### 🧠 Omni-SimpleMem: multimodal memory (text, image, audio, video) Omni-SimpleMem extends the compression-first philosophy to four modalities, built on three principles: **Selective Ingestion** (entropy-driven filtering per modality), **Progressive Retrieval** (hybrid FAISS + BM25 with pyramid token-budget expansion), and **Knowledge Graph Augmentation** (multi-hop cross-modal reasoning). Rather than being hand-designed, its architecture was *discovered* by an autonomous research pipeline that ran around 50 experiments across two benchmarks, diagnosing failure modes, proposing architectural changes, and even repairing data-pipeline bugs with no human in the inner loop. Tellingly, the bug fixes and architectural changes each contributed more than all hyperparameter tuning combined, taking the system from a naive baseline to state-of-the-art on both LoCoMo and Mem-Gallery. Full docs: [**Omni-SimpleMem →**](OmniSimpleMem/). ### 🧬 EvolveMem: self-evolving retrieval EvolveMem closes a blind spot shared by almost every memory system: the stored content evolves, but the *retrieval* machinery (scoring functions, fusion strategies, answer-generation policies) stays frozen after deployment. EvolveMem runs a closed-loop AutoResearch process (**Evaluate → Diagnose → Propose → Guard → Repeat**) in which an LLM diagnoses per-question failures and proposes configuration changes, guarded by automatic rollback on regression and exploration incentives during stagnation. It discovers new retrieval dimensions (query decomposition, entity-swap, answer verification) not in the original design, improves LoCoMo by 25.7% relative over the strongest baseline, and its evolved configurations transfer positively across benchmarks. Full docs: [**EvolveMem →**](EvolveMem/). ### How they fit together `from simplemem import SimpleMem` gives you the text core with automatic routing to the multimodal backend, and `simplemem.optimize(...)` taps EvolveMem to tune retrieval for your own data. One package, one mental model: compress losslessly, retrieve by intent, and let the system keep improving itself. --- ## 📦 Installation ### 📝 Notes for First-Time Users - Ensure you are using **Python 3.10+ in your active environment**, not just installed globally. - An OpenAI-compatible API key must be configured **before running any memory construction or retrieval**, otherwise initialization may fail. - When using non-OpenAI providers (e.g., Qwen or Azure OpenAI), verify both the model name and `OPENAI_BASE_URL` in `config.py`. - For large dialogue datasets, enabling parallel processing can significantly reduce memory construction time. ### 📋 Requirements - 🐍 Python 3.10+ - 🔑 OpenAI-compatible API (OpenAI, Qwen, Azure OpenAI, etc.) ### 🛠️ Setup ```bash # 📥 Clone repository git clone https://github.com/aiming-lab/SimpleMem.git cd SimpleMem # 📦 Install dependencies (pinned versions) pip install -r requirements.txt # — OR — install as an editable package pip install -e . # default: text + multimodal + evolver pip install -e ".[server]" # + MCP / HTTP server (mcp, fastapi, ...) pip install -e ".[all]" # everything, including dev tools # ⚙️ Configure API settings cp config.py.example config.py # Edit config.py with your API key and preferences ``` ### ⚙️ Configuration Example ```python # config.py OPENAI_API_KEY = "your-api-key" OPENAI_BASE_URL = None # or custom endpoint for Qwen/Azure LLM_MODEL = "gpt-4.1-mini" EMBEDDING_MODEL = "Qwen/Qwen3-Embedding-0.6B" # State-of-the-art retrieval ``` --- ## 🐳 Run with Docker The **MCP Server** can be run in Docker for a consistent, isolated environment. Data (LanceDB and user DB) is persisted in a host volume. ### Prerequisites - [Docker](https://docs.docker.com/get-docker/) and [Docker Compose](https://docs.docker.com/compose/install/) ### Quick run ```bash # From the repository root docker compose up -d ``` - **Web UI:** http://localhost:8000/ - **REST API:** http://localhost:8000/api/ - **MCP (SSE):** http://localhost:8000/mcp/sse?token=<TOKEN> Data is stored in `./data` on the host (created automatically). ### Custom configuration 1. Copy the environment template and edit it: ```bash cp .env.example .env # Edit .env: set JWT_SECRET_KEY, ENCRYPTION_KEY, LLM_PROVIDER, model URLs, etc. ``` 2. Run with the env file: ```bash docker compose --env-file .env up -d ``` ### Using Ollama on the host When `LLM_PROVIDER=ollama` and Ollama runs on your machine (not in Docker), set in `.env`: ```bash LLM_PROVIDER=ollama OLLAMA_BASE_URL=http://host.docker.internal:11434/v1 ``` On Linux, `host.docker.internal` is enabled automatically via the Compose file. ### Useful commands ```bash docker compose logs -f simplemem # Follow logs docker compose down # Stop and remove containers ``` > 📖 For self-hosting the MCP server (Docker or bare metal), see [MCP Documentation](MCP/README.md). --- ## 🔌 MCP Server *(text memory)* SimpleMem is available as a **cloud-hosted memory service** via the Model Context Protocol (MCP), enabling seamless integration with AI assistants like Claude Desktop, Cursor, and other MCP-compatible clients. **🌐 Cloud Service**: [mcp.simplemem.cloud](https://mcp.simplemem.cloud) — or self-host the MCP server locally using [Docker](#-run-with-docker). ### Key Features | Feature | Description | |---------|-------------| | **Streamable HTTP** | MCP 2025-03-26 protocol with JSON-RPC 2.0 | | **Multi-tenant Isolation** | Per-user data tables with token authentication | | **Hybrid Retrieval** | Semantic search + keyword matching + metadata filtering | | **Production Optimized** | Faster response times with OpenRouter integration | ### Quick Configuration ```json { "mcpServers": { "simplemem": { "url": "https://mcp.simplemem.cloud/mcp", "headers": { "Authorization": "Bearer YOUR_TOKEN" } } } } ``` > 📖 For detailed setup instructions and self-hosting guide, see [MCP Documentation](MCP/README.md) --- ## 📊 Reproduce Paper Results Reproduce the LoCoMo / MemBench / Mem-Gallery numbers from the papers. Each pillar has its own benchmark runner in its own directory. Install the benchmark extras first: `pip install -e ".[benchmark]"`. ### 📝 SimpleMem (text) — LoCoMo Run from the repository root: ```bash python test_locomo10.py # full LoCoMo benchmark python test_locomo10.py --num-samples 5 # quick subset python test_locomo10.py --result-file my_results.json ``` ### 🧬 EvolveMem — self-evolution + LoCoMo / MemBench Run from the `EvolveMem/` directory (see [`EvolveMem/README.md`](EvolveMem/README.md)): ```bash cd EvolveMem python run_evolution.py --data data/locomo10.json --max-rounds 7 python run_benchmark.py locomo --sample 0 --initial weak --max-rounds 3 python run_benchmark.py membench --agent FirstAgent --max-rounds 3 ``` ### 🧠 Omni-SimpleMem — LoCoMo / Mem-Gallery Run from the `OmniSimpleMem/` directory (see [`OmniSimpleMem/README.md`](OmniSimpleMem/README.md)): ```bash cd OmniSimpleMem python benchmarks/locomo/run_locomo.py --data-path /path/to/locomo10.json --model gpt-4o ``` --- ## 🗺️ Roadmap Current capability by integration channel: | Capability | Python (`pip install`) | MCP server (Claude Desktop, Cursor, ...) | |:--|:--:|:--:| | Text memory | ✅ | ✅ | | Multimodal (image / audio / video) | ✅ | ⬜ planned | | `optimize()` self-evolving retrieval | ✅ | ⬜ planned | Planned work to close the gap (the MCP server is a standalone multi-tenant text service; these are real features, not doc fixes): - [ ] **Multimodal over MCP.** Add `memory_add_image` / `memory_add_audio` / `memory_add_video` tools. Needs a file-upload path (base64 or URL, since MCP cannot pass local file paths), a multi-tenant adaptation of the Omni-SimpleMem storage backend, and server-side vision/audio model access. - [ ] **EvolveMem over MCP.** Expose `optimize()` as an MCP tool. More tractable than multimodal (text in, JSON config out, no file transport), but the MCP retriever currently honors only `semantic_top_k` / `keyword_top_k` of the ~10 dimensions EvolveMem evolves. Requires extending the MCP retriever to support the remaining knobs (structured top_k, fusion mode/weights, entity swap, query decomposition, answer verification), an adapter to run the evolution loop over a tenant's stored memories, per-tenant config persistence, and async execution (the loop is LLM-heavy and would time out a synchronous request). - [ ] **Docker** inherits both automatically once the MCP server supports them (add multimodal deps to the image and an Omni storage volume). For full multimodal and self-evolving retrieval today, use the Python API (see [Quick Start](#-quick-start)). --- ## 📝 Citation If you use SimpleMem in your research, please cite: ```bibtex @article{simplemem2026, title={SimpleMem: Efficient Lifelong Memory for LLM Agents}, author={Liu, Jiaqi and Su, Yaofeng and Xia, Peng and Zhou, Yiyang and Han, Siwei and Zheng, Zeyu and Xie, Cihang and Ding, Mingyu and Yao, Huaxiu}, journal={arXiv preprint arXiv:2601.02553}, year={2026}, url={https://arxiv.org/abs/2601.02553} } ``` ```bibtex @article{evolvemem2026, title={EvolveMem: Self-Evolving Memory Architecture via AutoResearch for LLM Agents}, author={Liu, Jiaqi and Ye, Xinyu and Xia, Peng and Zheng, Zeyu and Xie, Cihang and Ding, Mingyu and Yao, Huaxiu}, journal={arXiv preprint arXiv:2605.13941}, year={2026}, url={https://arxiv.org/abs/2605.13941} } ``` ```bibtex @article{omnisimplemem2026, title = {Omni-SimpleMem: Autoresearch-Guided Discovery of Lifelong Multimodal Agent Memory}, author = {Liu, Jiaqi and Ling, Zipeng and Qiu, Shi and Liu, Yanqing and Han, Siwei and Xia, Peng and Tu, Haoqin and Zheng, Zeyu and Xie, Cihang and Fleming, Charles and Ding, Mingyu and Yao, Huaxiu}, journal = {arXiv preprint arXiv:2604.01007}, year = {2026}, } ``` --- ## 📄 License This project is licensed under the **MIT License** - see the [LICENSE](LICENSE) file for details. --- ## 🙏 Acknowledgments We would like to thank the following projects and teams: - 🔍 **Embedding Model**: [Qwen3-Embedding](https://github.com/QwenLM/Qwen) - State-of-the-art retrieval performance - 🗄️ **Vector Database**: [LanceDB](https://lancedb.com/) - High-performance columnar storage - 📊 **Benchmark**: [LoCoMo](https://github.com/snap-research/locomo) - Long-context memory evaluation framework

simplemem-skill

Installation

SKILL.md