MultiGPT

Roadmap

MultiGPT is built milestone by milestone. After each milestone the tests run, there is a short report, and then work continues. As of version 0.4.0, milestones 1 to 8 including 4a, the runner (14, with a few open points) and the node (15) are done, images (9), operations (12) and the scratchpad (13, so far the context menu and storage of files from tools) partly. Each milestone lists its work packages with their status. Voice (10) and music (11) are still open.

Symbols for the work packages: ✓ done, ◐ partially done, ◔ in progress (not yet released), ○ planned.

  1. Milestone 1 Project skeleton

    Done

    Django project configured from the environment, Makefile, login with brute-force throttling, gunicorn (gthread), health check /healthz/, Debian package with systemd unit and mgpt-ctl, Docker files. With a local PostgreSQL the package creates the role, database and pgvector, migrates automatically, asks for host names and certificate via debconf and sets up nginx with TLS.

    Tests, linter and make deb pass, and versions are tagged with make release (currently 0.4.0). Still open for acceptance: a test installation on a fresh Debian 13 and the Docker build (untested).

    Work packages: 9 of 9 done
    • M1-01 Django project and dependencies
    • M1-02 Settings from the environment only
    • M1-03 PostgreSQL with pgvector
    • M1-04 Login with brute-force throttling
    • M1-05 Base layout with header and sidebar
    • M1-06 Makefile for setup, tests and packaging
    • M1-07 gunicorn and health check
    • M1-08 Debian package with service and admin command, Docker files
    • M1-09 Test setup and linter

  2. Milestone 2 Data model and admin

    Done

    Providers, models, roles and groups; accounts extend Django’s User and Group. Encrypted API keys (Fernet, the admin area shows only the last 4 characters), a central permission check, admin area for administrators only. Private chats appear there only as metadata. Four default roles: administrator, adult, teenager, guest.

    Complete. Teenagers and guests start without models; the administrator enables them per role.

    Work packages: 6 of 6 done
    • M2-01 Data model: providers, models, roles, groups
    • M2-02 Encrypted API keys
    • M2-03 Admin forms
    • M2-04 Four default roles and the “family” group
    • M2-05 Central permission check
    • M2-06 Create accounts with a role

  3. Milestone 3 First provider and streaming

    Done

    OpenAI-compatible adapter (OpenAI, OpenRouter, LM Studio), streamed answers with a stop button, regenerate, model per message, chat view with saved history.

    Tested end to end in the browser; streamed answers with a real OpenAI key work.

    Work packages: 6 of 6 done
    • M3-01 Common provider interface
    • M3-02 Adapter for OpenAI-compatible providers
    • M3-03 Streamed answers
    • M3-04 Stop button, partial text is kept
    • M3-05 Chat view with model choice and “regenerate”
    • M3-06 Tests: event order, errors, stopping, other users' chats

  4. Milestone 4 More providers and LM Studio

    Done

    Adapters for Anthropic and Google Gemini, fetching model lists with make sync-models, LM Studio as a local provider with online indicator and automatically offered models.

    LM Studio runs in a real home network. Since 0.3.0 new models get their capabilities automatically (for LM Studio from its model list), plus the capability matrix in the admin, greyed-out offline providers and a dedicated message for expired API keys. Still open for acceptance: real calls to Anthropic and Gemini.

    Work packages: 5 of 5 done
    • M4-01 Adapters for Anthropic and Google Gemini
    • M4-02 Fetch the providers' model lists
    • M4-03 Provider online status
    • M4-04 Status indicator in the chat, offline models greyed out
    • M4-05 Local models free of charge, no sending to offline providers

  5. Milestone 4a MCP tools

    Done

    Models use tools from connected MCP servers – with permission checks, confirmation before sensitive calls and display in the chat. Plus built-in tools for calculations and PDF sheets.

    Tool loop with at most 10 rounds, confirmation before unapproved tools (also after a reload), every call shown in the chat. Since 0.3.0: import from “mcpServers” JSON, online status per server, a tool table with classification in the admin, MCP access per model (none, all, selected; checked on the server) and the run_python tool in a bubblewrap sandbox. Since 0.3.1 the create_pdf tool for worksheets, letters and tables. MCP tools also work with local models via LM Studio. Since 0.3.2 a model with many tools gets only a core set and looks up more with find_tools; MultiGPT estimates the request before sending and drops older messages if needed. Since 0.3.5 administrators see offline MCP servers live in the status bar, and since 0.3.6 scipy is available in the sandbox.

    Work packages: 12 of 12 done
    • M4a-01 Bridge between web server and MCP
    • M4a-02 Tool calls in all three adapters
    • M4a-03 MCP client with connection test
    • M4a-04 Tool loop with round limit and permission check
    • M4a-05 Confirmation before sensitive tools
    • M4a-06 Collapsible tool calls in the chat
    • M4a-07 Tests with a built-in MCP test server
    • M4a-08 MCP access per model since 0.3.0
    • M4a-09 Online status and tool list per MCP server since 0.3.0
    • M4a-10 Calculations with Python in a sandbox since 0.3.0
    • M4a-11 Worksheets and texts as PDF since 0.3.1
    • M4a-12 Tools on demand (`find_tools`) and protection against a too small context since 0.3.2

  6. Milestone 5 Convenience

    Done

    Markdown with code highlighting and a copy button (libraries bundled locally, output sanitised), formulas with KaTeX, SVG preview, automatic chat titles, rename, archive, delete, search, Markdown export, system prompt per chat. Plus attachments, projects and shared chats.

    Complete. Follow-ups: editing your own messages with versions (“‹ 1/2 ›”) and a copy button. Version 0.3.0 added attachments with image input (EXIF removed), projects with instructions, default model and collections, chat sharing with RWUD permissions and a reworked interface (full-width answers, collapsible sidebar, font size A−/A+). Since 0.3.2 the history only follows a streaming answer while you are at the bottom; since 0.3.4 arrow up/down recalls earlier prompts, and an answer can be deleted together with its question and everything after it; since 0.3.5 there is a personal menu with a profile picture; since 0.3.6 notices to the owner of shared chats and the version display.

    Work packages: 7 of 7 done
    • M5-01 Markdown, code highlighting and formulas
    • M5-02 Titles, rename, archive, delete, search
    • M5-03 System prompt per chat
    • M5-04 Export as Markdown
    • M5-05 Edit messages, switch between versions
    • M5-06 Share chats with permissions since 0.3.0
    • M5-07 Projects since 0.3.0

  7. Milestone 6 Comparison, usage, budgets, family

    Done

    The same question to several models, usage overview, monthly budgets and the “Family” admin page.

    Implemented: comparison mode with two or three models, the answers stored as versions in the chat tree (tools are off in comparison); usage and monthly budgets with a warning from 80 % and a block from 100 % except for free models; a personal usage page; a “Family” page with accounts, roles, budgets and groups, plus read-only insight into teenagers’ chats only with an option and a permanent notice. Since 0.3.0 billing runs through accounts: monetary, tokens only or flat rate, prices with validity dates including cache and reasoning, exchange rates, bookings and budgets per account. Since 0.3.4 MultiGPT uses Anthropic prompt caching; since 0.3.6 the worker fetches the ECB USD→EUR reference rate automatically, and EUR accounts can be switched to USD.

    Work packages: 10 of 10 done
    • M6-01 Comparison mode with 2–3 models
    • M6-02 Usage overview
    • M6-03 Budgets: warning from 80 %, block at 100 %
    • M6-04 “Family” page
    • M6-05 Viewing teenagers' chats only with opt-in
    • M6-06 Billing accounts per provider since 0.3.0
    • M6-07 Prices with validity dates since 0.3.0
    • M6-08 Booking per answer since 0.3.0
    • M6-09 USD/EUR exchange rates since 0.3.0
    • M6-10 Budgets per billing account since 0.3.0

  8. Milestone 7 Your own documents (RAG)

    Done

    Collections, upload, background indexing, search with pgvector, sources with page numbers.

    Implemented: collections with sharing, upload (PDF, DOCX, TXT, MD), a worker with retries and OCR (Tesseract), search with access filtering in SQL plus full text, sources with page numbers, a document search tool and a dedicated RAG section in the admin. Embeddings locally via LM Studio (nomic-embed-text, 768 dimensions), OCR with olmOCR via LM Studio and Tesseract as a fallback, both testable with a button in the admin; directory sources from the server or NAS, limited to RAG_SOURCE_ROOTS and read in periodically. Since 0.3.0: citations with section, page and paragraph in six styles including bibliographic data and standards, figures described by a vision model, image files as documents, the document tools list_documents, document_info and read_document, viewing documents in the browser and cancelling jobs. Since 0.3.2 MultiGPT loads local models into LM Studio itself when needed, never beyond a memory budget; since 0.3.4 the indexing models stay loaded only for the run; since 0.3.5 several documents are indexed at once; since 0.3.6 runs can be paused and resumed.

    Work packages: 13 of 13 done
    • M7-01 Collections, documents and sharing
    • M7-02 Background worker for jobs
    • M7-03 Text extraction, chunking, embeddings
    • M7-04 Vector index and reindexing
    • M7-05 Search with access filter and sources
    • M7-06 Document search as a tool
    • M7-07 Tests for result quality and access limits
    • M7-08 Dedicated document admin
    • M7-09 Local embeddings and OCR via LM Studio; extended in 0.3.2: automatic model loading within a memory budget
    • M7-10 Directory sources
    • M7-11 Describe figures, images as documents since 0.3.0
    • M7-12 Tools to list and read documents since 0.3.0
    • M7-13 Citing with page, paragraph and section, citation styles since 0.3.0

  9. Milestone 8 Web search

    Done

    Swappable search backend, protected page fetching, sources below the answer.

    Implemented with a self-hosted SearXNG: a “web search” switch in the chat and a tool for models, “SearXNG testen” in the admin, page fetching without access to home-network addresses, web text only as marked source material. Since 0.3.0 the fetch_url and crawl_site tools (SSRF-protected, respecting robots.txt), URLs from the question, flagging of invented links and base rules for all models. No search API is connected.

    Work packages: 9 of 9 done
    • M8-01 Search via SearXNG
    • M8-02 Page fetching without access to the home network
    • M8-03 Web text as marked source material
    • M8-04 Sources below the answer, web search as a tool
    • M8-05 Tests against intranet addresses and redirects
    • M8-06 “Fetch page” tool since 0.3.0
    • M8-07 “Crawl website” tool since 0.3.0
    • M8-08 Safeguards against invented facts and links since 0.3.0
    • M8-09 Visible base rules, adjustable creativity and reasoning effort since 0.3.1

  10. Milestone 9 Images

    Partially done

    Image generation, editing a marked area (inpainting), variations and classic editing via a bundled MCP server.

    M9-01 done (0.3.0): image generation with OpenAI gpt-image and Gemini, as the chat model’s generate_image tool, in “image” mode and via a “generate with image model” hint for models without tools; images without metadata, cost per image. Still open: editing areas, variations and the MCP server for classic editing.

    Work packages: 1 of 5 done
    • M9-01 Image generation (OpenAI, Gemini) since 0.3.0
    • M9-02 Edit a marked area (mask) and variations
    • M9-03 Classic image editing via a bundled MCP server
    • M9-04 Protected delivery of images (delivery via nginx still open)
    • M9-05 Tests for image editing

  11. Milestone 10 Voice

    Planned

    Voice input (recording → text) and reading answers aloud, over HTTPS in the browser on PC and phone.

    Work packages: 0 of 3 done
    • M10-01 Voice input: recording becomes text
    • M10-02 Read answers aloud
    • M10-03 Limits for recording length and file size

  12. Milestone 11 Music

    Planned

    Newly planned; the exact scope is still being defined.

    The work packages will follow once the scope is settled.

  13. Milestone 12 Operations

    Partially done

    Worker unit, nginx example with TLS, backup, complete installation guide, verified package upgrade and removal.

    The worker unit multi-gpt-worker.service already ships in the package. The nginx site with TLS ships in the package as well: gunicorn listens on localhost only, and debconf asks for the host name and certificate. The mgpt-ctl admin commands are documented in the wiki, and debian/NEWS lists the steps after an update. Since 0.3.4 GitHub Actions builds the package for every version tag and publishes it in a signed apt repository. Still open: backup, a complete guide and verified upgrade and removal.

    Work packages: 1 of 4 done
    • M12-01 nginx with TLS in the package
    • M12-02 Backup of database, files and settings
    • M12-03 Complete installation guide
    • M12-04 Verified package upgrade and removal (automated tests exist, check on Debian 13 open)

  14. Milestone 13 Scratchpad

    Partially done

    Collect material from chats, your own documents, the web, tools, images and notes, each with its origin and citation data, and turn it into a fully referenced document. The model writes only from the collected entries. MultiGPT sets the in-text citations and the bibliography in the chosen citation style and checks verbatim quotes against the source. Export as Markdown, DOCX and PDF. Plus a context menu in the chat with “add to scratchpad”, follow-up questions, quick actions and your own prompt templates.

    The context menu in the chat has been available since 0.3.2: quote selected text with its origin, ask a follow-up, explain, simplify, translate, find counter-arguments and evidence. “Add to scratchpad” and your own templates are still missing. Prompt templates and the scratchpad itself follow; it attaches to projects.

    Work packages: 0 of 13 done
    • M13-01 Context menu in the chat since 0.3.2
    • M13-02 Prompt templates
    • M13-03 Citing for all kinds of sources
    • M13-04 Data model and permissions
    • M13-05 Collect material with its origin
    • M13-06 Scratchpad interface
    • M13-07 Outline
    • M13-08 Have sections written and revised
    • M13-09 Citation check
    • M13-10 In-text citations and bibliography
    • M13-11 Export as Markdown, DOCX and PDF
    • M13-12 Projects and sharing
    • M13-13 Privacy, security, documentation

  15. Milestone 14 Runner

    Done

    A container on the computer the browser runs on (rootless Podman; on Windows and macOS via WSL2 or a Linux VM) as a workspace for models: they read, search and edit files, run commands, test and fix – like a coding tool, but locked in. By default the container has no network (with the “package sources” permission only through a proxy to fixed package sources) and only a volatile in-memory workspace; the work is stored on the server as a git bundle with its full history and is backed up with the database. Pairing uses a token (stored only as a hash, revocable, with an expiry date), and the runner connects outgoing only, via WebSocket over TLS. Every access goes through MultiGPT: the server checks permissions, confirmation modes, quota and the audit log. In the chat there is a terminal in the browser, a preview for HTML pages in a sandboxed frame and the model’s task list. Devices on the home network can be controlled via ssh or telnet – only those the administrator approves, and with a confirmation before every command. Usage is billed through the “runtime” account type in container minutes per role or person and month, with a warning from 80 % and a stop at 100 % – the work is kept.

    Released with version 0.4.0. Changed from the first plan: the work is stored as a git bundle on the server instead of volumes on the client, the model works in many rounds like a coding tool instead of running single commands only, plus ssh and telnet to approved devices, package sources through a proxy with an allow list, a web server (lighttpd) and the agent as a Rust program multigpt-runner. The runner’s web ports are by default reachable by the whole home network without any login. Still open: Docker is not supported (Podman only), and incremental bundles, container logs on the “Runner” page and live output of tool runs in the chat terminal are missing.

    Work packages: 12 of 15 done
    • M14-01 Data model and permissions since 0.4.0
    • M14-02 Pairing via token on the web since 0.4.0
    • M14-03 Frame protocol since 0.4.0
    • M14-04 Runner service on the server since 0.4.0
    • M14-05 Delivery via PostgreSQL (LISTEN/NOTIFY) since 0.4.0
    • M14-06 Runner tools in the chat (MCP transport) since 0.4.0
    • M14-07 Packaging and nginx since 0.4.0
    • M14-08 Runner program, groundwork since 0.4.0
    • M14-09 Work container and tools since 0.4.0
    • M14-10 Workspaces and sessions (git bundle) since 0.4.0
    • M14-11 “Runtime” time quota since 0.4.0
    • M14-12 Management, audit log, admin since 0.4.0
    • M14-13 Container images and distribution since 0.4.0
    • M14-14 Security, privacy, documentation since 0.4.0
    • M14-15 End-to-end test with real Podman since 0.4.0

  16. Milestone 15 Node

    Done

    MultiGPT as an MCP server: every account can create API keys whose permissions never exceed its own role. Other programs such as n8n, Claude or Cursor use them to ask models, search and upload documents, start and follow indexing and fetch generated files. Keys are stored only as a hash, can be revoked and optionally expire; every call is recorded in the audit log without content. Plus command-line commands and chat tools for indexing.

    Done with version 0.3.2. Since 0.3.3 the tools return their results directly (text, images, PDF); since 0.3.6 keys with the “read chats” permission can read the account’s own chats and chats shared with it, indexing runs can be paused, and chats created by ask go into an “API” project. Still open for acceptance: n8n workflows against an installed package and the older transport with various clients in practice.

    Work packages: 12 of 12 done
    • M15-01 API keys per account with permissions within the role since 0.3.2
    • M15-02 MCP server with tools since 0.3.2
    • M15-03 “API keys” page since 0.3.2
    • M15-04 Overview in the admin area since 0.3.2
    • M15-05 Command line for indexing and API keys since 0.3.2
    • M15-06 Chat tools for indexing since 0.3.2
    • M15-07 nginx integration since 0.3.2
    • M15-08 Guide for n8n since 0.3.2
    • M15-09 Tests since 0.3.2
    • M15-10 Documentation since 0.3.2
    • M15-11 Older MCP transport (HTTP+SSE) since 0.3.2
    • M15-12 Results as MCP content (text, images, PDF) since 0.3.3

Order

The critical path is 1 → 2 → 3 → 4 → 4a: everything that uses tools (document search, web search, calculations, PDF sheets, image generation and the runner) depends on the MCP loop. Convenience (5) and usage/budgets (6) could run in parallel after milestone 3. HTTPS with nginx already ships in the package and is therefore ready before the voice features (10).

For the scratchpad (13), the context menu came first (0.3.2); prompt templates are next. The runner (14) builds on MCP, the billing accounts from milestone 6, nginx with TLS and the node (15): its tools run through the chat’s MCP loop, and agents too reach it only through a MultiGPT API key, never directly.

Later (after version 1)

Full-text search across all messages and a “memory across chats” (proposal, not yet approved).