How does an autonomous personal AI assistant actually work?
The PiEmbeddedRunner agent loop, 24+ channels, plugin architecture, SQLite memory, and security model, mapped from source.
Five boundaries: external channels and triggers, the Gateway on your machine, device nodes on your phone/Mac, external model providers, and external state the agent can reach via tools.
The PiEmbeddedRunner drives a continuous async loop. Messages arrive, context is compiled by prompt-builder.ts, the Pi agent core streams tool calls, and results route back through auto-reply.ts.
Each circle is a phase of the agent loop. Click to explore.
Eight subsystems. The Gateway orchestrates everything through a WebSocket control plane. Plugins extend channels, tools, memory, and model providers.
Each block is a subsystem. Click to explore.
Core channels ship with OpenClaw. Additional channels load as plugins from extensions/. All route through auto-reply.ts.
Unified routing via auto-reply.ts. DM policies, group rules, allowlists per channel.
The plugin loader (src/plugins/loader.ts) scans extensions/ for packages declaring openclaw.extensions in package.json. Four plugin types register into distinct slots.
Registers into the channel router. Implements auth, inbound parsing, access control, outbound formatting. MS Teams, Matrix, WeChat are all channel plugins.
Registers into the tool registry. Extends what the agent can do — browser, canvas, custom API integrations. Managed via ClawHub or workspace skills.
Registers into memory search. memory-core (extensions/memory-core/) provides SQLite-backed indexing and vector search. Replaceable with custom backends.
Registers into model providers. Claude, GPT, Gemini, local models. OAuth + API key auth with failover chains. TypeBox schema validation.
50+ built-in tools. Sandboxed via Docker for non-main sessions. Skills managed through ClawHub or workspace directory.
Not flat markdown files. OpenClaw uses SQLite-backed memory with indexing and search, powered by the memory-core plugin. The prompt-builder queries memory search to inject relevant context per task.
Persistent storage via memory-core plugin in extensions/memory-core/. Replaces append-only markdown with structured, queryable storage.
Facts and session history indexed for retrieval. Automatic extraction from conversations. Structured for fast lookup by the Context Compiler.
The prompt-builder queries src/memory/ to find relevant context per task. Only matching facts injected — never the full history. Prevents context overflow.
Immutable directives (SOUL.md), owner profile (USER.md), persona routing (AGENTS.md), and capability defs (TOOLS.md) still live as markdown in ~/.openclaw/workspace/.
resolveSessionKey() in sessions.ts determines session ID: main for direct, dm:channel:id for DMs, group:channel:id for groups.
1:1 with owner. Full host access. Tools run on machine. No sandbox. Session key: main
Isolated per-channel. Mention or always-on activation. Queue modes. Key: group:channel:id
Non-main in Docker via sandbox.ts. Allowlist: bash, read, write. Denylist: browser, canvas, nodes.
Three layers: inbound access control (DM pairing + allowlists), tool policy and exec security (profiles + ask mode), and sandbox isolation (Docker per-session). Run openclaw security audit to check posture.
Code issued, msg ignored
openclaw pairing approve
requireMention · activation
all · allowlist · quote
messaging · minimal · full
per-agent deny groups
full · deny · always · off
per-session toggle
Default for personal setups: security="full", ask="off" · Tighten per your threat model
Direct execution
per-session container
bash,read,write,edit ✓
browser,canvas,nodes ✗
Slash commands in any channel + CLI commands via Commander.js.
Four patterns. Same client interfaces (CLI, Web UI, mobile apps) across all.
Developer machine
~/.openclaw/
Loopback
macOS App
~/.openclaw/
Loopback + Tailscale
VPS/VM
~/.openclaw/
SSH tunnel
Docker container
Persistent volume
HTTPS ingress
Architecture is abstraction. Here's what it looks like running.
Cron fires. prompt-builder.ts loads SOUL + USER + memory search results. Weather, inbox, calendar skills. Summary to WhatsApp via auto-reply.ts.
"Hey Molty, first meeting?" Wake word triggers on macOS node. PiEmbeddedRunner fetches calendar, responds via ElevenLabs TTS.
Meeting request found. Agent drafts reply. Tool policy allows email send (exec security="full", ask="off" in personal mode). Email sends. If ask="always" were configured, owner would see a prompt first.
WhatsApp message from unknown. dmPolicy="pairing" kicks in. Code issued. Ignored until openclaw pairing approve.
"Analyse competitor pricing." Browser tool launches managed Chromium via CDP. Snapshots, extracts. All Read.
Research session uses sessions_send. Main session synthesises. Canvas push via A2UI.
Cron reviews. memory-core indexes new facts into SQLite. Session pruning compacts context. Tomorrow's prompt-builder has today.
The stack under the hood.