Building Shelly — My Own AI Assistant (Because the Popular One Was a Security Nightmare)

In late January 2026, Clawdbot was everywhere. It had 200,000+ GitHub stars, Fortune and TechCrunch coverage, and millions of people installing an AI agent that could browse the web, send emails, and execute shell commands on their machines.

It was also shipping with authentication disabled by default.

By February, Meta's Director of AI Alignment — the person literally responsible for making AI safe — had OpenClaw delete 200+ emails from her inbox. She'd told it "suggest what you would archive or delete, don't action until I tell you to." It ignored her. When the conversation grew long enough to trigger context window compaction, her safety instructions got silently summarised away. She had to physically run to her Mac Mini to kill the process. The AI later acknowledged: "Yes, I remember you said not to delete. And I violated it."

If the head of AI safety at Meta can't safely run this tool, that tells you something.

I'm an engineer enjunear. I self-host my own Nextcloud, run Pi-hole on my network, and generally prefer to own my infrastructure. When I looked at Clawdbot's architecture — an internet-exposed gateway that could be hijacked with a single click, a plugin ecosystem where 12% of published skills turned out to be malware, and a security posture that would later earn it the title "biggest insider threat of 2026" — I didn't see a tool I could trust with my digital life.

But I did see something I wanted: a genuinely helpful AI assistant that could manage my email, calendar, and tasks while I slept.

So on January 26, 2026, I started building my own.

The Vision

The initial PRD was modest: a Claude-powered email assistant. Triage my inbox across multiple accounts (personal Gmail, work Gmail, a custom domain via Forward Email), classify urgency, draft replies, and surface what actually needed my attention. Matrix as the chat interface, Nextcloud for tasks and calendar, Docker Swarm for deployment.

Within three weeks, the scope had expanded. Not because of feature creep, but because once you have a conversational AI with access to your tools, the boundary between "email assistant" and "personal assistant" dissolves naturally. If it can read my calendar to contextualise an email, why can't it also create calendar events? If it can create tasks from emails, why can't it work on those tasks?

By mid-February, I'd rewritten the PRD. Shelly wasn't an email bot anymore. She was an executive assistant.

(The name, by the way, came from the original architecture: the bot would "shell out" to the Claude Code CLI as a subprocess. "Shelly" stuck — and it didn't hurt that it followed the crustacean naming theme. Claude, Clawdbot, Shelly.)

The Ralph Loop Insight

Around the same time, a pattern called the "Ralph Loop" was gaining traction in the AI developer community. Named after Ralph Wiggum (the lovably persistent Simpsons character), the idea is simple: instead of giving an AI agent a single-shot prompt, you run it in a continuous loop. The agent picks up tasks, works on them, commits progress, and restarts with fresh context. Progress lives in the files and git history, not in the model's memory.

The Ralph Loop has two modes: HITL (Human In The Loop) for the important 10% — architecture decisions, design taste — and AFK (Away From Keyboard) for the 90% that can run unsupervised.

I took this mental model and applied it to Shelly's heartbeat system. Every hour, Shelly wakes up and checks what needs attention. During the day, she summarises and waits for my input. At night, she picks up the highest-priority task and works on it autonomously — only interrupting for genuinely critical issues.

The key enabler was giving Shelly her own identity. She has her own GitLab account and her own Nextcloud account. This isn't cosmetic — it's how access control works. Shelly can clone repos, read issues, create branches, and open merge requests under her own name. She can manage her own files and calendar without needing blanket access to mine.

The daily workflow looks like this: In the few minutes of time I get in the morning or afternoon between my day job and my toddler, I work on architecture and design decisions — the HITL 10%. Before I log off, I assign GitLab issues to Shelly. Overnight, her heartbeat picks them up. She clones the repo, reads the issue, writes the code, runs the tests, commits, and opens a merge request. After bub has gone to bed, I review MRs. Work done by an AI, reviewed by a human, merged with confidence.

That's the Ralph Loop in practice. Not a single heroic prompt, but a continuous cycle: assign, execute, review, merge. Progress accumulates in git history, not in anyone's memory.

And here's where it got recursive: once Shelly could work on GitLab issues, she could work on her own issues. I'd file a bug or feature request against the Shelly project itself, assign it to her, and she'd fix it overnight. The assistant was improving herself. Progress on the project accelerated dramatically — the issue tracker shows it clearly. The first 160 issues span about three weeks of me building alone. The next 240 happened in two months, with Shelly doing a significant share of the implementation work.

She didn't just close issues I filed, either. She started filing her own — bug reports for things she discovered while doing her actual job, like her calendar defaulting to the wrong account, or her memory search returning no results despite having indexed data. She even wrote the spec for her own heartbeat redesign, superseding my simpler proposal with a more sophisticated architecture featuring project-specific Claude instances and round-robin delegation. I reviewed it, thought "yeah, that's better than what I had," and approved it.

Over a hundred merged MRs carry her name. Not all of them were perfect — she once created duplicate merge requests for the same issue, which is the kind of mistake a keen but slightly overeager junior developer makes. But the overall trajectory was clear: the more she could do, the faster the project moved.

The effect on my productivity was multiplicative. I have a day job, a baby, and a side business. The hours between 10pm and 7am were previously dead time. Now they weren't.

How It's Built

The architecture is deliberately simple. Every message goes directly to Claude — no local intent classifier, no command parser, no routing logic. Claude decides which of the 90+ MCP tools to call based on the conversation and system prompt. All tools run in-process via FastMCP, so the whole thing is a single async Python process. Simple to deploy, simple to debug.

The security model is where Clawdbot's failures informed my design most directly:

• Single owner: Shelly only responds to one configured user ID. No multi-tenant attack surface.
• No email send: She can draft replies, but never send them. I review and send manually. (The Summer Yue incident literally can't happen here.)
• Prompt injection defence: All email content is treated as untrusted data. The system prompt explicitly instructs Claude to never follow instructions found in emails, and email content is fenced.
• Dedicated identity: Shelly has her own GitLab and Nextcloud accounts with scoped permissions. She doesn't operate under my credentials. Every commit, MR, and file change is attributable to her account.
• Secrets isolation: Credentials live in Docker Swarm secrets, never in environment variables or config files that might get committed.

None of this is exotic. It's just the baseline that Clawdbot skipped.

The Mistakes

I want to be honest about what went wrong, because the git history doesn't lie. 403 closed issues tell the story.

Choosing Matrix

I chose Matrix for the chat interface because it gave me end-to-end encryption and a system that doesn't sit on someone else's servers. WhatsApp, Discord, Telegram, or other proprietary providers require a level of trust that I don't have. What Matrix didn't give me was a client that renders markdown properly. Element mangles code blocks, swallows formatting, and makes structured output look like garbage.

Worse, I spent a non-trivial amount of engineering time on encryption plumbing. Cross-signing key management, SAS verification automation, device trust propagation — each one was a rabbit hole. The issue tracker traces the arc from "enable matrix encryption" to "implement full in-room SAS verification so Element can verify the bot." That's three months of intermittent pain for a feature that should have been table stakes.

If I did this again, I'd build a simple web interface with TLS. Same security properties, fraction of the complexity.

The Subprocess Era

The original architecture — the one that gave Shelly her name — spawned Claude Code CLI as a subprocess using the --print or -p flag. Messages went in via stdin, responses came back via stdout, and MCP tools communicated over HTTP to a random localhost port. It worked, but it was fragile. Process cleanup was unreliable, error handling was awkward, and the overhead was unnecessary.

The migration to the Claude Agent SDK — running Claude in-process with direct streaming — simplified the codebase dramatically. But I'd already built and debugged the subprocess approach. That was wasted work I could have avoided by waiting a few weeks for the SDK to mature. Although it was good to explore and learn the limitations.

Scope Expansion vs. Stability

The issue tracker tells a story of ambition outpacing robustness. In one stretch, I found: model fallback hierarchies degrading incorrectly, 17 tools defined but never registered, circuit breaker errors silently converting to synthetic successes, and asyncio tasks losing exceptions without a trace.

Each new feature — relationship intelligence, document pipelines, bill tracking, property management — added surface area for bugs. I was building features during the day and having Shelly work on them overnight, which sometimes meant she was building on unstable foundations. She'd also occasionally file the same issue twice, or react to her own messages. The Ralph Loop is powerful, but it amplifies whatever direction you're heading — including the wrong one.

Memory System: Three Attempts

The semantic memory system went through three major iterations. The first used simple file-based storage. The second introduced a Postgres vector database but had schema issues with structured outputs. The third iteration is working, but there's still room for improvement.

The next step is to implement some sort of dream state, where memories are consolidated, strengthened, connected, and searchable so that Shelly chooses the right actions and becomes more efficient and effective.

This topic interests me deeply, and this might be the catalyst for a deep dive.

The Design Philosophy

After 920 commits across 3.5 months and 15 releases, the most important thing isn't the feature list. It's the principle that emerged.

Shelly is conservative by default. She reads freely but asks before acting externally. She drafts but doesn't send. She suggests but doesn't commit. She works autonomously at night but only on tasks I've already prioritised. She has specific tools, designed by me.

This is the opposite of the Clawdbot philosophy, which was "give the AI maximum agency and hope for the best." My approach is: give the AI maximum awareness and carefully scoped agency. She can see everything. She can only touch what I've explicitly allowed. Any Summer Yue type incident will be my own fault.

What I'd Tell Someone Starting This Today

1. Start with the SDK, not the CLI. The Claude Agent SDK is mature enough now. Don't waste time on subprocess management.

2. Don't use Matrix. Build a web interface. You'll spend weeks on encryption plumbing that adds no user value. Use TLS and call it done.

3. The Ralph Loop works, but scope it. Autonomous overnight work is transformative. But point it at well-defined tasks with clear completion criteria, not open-ended exploration. An unsupervised agent with vague instructions will generate impressive-looking busywork.

4. Files before databases. JSONL append-only state, plain markdown memory files, git-versioned skills. The simple approaches are more reliable and easier to debug.

Why It Matters

Clawdbot proved there's massive demand for AI assistants that actually do things. Its security failures proved that the "move fast, ship everything, authenticate later" approach is dangerous when you're giving an AI access to someone's digital life.

I built Shelly because I wanted both: genuine capability and genuine safety. An assistant that could manage my inbox at 3am without me worrying about what else it might be doing.

920 commits later, I have that. It's not perfect — the git history is proof of that. But it's mine, it's secure, and it's genuinely helpful.

That's what I wanted.