Building memo — a fully offline AI assistant

In early 2026 I asked myself a simple question: does an AI assistant really need the cloud to know me? The answer became memo — 1200+ commits and the biggest project I've ever built.

The architecture

memo has two halves:

• Go backend — voice pipeline, agents, local file access.
• Flutter frontend — desktop UI, voice chat, memory browser.

Inference runs on llama.cpp, so the model lives on your machine and works with no internet. For retrieval I use sqlite-vec for embedding search over my documents and past messages — everything stays local.

// Starting the local model from Go
lm := llama.New("models/qwen-3b-q4.gguf")
reply, err := lm.Chat("What are we building today?")

The voice pipeline

Voice was the hardest part:

• VAD — detect when you stop talking, then reply.
• Barge-in — stop the reply when you start speaking.
• AEC (acoustic echo cancellation) — cancel the speaker's echo from the mic signal.

Together they create a conversation that feels live. Piper handles local TTS on the reply side.

What I learned

1. Local-first is a technical decision, not a philosophy. Data layout, model size and latency all change when you can't cheat with a server.
2. Conventional commits scale. feat(voice): barge-in keeps a 1203-commit history navigable.
3. Test culture pays off. 114+ Flutter tests and a Go suite — time-sensitive voice code needs deterministic tests.

memo is fully open source: github.com/BugraAkdemir/memo.