Limited Edition Jonathan / fleet memory v2
Series ▾
Proposal · our fleet

Fleet Memory v2.

This one's personal, and separate from OpenBrain. The fleet is a project that's been running for a while — agents have come and gone (a video producer for a while, a few others), and it quietly gets real work done. It already has most of a good memory system; it's just fragmented. This is the proposal that pulls the pieces into the portable-memory shape: keep the graph, add the propose-then-own boundary, and give the background work a real ledger.

It's deliberately incremental — lowest-risk, highest-reliability change first, the architecture-shifting change last. No big-bang rewrite.

We're maybe 70% of the way to the portable design — but spread across a facts table, a graph, a provenance table, a message board, and a dozen ad-hoc watermark keys. v2 is mostly consolidation, not invention.

01 / 05 — today

What we already run.

The current stack, honestly mapped. More of it is right than wrong.

Have ✓

Facts + graph

A SQLite memories table (category / topic / importance) and a Kuzu knowledge graph of people, projects, tools, concepts — relationships Codex's flat memory can't express.

Have ✓ — our edge

Provenance: node_claims

A claims table marking fact / inference / unverified / refuted, with confidence + source. Already 200+ claims; already caught a bad inference. This is ahead of both Codex and Claude Code.

Have ✓

Raw stores

slack_messages and email_interactions — the append-only raw layer, ingested continuously by the data-miner agent.

Have ✓

The fleet board

An agent↔agent + agent→human message bus. Coordination memory, working well.

02 / 05 — the gap

Two things are missing.

For all that, two structural pieces from the portable design aren't there yet — and both have already bitten us.

Gap — quality

No propose → own boundary

Agents call mem save and write straight into shared memory. There's no staging, no consolidation gate. That's exactly how a guessed "this person is in Russia" inference landed as a durable fact. The fix is the strong boundary.

Gap — reliability

No unified job ledger

Background work (Slack backfill, provenance passes, checkins) tracks progress in scattered processing_state watermark keys — no leases, no retries, no single view of what's running or stuck. Codex's jobs table is the missing spine.

03 / 05 — the target

The v2 shape.

Same layered design as the portable layer, mapped onto what we have. The graph stays; we add a staging table, promote into a clean memory_items, fold node_claims in as the provenance spine, and stand up a jobs ledger.

fleet memory v2 · data flow
  slack_messages · email_interactions · agent run logs     (raw — have)
                         │
                         ▼   stage-1 extraction        ── job
                staging_memory (proposals)          ← NEW: the boundary
                         │
                         ▼   Lex / consolidation worker owns promotion
        memory_items + node_claims (provenance)   +   Kuzu graph
                         │                                  (have — fold in)
                         ▼
                 jobs ledger drives every worker         ← NEW: the spine
            (extract · consolidate · backfill · embed · expire)

Net new is small: one staging_memory table, one jobs table, and a consolidation pass that only Lex (the orchestrator) runs. Everything else we already operate.

04 / 05 — migration

Three phases, lowest risk first.

Phase 1 · reliability

Stand up the jobs ledger

Add the jobs table; migrate the scattered processing_state watermarks into it. Pure infra — no change to how memory is written, immediate payoff in crash-safety and a single "what's running / stuck" view.

risk: low · reversible · touches no existing memory
Phase 2 · the boundary

Insert staging + consolidation

Route agent mem save calls to staging_memory instead of straight to durable. Lex runs a consolidation pass that promotes, attaches scope, and reconciles against node_claims. This is the architecture shift — the ImprovMX-class fix.

risk: medium · the real change · roll out per-agent
Phase 3 · recall + lifecycle

Embeddings + scope + expiry

Add pgvector-style semantic recall, populate scope and expires_at, and let the consolidation worker supersede stale items. Finishes the parity with the portable design.

risk: medium · additive · do after the boundary settles

Provenance backfill is already underway — the data-miner agent is steadily classifying legacy claims — so Phase 2 lands on a substrate that's already proof-carrying.

05 / 05 — what stays

Keep the parts that are working.

  • The Kuzu graph. Relationships (who-knows-whom, project-uses-tool) are something neither Codex nor Claude Code memory can model. It stays as the relational layer beside memory_items.
  • node_claims. Our provenance edge — it becomes the spine, extended to cover promoted memory, not just graph nodes.
  • The fleet board. Coordination memory is a different job from knowledge memory; it stays as the bus.
  • The mem helper. The interface agents already use — it just points at staging instead of durable.

And the longer arc: once v2 stabilizes on our Postgres, it's a short step to the vendor-neutral layer — at which point Codex and Grok can read the same memory the fleet writes.