01 / the opportunity
Lost in the agents' output. Memory could not stay in the black box.
the problem
I was running real work with agents: products, design, code. The useful parts were buried in the output. I left context and memory to the model. It did not hold. Feedback, decision junctions, and judgement calls sat in that flood, and they were hard to trace.
the gap
Chat tools remember a thread, but a thread is no place to trace a decision, a judgement, or feedback. Nothing sat outside the model. The model that wrote the work was its only judge, with no spec, no procedure, and no pass or fail. I needed a source of truth that was not the chat, and not the model.
why it's worth building
- Why now: agents already do the work. A reliable source of truth turns that work into insight and learning. I get better deliverables with AI, and I learn the process from the ground up.
- The process is the product: load, work, write back. A correction becomes a rule. I stay in the loop. The next session starts from there, not from zero.
- Self-hosted: context, memory, and retrieval optimization, on infrastructure I own. Every agent, every client, one source.
- The bet: when capture is automatic and judgement lives outside the model, agents learn. They load the source of truth, capture a decision, and turn a correction into a rule. The result is a multi-agent feedback loop that keeps learning.
the vision
A cloud-hosted brain as the source of truth, tuned for many agents. It keeps shipping better tools and methods for curating knowledge and running the learning loop. Live insight, better deliverables, and metrics that show whether the work improved.
02 / product decisions
Product decisions that shaped Orchestra.
-
product boundary
A private brain, shared across the tools.
Work is done in Cursor, Claude, and cloud agents, and each client holds only part of the memory. One brain was needed: loaded before work, written back to as it happens. Orchestra stays private, with one owner, so effort goes to the loop, not to accounts. Capture is written during the session, and the live notebook is shown on the website.
- trade-off
- No other users. No teams, no billing, no login for anyone else.
- payoff
- The next session, in any tool, is started from the same source of truth. A correction does not have to be said twice. Real data is shown on the screens, not a demo.
-
structure
Memory objects, each with a type.
The brain holds different memory objects: what happened, what was decided, what to watch for, and a few pages that have to be read whole. Each object is given a type and a shape. Records for the many. A document only when one living copy is needed. Anything that can be built from records is generated on request.
- trade-off
- A type is picked on every write. Fields are filled, and a shape check refuses an empty note or a decision without its reason.
- payoff
- A decision can be found later with its reason still on it. The living pages stay one source. Generated views do not go stale.
-
process
Keep the reasoning, not the transcript.
A transcript cannot be the source of truth. It is long, mixed, and gone when the tab is closed. So the record stays short, and the process is attached: the trigger, the alternatives, and the evidence. Reasoning is kept. The chat is not.
- trade-off
- No full replay in Orchestra. What was not written on the record cannot be opened later.
- payoff
- A decision can be read with the why still on it. The next session does not have to be taught again.
-
approval
A decision distribution system.
Judgements have to reach every agent. The app takes the decision: approve, edit, or reject, with the reason in the owner's words. Then it is distributed into the process the next model loads.
- trade-off
- Resolving happens in the app, outside the session that proposed the change.
- payoff
- The next model is guided by the decision. The judgement lives in the process.
03 / how it works
The brain, loaded before work and written as it goes.
An agent loads the brain through MCP, does the work, and capture writes most of it back on its own. I read what landed, and I decide what becomes a rule. The next agent starts from there.
-
01 / load
Open the brain
Guidelines, memory and skills for this job.
-
02 / work
Do the work
A session, a cloud agent, or both.
-
03 / capture
Write back, mostly on its own
Decisions, gotchas and what was done.
-
04 / decide
I stay in the loop
Lessons, proposals and what the team may trust.
30 mcp tools
- start_session
- get_guidelines
- recall
- remember
- list_activity
- groom
- list_lessons
- get_lesson
- write_lesson
- retire_lesson
- list_comments
- reply_to_comment
- get_skill
- begin_skill_run
- run_skill_acceptance
- list_proposals
- propose_change
- resolve_proposal
- read_project
- create_project
- update_project
- get_roadmap
- add_roadmap_item
- resolve_roadmap_item
- read_document
- update_document
- help
- invoke_agent
- create_agent
- validate_agent_reply
04 / design & interface
The brain has a process I can see.
Orchestra enables me to see the learning loop: capture, feedback, lessons, guidelines. I read the reasoning, and I step in when an approval, a proposal, or a comment is needed.
process
-
01 research
The work started by auditing real sessions and production data: reasoning, decision points, and the feedback loop. What had to be captured, and where the gaps were. The process was mapped. The map became the product, and the pages that make it visible.
-
02 product map
Define the product as data and process, mapped together. I use that map to design the interface. The right data in the right structure shows how the work ran, and where I need to step in.
-
03 journeys
A journey was mapped for each actor: the session agent, the cloud agent, and the owner. Goal, steps, and what they need to see or do at each point. That is how the product map becomes a flow.
-
04 design
A cohesive design system supports the data and experience layers from the product map and the journeys. Cohesion and hierarchy were the emphasis: one language from spec to screen. The experience makes the process visible, shows where the owner is, and gives a clear moment to decide.
-
05 build
The live product is the base. Test is part of building: each view sits on real records as it ships. If the data is missing, the product changes. Nothing dummy on screen.
-
06 iteration
Iteration and experimenting with new features and modules are core to how this product is made. A new module or view goes back through journeys, design, and build. Skills, Learning, Modules: the interface grows with the brain. Everything is tested live.
journeys
session agent
- load guidelines
- open a skill
- do the work
- capture writes back
session agent
- user corrects
- write a lesson
- next session loads it
session agent
- hand a job off
- context through memory
- check the result
session agent
- propose a document change
- I decide
- every agent reads it
cloud agent
- take a task
- work in the cloud
- write back to the brain
cloud agent
- coordinator plans
- worker runs
- report verified
owner
- Overview
- Needs you
- approve or deny
owner
- Skills
- open a skill
- read the procedure
owner
- Learning
- comments and proposals
- I decide the rule
-
overview
I stay in the loop.
Overview is the ops face of the brain. Fleet health, live work, and the things that need me. An agent can propose a destructive command. I deny it, or I approve and run.
-
skills
Skills live in the brain.
Agents open a skill when its description matches the job. One row is the job: name, when to use it, valid, who uses it, runs, lessons.
-
a skill
A skill is a process.
Why it exists, when it runs, then the steps. I read that. Checks sit on the process, not on a prompt dump. The file in the repo is the source.
-
capture
Most sessions write back on their own.
One Cursor session: an event, a gotcha, a lesson. Capture did that. I read what landed. The next session already has it.
-
modules
A visual guidance layer, from the live code.
Most of the work with agents is code. Modules tracks the project as it is built: layers, imports, the job of each file, and the problems on it. The map is drawn from the deployed code: a graph for structure, a file list for the work. Open a file, and what it pulls in is shown.
05 / outcome & next steps
Designed, built and run a cloud brain.
A working cloud brain for my agents, in daily use. Capture is automatic in most sessions. The process around it is how I run the work.
built platform
Hosted MCP brain, a web app to read and decide, a skills library, and a cloud agent team.
tech stack
Python MCP server; SQLite with full-text search; a Vite, React and Tailwind web app; Litestream to Tigris; hosted on Fly.io.
- Decisions, lessons and skills in one place every agent loads before work
- One brain
- Most sessions write back on their own. I read what landed.
- Automatic capture
- Load, work, write back, validate. I decide what becomes a rule.
- The process
What it enabled
Work runs with an agent team on the same brain. A correction given once is a rule the next session already has. Cloud agents take a task, work within a budget, and write the result back into the process.