Why we rebuilt OCTA Flow’s execution engine from the ground up
By Asif Iqbal, Senior Software Engineer
Why OCTA replaced managed agent infrastructure with an execution engine it controls end to end—from sandboxing and durable workflows to caching and observability.
Over the past few months we replaced the foundation OCTA Flow runs on. We moved away from Anthropic’s managed sessions and managed sandbox for code execution, and replaced them with infrastructure we own end to end: our own agent loop, our own execution sandbox, our own durability layer, and our own observability pipeline.
This wasn’t a rewrite for its own sake. Managed infrastructure is convenient until you need control it doesn’t give you—over dependency staleness, over caching behavior, over what happens when a step fails halfway through a long-running job, over what data leaves your systems. We hit all four of those walls, so we built the layer underneath them ourselves.
Here’s what changed, and why each piece matters.
Owning the sandbox
OCTA Flow’s skills run code—Python and Node scripts that read source documents, transform them, and produce workpapers. Under the old architecture, that code was executed inside Anthropic’s managed sandbox. We didn’t control what was installed in it, when it was rebuilt, or how it aged.
We now run this code in Vercel Sandbox, against a Node and Python image that we bake and own ourselves. That means dependency versions are pinned by us, on our schedule, not silently rotated underneath us. It also means we can enforce our own zero data retention policy on execution, and get step-level logging we control end to end rather than whatever a third party’s sandbox happens to expose.
A provider-agnostic tool contract
We also decoupled our tool-calling layer from any single model provider. Core now owns the tool contract; Flow generates code against that contract with per-skill tool scoping. Practically, this means OCTA Flow isn’t structurally tied to Anthropic’s specific tool-use format—if we want to route work to a different model provider for a given skill, the contract layer makes that a configuration decision, not a rewrite.
Durable execution with DBOS
The biggest reliability change is durable workflow execution, built on DBOS. Every non-deterministic step—a model call, a tool call, a sandbox execution—has its input, parameters, and output captured as it happens. If a workflow crashes or a process dies mid-run, it resumes from the last completed step instead of starting over.
We didn’t take this on faith. Before adopting it, we ran a pilot with an explicit go/no-go bar: kill a running workflow mid-execution and confirm it resumes from the correct point without re-running already-completed steps, and confirm the storage overhead of checkpointing stays bounded as workflow volume grows. It passed on both counts—a mid-workflow crash resumed cleanly with zero completed steps re-executed, and checkpoint storage grew linearly (a few kilobytes per workflow) up through thousands of workflows, not superlinearly. That gave us a durability layer that behaves predictably at scale, not just in a demo.
On top of that, we scope authority and artifacts at the level of an individual run, backed by an append-only audit store. Every run has a tamper-evident record of exactly what happened—which steps executed, what they read and wrote, and what they cost.
Finding a real caching bug, and what it taught us
Prompt caching is where “own the request” earns its keep. Anthropic’s API lets you mark stable prefixes of a conversation so repeated context is billed at a fraction of the cost of a fresh read—but only if the exact same token prefix is sent turn after turn. Under the old architecture we didn’t have this level of control over how requests were assembled; under the new one, we do, and we instrumented cost per model call closely enough to see when caching wasn’t behaving as expected.
That instrumentation caught a real bug: two message-rewriting steps in our conversation pipeline were quietly changing the exact token prefix of history on every turn, which broke cache matching every single time. On one measured 58-turn run, that meant 88.5% of the run’s cost was billed at the cache-write rate instead of the cache-read rate—a difference of over 12x on the tokens affected, because the system was silently re-paying for context it should have been reusing. Fixing the two transforms restored cache hits to the conversation history they belonged to.
We’re sharing this not as a “we cut costs by X%” headline, but because it’s a good example of the payoff of owning this layer: caching correctness isn’t something you can verify from outside a managed API, and a single silent regression in message construction can erase the entire benefit of prompt caching without any error ever being thrown. Owning the request pipeline is what lets us catch it.
Seeing what actually happens in a run
None of the above is worth much if you can’t inspect it after the fact. We now generate traces and spans for every session and feed them into Langfuse, so any run’s turns and tool calls are inspectable end to end—which step ran, what it called, what it cost.
What’s next: grading run quality automatically
The last piece we’re building is a quality layer: a verifier, a structured findings taxonomy, and an eval harness that runs skills against golden baselines to score output quality automatically. This exists today and is actively used to catch regressions during development, but it’s not yet wired into production as an automatic release gate—we’re being deliberate about earning that before we let it block a deploy. We’d rather ship it slower and have it be trustworthy than turn it on early and have it be noise.
Owning every layer, end to end
Taken together, this is a shift from “rent someone else’s black box” to “own every layer we depend on for correctness.” We now control the sandbox our code runs in, the durability guarantees behind every run, the shape of every request we send to a model, and the visibility we have into what happened afterward. That’s the foundation the next phase of OCTA Flow builds on.