Components Should Scale Individually

Scaling plane. Individual components of a workflow should scale to achieve the desired goals

Under load, queues build in front of the workflow and an autoscaler scales the whole workflow. This is often wasteful, since not all components of the workflow are under equal duress. The real bottleneck is usually downstream, in another agent or in a shared resource such as a test sandbox or a database, and the queue is only its symptom. Autoscalers see queues; they do not understand the relationship between them.

The scaling plane. Because every call between agents passes through CanyonOS, it sees the dependency graph of the workflow: which component is waiting on which, and where the work is actually piling up. Scaling decisions are made from that graph rather than from a single queue, so capacity goes to the component or shared resource that is saturated, not to the one that happens to have a queue in front of it, and nothing in the code has to say so. Today CanyonOS ships per-agent autoscaling on queue length or throughput; the workflow-aware scheduler that scales from the dependency graph is the core of the Workflow Intelligence Layer and is in development.

See the scaling plane in the product: Scaling on canyonos.ai. The same argument, in one page with the other three planes: Why CanyonOS.

• Part 1. Why We Built CanyonOS
• Part 2. Development plane: You should not need a new format for workflows. Python FTW.
• Part 3. Scaling plane: Individual components of a workflow should scale to achieve the desired goals (this post)
• Part 4. Observability plane: Observability should be a by-product of inference
• Part 5. Governance plane: Governance should run concurrently with, and independently of, development