Part 8

Where state lives

This page shows where each project stores configuration, runtime state, and snapshots, and what that choice means for the tools you operate with.

After this page you can:

State why ax keeps all of its state out of etcd.
State which kagent data lives in Kubernetes and which lives in PostgreSQL.
List the stores Substrate adds underneath both projects.

The same conclusion, a different line

Both projects keep high-churn agent state out of etcd. They draw the line in different places. ax keeps everything out of etcd, while kagent keeps the declarative definitions in Kubernetes and everything that changes per conversation in PostgreSQL.²⁹

Figure: ax puts configuration and runtime state in Redis, while kagent puts configuration in Kubernetes and runtime state in PostgreSQL; both rely on Substrate for snapshots and private Actor state.

ax: Tasks, Workspaces, and Models, plus locks and PubSub, are in Redis. kagent: Agents, Harnesses, AgentTemplates, and other CRDs are in etcd through the Kubernetes API; Sessions, A2A tasks and events, revisions, and checkpoints are in PostgreSQL. Below both, Substrate keeps Actor snapshots and each Actor's durable directory, and its install includes PostgreSQL and rustfs.

ax: Redis, no CRDs

ax DESIGN.md gives the reason directly: "Storing millions of short-lived tasks as Kubernetes CRDs pushes etcd past its comfort zone (single-digit GB storage limits, write-rate bottlenecks, control plane degradation)." ax stores its state in Redis and reconciles directly with Agent Substrate under fine-grained distributed locks.²

Because ax resources are not Kubernetes objects, kubectl cannot list them. You use the ax CLI, which is shaped like kubectl with apply, get, describe, watch, and delete.¹ GitOps tools that reconcile Kubernetes objects, such as Argo CD or Flux, have no ax objects to watch, so applying ax manifests from Git needs a step that calls ax apply.

kagent: CRDs for configuration, PostgreSQL for runtime

kagent's architecture doc states the split: "Kubernetes stores desired agent and sandbox configuration. PostgreSQL stores runtime identity, A2A history, and lifecycle state."⁹ A Session is PostgreSQL-backed control-plane state exposed through gRPC, and it is not a Kubernetes resource.¹⁰

Agent definitions are ordinary Kubernetes objects, so kubectl apply and GitOps workflows work on them.⁸ High-churn data, such as turns and events, stays out of etcd, and you operate a PostgreSQL database for it.

Placement by data type

Configuration: what you declare
Data	ax	kagent
Agent and runtime definitions	ax: Redis (Task, Workspace, Model).²	kagent: Kubernetes (Agent, Harness, AgentTemplate, SandboxTemplate).⁹
Compiled runtime input	ax: No equivalent. The Task runs the image as declared.³	kagent: the prepared revision, in PostgreSQL and ate-api as a Substrate ActorTemplate.⁹

Runtime: what changes while agents run
Data	ax	kagent
Execution record and status	ax: Redis, with locks and PubSub for watches.²	kagent: PostgreSQL Session rows, exposed by gRPC.⁹
Conversation history	ax: No equivalent. History is whatever your agent writes.³	kagent: A2A context, tasks, and events in PostgreSQL.⁹

Snapshots and private state: what Substrate holds
Data	ax	kagent
Sandbox filesystem	ax: workspaces mounted at their own paths in the Task.³	kagent: a Substrate `DurableDir` at `/data`, private to the runtime.¹⁰
Restart point	ax: the actor checkpoint taken by `SuspendTask`.²	kagent: a named checkpoint in PostgreSQL plus a Substrate snapshot tag.⁹

The stores Substrate adds

The Substrate install includes its own PostgreSQL and the rustfs object store.¹² Saying "ax uses Redis" is therefore incomplete: an ax install runs Redis plus Substrate's PostgreSQL, and a kagent install runs its own PostgreSQL plus Substrate's. Plan backups and capacity for every one of these stores.