Part 5

Run a first kagent agent

Create a kind cluster with kagent's build of Substrate, install kagent v1.0.0-alpha6 on it, declare a Harness, an AgentTemplate, and an Agent, then create a Session and converse with it.

After this page you can:

Install the Substrate fork kagent needs, with the credential provider that injects the model API key.
Declare a Harness, an AgentTemplate, and an Agent, and read the Agent's conditions until it is Ready.
Create a Session, send it two turns, and show that it keeps its history.
Explain which revision a Session runs after you change the Agent.

Before you start

Complete the shared setup checklist first. This lab does not use the upstream Substrate cluster from that page: kagent v1.0.0-alpha6 requires its own Substrate fork, kagent-dev/substrate v0.3.0-alpha3, and both builds install into ate-system.³²³⁰ You need Docker, kind, kubectl, Helm, jq, openssl, and an OpenAI API key, because the kagent chart's default model provider is OpenAI.³¹²⁷ Everything below targets kagent v1.0.0-alpha6 (commit 9708e79) and follows kagent's own scripts/setup-cluster/setup-cluster.sh at that commit, using the published charts and images instead of local builds.³¹²³²⁵²⁶

Create the cluster and install kagent's Substrate

kagent never hands the model API key to the agent runtime. Its compiler emits egress bindings, and Substrate's gateway fetches the referenced Kubernetes Secret and replaces the outgoing HTTP header. That needs the fork's credential provider, a namespace policy that lets atespace kagent read Secrets in namespace kagent ("Empty grants deny all credential fetches"), and an egress-mitm-ca-pool CA in ate-system.³²

Create a kind cluster named kagent from a kagent checkout at the release tag. The Make target creates a local registry, a cluster with the certificates beta APIs enabled, and MetalLB; the kubectl context is kind-kagent.³¹³⁴
```
git clone --branch v1.0.0-alpha6 https://github.com/kagent-dev/kagent.git
cd kagent
make create-kind-cluster
kubectl config use-context kind-kagent
```

Download kubectl-ate from the fork's release. It runs on your machine and mints the CA and JWT pools.³¹

SUBSTRATE_VERSION=0.3.0-alpha3
OS="$(uname -s | tr '[:upper:]' '[:lower:]')"
ARCH="$(uname -m)"; [ "$ARCH" = x86_64 ] && ARCH=amd64; [ "$ARCH" = aarch64 ] && ARCH=arm64
curl -fsSL -o kubectl-ate \
  "https://github.com/kagent-dev/substrate/releases/download/v${SUBSTRATE_VERSION}/kubectl-ate-${OS}-${ARCH}"
chmod +x kubectl-ate

Install the fork's CRD chart and main chart, with the credential provider grant. The main chart also deploys Substrate's PostgreSQL and a RustFS object store with the bucket ate-snapshots.³¹³² kagent's script also passes two extraArgs for local image builds and fast CI resyncs; this lab uses published images and leaves them out.

helm upgrade --install substrate-crds \
  oci://ghcr.io/kagent-dev/substrate/helm/substrate-crds --version "$SUBSTRATE_VERSION" \
  --namespace ate-system --create-namespace
helm upgrade --install substrate \
  oci://ghcr.io/kagent-dev/substrate/helm/substrate --version "$SUBSTRATE_VERSION" \
  --namespace ate-system \
  --set 'credentialProvider.namespacePolicies[0].atespace=kagent' \
  --set 'credentialProvider.namespacePolicies[0].allowedNamespaces[0]=kagent'

Create the CA and JWT pools, including the egress gateway CA, then wait until the Secret is readable. kubectl-ate exits before the Secret can be read, which is why kagent's script polls for it.³¹

kubectl create namespace podcertificate-controller-system --dry-run=client -o yaml | kubectl apply -f -
./kubectl-ate --context kind-kagent admin make-ca-pool  --ca-id=1 --name=service-dns-ca-pool  --secret-namespace=podcertificate-controller-system
./kubectl-ate --context kind-kagent admin make-ca-pool  --ca-id=1 --name=pod-identity-ca-pool --secret-namespace=podcertificate-controller-system
./kubectl-ate --context kind-kagent admin make-jwt-pool --key-id=1 --name=actor-id-jwt-pool   --secret-namespace=ate-system
./kubectl-ate --context kind-kagent admin make-ca-pool  --ca-id=1 --name=actor-id-ca-pool     --secret-namespace=ate-system
./kubectl-ate --context kind-kagent admin make-ca-pool  --ca-id=1 --name=egress-mitm-ca-pool  --secret-namespace=ate-system --key-type=ECDSAP256
until kubectl get secret actor-id-ca-pool -n ate-system; do sleep 2; done

Publish the actor identity CA certificate and the Control API authentication config, then let the Substrate release finish rolling out.³¹

actor_id_ca_root="$(kubectl get secret actor-id-ca-pool -n ate-system -o jsonpath='{.data.pool}' \
  | base64 --decode | jq -r '.CAs[0].RootCertificateDER' | base64 --decode \
  | openssl x509 -inform der -outform pem)"
kubectl create secret generic actor-id-ca-certs -n ate-system \
  --from-literal=ca.crt="${actor_id_ca_root}" --dry-run=client -o yaml | kubectl apply -f -

k8s_issuer="$(kubectl get --raw /.well-known/openid-configuration | jq -r '.issuer')"
kubectl create configmap ate-api-authentication -n ate-system \
  --from-literal=authentication.yaml="$(cat <<EOF
actorIdentityJWTProvider: kubernetes
jwtProviders:
- name: kubernetes
  issuer: ${k8s_issuer}
  audiences: [api.ate-system.svc]
  certificateAuthorityFile: /var/run/secrets/kubernetes.io/serviceaccount/ca.crt
  discoveryTokenFile: /var/run/secrets/kubernetes.io/serviceaccount/token
EOF
)" \
  --dry-run=client -o yaml | kubectl apply -f -

helm upgrade substrate oci://ghcr.io/kagent-dev/substrate/helm/substrate \
  --version "$SUBSTRATE_VERSION" --namespace ate-system --reuse-values --wait --timeout 5m

Expected result: the last command returns without a timeout, and the two checks on the Setup page list the api Service and an atelet DaemonSet. On this cluster the SUBSTRATE-VERSION column is empty, because the fork chart does not set that label; that is expected. The certificate is public material, so passing it with --from-literal exposes nothing secret. Both commands in this step use --dry-run=client -o yaml | kubectl apply -f -, as kagent's script does, so you can rerun the step safely.³¹

If it does not work: if helm upgrade times out, run kubectl get pods -n ate-system. Pods stuck in ContainerCreating usually mean one of the pool Secrets from the previous step is missing; run kubectl get secrets -n ate-system and kubectl get secrets -n podcertificate-controller-system and create any pool that is absent.

Install kagent

Install the CLI at the pinned release. The installer script accepts --version.²⁶

curl -fsSL https://raw.githubusercontent.com/kagent-dev/kagent/v1.0.0-alpha6/scripts/get-kagent \
  | bash -s -- --version v1.0.0-alpha6
kagent version

Write a values file. Substrate integration is off by default in the chart, so turn it on and point it at the Substrate API and router. The chart can also create the WorkerPool that agents run on, named kagent-default; it needs the fork's gVisor worker image. The values below match the ones kagent's setup script passes, including 8 worker replicas.²⁷³¹
```
controller:
  substrate:
    enabled: true
    ateApiEndpoint: "dns:///api.ate-system.svc:443"
    atenetRouterURL: "http://atenet-router.ate-system.svc:80"
substrateWorkerPool:
  create: true
  name: kagent-default
  replicas: 8
  sandboxClass: gvisor
  workerImage: "ghcr.io/kagent-dev/substrate/ateom-gvisor:v0.3.0-alpha3"
```
Install the CRD chart, which creates the kagent namespace. Then store the OpenAI key in the Secret the chart expects. The chart's defaults name it kagent-openai with key OPENAI_API_KEY, and its ModelConfig always references that Secret.²⁷ read -rs prompts for the key without echoing it, and printf %s (a shell builtin) passes it on stdin without a trailing newline, so the key never appears in the process list, your shell history, or the Helm release record. The gateway copies the Secret value into the request header as it is, so a stray newline would break the model call.³² Paste the key at the prompt and press Enter.
```
helm install kagent-crds oci://ghcr.io/kagent-dev/kagent/helm/kagent-crds \
  --version 1.0.0-alpha6 --namespace kagent --create-namespace
read -rs KEY && printf %s "$KEY" | kubectl create secret generic kagent-openai -n kagent --from-file=OPENAI_API_KEY=/dev/stdin; unset KEY
```
Install the main chart from the same OCI registry the CLI uses. It creates the default ModelConfig, named default-model-config, pointing at the Secret.²⁶²⁷
```
helm install kagent oci://ghcr.io/kagent-dev/kagent/helm/kagent \
  --version 1.0.0-alpha6 --namespace kagent \
  --values values-substrate.yaml
```
Expected result: kubectl get pods -n kagent shows the controller, UI, PostgreSQL, tools, kmcp, and grafana-mcp pods Running, because the chart enables kagent-tools, kmcp, and grafana-mcp by default. kubectl get workerpools -n kagent lists kagent-default.²⁷

If it does not work: Helm fails with "substrateWorkerPool.workerImage is required" only when workerImage is empty. A wrong or mistyped image passes Helm, and the kagent-default worker pods then sit in ImagePullBackOff; check the value against the file above. If default-model-config reports a missing Secret, check that kagent-openai exists in namespace kagent.²⁷

Declare a Harness, an AgentTemplate, and an Agent

All three are api.kagent.dev/v1alpha3 Kubernetes resources. References resolve in the Agent's namespace, and the reusable Harness and AgentTemplate have no binding to each other; only the Agent pairs them.²⁵ Save the blocks into one file, agent.yaml, separated by --- lines.

Harness: how agents run

apiVersion: api.kagent.dev/v1alpha3
kind: Harness
metadata:
  name: my-first-harness
  namespace: kagent
spec:
  # Selects the kagent runtime adapter. An empty map is required: exactly
  # one of kagent, codex, claude, or byo must be present.
  kagent: {}
  workload:
    # golang-adk is the chart's image for declarative agents. This digest is the
    # one ghcr.io serves for tag 1.0.0-alpha6.
    image: ghcr.io/kagent-dev/kagent/golang-adk@sha256:d8defe7a6d163b11fc421cb18c51bd498e7ffab43424aa029f19bd1848390dcd
  substrate:
    workerPoolRef:
      name: kagent-default
    snapshotPolicy:
      # The bucket the Substrate chart's RustFS creates, with the path
      # kagent's own setup script uses.
      location: s3://ate-snapshots/kagent

kagent selects the runtime variant. The API rejects a Harness that sets none, or more than one, of kagent, codex, claude, and byo.²⁵
workload.image must be pinned by a full 64-character sha256 digest; the schema rejects tags.^{harness_types.go} The digest above is the one the registry returned for ghcr.io/kagent-dev/kagent/golang-adk:1.0.0-alpha6 on 2026-10-01; to look it up yourself, run crane digest ghcr.io/kagent-dev/kagent/golang-adk:1.0.0-alpha6 or docker buildx imagetools inspect on the same reference.²⁷
The snapshot location is the ate-snapshots bucket that the Substrate chart's RustFS creates, under the kagent path that kagent's setup script uses.³¹
substrate.workerPoolRef and substrate.snapshotPolicy.location are both required.^{runtime_types.go}

AgentTemplate: what the agent does

apiVersion: api.kagent.dev/v1alpha3
kind: AgentTemplate
metadata:
  name: my-first-template
  namespace: kagent
spec:
  description: My first kagent agent
  modelConfig:
    name: default-model-config
  systemPrompt: You are a concise, helpful assistant.

An AgentTemplate holds model configuration, description and prompt, MCP tool bindings, skills, plugins, and subagent bindings. It has no runtime status of its own.²⁵

Agent: the pairing

apiVersion: api.kagent.dev/v1alpha3
kind: Agent
metadata:
  name: my-first-agent
  namespace: kagent
spec:
  templateRef:
    name: my-first-template
  harnessRef:
    name: my-first-harness

Each side takes either a reference (templateRef, harnessRef) or an inline spec (template, harness), and exactly one choice is required per side.²³²⁵

Apply and wait for the Agent to compile

Apply the file with kubectl. The CLI also offers kagent apply -f for Agent and AgentTemplate manifests.²⁶
```
kubectl apply -f agent.yaml
kagent agent get my-first-agent
```
Expected result (columns from the CLI source; the timestamp varies). READY stays UNKNOWN or False until the golden snapshot is ready, which takes a minute or so.²⁶²⁵³¹ The CLI prints UTC: the sample time 07:00Z is 17:00 AEST.
```
NAME             READY   CREATED
my-first-agent   True    2026-10-01T07:00:00Z
```
If it does not work: read the Agent's conditions with kubectl get agents.api.kagent.dev my-first-agent -n kagent -o yaml. The Agent reports Accepted, ResolvedRefs, Compatible, and Ready.²³ A missing WorkerPool reports WorkerPoolNotFound, and a Substrate failure reports Ready=False with reason RuntimePreparationFailed.²⁵ Use the qualified name if the legacy kagent.dev Agent CRD is also installed.²⁵

Figure: the controller compiles an Agent into an immutable revision, applies it as a Substrate ActorTemplate, and only advances the latest successful revision once the golden snapshot is ready.

AgentTemplate and Harness are resolved through the Agent (Accepted, ResolvedRefs). The harness compiler produces an immutable revision and digest (Compatible). The revision is applied as an ate-api ActorTemplate, which builds a golden snapshot. When the snapshot is ready, the Agent's latest successful revision advances (Ready). Sessions are created from that revision.

Create a Session and converse

Create a Session from the Agent. The CLI port-forwards to service/kagent-controller on 8083 when the default local endpoint is unreachable, and its namespace defaults to kagent.²⁶
```
kagent agent session create --agent my-first-agent
```
Expected result (columns from the CLI source; the ID and time vary). The CLI prints UTC: the sample time 07:05Z is 17:05 AEST.²⁶
```
ID                                     AGENT            STATE   CREATED
8bd650a8-9775-488f-8bc1-0d52bf7bdcab   my-first-agent   READY   2026-10-01T07:05:00Z
```
Creation selects the Agent's latest successful revision, creates its Actor initially suspended, and marks the Session ready once Substrate accepts it.¹⁰ STATE moves from CREATING to READY; other values are SUSPENDED, FAILED, DELETING, and DELETED.^{runtime.proto}
Copy the ID from the ID column into a shell variable, then send two turns. The second turn shows that the Session keeps history.²⁶⁸
```
SESSION_ID=PASTE_THE_ID_HERE
kagent agent invoke --session "$SESSION_ID" --task "What is 2+2?"
kagent agent invoke --session "$SESSION_ID" --task "What did I just ask you?"
```
invoke also accepts --file instead of --task, and --stream to stream the response.²⁶

If it does not work: if the Session is ready but the first turn fails at the model call, the gateway did not inject the key. Check that the Secret kagent-openai exists in namespace kagent, that the Substrate release carries the credentialProvider.namespacePolicies grant (helm get values substrate -n ate-system), and that egress-mitm-ca-pool exists in ate-system. The gateway's credential cache can take up to five minutes to pick up a changed Secret.³²

Clean up when you are done.²⁶

kagent agent session delete "$SESSION_ID"

Figure: a turn goes from the CLI to the controller on port 8083, which records history in PostgreSQL and forwards the turn over private A2A to the Session's Actor.

The kagent CLI calls the controller on port 8083 (gRPC and A2A). The controller looks up the Session in PostgreSQL, resumes its Substrate Actor through ate-api if needed, and forwards the turn to the Harness runtime over A2A. The reply and task history are stored in PostgreSQL.

The controller's port 8083 serves gRPC, gRPC-Web, A2A, MCP, and health, and the Actor's A2A endpoint is private on port 80.¹⁰ An idle Session expires after the default sessionIdleTTL of 168 hours; audit history and checkpoints remain.²⁷

Revision pinning

Edit the AgentTemplate's systemPrompt and apply it again. The controller compiles a new revision, but the Session you already created keeps the revision it was created from; only Sessions created afterwards use the new prompt.²⁵ A failed compile leaves the previous successful revision available.²⁵

This pinning has no direct counterpart in ax, where a Task is simply a new unit of work.³