This playbook runs MetaLoom in a cluster: the Loom server, a pool of Cortex workers, and — the part that needs real attention — the service account and RBAC that let the chat agent spawn per-session sandbox Pods. It ends with how to package the whole thing as a Helm chart.
If you have not deployed MetaLoom before, read Deploying with Docker first. The constraints are the same; only the packaging changes.
What You Will Build
Two namespaces is a deliberate choice: runner Pods are created by the agent at runtime from untrusted model output, so they get their own namespace with its own quota and network policy, and the RBAC that creates them is scoped to that namespace only.
Prerequisites
-
A cluster and
kubectlcontext with permission to create namespaces, RBAC and workloads. -
A PostgreSQL instance. The manifests below assume a managed database or an operator-provisioned one reachable at
postgres.loom.svc; running your own is out of scope here. -
A
ReadWriteManyvolume for the media library if more than one worker replica must see it, plus a volume for Loom’s uploads. -
The images from Container Images, reachable from the cluster.
Step 1 — Namespaces and Secret
kubectl create namespace loom
kubectl create namespace loom-runners
kubectl -n loom create secret generic loom-secrets \
--from-literal=db-password='change-me' \
--from-literal=initial-password='first-login-password'
Step 2 — Loom Server
apiVersion: v1
kind: ServiceAccount
metadata:
name: loom
namespace: loom
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: loom-config
namespace: loom
spec:
accessModes: [ReadWriteOnce]
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
name: loom-uploads
namespace: loom
spec:
accessModes: [ReadWriteMany]
resources:
requests:
storage: 200Gi
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: loom-server
namespace: loom
spec:
replicas: 1
selector:
matchLabels: { app: loom-server }
template:
metadata:
labels: { app: loom-server }
spec:
serviceAccountName: loom # the sandbox backend authenticates with this SA's token
securityContext:
runAsNonRoot: true
runAsUser: 1000
fsGroup: 0
containers:
- name: loom
image: metaloom/loom-server:latest
env:
- name: LOOM_DB_HOST
value: postgres.loom.svc
- name: LOOM_DB_NAME
value: loom
- name: LOOM_DB_USERNAME
value: loom
- name: LOOM_DB_PASSWORD
valueFrom:
secretKeyRef: { name: loom-secrets, key: db-password }
- name: LOOM_INITIAL_PASSWORD
valueFrom:
secretKeyRef: { name: loom-secrets, key: initial-password }
ports:
- { name: rest, containerPort: 8092 }
- { name: metrics, containerPort: 8989 }
livenessProbe:
httpGet: { path: /api/v1/health, port: rest }
initialDelaySeconds: 30
periodSeconds: 20
readinessProbe:
httpGet: { path: /api/v1/health, port: rest }
initialDelaySeconds: 10
periodSeconds: 10
volumeMounts:
- { name: config, mountPath: /config }
- { name: uploads, mountPath: /uploads }
volumes:
- name: config
persistentVolumeClaim: { claimName: loom-config }
- name: uploads
persistentVolumeClaim: { claimName: loom-uploads }
---
apiVersion: v1
kind: Service
metadata:
name: loom
namespace: loom
spec:
selector: { app: loom-server }
ports:
- { name: rest, port: 8092, targetPort: rest }
- { name: metrics, port: 8989, targetPort: metrics }
Notes that matter more than they look:
-
/configmust be persistent. On first start Loom writesloom.ymlthere with a generated keystore password and createskeystore.jceksbeside it. That keystore signs every JWT — if the volume is recreated, all issued tokens (including your Cortex workers') stop verifying. -
Probe on
8092, not8989. The monitoring port serves/metricsonly; the health check that also verifies database connectivity isGET /api/v1/healthon the REST port. -
Keep
replicas: 1unless you have verified otherwise. The pipeline run engine keeps run state in the process that started the run, and/configisReadWriteOnce.
Step 3 — Cortex Workers
Workers are long-running daemons that dial out to Loom, so they are a plain Deployment and you scale
capacity with the replica count.
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: cortex-cpu
namespace: loom
spec:
serviceName: cortex-cpu
replicas: 3
selector:
matchLabels: { app: cortex-cpu }
template:
metadata:
labels: { app: cortex-cpu }
spec:
containers:
- name: cortex
image: metaloom/cortex-server:latest
env:
- name: LOOM_HOST
value: loom.loom.svc
- name: LOOM_PORT
value: "8092"
- name: LOOM_TOKEN
valueFrom:
secretKeyRef: { name: cortex-token, key: token }
- name: CORTEX_NODE_ID # unique + stable: the ordinal from the StatefulSet
valueFrom:
fieldRef: { fieldPath: metadata.name }
- name: CORTEX_META_PATH
value: /meta
- name: CORTEX_NODE_BLACKLIST
value: whisper,llm,vlm,captioning,facedescription
ports:
- { name: mon, containerPort: 8093 }
livenessProbe:
httpGet: { path: /api/health, port: mon }
readinessProbe:
httpGet: { path: /api/ready, port: mon }
periodSeconds: 10
volumeMounts:
- { name: meta, mountPath: /meta }
- { name: media, mountPath: /media }
volumes:
- name: media
persistentVolumeClaim: { claimName: media }
volumeClaimTemplates:
- metadata: { name: meta }
spec:
accessModes: [ReadWriteOnce]
resources: { requests: { storage: 5Gi } }
Why a StatefulSet rather than a Deployment: CORTEX_NODE_ID must be unique per worker and stable
across restarts — Loom keys registration, node-kind restrictions and run attribution on it and rejects
a second worker announcing an id already in use. The worker refuses to start without one. A StatefulSet
gives you cortex-cpu-0, cortex-cpu-1, … for free, plus a per-replica /meta volume so the
filesystem-source index survives a reschedule. With a Deployment you would have to inject an id another
way.
LOOM_TOKEN should be a long-lived API key (POST /api/v1/tokens, or Admin → API Keys in the UI),
stored in a Secret. Without a token a worker still registers and runs tasks but cannot persist results.
A GPU pool for the heavy kinds
env:
- name: CORTEX_NODE_WHITELIST
value: whisper,facedetect,captioning,vlm
resources:
limits:
nvidia.com/gpu: 1
nodeSelector:
accelerator: nvidia
Loom routes each node task to a worker that accepts its kind, so a whitelisted GPU pool plus a blacklisted CPU pool splits the work without any change to the pipeline. See Nodes → Requirements at a glance for which kinds need what.
|
Important
|
Every worker that might receive a task for a given asset needs the media mounted at the
same path. Loom dispatches a path reference, never the bytes. In-cluster that means one ReadWriteMany
PVC mounted at an identical mountPath in every worker pool.
|
Step 4 — The Loom Service Account
The Deployment above already sets serviceAccountName: loom. On its own that account needs no
permissions — Loom does not talk to the Kubernetes API for normal operation.
It becomes load-bearing the moment you enable the coding sandbox: the Kubernetes backend creates
runner Pods using the token mounted into the Loom pod at
/var/run/secrets/kubernetes.io/serviceaccount/token, and reaches the API server at
KUBERNETES_SERVICE_HOST:KUBERNETES_SERVICE_PORT. So do not set automountServiceAccountToken:
false on the Loom pod, and grant that account exactly the pod verbs it needs — nothing more.
Step 5 — Session Sandboxes
The coding sandbox gives the chat agent run_shell,
write_file, read_file and list_files inside a per-chat Session Runner Pod. It is off by
default; everything below is what turning it on requires.
5a. RBAC in the runner namespace
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
name: loom-session-runner-manager
namespace: loom-runners
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["create", "get", "delete"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
name: loom-session-runner-manager
namespace: loom-runners
subjects:
- kind: ServiceAccount
name: loom
namespace: loom # the SA the Loom pod runs as, in ITS namespace
roleRef:
kind: Role
name: loom-session-runner-manager
apiGroup: rbac.authorization.k8s.io
A Role + RoleBinding (not a ClusterRole) is the point: the grant is confined to loom-runners.
create, get and delete on pods is the complete set the backend uses — creating the runner, polling
it to readiness, and reaping it.
5b. Turn it on
- name: LOOM_AGENT_SANDBOX_ENABLED
value: "true"
- name: LOOM_AGENT_SANDBOX_BACKEND
value: kubernetes
- name: LOOM_AGENT_SANDBOX_NAMESPACE
value: loom-runners
- name: LOOM_AGENT_SANDBOX_IMAGE
value: metaloom/loom-session-runner:latest
- name: LOOM_AGENT_SANDBOX_MAX_CONCURRENT
value: "10"
- name: LOOM_AGENT_SANDBOX_IDLE_TTL_S
value: "900"
- name: LOOM_AGENT_SANDBOX_MAX_SESSION_S
value: "3600"
| Variable | Default | Effect in a cluster |
|---|---|---|
|
|
Master switch for the coding tools. |
|
|
Set to |
|
the SA’s own namespace |
Where runner Pods are created. Point it at the isolated namespace, or the grant above will not match. |
|
|
Must be pullable by the runner namespace — copy the imagePullSecret there if the registry is private. |
|
|
Per-deployment cap on live runners. Size it together with the ResourceQuota below. |
|
|
Idle reap. Lower it on a busy cluster. |
|
|
Hard time-box, also written into the Pod’s |
|
|
Per-runner CPU. |
|
|
Per-runner memory. |
|
|
|
|
|
How long Loom waits for the runner to answer |
You do not apply a runner manifest yourself — Loom POSTs one per session. It is already hardened:
runAsNonRoot, all capabilities dropped, read-only root filesystem,
automountServiceAccountToken: false, seccomp RuntimeDefault, emptyDir workspace and tmp, and an
activeDeadlineSeconds backstop. The annotated reference is
loom/agent/deploy/session-runner-pod-template.yaml in the source repository. No credentials ever enter
a runner: Loom drives it over HTTP with a per-session bearer token.
5c. Guardrails on the runner namespace
RBAC decides who may create runner Pods. These three objects decide what a runner can do once it exists. Treat them as part of enabling the feature, not as optional hardening.
apiVersion: v1
kind: ResourceQuota
metadata:
name: loom-session-runners
namespace: loom-runners
spec:
hard:
pods: "12" # >= LOOM_AGENT_SANDBOX_MAX_CONCURRENT, with headroom for reaping
requests.cpu: "2"
requests.memory: 2Gi
limits.cpu: "12"
limits.memory: 6Gi
---
apiVersion: v1
kind: LimitRange
metadata:
name: loom-session-runners
namespace: loom-runners
spec:
limits:
- type: Container
default: { cpu: "1", memory: 512Mi }
defaultRequest: { cpu: 100m, memory: 128Mi }
max: { cpu: "2", memory: 1Gi }
---
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
name: loom-session-runner-isolation
namespace: loom-runners
spec:
podSelector:
matchLabels: { app: loom-session-runner }
policyTypes: [Ingress, Egress]
ingress:
- from:
- namespaceSelector:
matchLabels: { kubernetes.io/metadata.name: loom }
podSelector:
matchLabels: { app: loom-server }
ports:
- { protocol: TCP, port: 8080 }
egress: [] # deny all egress: a runner has no reason to reach anything
The NetworkPolicy is the one that needs care in both directions. Loom talks to the runner directly on
its pod IP, so ingress from the Loom pod on 8080 must be allowed or every sandbox session dies at the
readiness wait. Everything else — including all egress — stays denied, which is what keeps arbitrary
model-generated code from reaching your cluster, your database or the internet.
|
Note
|
If you run a CNI without NetworkPolicy enforcement, these rules are silently inert. Verify before you rely on them. |
5d. OpenShift
The kubernetes backend covers OpenShift as well. The runner Pod already complies with the
restricted-v2 SCC (non-root, no privilege escalation, all capabilities dropped, seccomp
RuntimeDefault), so no custom SCC is needed. Grant the same namespaced Role, and make sure the runner
image is pullable from the runner namespace’s image pull secrets.
Step 6 — Verify
kubectl -n loom rollout status deploy/loom-server
kubectl -n loom rollout status statefulset/cortex-cpu
# Loom sees the workers
kubectl -n loom exec deploy/loom-server -- \
curl -sf -H "Authorization: Bearer $TOKEN" localhost:8092/api/v1/processors | jq '.[].nodeId'
Then exercise the sandbox from the chat UI — ask the agent to run a shell command — and watch the Pod appear and disappear:
kubectl -n loom-runners get pods -l app=loom-session-runner -w
A runner is created lazily on the first coding tool call of a chat, and reaped on idle-TTL or when the session time-box elapses. Seeing one appear, become ready, and vanish some minutes later is the whole feature working.
Packaging as a Helm Chart
The manifests above are the deployable truth; a chart is a way to parameterize them per environment.
MetaLoom ships two charts in the source repository — helm/loom and helm/cortex. See the
Helm Charts guide for the full install flow and values reference; this
section explains how those values map onto the manifests above.
A values file for the loom chart looks like:
image:
repository: metaloom/loom-server
tag: latest
pullPolicy: IfNotPresent
database:
host: postgres.loom.svc
name: loom
username: loom
existingSecret: loom-secrets # key: db-password
persistence:
config: { size: 1Gi, accessMode: ReadWriteOnce } # keystore + loom.yml — never drop this
uploads: { size: 200Gi, accessMode: ReadWriteMany }
serviceAccount:
create: true
name: loom
ai: # chat agent; needs a reachable provider to answer
enabled: true
providerType: OLLAMA
url: http://ollama.loom.svc:11434
modelId: gpt-oss:20b
sandbox: # renders the RBAC, quota, LimitRange and NetworkPolicy
enabled: false
namespace: loom-runners
image: metaloom/loom-session-runner:latest
maxConcurrent: 10
idleTtlSeconds: 900
maxSessionSeconds: 3600
rbac:
create: true
networkPolicy:
create: true
cortex:
pools:
- name: cpu
replicas: 3
nodeBlacklist: [whisper, llm, vlm, captioning, facedescription]
- name: gpu
replicas: 1
nodeWhitelist: [whisper, facedetect, captioning, vlm]
gpu: 1
media:
existingClaim: media
mountPath: /media
Two rules for whoever writes the templates:
-
Everything under
sandboxis a unit. RenderingLOOM_AGENT_SANDBOX_ENABLED=truewithout the Role, the RoleBinding and the NetworkPolicy produces a deployment that either cannot create runners at all or creates unconstrained ones. -
CORTEX_NODE_IDmust come from a stable per-replica identity (a StatefulSet ordinal), never from a random suffix — see Step 3.
Troubleshooting
| Symptom | Cause and fix |
|---|---|
Sandbox tool calls fail with a |
The RoleBinding subject does not match. The |
Runner Pods are created but never become ready |
The NetworkPolicy is blocking Loom → runner on |
Runner creation fails with |
|
Cortex pod restarts with "No worker id configured" |
|
Two workers, only one visible in |
Duplicate |
Everyone logged out after a Loom pod restart |
The |
Node tasks fail with missing files |
The media PVC is mounted at different paths in different pools, or not mounted in one of them. |
Next Steps
-
Give the workers something to do — Transcribing for the Chat Agent.
-
Scrape the stack — Loom Metrics, Cortex Metrics.
-
Full sandbox reference — Chat & AI Agent → Coding Sandbox.