Operation

How Loom and Cortex operate together at runtime. Loom is the brain; Cortex is the muscle. Loom stores assets, owns pipeline definitions and drives processing runs; Cortex is a fleet of worker daemons that register with Loom and execute the individual tasks Loom hands them.

Architecture Overview

The platform is a set of separately deployed components. The dashed boxes below are deployment boundaries (containers / processes); solid boxes are the parts inside them. Loom and Cortex both read the shared media storage at the bottom.

MetaLoom component architecture The browser talks to the Loom server over REST, WebSocket and SSE. Loom persists to PostgreSQL, dispatches processing tasks to Cortex workers over a WebSocket and receives results over REST, drives per-chat Session Runner sandboxes over HTTP, and calls an external LLM service. Loom and Cortex both read a shared media storage that spans the bottom. Browser / Loom App Loom UI chat · assets · pipelines PostgreSQL persistent metadata store Loom Server metaloom/loom-server REST · WebSocket · SSE API :8092 · Auth (JWT / OAuth2) Pipeline Run Engine owns the DAG · dispatches tasks AI Agent Loop chat · skills · memory · sandbox DAOs (jOOQ) assets · pipelines · chat Cortex Worker(s) metaloom/cortex-server Node Runtime hash · whisper · face · ocr · scene … Session Runner loom-session-runner per-chat coding sandbox LLM / model svc speech · vision · language (external) Shared media storage files referenced by path — read by both Loom and Cortex REST · WS · SSE jOOQ / SQL Processor WS · tasks ⇄ results HTTP + token provider API

Two planes connect Loom and Cortex — do not confuse them:

Plane Transport Carries

Control

Processor WebSocket (/api/v1/processors/ws)

Registration, heartbeats, task dispatch (source/node/segment), task results, live tracking events.

Data

REST (LoomClient)

The durable result payloads written back into Loom, keyed by asset.

Cortex always dials out to Loom on port 8092; Loom never connects to Cortex.

How Loom Dispatches Work

Loom owns the pipeline graph and dispatches a task per asset to the Cortex fleet. Cortex A runs the source node, scans the shared filesystem and reports the discovered assets to Loom (Loom reads). Loom then dispatches processing tasks to Cortex B and C, which read their file and process it in parallel — each node colouring in at its own speed. Results return asynchronously: the faster worker (B) finishes and returns first, C follows later (Loom writes). The pipeline strip mirrors the active nodes.

Worker Lifecycle

A Cortex worker connects to Loom and serves the tasks Loom dispatches:

  1. Registers — opens the processor WebSocket, authenticates (?token=<jwt>), and announces the node kinds it is willing to run. Loom persists it as a durable cortex_instance.

  2. Waits for work — Loom’s pipeline engine dispatches a source task to enumerate media, then a node task per graph node per item (affinity work goes out as segment tasks).

  3. Runs one task at a time — the worker executes the requested node and replies with the outcome (COMPLETED / FAILED / SKIPPED) plus outputs for downstream nodes.

  4. Persists results — the worker writes the durable payload to Loom over REST (see persistence path).

  5. Streams progress — tracking events flow back over the WebSocket; Loom re-broadcasts them to UI clients on /api/v1/pipelines/events/ws.

Loom, not the worker, decides what runs next. Every dispatched task gets exactly one result — a node that throws is reported FAILED, never dropped.

Connecting Cortex to Loom

LOOM_HOST=loom.internal LOOM_PORT=8092 LOOM_TOKEN=<jwt> \
  docker run --rm -e LOOM_HOST -e LOOM_PORT -e LOOM_TOKEN metaloom/cortex-server:latest

See Cortex Configuration and Cortex Containers.

Deployment Patterns

Pattern Description

Worker Deployment

One or more Cortex replicas run as a long-lived Deployment; scale replicas to add capacity.

Specialised pool

A separate Cortex Deployment advertises only heavy kinds (e.g. facedetect, whisper) and is scheduled onto GPU nodes; Loom routes those kinds to it.

Side-car

Cortex runs next to Loom in the same compose stack for small setups.