Running in containers

The firmware always runs on the board. Everything else, from data fetches to training, conversion, firmware builds, westwind serve and the Flower SuperLink, can run on the developer's machine, in Docker Compose, on a Kubernetes cluster, or split between a site and a cluster. The pipeline's compute section picks one:

compute:
  mode: kubernetes                # local, compose, hybrid or kubernetes
  image: 123456789012.dkr.ecr.eu-west-1.amazonaws.com/westwind:0.1.0
  storage: s3://westwind-artifacts/runs
  kubernetes:
    namespace: westwind
    region: eu-west-1
    gpu: true

westwind deploy -o deploy/ writes the files for the chosen mode with the pipeline's values filled in, and westwind submit runs stages there.

Images

The Dockerfile has two targets:

runtime (default)

The CLI with PyTorch, ExecuTorch, LiteRT and Flower, for linux/amd64 and linux/arm64. It runs fetch, train, convert, westwind serve and the Flower SuperLink and SuperNodes. About 3 GB.

firmware

The runtime image plus a Zephyr workspace from the westwind-zephyr manifest and the Zephyr SDK toolchains in ZEPHYR_TOOLCHAINS. It runs the generate and build stages.

docker build -t westwind:0.1.0 .
docker build --target firmware --secret id=github_token,env=GITHUB_TOKEN -t westwind:0.1.0-firmware .
docker build --build-arg TORCH_INDEX=https://download.pytorch.org/whl/cu128 -t westwind:0.1.0-cuda .

.github/workflows/image.yml publishes the runtime and firmware images to ghcr.io/eoinjordan/westwind on version tags. The github_token secret is only needed while westwind-zephyr is private.

Docker Compose

deploy/compose.yaml has one profile per job. westwind deploy --mode compose writes it with a .env that points at the pipeline:

westwind deploy -f westwind.yaml --mode compose -o deploy
docker compose -f deploy/compose.yaml --profile train run --rm train
docker compose -f deploy/compose.yaml --profile firmware run --rm firmware
docker compose -f deploy/compose.yaml --profile serve up -d
docker compose -f deploy/compose.yaml --profile flower up --abort-on-container-exit --exit-code-from federate

westwind submit with compute.mode: compose does the first two in one step, sending fetch, train and convert to the train service and generate and build to the firmware service:

westwind submit -f westwind.yaml --stages fetch,train,convert

The flower profile starts a SuperLink and two SuperNodes, each training on half of the pipeline's dataset, then submits the run. The aggregated model is written to <out>/model/<name>.pte, where the firmware profile picks it up.

Credentials (HF_TOKEN, KAGGLE_USERNAME, KAGGLE_KEY, EI_API_KEY) come from the environment of the shell that runs docker compose.

Kubernetes on AWS

The cluster runs westwind serve, the Flower SuperLink and, optionally, SuperNodes for data that lives in the cloud. Training and firmware builds are Jobs created by westwind submit: the pipeline and the files it names are packed into a ConfigMap, the Job runs westwind job on them and uploads the model, the firmware images and the manifests to compute.storage.

Setup with eksctl and EKS Auto Mode, which manages the nodes, load balancers and volumes:

westwind deploy -f westwind.yaml --mode kubernetes -o deploy
aws s3 mb s3://westwind-artifacts --region eu-west-1
aws ecr create-repository --repository-name westwind --region eu-west-1
eksctl create cluster -f deploy/aws/cluster.yaml
kubectl apply -f deploy/aws/cluster-resources.yaml
kubectl apply -k deploy/k8s/overlays/aws
kubectl -n westwind create secret generic westwind-secrets \
    --from-literal=HF_TOKEN=... --from-literal=SERVE_TOKEN=...

Push the images to the ECR repository, or point compute.image at GHCR. Then:

westwind submit -f westwind.yaml --stages fetch,train,convert --wait
westwind submit -f westwind.yaml --stages generate,build --wait
westwind submit -f westwind.yaml --dry-run       # print the manifests only

With compute.kubernetes.gpu: true train Jobs use the -cuda image and land on the GPU NodePool from cluster-resources.yaml (g6 or g5, spot first, removed when idle). A Job cannot train on a GPU and build firmware at once; submit the two groups of stages separately.

The cluster's Pod Identity association lets the westwind service account read and write the bucket and nothing else.

Exposure

westwind serve is published through an ALB with HTTPS and the SuperLink through an NLB that terminates TLS on port 443, both with an ACM certificate. Both start closed: inbound-cidrs and loadBalancerSourceRanges are 0.0.0.0/32, which matches no address. Replace them with the sites' public addresses and set the certificate ARNs in deploy/k8s/overlays/aws/serve-ingress.yaml and superlink-nlb.yaml.

The SuperLink's Control API shares its port, so a client that reaches it can submit runs. Keep the source ranges tight, and consider SuperNode authentication (flower-superlink --enable-supernode-auth).

Hybrid

In a hybrid setup the boards and the data stay on site and the heavy work runs on the cluster. The site runs the site profile of the compose file, for example on the Linux side of an Arduino UNO Q or on a Raspberry Pi:

site-bridge

westwind serve --serial reads WW1 requests from the board and forwards them to the westwind serve on the cluster with --upstream, sending WESTWIND_UPSTREAM_TOKEN as a bearer token. The board keeps its placement (edge, cloud or split) and needs no network access of its own.

site-supernode

A Flower SuperNode that trains on the site's data (compute.site.data_path) and joins the SuperLink on the cluster over TLS. The data never leaves the site; only weights do.

compute:
  mode: hybrid
  image: ghcr.io/eoinjordan/westwind:0.1.0
  storage: s3://westwind-artifacts/runs
  site:
    serial: /dev/ttyACM0
    upstream: https://westwind.example.com
    data_path: /srv/heater-log
federated:
  framework: flower
  superlink: flower.example.com:443
  rounds: 5
  min_clients: 2

# on the cluster side, once
westwind deploy -f westwind.yaml --mode hybrid -o deploy
kubectl apply -k deploy/k8s/overlays/aws

# on each site
westwind deploy -f westwind.yaml --mode compose -o deploy
WESTWIND_UPSTREAM_TOKEN=... docker compose -f deploy/compose.yaml --profile site up -d

# from a machine inside the SuperLink's source ranges
westwind federate -f westwind.yaml --submit
westwind submit -f westwind.yaml --stages generate,build --wait

Serial devices can only be passed into containers on Linux hosts.

Measured

On a Windows 11 laptop (8 cores, Docker Desktop 29.8) with the AnomalyNet pipeline and the eoinedge/heater-anomaly-windows dataset:

The Kubernetes manifests were rendered with kubectl kustomize; they have not been applied to an EKS cluster yet.