Recipe: background worker queue

Boot a redis-7 VM as the broker (AUTH on, creds at /root/.redis-credentials) and a nodejs-24 VM running your worker (BullMQ, Celery, Sidekiq, etc.). The worker connects to Redis over its TCP endpoint; resize the worker or add more of them as load grows.

Offload slow work (image processing, email sends, scheduled syncs) to a background worker behind a queue.

One prompt

set up a redis-backed job queue with a node worker

Your agent chains:

1. create_vm with image: redis-7, named e.g. queue — Redis 7 with AUTH enabled by default, credentials at /root/.redis-credentials. Read them with exec cat /root/.redis-credentials.
2. create_vm with image: nodejs-24 for the worker, in the same project; upload your worker code to /root/app. The supervisor keeps the worker process running.
3. Point the worker at Redis privately by name — redis://default:<pw>@queue:6379. Same-project VMs share a private network, so there’s no public port (Project networking). The redis-7 image is built for exactly this — don’t expose 6379 publicly.

Scale the workers

When the queue backs up, you have two moves:

give the worker vm more cpu

Up — your agent calls resize_vm with a bigger vcpuMillicores / memMib. It applies live, with no rebuild and no dropped connections, so an in-flight job survives it (VM sizing).

Out — create_vm another worker in the same project and point it at the same Redis. Queue workers scale out cleanly; that’s usually the better move once one VM is busy.

Watch throughput with tail_vm_log.

Notes

• Simplest of all: run Redis and the worker on the same VM (127.0.0.1:6379). Split them across two VMs (as above) when you want to scale workers independently of the broker.
• Need durability for the queue itself? Redis persistence lives on the VM disk (automatic daily backups).
• Snapshot a fully-wired worker as a template with save_image to clone more workers fast — see Snapshots & templates.