QueueFlow is a PostgreSQL-native distributed job queue and workflow engine. No Redis, no broker, no separate state store. The jobs table is the queue, and workers in any language talk to it over HTTP.
import { QueueFlow, wf } from "@queueflow/sdk"; const qf = new QueueFlow({ baseUrl: "http://localhost:8000", token: process.env.QUEUEFLOW_TOKEN }); // Enqueue a job with retries and a timeout, then wait for the result. const job = await qf.jobs.create({ task: "resize_image", payload: { id: 42 }, maxRetries: 3, timeout: 30 }); const done = await qf.jobs.waitFor(job.id); // Declare a DAG. Each step runs once its dependencies complete. const run = await qf.workflows.create( wf("etl") .step("extract", "fetch_rows") .step("transform", "normalize", { after: ["extract"] }) .step("load", "upsert", { after: ["transform"], onFailure: "halt" }), ); const finished = await qf.workflows.waitFor(run.id); console.log(finished.status, finished.context);
Features
Any plain PostgreSQL 13 or newer works: RDS, Cloud SQL, Azure, or your laptop. No extensions required.
Workers claim rows with FOR UPDATE SKIP LOCKED and own them through lease tokens. Durable, transactional, at-least-once delivery with no extra infrastructure.
Declare steps and dependencies. The engine gates each step, threads results through a shared context, and applies per-step failure policies: halt, skip, or continue.
Fixed, linear, or exponential backoff with caps and jitter. A retry is just a row scheduled in the future, so delays survive restarts.
Exhausted and non-retryable jobs land in an inspectable DLQ. Replay any entry as a fresh job from the API.
Recurring enqueues from standard crontab expressions, deduplicated across servers, with pause and resume.
A lease, heartbeat, complete, and fail HTTP protocol lets handlers run in any language with the same retry, DLQ, and workflow semantics as in-process Rust handlers.
Prometheus metrics, structured JSON logs, health and readiness probes, and a Server-Sent Events stream of job status changes.
The OpenAPI 3.1 spec is derived from the handlers and ships with every release, so clients never drift from the server.
Tenant isolation is enforced on every job and workflow endpoint. An Idempotency-Key header makes client retries safe.
How it works
The engine applies migrations on startup and exposes a REST API, Prometheus metrics, and interactive docs.
Start the server with a DATABASE_URL. It applies its own migrations, serves the API on port 8000, and runs workers in the same process or separately.
Use the REST API or an SDK. Set priority, retries, backoff, timeouts, and a future run_at. Workflows are plain JSON with depends_on edges; cycles are rejected at creation.
Handlers in any language lease jobs from a queue, extend the lease while working, and complete or fail with a retryable flag. Crashed workers consume retry budget instead of crash-looping.
# Start the engine (any plain Postgres 13+). --dev accepts any bearer token; # in production pass --api-keys and --worker-token instead. export DATABASE_URL=postgres://postgres:postgres@localhost:5432/postgres queueflow serve --dev --mode all --workers 10 --api-port 8000 # Enqueue a job curl -X POST http://localhost:8000/api/v1/jobs \ -H 'Authorization: Bearer dev' -H 'Content-Type: application/json' \ -d '{"task_name":"echo","payload":{"hello":"world"}, "config":{"priority":5,"max_retries":3,"timeout":30}}' # => {"job_id":"…"} # Lease work from any language (long-polls when the queue is empty) curl -X POST http://localhost:8000/api/v1/queues/default/lease \ -H 'Authorization: Bearer dev' -H 'Content-Type: application/json' \ -d '{"max_jobs":1,"lease_secs":60,"wait_secs":20}' # Report the outcome curl -X POST http://localhost:8000/api/v1/jobs/<id>/complete \ -H 'Authorization: Bearer dev' -H 'Content-Type: application/json' \ -d '{"lease_token":"…","result":{"ok":true}}'
Dashboard
Every server ships a read-only dashboard at /ui/: per-queue backlog with the age of the oldest waiting job, jobs with live status, workflow graphs coloured by step state, dead letters, and cron schedules. No extra service, no build step. Read about the dashboard.
SDKs
Each client pairs a core generated from the released OpenAPI spec, so types never drift from the server, with a hand-written facade for the ergonomics codegen cannot give you. Anything else can be generated from the spec in minutes.
npm install @queueflow/sdk
Typed end to end, zero runtime dependencies, ESM and CJS. Ships a worker runtime (qf.worker.run) with concurrency and graceful drain, an SSE watch() stream, a workflow builder DSL, and a waitFor() poller.
pip install queueflow
Python 3.9 and newer. Generated models and API classes for every endpoint, plus a facade that ships a worker runtime (run_worker) with concurrency and graceful drain, waiters, and a workflow builder.
go get github.com/elision-labs/queueflow-sdk-go
A Go client generated from the spec, with context-aware requests, typed errors, and a facade that ships a worker runtime (Run) with concurrency and graceful drain, waiters, and a workflow builder.
cargo add queueflow-sdk
Generated async client with a facade that ships a worker runtime (run_worker) with concurrency and graceful drain, waiters, and a workflow builder. The first-party queueflow-client crate shares the engine's own types.
Start Postgres in Docker, run the quick start, and have a DAG running in a few minutes. Embedding the engine as a Rust library? It also runs against an in-memory store for tests.