Getting Started

This page boots BullMQ (Node.js's modern job queue) on Redis, defines a job, processes it, and schedules a recurring task.

Same shapes work for Sidekiq (Ruby), Celery (Python), and most other queue libraries — different syntax, same concepts.

Bring Up Redis

# docker-compose.yml
services:
  redis:
    image: redis:7-alpine
    ports: ["6379:6379"]
    volumes:
      - redis-data:/data
    command: redis-server --appendonly yes

volumes:
  redis-data:

docker compose up -d
docker compose exec redis redis-cli ping
# PONG

Define and Enqueue a Job

npm install bullmq

// queue.js — the producer side
const { Queue } = require('bullmq');

const emailQueue = new Queue('emails', {
  connection: { host: 'localhost', port: 6379 },
});

// Enqueue
async function sendWelcomeEmail(userId, email) {
  await emailQueue.add('welcome', { userId, email }, {
    attempts: 3,
    backoff: { type: 'exponential', delay: 1000 },
    removeOnComplete: 100,        // keep last 100 successful
    removeOnFail: 1000,           // keep last 1000 failed
  });
}

// From your web handler:
app.post('/signup', async (req, res) => {
  const user = await createUser(req.body);
  await sendWelcomeEmail(user.id, user.email);   // returns instantly
  res.json({ ok: true });
});

The web request returns in milliseconds. The actual email send happens in a worker process.

Process the Job

// worker.js — runs as a separate process
const { Worker } = require('bullmq');

const worker = new Worker(
  'emails',
  async (job) => {
    console.log(`Processing job ${job.id}: ${job.name}`);
    const { userId, email } = job.data;

    // do the actual work
    await sendEmail(email, await renderWelcomeTemplate(userId));

    return { sent: true, at: new Date().toISOString() };
  },
  {
    connection: { host: 'localhost', port: 6379 },
    concurrency: 5,               // process 5 jobs in parallel
    limiter: {
      max: 100,
      duration: 60_000,           // 100 jobs/minute max
    },
  }
);

worker.on('completed', (job, result) => console.log(`Done ${job.id}:`, result));
worker.on('failed', (job, err) => console.error(`Failed ${job.id}:`, err.message));

Run the worker in its own process:

node worker.js

Now web servers handle web traffic; worker fleet handles jobs. Scale them independently — you can have 2 web servers and 10 workers if you're send-heavy.

Retries and Backoff

BullMQ retries automatically based on the options at enqueue time:

await queue.add('process-payment', { orderId }, {
  attempts: 5,
  backoff: { type: 'exponential', delay: 2000 },   // 2s, 4s, 8s, 16s, 32s
});

If the worker function throws, BullMQ retries with the configured delay. After attempts exhaust, the job goes to the failed state.

Dead-Letter Pattern

Failed jobs are visible in BullMQ's UI / getFailed() API. For automatic dead-lettering:

const worker = new Worker('emails', processJob, {
  // ...
});

worker.on('failed', async (job, err) => {
  if (job.attemptsMade >= job.opts.attempts) {
    // exhausted retries — move to a "dead letter" queue
    await deadLetterQueue.add('dead', {
      originalQueue: 'emails',
      originalJob: job.data,
      error: err.message,
      failedAt: new Date().toISOString(),
    });
  }
});

Now you have a separate queue of failed jobs you can inspect, fix, and replay.

Scheduled Jobs

Run a job at a specific time:

// 1 hour from now
await queue.add('reminder', { userId }, { delay: 60 * 60 * 1000 });

// Specific timestamp
await queue.add('publish-post', { postId }, {
  delay: new Date('2025-06-01T09:00:00Z').getTime() - Date.now(),
});

Recurring (Cron) Jobs

// Run every day at 06:00 UTC
await queue.add('daily-summary', {}, {
  repeat: { pattern: '0 6 * * *' },           // cron pattern
});

// Or every 30 minutes
await queue.add('check-pending', {}, {
  repeat: { every: 30 * 60 * 1000 },
});

This replaces cron + a script that POSTs to your app. The job runs in your worker fleet with the same retry / observability story as any other job.

Priority Queues

BullMQ supports priorities:

await queue.add('email', emailData, { priority: 1 });   // higher priority
await queue.add('email', emailData, { priority: 10 });  // lower priority (default)

Lower number = higher priority. Useful for "premium users get jumped to the front of the line."

Rate Limiting

Limit how fast a queue's workers process — useful when calling a rate-limited external API:

const worker = new Worker('email', processEmail, {
  limiter: { max: 10, duration: 1000 },     // 10 jobs/second across this worker
});

For cross-worker rate limiting (3rd-party API quotas), wrap the API call in a separate rate-limited service or use Redis-based rate limiters.

The Dashboard

BullMQ ships Bull Board for inspecting queues:

npm install @bull-board/api @bull-board/express

const { createBullBoard } = require('@bull-board/api');
const { BullMQAdapter } = require('@bull-board/api/bullMQAdapter');
const { ExpressAdapter } = require('@bull-board/express');

const serverAdapter = new ExpressAdapter();
serverAdapter.setBasePath('/admin/queues');

createBullBoard({
  queues: [new BullMQAdapter(emailQueue), new BullMQAdapter(reportsQueue)],
  serverAdapter,
});

app.use('/admin/queues', adminAuthMiddleware, serverAdapter.getRouter());

Now /admin/queues shows you live queues, in-progress jobs, completed/failed/scheduled jobs. Put it behind auth.

Sidekiq has the analogous Sidekiq::Web; Celery has Flower.

Equivalent in Other Languages

Ruby / Sidekiq

# Enqueue
WelcomeEmailJob.perform_async(user_id, email)

# Worker
class WelcomeEmailJob
  include Sidekiq::Job
  sidekiq_options retry: 3

  def perform(user_id, email)
    UserMailer.welcome(user_id, email).deliver_now
  end
end

Python / Celery

# Enqueue
welcome_email.delay(user_id, email)

# Worker
@app.task(bind=True, autoretry_for=(Exception,), retry_kwargs={'max_retries': 3})
def welcome_email(self, user_id, email):
    send_email(email, render_welcome(user_id))

Tear Down

docker compose down -v

What's Next

You can enqueue and process jobs reliably. When jobs grow into multi-step workflows, you graduate:

Jobs vs Workflows — when to reach for Temporal, what durable execution gives you
Best Practices — idempotency, retries, dead-letter, observability, scaling

Getting Started

On this page