Jobs vs Workflows

A job queue is enough for most apps. When you find yourself wiring multiple jobs together with database state to track progress, you've reinvented an orchestrator. This page is about knowing when to graduate.

The Symptom

You started with sendWelcomeEmail. Then product asks for:

Onboard new customer:
  1. Create account
  2. Wait 1 day, send reminder if not verified
  3. Wait 5 more days, deactivate if still not verified
  4. Once verified, send welcome bonus
  5. Charge first payment 14 days after signup
  6. If charge fails, retry 3 times over 3 days
  7. If still failing, suspend account and notify support

With a basic job queue, you build:

• A pending_onboarding table tracking state per user.
• A scheduled job (cron) that runs every hour, looks at the table, decides what to do.
• Per-state functions to advance to the next step.
• Manual retry logic.
• Observability bolted on top.

You've built a state machine on top of a database. It works but is brittle, hard to test, and impossible to evolve.

The orchestrator is for this.

What Durable Execution Gives You

A workflow orchestrator (Temporal, Inngest, Trigger.dev, Step Functions) gives you:

Your code looks normal:

// Temporal-style workflow
async function onboardCustomer(customerId) {
  const account = await activities.createAccount(customerId, { startToCloseTimeout: '30s' });

  await sleep('1 day');
  const verified = await activities.checkVerification(account.id);
  if (!verified) {
    await activities.sendReminderEmail(account.email);
  }

  await sleep('5 days');
  const stillUnverified = !await activities.checkVerification(account.id);
  if (stillUnverified) {
    await activities.deactivateAccount(account.id);
    return;
  }

  await activities.sendWelcomeBonus(account.id);

  await sleep('14 days');

  for (let attempt = 0; attempt < 3; attempt++) {
    try {
      await activities.chargeFirstPayment(account.id);
      return;
    } catch (e) {
      await sleep('1 day');
    }
  }

  await activities.suspendAccount(account.id);
  await activities.notifySupport(account.id);
}

This is the workflow — there's no state-machine table, no cron job, no manual retries. The orchestrator persists execution state, handles all sleeps and retries, and shows you a complete timeline of every customer's onboarding in its UI.

How It Works (the Magic)

Workflow code runs once and is "replayed" from history on restart:

1. Workflow starts. The orchestrator records every step in history.
2. Worker crashes after step 3.
3. Worker restarts.
4. Orchestrator replays workflow code from the beginning — but each await activities.X(...) returns the cached result from history rather than re-executing.
5. At step 4 (where the crash happened), there's no cached result, so the activity actually runs.

That's why workflow code must be deterministic — same inputs, same code path. You can't use Math.random(), new Date(), or call non-determinstic external APIs directly. Use activities for those — activities are normal functions whose outputs are recorded.

The Players

Temporal

The leader. Open-source core, hosted Temporal Cloud, SDKs in Go, TypeScript, Java, Python, .NET, PHP.

Use when: complex workflows, long-running, you can run a Temporal cluster (or pay for Cloud), team will invest in learning.

Don't use when: the operational cost outweighs the workflow complexity.

Inngest

Hosted-first. JavaScript / TypeScript optimized; growing multi-language support. Event-driven by design.

import { Inngest } from "inngest";
const inngest = new Inngest({ id: "my-app" });

export const onboardCustomer = inngest.createFunction(
  { id: "onboard-customer" },
  { event: "user/signup" },
  async ({ event, step }) => {
    const account = await step.run("create-account", () => createAccount(event.data.userId));

    await step.sleep("wait-day", "1d");

    const verified = await step.run("check-verified", () => isVerified(account.id));
    if (!verified) {
      await step.run("send-reminder", () => sendReminder(account.email));
    }
    // ...
  }
);

Use when: modern Node/TS stack, want hosted, like the event-driven model.

Trigger.dev

Similar niche to Inngest. Hosted; great DX for Next.js / Node teams; visual UI for runs.

AWS Step Functions

JSON-defined state machine. AWS-native. Tight integration with Lambda, SQS, DynamoDB.

Use when: AWS-heavy stack; willing to write JSON DSL; want AWS-managed.

Don't use when: complex branching (Step Functions JSON gets unwieldy fast).

Apache Airflow / Dagster / Prefect

For data pipelines (ETL, ML, batch). Different niche from transactional workflows — these are about DAGs of data transformations, scheduled.

Use when: batch data engineering.

Don't use when: transactional user-facing flows (use Temporal/Inngest).

Argo Workflows

Kubernetes-native. Each step is a container. Good for ML, batch processing in K8s.

Cadence

The predecessor to Temporal (same authors). Still around; pick Temporal for new projects.

When to Stay With Just Job Queue

You don't need an orchestrator. Stay with BullMQ/Sidekiq/Celery if:

• Jobs are independent. No multi-step state machines.
• No long sleeps. "Send email" is a job. "Wait 7 days then send email" is borderline; "wait 7 days, check state, decide next step" needs orchestration.
• Failure handling is simple. Retry-with-backoff is enough.
• The team is small. Orchestrators have a learning curve.

You can always graduate later — start jobs, move to workflows when the pain shows up.

A Hybrid Pattern

Real systems often have both:

• Job queue for short, independent tasks: send email, generate thumbnail, refresh cache, charge card.
• Orchestrator for multi-step business processes: customer onboarding, order fulfillment, subscription renewals, complex deploys.

The orchestrator calls activities; some activities enqueue jobs; some jobs trigger workflows. They compose.

Comparing the Choices

When Workflow Code Gets Hard

Workflow orchestration is powerful but has sharp edges:

• Determinism rules — Math.random(), new Date(), fetch() outside activities all break replay.
• Versioning workflows in flight — code changes can break running workflows that started on the old version.
• Testing — replay-based execution makes unit tests tricky; SDKs provide test frameworks.
• Debugging — when something looks wrong, you read execution history rather than a stack trace.

Plan a few weeks of learning curve before adopting. The payoff is real, but Temporal is not a casual choice.

What's Next

You know when to reach for an orchestrator. Best Practices covers operational concerns — idempotency, retry strategy, dead-letter, observability, scaling — that apply across job queues and orchestrators.