Job Chaining and Workflows

Multi-step processes

Processing an order is not one job — it is a sequence: charge payment → send confirmation → generate invoice → sync inventory → notify warehouse. Each step depends on the previous one. If payment fails, do not send the confirmation. If the invoice fails, still sync inventory.

Job chaining means one job enqueues the next job when it completes. This creates a pipeline of sequential steps.

Simple chaining: enqueue in the handler

const handlers: Record<string, JobHandler> = {
  process_order: async (payload) => {
    const { orderId, paymentToken } = payload;

    // Step 1: Charge payment
    await chargeCard(getOrderTotal(orderId), paymentToken);
    updateOrderStatus(orderId, "paid");

    // Chain: enqueue the next step
    enqueue("send_order_confirmation", { orderId }, { priority: PRIORITY.HIGH });
  },

  send_order_confirmation: async (payload) => {
    const order = getOrder(payload.orderId);
    const user = getUser(order.userId);
    await sendEmail(user.email, "Order Confirmed", `Order ${order.id} confirmed.`);

    // Chain: enqueue the next steps (can fan out)
    enqueue("generate_invoice", { orderId: payload.orderId });
    enqueue("sync_inventory", { orderId: payload.orderId });
  },

  generate_invoice: async (payload) => {
    await generatePDF(payload.orderId);
    // No more steps — end of this branch
  },

  sync_inventory: async (payload) => {
    const items = getOrderItems(payload.orderId);
    for (const item of items) {
      await syncInventory(item.productId, item.quantity);
    }
  },
};

Each handler does its work, then enqueues the next job(s). send_order_confirmation fans out into two parallel jobs: generate_invoice and sync_inventory run independently.

Workflow visualization

process_order
  └─ send_order_confirmation
       ├─ generate_invoice
       └─ sync_inventory

If process_order fails (payment declined), nothing downstream runs. If send_order_confirmation fails, invoices and inventory sync do not run either. Each step only proceeds if the previous step succeeds.

Tracking workflow progress

Add a workflow_id to group related jobs:

function startWorkflow(type: string, payload: Record<string, unknown>): string {
  const workflowId = crypto.randomUUID();
  enqueue(type, { ...payload, workflowId });
  return workflowId;
}

// In handlers, pass the workflow ID forward
const handlers = {
  process_order: async (payload) => {
    await chargeCard(/* ... */);
    enqueue("send_order_confirmation", {
      orderId: payload.orderId,
      workflowId: payload.workflowId,
    });
  },
  // ...
};

Query all jobs in a workflow:

SELECT type, status, attempts, completed_at
FROM jobs
WHERE payload LIKE '%"workflowId":"abc-123"%'
ORDER BY created_at;

Error handling in chains

When a step fails, the chain stops at that point. Downstream jobs are never enqueued. The failed job retries according to its retry configuration. If it permanently fails (moves to the DLQ), the workflow is incomplete.

For some workflows, a failed step should not block everything. Use conditional chaining:

send_order_confirmation: async (payload) => {
  try {
    await sendEmail(/* ... */);
  } catch {
    // Email failed — log but continue the workflow
    console.error("Confirmation email failed, continuing workflow");
  }

  // Always enqueue the next steps, regardless of email success
  enqueue("generate_invoice", { orderId: payload.orderId });
  enqueue("sync_inventory", { orderId: payload.orderId });
},

This matches the dependency classification from the Error Handling course: email is nice-to-have, invoice and inventory are important. The workflow continues even if the email fails.

Exercises

Exercise 1: Build the order processing workflow. Enqueue process_order. Verify each step runs in sequence.

Exercise 2: Make send_order_confirmation fail. Verify generate_invoice and sync_inventory do not run (because they were never enqueued).

Exercise 3: Add conditional chaining: email failure should not block invoice generation. Verify the workflow continues.

Why does job chaining enqueue the next step inside the handler instead of defining the full workflow upfront?