Answer reads from the last good answer when a vendor is down or rate limiting

An instance that serves repeat reads from a cache, caps what it spends, and returns the last successful answer marked stale when the vendor fails, with the cases where it refuses to.

A page that shows campaign numbers makes the same reads over and over. When the vendor’s API is down, or has started answering 429, those reads fail and the page goes blank. Three plugins change that:

• cache serves a repeat read from a store for a few seconds or minutes, so most reads never reach the vendor.
• budget caps how many vendor requests the instance may make, so one busy page cannot spend the whole rate limit.
• failover keeps every successful answer in a store you supply. When a later read fails because the vendor is unavailable or rate limiting, it returns the kept answer with meta.stale: true and the time it was fetched.

Failover serves only a read that succeeded before, and only for those two kinds of failure. It never invents an answer, and it never hides a rejected key or a missing record. A stale answer is old data: this guide shows how to tell, so you can say so on the page.

Before you start

• Node 22 or later.
• The code below runs on the in-memory adapter, so it needs no vendor key. Each sample on this page is a file in the repository that the test suite runs; the output shown is what it printed.
• For real data: an Instantly, Smartlead or HeyReach API key. A read-only key is enough. No method of the SDK writes to a vendor.

What you get

One read, outbound.campaigns.list(), repeated over half an hour while the vendor goes down, rate limits and comes back.

12:00 vendor up: stale false, fetched 12:00:00, requests 1
12:00 same read: stale false, fetched 12:00:00, requests 0
12:10 vendor down: stale true, fetched 12:00:00, requests 1
12:10 first-time read: Provider - memory responded 503
12:20 rate limited: stale true, fetched 12:00:00, requests 1
12:30 vendor up: stale false, fetched 12:30:00, requests 1

Read it line by line:

• 12:00, vendor up. A fresh answer. One request.
• 12:00, same read. Served by cache. No request, and fetchedAt is still when it was fetched.
• 12:10, vendor down. The cached answer has expired, the vendor answers 503, and failover serves the last good answer with stale: true.
• 12:10, a read never made before. There is no last good answer. The error reaches the caller.
• 12:20, rate limited. Treated the same as down.
• 12:30, vendor up. Fresh again, and the new answer replaces the kept one.

Build the instance

import { createOutbound } from "outbound-sdk";
import type { Adapter } from "outbound-sdk";
import { budget } from "outbound-sdk/budget";
import { cache, memoryCacheStore } from "outbound-sdk/cache";
import { failover } from "outbound-sdk/failover";
import { memory } from "outbound-sdk/memory";

import { outage } from "./outage.ts";
import { seed } from "./seed.ts";

const MINUTE = 60_000;

export interface Options {
  readonly adapter?: Adapter;
  /** Vendor requests the instance may make before `requestBudget.reset()`. */
  readonly maxRequests?: number;
  /** How long failover may serve an answer, in seconds. */
  readonly maxAge?: number;
}

export const build = (options: Options = {}) => {
  // The samples move this clock by hand. Leave `now` out in real code.
  let now = Date.parse("2026-11-14T12:00:00.000Z");
  const vendor = outage();

  const outbound = createOutbound({
    adapter: options.adapter ?? memory({ data: seed, workspaceId: "acme" }),
    plugins: [
      // Outermost: sees every failure from the layers inside it.
      failover({
        maxAge: options.maxAge,
        now: () => now,
        store: memoryCacheStore(),
      }),
      // A repeat read within a minute never reaches the vendor.
      cache({ now: () => now, ttl: 60 }),
      // Stops the instance from spending the vendor's whole rate limit.
      budget({ maxRequests: options.maxRequests ?? 100, per: "run" }),
      vendor.plugin,
    ],
    runtime: { now: () => now },
  });

  return {
    outbound,
    /** Moves the clock forward. */
    pass: (minutes: number) => {
      now += minutes * MINUTE;
    },
    vendor,
  };
};

plugins[0] is the outermost layer, and the order here matters:

1. failover first, so it sees a failure from any layer inside it. It answers only for the vendor’s failures: a budget that has run out is your own cap, and that error passes through.
2. cache inside it. A cache hit returns before anything further in runs.
3. budget last of the three, next to the vendor, so it counts only requests that are really made.

Failover takes a store and has no default: where the last good answers are kept is your decision. memoryCacheStore() from outbound-sdk/cache keeps up to 500 entries in the process and is empty after a restart. A CacheStore of your own is three functions (get, set, delete) over Redis, a table or anything else. maxAge is how old an answer may be and still be served, in seconds. The default is 7 days.

What counts as “the same read”: the provider, the workspace and the whole operation, filters and page included. campaigns.list() and campaigns.list({ status: "paused" }) are two reads, and each page of a listAll() walk is kept separately.

Simulate the outage

The in-memory adapter never fails, so the samples put a small plugin of their own innermost, where the vendor would be. It throws the error a real adapter throws for a 503, 429, 401 or 404. Because the in-memory adapter makes no HTTP request, it also counts one request per operation, so that budget and meta.requests have something to count.

// A stand-in for the vendor's API. The in-memory adapter never fails and
// makes no HTTP request, so this plugin, listed last (innermost), fails the
// way a real adapter would and counts one request per operation.
import { OutboundError } from "outbound-sdk";
import type { OutboundErrorCode, OutboundPlugin } from "outbound-sdk";

const STATUS: Partial<Record<OutboundErrorCode, number>> = {
  NotFound: 404,
  Provider: 503,
  RateLimited: 429,
  Unauthorized: 401,
};

export interface Outage {
  readonly plugin: OutboundPlugin;
  /** Every operation fails with this code until `recover()`. */
  readonly fail: (code: OutboundErrorCode) => void;
  readonly recover: () => void;
}

export const outage = (): Outage => {
  let failing: OutboundErrorCode | undefined;

  return {
    fail: (code) => {
      failing = code;
    },
    plugin: {
      name: "outage",
      wrap: (operation, next, info) => {
        info.state.requests += 1;

        if (failing !== undefined) {
          const status = STATUS[failing];

          throw new OutboundError(
            failing,
            `${info.provider} responded ${status}`,
            { provider: info.provider, status }
          );
        }

        return next(operation);
      },
    },
    recover: () => {
      failing = undefined;
    },
  };
};

// One client with two campaigns and a week of results for each.
import type { MemoryData } from "outbound-sdk/memory";

const DAYS = [7, 8, 9, 10, 11, 12, 13];

export const seed: MemoryData = {
  campaigns: [
    { id: "cmp_founders", name: "SaaS founders" },
    { id: "cmp_agencies", name: "Agency owners" },
  ],
  stats: DAYS.flatMap((day) => [
    {
      campaignId: "cmp_founders",
      counts: { bounced: 1, replied: 3, sent: 60 },
      date: `2026-11-${String(day).padStart(2, "0")}`,
    },
    {
      campaignId: "cmp_agencies",
      counts: { bounced: 2, replied: 6, sent: 80 },
      date: `2026-11-${String(day).padStart(2, "0")}`,
    },
  ]),
};

import { OutboundError } from "outbound-sdk";

import { build } from "./instance.ts";

const { outbound, pass, vendor } = build();

// The same read every time. What changes is where the answer comes from.
const read = async (label: string): Promise<void> => {
  const { meta } = await outbound.campaigns.list();

  console.log(
    `${label}: stale ${meta.stale}, fetched ${meta.fetchedAt.slice(11, 19)}, requests ${meta.requests}`
  );
};

// 12:00. The vendor is up.
await read("12:00 vendor up");

// 12:00:30. The same read again is served by the cache.
pass(0.5);

await read("12:00 same read");

// 12:10. The vendor is down and the cached answer has expired.
pass(9.5);

vendor.fail("Provider");

await read("12:10 vendor down");

// A read that never succeeded has no last good answer.
try {
  await outbound.campaign("cmp_founders").stats();
} catch (error) {
  if (OutboundError.is(error)) {
    console.log(`12:10 first-time read: ${error.code} - ${error.message}`);
  }
}

// 12:20. The vendor answers 429 instead. Same treatment.
pass(10);

vendor.fail("RateLimited");

await read("12:20 rate limited");

// 12:30. The vendor is back.
pass(10);

vendor.recover();

await read("12:30 vendor up");

That is the script that printed the timeline above.

Say that the data is old

Every result has meta.stale and meta.fetchedAt. When stale is true, show the age: “as of 12:00” is honest, a number with no date is not.

fetchedAt is when the vendor was asked, whatever served the answer. In the output above the stale answers say 12:00:00, the time of the one vendor request, although the cache served the same answer again at 12:00:30 and failover saw that hit pass through. With cache outside failover instead, the age is just as true: the cache never stores an answer marked stale.

When the budget runs out

budget({ maxRequests, per: "run" }) counts every vendor request the instance makes until you call requestBudget.reset(). Once it is spent, the next operation is refused with RateLimited before it reaches the vendor. Failover does not answer for that: the cap is yours, not an outage, so the error reaches the caller even for a read that succeeded before. The same goes for maxRequests and for a deadline that passes while the SDK’s own rate budget is full.

import { OutboundError } from "outbound-sdk";

import { build } from "./instance.ts";

// This run may make three vendor requests.
const { outbound, pass } = build({ maxRequests: 3 });

await outbound.campaigns.list();

await outbound.campaign("cmp_founders").stats();

await outbound.campaign("cmp_agencies").stats();

console.log(
  `used ${outbound.requestBudget.used()}, remaining ${outbound.requestBudget.remaining()}`
);

// Ten minutes on the cache has expired, and the budget is spent. The cap is
// yours, not an outage, so failover does not answer for it: a read that
// succeeded before is refused like one that never did.
pass(10);

for (const [label, read] of [
  ["seen before", () => outbound.campaign("cmp_founders").stats()],
  ["not seen before", () => outbound.senders.list()],
] as const) {
  try {
    // oxlint-disable-next-line no-await-in-loop -- one read at a time keeps the output in order.
    await read();
  } catch (error) {
    if (OutboundError.is(error)) {
      console.log(`${label}: ${error.code} - ${error.message}`);
    }
  }
}

// A new run starts with a full budget.
outbound.requestBudget.reset();

const fresh = await outbound.campaign("cmp_founders").stats();

console.log(`after reset: stale ${fresh.meta.stale}`);

used 3, remaining 0
seen before: RateLimited - The run used its budget of 3 request(s) to memory
not seen before: RateLimited - The run used its budget of 3 request(s) to memory
after reset: stale false

Reset the budget where a “run” starts for you: per incoming request, per cron tick, per minute. With per: "call" (the default) the cap applies to each SDK call on its own and nothing needs resetting. Retries count, because each one is a vendor request.

Point it at your vendor

Swap the adapter and drop the stand-in plugin.

import { createOutbound } from "outbound-sdk";
import { budget } from "outbound-sdk/budget";
import { cache, memoryCacheStore } from "outbound-sdk/cache";
import { failover } from "outbound-sdk/failover";
import { instantly } from "outbound-sdk/instantly";

// Reads INSTANTLY_API_KEY. Swap the import for smartlead() or heyreach().
export const outbound = createOutbound({
  adapter: instantly({ workspaceId: "acme" }),
  // Give up on a failing vendor after 5 seconds, retries included. Failover
  // answers only once the call has failed.
  deadline: 5000,
  plugins: [
    // This store is the process's memory: it is empty after a restart. Pass
    // your own CacheStore (Redis, a table) to keep answers across deploys.
    failover({ maxAge: 24 * 60 * 60, store: memoryCacheStore() }),
    cache({ ttl: 60 }),
    budget({ maxRequests: 200, per: "run" }),
  ],
});

On a real adapter the request engine comes first: a 503 or a 429 is retried, and a 429 is waited out, before the call fails. Failover answers only after that. Without a deadline, a page waits through every retry before it gets its stale answer. Set deadline to how long a reader should wait.

What each vendor can serve

The same instance and outage, run on the in-memory adapter limited to what each vendor’s adapter declares. This shows which calls a vendor’s adapter would refuse. It does not show what that vendor’s API returns.

import { OutboundError } from "outbound-sdk";

import { asVendor, VENDORS } from "../_shared/as-vendor.ts";
import type { Vendor } from "../_shared/as-vendor.ts";
import { build } from "./instance.ts";
import { seed } from "./seed.ts";

const outageAs = async (vendor: Vendor): Promise<string> => {
  const {
    outbound,
    pass,
    vendor: api,
  } = build({ adapter: asVendor(vendor, seed, "acme") });

  try {
    await outbound.campaigns.list();
    await outbound.stats();
    pass(10);
    api.fail("Provider");

    const campaigns = await outbound.campaigns.list();
    const stats = await outbound.stats();

    return `${vendor}: campaigns stale ${campaigns.meta.stale}, workspace stats stale ${stats.meta.stale}`;
  } catch (error) {
    if (!OutboundError.is(error)) {
      throw error;
    }

    return `${vendor}: ${error.code}: ${error.message}`;
  }
};

const lines = await Promise.all(VENDORS.map(outageAs));

console.log(lines.join("\n"));

instantly: campaigns stale true, workspace stats stale true
smartlead: campaigns stale true, workspace stats stale true
heyreach: campaigns stale true, workspace stats stale true

No vendor is refused. The three plugins wrap whatever reads an adapter serves and need no capability of their own. A read the adapter does not support is still Unsupported, before any plugin runs.

What it costs in vendor requests

• A cache hit costs none.
• A stale answer costs the failed attempts that came before it: every retry is a request.
• A read refused by a spent budget costs none.
• Failover adds one store write per successful read and one store read per failure. cache adds one store read per call and one write per miss.

When it cannot answer

import { OutboundError } from "outbound-sdk";

import { build } from "./instance.ts";

const attempt = async (
  label: string,
  read: () => Promise<object>
): Promise<void> => {
  try {
    await read();
    console.log(`${label}: answered`);
  } catch (error) {
    if (!OutboundError.is(error)) {
      throw error;
    }

    console.log(`${label}: ${error.code}, permanent ${error.permanent}`);
  }
};

// Every case starts with one good read, so a last good answer exists.

// 1. The key is rejected. Serving old data would hide a problem only you
//    can fix.
const badKey = build();

await badKey.outbound.campaigns.list();

badKey.pass(10);

badKey.vendor.fail("Unauthorized");

await attempt("rejected key", () => badKey.outbound.campaigns.list());

// 2. The record is gone. The old copy is not served in its place.
const gone = build();

await gone.outbound.campaigns.get("cmp_founders");

gone.pass(10);

gone.vendor.fail("NotFound");

await attempt("deleted record", () =>
  gone.outbound.campaigns.get("cmp_founders")
);

// 3. The last good answer is older than `maxAge` (here 5 minutes).
const old = build({ maxAge: 300 });

await old.outbound.campaigns.list();

old.pass(10);

old.vendor.fail("Provider");

await attempt("answer too old", () => old.outbound.campaigns.list());

// 4. The same listing with another filter is another read. It was never
//    answered, so there is nothing to serve.
const other = build();

await other.outbound.campaigns.list();

other.pass(10);

other.vendor.fail("Provider");

await attempt("different filter", () =>
  other.outbound.campaigns.list({ status: "paused" })
);

await attempt("same read as before", () => other.outbound.campaigns.list());

rejected key: Unauthorized, permanent true
deleted record: NotFound, permanent true
answer too old: Provider, permanent false
different filter: Provider, permanent false
same read as before: answered

Limits

• Failover works one read at a time. Check meta.stale on each result: a page built from several reads can mix fresh and stale answers.
• “Same read” is exact. A different filter, page size or cursor is a different read.
• Failover answers for the vendor’s failures only. A read stopped by your own cap (budget, maxRequests, a deadline on a full rate budget) fails with RateLimited, with or without a last good answer.
• The stores hold whole answers as the adapter returned them, personal data included. That is also true with redact on the instance: it is the outermost plugin and masks on the way out, after the stores have been written. Put the stores where that data is allowed to be.
• A run budget is one counter per instance in one process. It is not shared between processes.
• Stale numbers are still only as good as the vendor’s were when they were fetched.