Find why one campaign is underperforming

One function that runs a fixed list of checks on a campaign and returns the findings most serious first, with the days that broke from its own trend and its rates beside the rest of the workspace.

“This campaign is not working” has a short list of usual causes: too few sends to tell, bounces, a mailbox that stopped, a schedule that barely opens, a sequence with no follow-ups. campaignDiagnosis checks one campaign for each of them and returns findings with the numbers behind them and what to look at next. Two more reads give context. anomalies finds the days whose counts broke from the days before. The benchmarks plugin puts the campaign’s rates beside the whole workspace’s.

The checks are fixed rules on counts. No AI model is called, the copy is not read, and nothing is changed. An empty list of findings means nothing was found in what could be checked, which is why the answer also lists what could not be.

Before you start

• Node 22 or later.
• The code below runs on the in-memory adapter, so it needs no vendor key. Each sample on this page is a file in the repository that the test suite runs; the output shown is what it printed.
• For real data: an Instantly, Smartlead or HeyReach API key. A read-only key is enough.

What you get

Ecommerce ops (active): 8 findings. Most serious: Bounce rate is 4.1% (117 of 2836 sends), over the 3% limit.

[high] high_bounce_rate: Bounce rate is 4.1% (117 of 2836 sends), over the 3% limit.
    next: Use bounce_watch to find the senders behind it, and verify the lead list.
[high] inactive_sender: 1 of 2 senders are not active: dan@example.com (paused).
    next: Reconnect or replace them; warmup_readiness shows which mailboxes are fit to send.
[high] low_reply_rate: Reply rate is 0.2% (2 of 945), under the 1% floor. The 95% interval is 0.1% to 0.8%, so it is below the floor beyond chance.
    next: Use step_funnel and copy_audit to see where replies are lost.
[medium] falling_sends: Sends fell from 628 to 316 over the last 7 days against the 7 before, a drop of 49.7%.
    next: Use pipeline_forecast to check the leads left, and sender_capacity for the mailboxes.
[medium] few_positive_replies: Only 0 of 2 replies (0%) are positive, under the 20% expected. With 2 replies the 95% interval is 0% to 65.8%.
    next: Read the replies: the offer or the audience may be off.
[medium] short_sequence: The sequence has 2 steps; follow-ups usually bring a large share of the replies.
    next: Add follow-up steps; step_funnel shows what each step earns.
[low] schedule_limits_volume: The sending windows are open 15 hours a week, under 20.
    next: Use settings_audit to review the schedule and the limits.
[low] tracking_off: Open tracking is off, so the missing open count is expected and not a fault.
    next: Judge this campaign on replies, not opens.

Unusual days in the last 14:
  - bounced was 14 on 2026-11-10, against an average of 3 over the 14 days with data before it (z = 12.5419).
  - sent was 44 on 2026-11-08, against an average of 86.7143 over the 14 days with data before it (z = -3.5934).
  - sent was 46 on 2026-11-07, against an average of 90 over the 14 days with data before it (z = -21.1997).
  - replied was 1 on 2026-11-02, against an average of 0.0714 over the 14 days with data before it (z = 3.4744).

Against the whole workspace:
  reply rate 0.2% against 6.3% (0.03 times the workspace)
  bounce rate 4.1% against 2.4% (1.71 times the workspace)

Write the diagnosis

import type { Outbound } from "outbound-sdk";
import type { BenchmarksExtension } from "outbound-sdk/benchmarks";
import type { InsightsExtension } from "outbound-sdk/insights";

const percent = (rate: number | null): string =>
  rate === null ? "not reported" : `${(rate * 100).toFixed(1)}%`;

const times = (ratio: number | null): string =>
  ratio === null ? "no comparison" : `${ratio.toFixed(2)} times the workspace`;

/** Why one campaign is underperforming, as plain text. */
export const diagnose = async (
  outbound: Outbound & InsightsExtension & BenchmarksExtension,
  campaign: string
): Promise<string> => {
  const { data, meta } = await outbound.insights.campaignDiagnosis({
    campaign,
  });

  const lines = [`${data.name} (${data.status}): ${data.summary}`, ""];

  for (const finding of data.findings) {
    lines.push(
      `[${finding.severity}] ${finding.code}: ${finding.finding}`,
      `    next: ${finding.next}`
    );
  }

  // An empty list of findings means nothing was found in what could be
  // checked. This is the other half of that sentence.
  if (data.notChecked.length > 0) {
    lines.push(
      "",
      "Not checked:",
      ...data.notChecked.map((what) => `  - ${what}`)
    );
  }

  // Context 1: the days that broke from the campaign's own trend.
  if (outbound.capabilities["stats.byDay"] !== false) {
    const unusual = await outbound.insights.anomalies({
      campaign: data.campaign,
      days: 14,
    });

    lines.push(
      "",
      "Unusual days in the last 14:",
      ...unusual.data.anomalies.map((anomaly) => `  - ${anomaly.finding}`),
      ...unusual.data.metrics.flatMap((metric) =>
        metric.reason === null
          ? []
          : [`  - ${metric.metric} could not be tested: ${metric.reason}`]
      )
    );
  }

  // Context 2: the campaign beside the whole workspace it runs in.
  const { comparisons } = await outbound.benchmarks.campaign(data.campaign);

  lines.push(
    "",
    "Against the whole workspace:",
    `  reply rate ${percent(comparisons.replyRate.value)} against ${percent(comparisons.replyRate.baseline)} (${times(comparisons.replyRate.ratio)})`,
    `  bounce rate ${percent(comparisons.bounceRate.value)} against ${percent(comparisons.bounceRate.baseline)} (${times(comparisons.bounceRate.ratio)})`
  );

  if (!meta.complete) {
    lines.push(
      "",
      "The diagnosis covers part of the data:",
      ...meta.notes.map((note) => `  - ${note}`)
    );
  }

  return lines.join("\n");
};

The checks, with their defaults. Each bar is an input.

Things to keep when you format the answer:

• Under min_sent, rates are not judged. too_little_volume replaces the bounce, reply and positive-reply checks. It does not mean they passed.
• notChecked is part of the answer. A read the adapter cannot serve, or a count the vendor does not report, is named there.
• The reply rate is replies divided by leads contacted, or by sends when the vendor reports no contacted count. The bounce rate is bounces divided by sends.
• from and to limit the totals. Left out, the totals cover the campaign’s whole history. The recent-sends checks always use the last recent_days complete days, ending yesterday.

For context:

• anomalies holds each of the last days complete days (default 30; 14 above) to the baseline_days (14) before it. A count threshold (2.5) standard deviations or more from that baseline’s mean is returned. A day needs at least min_baseline_points (5) earlier days with data. When those days all have the same count there is no standard deviation, and a day with any other count is returned with judgedBy: "flat_baseline". Days the vendor has no count for are skipped, never read as zero.
• benchmarks.campaign returns each rate as value (the campaign), baseline (the workspace over the same dates) and ratio. The baseline is the whole workspace, this campaign included, so a campaign that is most of the workspace is compared mostly with itself. No outside benchmark is used.

Try it without a key

This seed is five weeks of results for three campaigns, one of which lost a mailbox a week ago:

// Five weeks of made-up results for the client "acme": two campaigns that
// are doing fine and one, "Ecommerce ops", that is not.
import type { MemoryData, MemoryStats } from "outbound-sdk/memory";

const DAY = 86_400_000;
const TODAY = Date.parse("2026-11-14T00:00:00.000Z");

const day = (daysAgo: number): string =>
  new Date(TODAY - daysAgo * DAY).toISOString().slice(0, 10);

const daily = (
  campaignId: string,
  perDay: (daysAgo: number) => MemoryStats["counts"]
): MemoryStats[] =>
  Array.from({ length: 35 }, (_, index) => ({
    campaignId,
    counts: perDay(index + 1),
    date: day(index + 1),
  }));

export const CAMPAIGN = "cmp_ecom";

export const seed: MemoryData = {
  campaigns: [
    { id: "cmp_founders", name: "SaaS founders" },
    { id: "cmp_agencies", name: "Agency owners" },
    {
      id: CAMPAIGN,
      name: "Ecommerce ops",
      settings: {
        dailyLimit: 100,
        // Monday to Friday, three hours a day.
        schedule: [
          {
            days: [1, 2, 3, 4, 5],
            from: "09:00",
            timezone: "Europe/Berlin",
            to: "12:00",
          },
        ],
        trackOpens: false,
      },
      steps: [
        { variants: [{ body: "Hi {{firstName}}.", subject: "Returns" }] },
        {
          delayDays: 4,
          variants: [{ body: "Any thoughts?", subject: "Returns" }],
        },
      ],
    },
  ],
  senders: [
    {
      address: "cara@example.com",
      campaignIds: [CAMPAIGN],
      dailyLimit: 50,
      id: "snd_cara",
    },
    {
      address: "dan@example.com",
      campaignIds: [CAMPAIGN],
      dailyLimit: 50,
      id: "snd_dan",
      status: "paused",
    },
  ],
  stats: [
    ...daily("cmp_founders", (daysAgo) => ({
      bounced: 1,
      contacted: 20,
      positive: daysAgo % 2,
      replied: 2,
      sent: 60,
    })),
    ...daily("cmp_agencies", (daysAgo) => ({
      bounced: 1,
      contacted: 25,
      positive: 1,
      replied: 2 + (daysAgo % 2),
      sent: 80,
    })),
    // Sends halved a week ago, when one mailbox was paused. Bounces spiked
    // on 2026-11-10. Two replies in five weeks, neither of them positive.
    ...daily(CAMPAIGN, (daysAgo) => ({
      bounced: daysAgo === 4 ? 14 : 2 + (daysAgo % 3),
      contacted: daysAgo <= 7 ? 15 : 30,
      positive: 0,
      replied: daysAgo % 12 === 0 ? 1 : 0,
      sent: daysAgo <= 7 ? 44 + (daysAgo % 2) * 2 : 88 + (daysAgo % 2) * 4,
    })),
  ],
};

import { createOutbound } from "outbound-sdk";
import { benchmarks } from "outbound-sdk/benchmarks";
import { insights } from "outbound-sdk/insights";
import { memory } from "outbound-sdk/memory";

import { diagnose } from "./diagnose.ts";
import { CAMPAIGN, seed } from "./seed.ts";

const outbound = createOutbound({
  adapter: memory({ data: seed, workspaceId: "acme" }),
  plugins: [insights(), benchmarks()],
});

console.log(await diagnose(outbound, CAMPAIGN));

That is the script that printed the diagnosis above. The paused mailbox, the halved sends and the two unusual days of sends on 2026-11-07 and 2026-11-08 are the same event seen three ways. The findings do not connect them: that reading is yours.

Point it at your vendor

Swap the adapter. Nothing else changes.

import { createOutbound } from "outbound-sdk";
import { benchmarks } from "outbound-sdk/benchmarks";
import { insights } from "outbound-sdk/insights";
import { instantly } from "outbound-sdk/instantly";

// Reads INSTANTLY_API_KEY. Swap the import for smartlead() or heyreach().
export const outbound = createOutbound({
  adapter: instantly({ workspaceId: "acme" }),
  // One diagnosis with its context may spend at most 30 vendor requests.
  maxRequests: 30,
  plugins: [insights(), benchmarks()],
});

Pass the campaign’s vendor id or its ref to the function.

What each vendor can serve

The same function, run on the in-memory adapter limited to what each vendor’s adapter declares. This shows which calls a vendor’s adapter would refuse. It does not show what that vendor’s API returns.

import { createOutbound, OutboundError } from "outbound-sdk";
import { benchmarks } from "outbound-sdk/benchmarks";
import { insights } from "outbound-sdk/insights";

import { asVendor, VENDORS } from "../_shared/as-vendor.ts";
import type { Vendor } from "../_shared/as-vendor.ts";
import { diagnose } from "./diagnose.ts";
import { CAMPAIGN, seed } from "./seed.ts";

const tryAs = async (vendor: Vendor): Promise<string> => {
  const outbound = createOutbound({
    adapter: asVendor(vendor, seed, "acme"),
    plugins: [insights(), benchmarks()],
  });

  try {
    const diagnosis = await diagnose(outbound, CAMPAIGN);

    const findings = diagnosis
      .split("\n")
      .filter((line) => line.startsWith("["));

    const notChecked = diagnosis.includes("Not checked:")
      ? "some checks not run"
      : "every check run";

    return `${vendor}: ok, ${findings.length} findings, ${notChecked}`;
  } catch (error) {
    if (!OutboundError.is(error)) {
      throw error;
    }

    return `${vendor}: ${error.code}: ${error.message}`;
  }
};

const lines = await Promise.all(VENDORS.map(tryAs));

console.log(lines.join("\n"));

instantly: ok, 8 findings, every check run
smartlead: ok, 8 findings, every check run
heyreach: ok, 6 findings, some checks not run

campaignDiagnosis needs only stats.campaign. Every other read is skipped and named when the adapter cannot serve it. The HeyReach adapter cannot read campaign settings, so the schedule and tracking checks are not run there and the two findings that depend on them are missing, not passed.

What it costs in vendor requests

• campaignDiagnosis: up to 7 requests, by its own description: the campaign, its totals, its settings, its sequence, a listing of its senders (up to max_senders, default 50) and its daily stats.
• anomalies: one stats read.
• benchmarks.campaign: two stats reads, the campaign and the workspace. Its result carries the count in requests.

None of them reads leads or messages, so the cost does not grow with the size of the campaign.

When it cannot answer

import { createOutbound, OutboundError } from "outbound-sdk";
import { benchmarks } from "outbound-sdk/benchmarks";
import { insights } from "outbound-sdk/insights";
import { memory } from "outbound-sdk/memory";

import { CAMPAIGN, seed } from "./seed.ts";

const outbound = createOutbound({
  adapter: memory({ data: seed, workspaceId: "acme" }),
  plugins: [insights(), benchmarks()],
});

// 1. A campaign that does not exist.
try {
  await outbound.insights.campaignDiagnosis({ campaign: "cmp_missing" });
} catch (error) {
  if (OutboundError.is(error)) {
    console.log("unknown campaign:", error.code, "-", error.message);
  }
}

// 2. Too few sends in the range to judge a rate. The reply and bounce
//    checks are skipped, and a finding says so.
const thin = await outbound.insights.campaignDiagnosis({
  campaign: CAMPAIGN,
  from: "2026-11-13",
});

console.log(
  "one day:",
  thin.data.findings.map((finding) => finding.code)
);

// 3. Daily counts that never vary are still tested. With no spread there is
//    no z-score, so a day is flagged only when its count differs at all.
const flat = await outbound.insights.anomalies({
  campaign: "cmp_founders",
  metrics: ["sent"],
});

console.log(
  "flat baseline:",
  flat.data.metrics[0]?.testable,
  "-",
  flat.data.anomalies.length,
  "unusual days"
);

// 4. A campaign the vendor returns no stats for cannot be benchmarked.
const empty = createOutbound({
  adapter: memory({ data: { campaigns: [{ id: "cmp_new" }] } }),
  plugins: [insights(), benchmarks()],
});

try {
  await empty.benchmarks.campaign("cmp_new");
} catch (error) {
  if (OutboundError.is(error)) {
    console.log("no stats:", error.code, "-", error.message);
  }
}

// 5. The same campaign diagnosed: nothing to judge, and it says what it
//    could not check.
const blank = await empty.insights.campaignDiagnosis({ campaign: "cmp_new" });

console.log(
  "nothing reported:",
  blank.data.findings.map((finding) => finding.code),
  blank.data.notChecked
);

// 6. An adapter without campaign stats refuses the diagnosis.
const noStats = createOutbound({
  adapter: memory({ capabilities: { "stats.campaign": false }, data: seed }),
  plugins: [insights()],
});

try {
  await noStats.insights.campaignDiagnosis({ campaign: CAMPAIGN });
} catch (error) {
  if (OutboundError.is(error)) {
    console.log("no campaign stats:", error.code, "-", error.message);
  }
}

unknown campaign: NotFound - Campaign cmp_missing was not found
one day: [
  'inactive_sender',
  'falling_sends',
  'too_little_volume',
  'short_sequence',
  'schedule_limits_volume',
  'tracking_off'
]
flat baseline: true - 0 unusual days
no stats: NotFound - No stats were returned for campaign cmp_new
nothing reported: [ 'no_recent_sends', 'too_little_volume' ] [
  'Reply rate: the vendor reports no reply or volume count.',
  'Bounce rate: the vendor reports no bounce or send count.',
  'Positive replies: the vendor reports no positive reply count.'
]
no campaign stats: Unsupported - campaign_diagnosis is not available: the memory adapter does not support stats.campaign

Limits

• One campaign per call. To find which campaign to diagnose, start from campaignRanking or the weekly client report.
• The findings are independent checks. Two findings with one cause are reported as two.
• It does not read the copy or the leads. Follow a finding’s next to the insight that does: stepFunnel and copyAudit for replies, bounceWatch for bounces, pipelineForecast for leads left.
• Against a flat baseline anomalies has no z-score to go by. A campaign that sends the same number every day is flagged on the first day it sends any other number, however small the difference, and threshold does not apply.
• The benchmark is your own workspace. It says “worse than your other campaigns”, not “bad”.