--- title: Copy and sequence insights description: Reference for the five insights about copy (stepFunnel, variantTest, copyAudit, copyReuse, replyTiming), with every input, what the answer holds and a sample that runs. sidebar: label: Copy and sequence related: - /docs/insights - /guides/ab-test-winner - /guides/pre-launch-audit --- Five insights about what a campaign says and when. Each is a method on `outbound.insights` once the [insights plugin](/docs/insights) is added. What every answer carries and how inputs are checked is on the [overview](/docs/insights). Three of them need per-step stats (`stats.byStep`). An adapter that does not declare it refuses them as `Unsupported`: see [which adapter serves which](/docs/insights#which-adapter-serves-which). A reply rate here is replies divided by leads contacted, or by messages sent when the vendor does not count leads. Each answer says which in `basis` or `replyRateBasis`. ## The data the samples run on Two email campaigns. "SaaS founders" tests two variants on its first two steps and has a careless third step. "Agency owners" reuses two pieces of its copy. ```ts title="seed.ts" // Made-up data for one email tool: a campaign with two A/B tested steps and a // careless third step, and a second campaign that reuses two pieces of copy. import type { MemoryData, MemoryMessage, MemoryStats, } from "outbound-sdk/memory"; const OPENER = { body: "Hi {{firstName}}, I saw {{company}} is hiring two account executives. Worth a short call next week?", subject: "Quick question about {{company}}", }; const FOLLOW_UP = { body: "Hi {{firstName}}, did my note reach you?" }; const pair = ( lead: string, sentAt: string, ...repliedAt: string[] ): MemoryMessage[] => [ { campaignId: "founders", direction: "out", id: `${lead}_out`, leadId: lead, sentAt, threadId: `thread_${lead}`, }, ...repliedAt.map((at, index) => ({ campaignId: "founders", direction: "in" as const, id: `${lead}_in${index}`, leadId: lead, sentAt: at, threadId: `thread_${lead}`, })), ]; /** One step's counts. A row with a `variantKey` is one variant of the step. */ const step = ( campaignId: string, stepIndex: number, counts: { replied: number; sent: number } ): MemoryStats => ({ campaignId, counts, date: "2026-11-10", stepIndex }); export const seed: MemoryData = { campaigns: [ { id: "founders", name: "SaaS founders", steps: [ { variants: [ OPENER, { body: "Hi {{firstName}}, most teams your size lose a day a week to manual follow-up. Want to see how {{company}} could get it back?", subject: "Idea for {{company}}", }, ], }, { delayDays: 3, variants: [ FOLLOW_UP, { body: "Hi {{firstName}}, is this on your list this quarter?" }, ], }, { delayDays: 5, variants: [ { body: "Hi {{firstName}, see https://example.com/pricing and https://example.com/demo before FRIDAY. This offer ENDS soon!!", subject: "Last chance: a free trial for {{company}}, with no obligation at all", }, ], }, ], }, { id: "agencies", name: "Agency owners", steps: [{ variants: [OPENER] }, { delayDays: 3, variants: [FOLLOW_UP] }], }, ], messages: [ // 2026-11-09 is a Monday. Replies after 0.5, 6, 24, 48 and 96 hours. ...pair("lead1", "2026-11-09T09:00:00.000Z", "2026-11-09T09:30:00.000Z"), ...pair("lead2", "2026-11-09T09:00:00.000Z", "2026-11-09T15:00:00.000Z"), // The second message from lead3 is a follow-on, not a second reply time. ...pair( "lead3", "2026-11-10T09:00:00.000Z", "2026-11-11T09:00:00.000Z", "2026-11-11T10:00:00.000Z" ), ...pair("lead4", "2026-11-10T10:00:00.000Z", "2026-11-12T10:00:00.000Z"), ...pair("lead5", "2026-11-09T09:00:00.000Z", "2026-11-13T09:00:00.000Z"), // A reply with no send before it in what was read. { campaignId: "founders", direction: "in", id: "lead6_in0", leadId: "lead6", sentAt: "2026-11-12T15:00:00.000Z", threadId: "thread_lead6", }, ], stats: [ step("founders", 0, { replied: 60, sent: 1000 }), // How the vendor splits the first two steps of "SaaS founders" between their variants. { ...step("founders", 0, { replied: 40, sent: 500 }), variantKey: "A" }, { ...step("founders", 0, { replied: 20, sent: 500 }), variantKey: "B" }, step("founders", 1, { replied: 16, sent: 800 }), { ...step("founders", 1, { replied: 10, sent: 400 }), variantKey: "A" }, { ...step("founders", 1, { replied: 6, sent: 400 }), variantKey: "B" }, step("founders", 2, { replied: 1, sent: 600 }), step("agencies", 0, { replied: 12, sent: 600 }), step("agencies", 1, { replied: 9, sent: 450 }), ], }; ``` A stats row in the in-memory adapter's seed can name a variant as well as a step. "SaaS founders" has a row per step and, for its first two steps, a row per variant, which is what a vendor that splits a step by variant returns. The instance is the in-memory adapter with the insights plugin: ```ts title="outbound.ts" import { createOutbound } from "outbound-sdk"; import { insights } from "outbound-sdk/insights"; import { memory } from "outbound-sdk/memory"; import { seed } from "./seed.ts"; export const outbound = createOutbound({ adapter: memory({ data: seed, workspaceId: "acme" }), plugins: [insights()], }); ``` ## `stepFunnel` Where a campaign's replies come from, one row per sequence step. | Input | Default | Range | What it is | | --- | --- | --- | --- | | `campaign` | required | a campaign ref | The campaign to read. | | `from` | `null` | `YYYY-MM-DD` | First day to count. `null` counts from the start. | | `to` | `null` | `YYYY-MM-DD` | Last day to count. `null` counts to today. | | `min_reply_rate` | 0 | 0 to 1 | A step earns its place when the low end of its reply rate interval is above this. 0 asks only that the rate is reliably above zero. | `data` (`StepFunnel`): - `finding`: what the numbers say, in one or two sentences. - `topStep`: the step with the most replies, or `null`. - `lastStepEarningPlace`: the last step whose reply rate is reliably above `min_reply_rate`, or `null`. - `steps`: one entry per step, counted from 0, with `action`, `delayDays`, `inSequence`, `sent`, `replies`, `replyRate`, `replyRateInterval`, `replyRateBasis`, `shareOfReplies`, `sentDrop` and `sentDropRate` (how far sends fell from the step before), `earnsPlace`, and `variants` (empty when the vendor does not split step stats by variant). - `totals`: `sent` and `replies`. - `unattributed`: counts the vendor tied to no step (`rows`, `sent`, `replies`). They are in no step's row. - **Needs:** `sequence`, `stats.byStep`. - **Costs:** 2 requests. - **Notes it can add:** some counts are tied to no step by the vendor and are reported as unattributed. - **Does not:** tell you why a step is weak. A step whose stats name it but which the sequence no longer has is listed with `inSequence: false`. A `null` count means the vendor does not report it. ```ts title="step-funnel.ts" import { outbound } from "./outbound.ts"; const { data } = await outbound.insights.stepFunnel({ campaign: "memory:acme:founders", min_reply_rate: 0.01, }); console.log(data.finding); for (const step of data.steps) { console.log( `step ${step.step}: sent ${step.sent}, replies ${step.replies}, rate ${step.replyRate} (${step.replyRateInterval?.low} to ${step.replyRateInterval?.high}), share ${step.shareOfReplies}, drop ${step.sentDropRate}, earns place ${step.earnsPlace}` ); } console.log( "step 0 variants:", data.steps[0]?.variants.map((each) => `${each.variantKey} ${each.replyRate}`) ); ``` ```text title="Output" Step 0 (counted from 0) brings the most replies: 60 of 77. Step 1 is the last whose reply rate is reliably above the bar; the 1 later step(s) sent 600 messages for 1 replies. step 0: sent 1000, replies 60, rate 0.06 (0.0469 to 0.0765), share 0.7792, drop null, earns place true step 1: sent 800, replies 16, rate 0.02 (0.0123 to 0.0322), share 0.2078, drop 0.2, earns place true step 2: sent 600, replies 1, rate 0.0017 (0.0003 to 0.0094), share 0.013, drop 0.25, earns place false step 0 variants: [ 'A 0.08', 'B 0.04' ] ``` Step 2 has a reply, but one reply from 600 sends gives an interval of 0.03% to 0.94%, which does not clear the 1% bar set here. ## `variantTest` Whether a step's A/B leader is ahead by more than chance. | Input | Default | Range | What it is | | --- | --- | --- | --- | | `campaign` | required | a campaign ref | The campaign to read. | | `metric` | `replied` | `replied`, `positive`, `opened`, `clicked` | What a variant is judged on. Replies and positive replies are divided by leads contacted (or sends); opens and clicks by messages delivered (or sends). | | `alpha` | 0.05 | above 0, up to 0.5 | A difference with a p-value under this is a winner. | | `min_sent` | 100 | 1 or more, whole | A variant with a smaller base than this is not compared. | | `from` | `null` | `YYYY-MM-DD` | First day to count. | | `to` | `null` | `YYYY-MM-DD` | Last day to count. | `data` (`VariantTest`): - `campaign`, `metric`, `alpha`: what was asked. - `variantStatsReported`: `false` when the vendor's step stats carry no variant rows at all. - `steps`: one entry per step with two or more variants, with `verdict` (`winner`, `no_difference_yet` or `too_little_data`), `leader`, `pValue` (the leader against its closest rival), `moreSendsNeeded`, a `reason` sentence, and `variants` (each with `sent`, `count`, `trials`, `basis`, `rate`, `interval`, `enoughData` and `pValueVsLeader`). Winners first. - `totals`: `stepsTested`, `winners`, `noDifferenceYet` and `tooLittleData`. - **Needs:** `stats.byStep`. - **Costs:** 1 request. - **Notes it can add:** the step stats carry no variant rows, so there is nothing to compare. - **Does not:** correct for comparing three or more variants. A leader must beat every other variant with enough data to be called a winner. `moreSendsNeeded` is rough: it assumes both rates stay as they are. It pauses nothing. ```ts title="variant-test.ts" import { outbound } from "./outbound.ts"; const { data } = await outbound.insights.variantTest({ campaign: "memory:acme:founders", }); for (const step of data.steps) { console.log(`step ${step.step}: ${step.verdict}, p-value ${step.pValue}`); console.log(` ${step.reason}`); console.log(" more sends needed:", step.moreSendsNeeded); } console.log("totals:", data.totals); ``` ```text title="Output" step 0: winner, p-value 0.0077 Variant A (0.08) beats every other variant by more than chance at alpha 0.05. more sends needed: null step 1: no_difference_yet, p-value 0.3124 Variant A (0.025) leads variant B (0.015), but a gap this size could be chance. more sends needed: 2213 totals: { noDifferenceYet: 1, stepsTested: 2, tooLittleData: 0, winners: 1 } ``` On step 1, variant A leads 2.5% to 1.5%, but with 400 sends each that gap has a p-value of 0.31. Pausing B now would be a guess. ## `copyAudit` Problems in a campaign's copy, found from fixed rules before it sends. | Input | Default | Range | What it is | | --- | --- | --- | --- | | `campaign` | required | a campaign ref | The campaign to audit. | | `max_subject_chars` | 60 | 0 or more, whole | The most characters a subject may have. | | `max_body_words` | 150 | 0 or more, whole | The most words a body may have. | | `max_links` | 1 | 0 or more, whole | The most links an email body may have. | | `max_caps_words` | 1 | 0 or more, whole | How many words of four or more letters may be in capitals, so one acronym is not a finding. | | `max_exclamations` | 1 | 0 or more, whole | How many exclamation marks a message may have. | | `max_connection_chars` | 300 | 0 or more, whole | The most characters a LinkedIn connection request may have. | | `spam_words` | 14 phrases | at least 1 character each | Words and phrases to flag in email copy, matched whole and without case. Replaces the default list. | The default `spam_words` are: 100% free, act now, buy now, cash bonus, click here, double your, earn money, free trial, guarantee, limited time, no obligation, risk-free, urgent, winner. `data` (`CopyAudit`): - `campaign`, `channel`. - `clean`: `true` when no check found anything in the copy that could be read. - `findings`: one entry per problem, with `step`, `variantKey`, `action`, `check`, `severity`, a `finding` sentence and the `text` that triggered it. High severity first. - `byCheck`: every check that was run, with how often it fired. - `bySeverity`: `high`, `medium`, `low`. - `stepsChecked`, `variantsChecked`, `variantsWithoutBody`. The checks: | Check | Severity | Fires when | | --- | --- | --- | | `malformed_variable` | high | Braces are unbalanced, or a variable has no name. | | `missing_subject` | high | The first email has no subject. Email only. | | `connection_request_too_long` | high | A connection request is over the limit. LinkedIn only. | | `spam_words` | medium | The copy uses a listed word or phrase. Email only. | | `too_many_links` | medium | The body has more links than allowed. Email only. | | `image` | medium | The body holds an image or a tracking pixel. Email only. | | `subject_breaks_thread` | medium | A follow-up's subject differs from the first email's, so it starts a new thread. Email only. | | `subject_too_long` | low | Email only. | | `body_too_long` | low | | | `all_caps` | low | | | `exclamation_marks` | low | | - **Needs:** `sequence`. - **Costs:** 2 requests. - **Notes it can add:** some variants have no body in the vendor's answer, so their body checks were skipped. - **Does not:** read the copy with a model, judge tone or check facts. It reads template text, never a rendered message. A variant whose body the vendor does not return is counted in `variantsWithoutBody`, not judged. ```ts title="copy-audit.ts" import { outbound } from "./outbound.ts"; const { data } = await outbound.insights.copyAudit({ campaign: "memory:acme:founders", }); for (const finding of data.findings) { console.log( `step ${finding.step} ${finding.severity} ${finding.check}: ${finding.finding}` ); } console.log("by severity:", data.bySeverity); console.log( `checked ${data.variantsChecked} variants in ${data.stepsChecked} steps, clean: ${data.clean}` ); ``` ```text title="Output" step 2 high malformed_variable: A variable's braces are not balanced, so it sends as raw text. step 2 medium spam_words: The copy uses 2 word(s) or phrase(s) that spam filters weigh. step 2 medium subject_breaks_thread: This follow-up has a subject that differs from the first email's, so it starts a new thread instead of continuing the first. step 2 medium too_many_links: The body has 2 links; the limit is 1. step 2 low all_caps: 2 word(s) of four or more letters are in capitals; the limit is 1. step 2 low exclamation_marks: The copy has 2 exclamation marks; the limit is 1. step 2 low subject_too_long: The subject is 68 characters; the limit is 60. by severity: { high: 1, low: 3, medium: 3 } checked 5 variants in 3 steps, clean: false ``` ## `copyReuse` How the same piece of copy performs in different campaigns. Variants are matched by the hash of their subject and body. | Input | Default | Range | What it is | | --- | --- | --- | --- | | `statuses` | `["active", "paused", "completed"]` | at least one campaign status | The campaign statuses to read. | | `max_campaigns` | 20 | 1 to 200 | The most campaigns to read. | | `min_sent` | 100 | 1 or more, whole | The smallest base a campaign needs to count in the spread, and a piece of copy needs to be named the best. | `data` (`CopyReuse`): - `reused`: every piece of copy used in two or more campaigns, with `contentHash`, `subject`, `bodyPreview` (the first 120 characters), `campaigns`, pooled `sent`, `replies`, `trials`, `replyRate` and `replyRateInterval`, `uses` (per campaign: `places` as `step:variant`, counts and rate), `spread` (best minus worst reply rate), `best`, `worst`, `spreadPValue`, `spreadIsReal` and a `finding` sentence. Widest spread first. - `bestCopy`: the copy with the highest pooled reply rate among those with a base of `min_sent`, or `null`. - `campaignsRead`, `piecesOfCopy`, `variantsWithoutStats`. - **Needs:** `sequence`, `stats.byStep`. - **Costs:** 2 requests per campaign, plus the campaign listing. - **Notes it can add:** some variants have no counts of their own, because the vendor reports their step as a whole or not at all, so their copy is listed without numbers. - **Does not:** match copy that differs by one character: the hash is of the exact subject and body. The spread test is always at the 0.05 bar. When a step has several variants and the vendor does not split them, those variants have no numbers. ```ts title="copy-reuse.ts" import { outbound } from "./outbound.ts"; const { data } = await outbound.insights.copyReuse({}); for (const copy of data.reused) { console.log(`"${copy.bodyPreview?.slice(0, 40)}..."`); console.log(` ${copy.finding}`); console.log( ` spread ${copy.spread}, p-value ${copy.spreadPValue}, real: ${copy.spreadIsReal}` ); } console.log( `best copy: "${data.bestCopy?.subject}" at ${data.bestCopy?.replyRate} over ${data.bestCopy?.trials} sends` ); console.log( `${data.piecesOfCopy} pieces of copy in ${data.campaignsRead} campaigns` ); ``` ```text title="Output" "Hi {{firstName}}, I saw {{company}} is h..." The same copy gets 0.08 in SaaS founders and 0.02 in Agency owners; the gap is more than chance, so the audience or the sender differs, not the copy. spread 0.06, p-value 0, real: true "Hi {{firstName}}, did my note reach you?..." The same copy gets 0.025 in SaaS founders and 0.02 in Agency owners; a gap this size could be chance. spread 0.005, p-value 0.6226, real: false best copy: "Quick question about {{company}}" at 0.0473 over 1100 sends 5 pieces of copy in 2 campaigns ``` The opener gets 8% in one campaign and 2% in the other. Same words, so the audience or the sender differs. The best copy's 4.73% is pooled: 52 replies over 1,100 sends in both campaigns. ## `replyTiming` How fast people reply, and on which days and hours. | Input | Default | Range | What it is | | --- | --- | --- | --- | | `campaign` | `null` | a campaign ref | Look at one campaign. `null` looks at every campaign. | | `since` | `null` | ISO 8601 date and time with offset | Only messages from this time on. `null` means 30 days before now. | | `max_messages` | 2,000 | 1 to 50,000 | The most messages to read, sent and received together. | | `timezone_offset_minutes` | 0 | -720 to 840, whole | Minutes to add to UTC for the weekday and hour tables, for example -300 for New York in winter. | `data` (`ReplyTiming`): - `campaign`, `since`, `timezoneOffsetMinutes`: what was used. - `timeToReply`: `sample` (replies paired with the message they answer), `medianHours`, `p75Hours`, `p90Hours`, and the share `within1Hour`, `within24Hours` and `within72Hours`. - `replies`: when replies arrive, as `byWeekday` (Monday first), `byHour` (0 to 23), `topWeekday` and `topHour`. - `sendsThatDrewReplies`: the same tables for the sends that drew a reply. - `counts`: `messagesRead`, `sent`, `received`, `paired`, `followOnReplies` and `unpaired`. - **Needs:** `messages.list`. - **Costs:** 1 request per page of messages, up to `max_messages`. - **Notes it can add:** the adapter cannot filter messages by date, so the cap applies to all messages and older ones were dropped after reading; a cap was reached. - **Does not:** separate automatic replies, because messages carry no automatic flag. A reply is paired with the last message sent before it in the same thread, or to the same lead. A further message from someone who already replied is counted in `followOnReplies` and not timed. A reply with no earlier send in what was read is `unpaired`. The offset is fixed, so it does not follow daylight saving. ```ts title="reply-timing.ts" import { outbound } from "./outbound.ts"; const { data } = await outbound.insights.replyTiming({}); console.log("time to reply:", data.timeToReply); console.log( `replies arrive most on ${data.replies.topWeekday} and at hour ${data.replies.topHour}` ); console.log( `sends that drew a reply went out most on ${data.sendsThatDrewReplies.topWeekday} at hour ${data.sendsThatDrewReplies.topHour}` ); console.log("counts:", data.counts); ``` ```text title="Output" time to reply: { medianHours: 24, p75Hours: 48, p90Hours: 76.8, sample: 5, within1Hour: 0.2, within24Hours: 0.6, within72Hours: 0.8 } replies arrive most on Monday and at hour 9 sends that drew a reply went out most on Monday at hour 9 counts: { followOnReplies: 1, messagesRead: 12, paired: 5, received: 7, sent: 5, unpaired: 1 } ``` Five replies could be paired, after 0.5, 6, 24, 48 and 96 hours. Every figure in `timeToReply` rests on those five: read `sample` before trusting a percentile.