Webhook events

Which vendor event becomes which normalized type, how event ids stay stable when a vendor delivers twice, and how to drop events you already handled.

What each vendor event becomes

The table below was produced by running every documented webhook payload in the repository’s fixtures/ folder through parse. The payloads are built from each vendor’s documentation with personal data removed. They are not captured deliveries, and no parser has received a live one. See what is verified.

import { parse, webhookProviders } from "outbound-sdk/webhooks";

import { payload, payloadNames, post } from "./fixtures.ts";

const row = (cells: readonly string[]) =>
  [cells[0]?.padEnd(10), cells[1]?.padEnd(36), cells[2]?.padEnd(25), cells[3]]
    .join(" ")
    .trimEnd();

console.log(row(["provider", "documented payload", "type", "data.category"]));

for (const provider of webhookProviders()) {
  for (const name of payloadNames(provider)) {
    // oxlint-disable-next-line no-await-in-loop -- one payload at a time keeps the table in order.
    const events = await parse(post(payload(provider, name)), { provider });

    for (const event of events) {
      console.log(
        row([
          provider,
          name.slice("webhook.".length, -".json".length),
          event.type,
          String(event.data.category ?? ""),
        ])
      );
    }
  }
}

provider   documented payload                   type                      data.category
heyreach   message_sent.THIRD_PARTY             linkedin.message_sent
heyreach   voice_note_received                  linkedin.message_received
instantly  account_error                        sender.status_changed
instantly  campaign_completed                   campaign.status_changed
instantly  email_bounced                        message.bounced
instantly  email_link_clicked                   message.clicked
instantly  email_opened                         message.opened
instantly  email_sent                           message.sent
instantly  lead_closed                          reply.categorized         positive
instantly  lead_interested                      reply.categorized         positive
instantly  lead_meeting_booked                  meeting.booked
instantly  lead_meeting_completed               lead.status_changed
instantly  lead_neutral                         reply.categorized         neutral
instantly  lead_not_interested                  reply.categorized         negative
instantly  lead_out_of_office                   reply.categorized         ooo
instantly  lead_unsubscribed                    lead.unsubscribed
instantly  lead_wrong_person                    reply.categorized         negative
instantly  reply_received                       reply.received
smartlead  CAMPAIGN_STATUS_CHANGED              campaign.status_changed
smartlead  EMAIL_BOUNCE                         message.bounced
smartlead  EMAIL_LINK_CLICK                     message.clicked
smartlead  EMAIL_OPEN                           message.opened
smartlead  EMAIL_REPLY                          reply.received
smartlead  EMAIL_REPLY.newdocs                  reply.received
smartlead  EMAIL_REPLY.threaded                 reply.received
smartlead  EMAIL_SENT                           message.sent
smartlead  LEAD_CATEGORY_UPDATED                reply.categorized         positive
smartlead  LEAD_UNSUBSCRIBED                    lead.unsubscribed
smartlead  MANUAL_STEP_REACHED                  unknown
smartlead  UNTRACKED_REPLIES                    reply.received
smartlead  campaign.bounce_threshold.breached   campaign.status_changed
smartlead  email.account.disconnected           sender.status_changed

How close each set of payloads is to the vendor’s documentation differs:

• Instantly documents one generic payload schema for all events, not one example per event. The payloads were built from that schema, so which optional fields a given event really carries is not verified.
• Smartlead documents an example for each event in the table. The three EMAIL_REPLY rows are three documented variants of the same event. The last two rows are account-level notices with a different envelope.
• HeyReach documents one payload, voice_note_received. The message_sent payload was published by a third party, not by HeyReach.

The parsers also map these event names, for which the repository has no documented payload. The mapping is by name only and was not run for this page:

Any other event name comes back as type: "unknown" with the vendor’s name in data.vendorEvent and the body in raw.

Reading the type

• reply.categorized carries data.category where the vendor’s event implies one: positive, negative, neutral or ooo. The category is the vendor’s. The SDK calls no model.
• Several vendor events share a type. Use data.vendorEvent when you need to tell lead_interested from lead_closed.
• Two events can describe one thing. HeyReach can send both message_reply_received and every_message_reply_received for one reply. They have different ids, so they are two events to you.

Idempotency

event.id is the same every time the vendor delivers the same event, so handling an event twice can be made a no-op. How the id is built depends on what the vendor sends:

An id built from the payload’s fields is stable only as far as the vendor resends the same values. Whether a vendor’s retry repeats its timestamp was not checked against live deliveries.

Within one request, parse already drops an event whose id repeats.

Drop events you already handled

Pass a dedupe store and parse returns only events whose id it has not seen.

import { memoryDedupeStore, parse } from "outbound-sdk/webhooks";
import type { DedupeStore } from "outbound-sdk/webhooks";

import { payload, post } from "./fixtures.ts";

const reply = payload("instantly", "webhook.reply_received.json");
const sent = payload("instantly", "webhook.email_sent.json");

// The same payload delivered twice gives the same id.
const [first] = await parse(post(reply), { provider: "instantly" });
const [again] = await parse(post(reply), { provider: "instantly" });

console.log(first?.id);

console.log("same id on redelivery:", first?.id === again?.id);

// Without a store, a redelivery is returned again.
console.log("no store, second delivery:", again === undefined ? 0 : 1, "event");

// With a store, an event already claimed is dropped.
const count = async (body: string, dedupe: DedupeStore): Promise<number> => {
  const events = await parse(post(body), { dedupe, provider: "instantly" });

  return events.length;
};

const store = memoryDedupeStore();

console.log("first delivery:", await count(reply, store));

console.log("redelivery:", await count(reply, store));

console.log("another event:", await count(sent, store));

// The store is one method. This one keeps ids in a Map; yours would use a
// database or Redis, where recording the id and testing it is one atomic step.
const seen = new Map<string, number>();

const custom: DedupeStore = {
  claim: (id) => {
    const fresh = !seen.has(id);

    if (fresh) {
      seen.set(id, Date.now());
    }

    return Promise.resolve(fresh);
  },
};

console.log(
  "custom store:",
  await count(reply, custom),
  await count(reply, custom)
);

ab72ea7bb4fd8b9ba91a3a2b3e593a10
same id on redelivery: true
no store, second delivery: 1 event
first delivery: 1
redelivery: 0
another event: 1
custom store: 1 0

memoryDedupeStore

memoryDedupeStore(maxIds) keeps ids in the memory of one process. It holds 10,000 ids by default and forgets the oldest when full. It is enough for one long-running process. It is wrong for anything with more than one instance, and for serverless functions, where each invocation may start with an empty store.

Your own store

DedupeStore is one method:

Recording and testing must be one atomic step, or two deliveries that arrive together will both be claimed. A unique-key insert in a database, or SET with NX in Redis, does that. The SDK stores nothing itself; the store is yours.

Events are hints

Deduplication handles the event that arrives twice. It does nothing for the event that never arrives. Do not treat a stream of events as complete: check event-driven numbers against list calls, which the reconcile plugin does.