Finding, qualifying, and auditing leads before a human touches them.

The pipeline

A campaign starts from search criteria (e.g. "plumbers in Ealing") and runs a fixed sequence: discover leads from Google Places Text Search and Companies House, merged and deduplicated; classify each lead's web presence (own domain, broken site, social-only, marketplace-only, or none); audit own-domain and broken sites against PageSpeed/Lighthouse; score every lead into HOT / WARM / LOW / POOR_FIT bands; then generate an AI-drafted audit summary, proposal, and outreach asset for the ones worth pursuing. Leads with no usable website skip straight to a presence-gap path — OSINT enrichment (Hunter, urlscan, Companies House) instead of a site audit — so "no website" is scored as an opportunity, not dropped.

Each stage runs as its own queue with its own concurrency and rate limits — discovery capped at 6 keyword searches and up to 60 results per keyword; OSINT enrichment throttled to 6/min; the Playwright-driven deep audit capped at 10/min and concurrency 1, because it's the CPU/RAM-heavy stage. Leads only reach outreach generation after passing the qualification gate, not automatically on discovery.

Scoring, not a coin flip

Opportunity score is a weighted sum of objective signals, clamped to 0–100: no website (+40), a mobile PageSpeed score at or below 49 (+30) or in the 50–69 band (+15), 50+ reviews (+30) or 10+ reviews (+15), plus specific defects pulled straight from the audit's own findings — broken images, broken links, mixed content, missing HTTPS, no contact method. Off-anchor-vertical leads are soft-capped below the WARM threshold so a good score on the wrong kind of business doesn't queue outreach it shouldn't. HOT is 80+, WARM is 60+, LOW is 40+, everything else is POOR_FIT and stops there.

The compliance gate on the send path

Outreach doesn't go out because a draft exists. SendOutreachDraft runs a suppression check, then an approval gate that blocks the DRAFT → SENDING transition until a human clears it, then a dry-run gate — OUTREACH_SEND_DRY_RUN=true in production by default, so sends stay drafts-only unless that's deliberately switched off — before anything reaches Resend. Cold outreach is restricted to Ltd/LLP entities under PECR, which is why Companies House legal-form data feeds directly into whether a lead is "sendable," not just whether it has an email address.

When a step fails halfway

Every queue job carries a deterministic id derived from what it acts on — discover-{campaignId}, deep-audit-{leadId} — so a retry or a duplicate trigger lands on the job already in flight instead of starting a second one. State-changing use cases (start, pause, approve, suppress, complete) are idempotent: running one twice is a no-op, not a second effect.

The hard case is an email whose send returned nothing usable. The draft is left in SENDING and ReconcileOutreachSend replays the original request under the same idempotency key. Resend either hands back the original message id — the receipt that was lost — or performs the send that never landed. Neither is a second email. That only holds while the provider still remembers the key (24 hours), so outside that window the draft goes to a person instead of being resolved by code. Reconciliation resolves in one direction only: it can produce a receipt, and it never marks a draft FAILED because it could not find out. A refused replay says nothing about whether the original was delivered.

Campaign completion had the same kind of gap. It was re-checked only when a job succeeded, so a campaign whose last lead exhausted its audit retries sat RUNNING for 23 days in production. A 15-minute sweep now re-runs the same terminal check across every running campaign, treating a lead untouched for 6 hours as settled. The lead keeps its honest non-terminal stage; only the campaign is unblocked. Replacing out-of-scope leads with fresh discovery is capped at 3 cycles per campaign, and the counter is persisted before the enqueue so a crash cannot reset it.

Trade-offs I chose

Slow over parallel on the heavy stage. The Playwright deep audit runs at concurrency 1 and 10 per minute. A campaign takes longer, but one worker cannot starve the queue stages around it of memory.

Safe by default over sending by default. OUTREACH_SEND_DRY_RUN defaults to true in the config schema, and switching it off without a mail provider key fails at startup rather than at the first send. A misconfigured deploy produces drafts, never stray email.

A person over a guess. An ambiguous send outside the idempotency window, a lead that is not a Ltd or LLP, a draft nobody approved: each stops and waits. The cost is manual work on edge cases; the alternative is emailing someone twice, or emailing someone the law says not to contact.

Scope over score. A strong score on the wrong kind of business is soft-capped below WARM, so the pipeline never queues outreach it cannot follow up.

What the production numbers actually say

Those figures are a dated, point-in-time snapshot read directly from the production database on 4 August 2026, after a specific backfill (PR #120) that started persisting Hunter-sourced email addresses instead of discarding them. They describe what one engineering change moved, not a live counter — treat them as a measurement of that day, not today's number.

Measured, not claimed

This is an internal tool with no public URL — there's nothing to hand a recruiter to click through. At independent QA sign-off (16 May 2026) the MVP passed all 10 quality gates, 759/759 tests, global coverage in the 94.8–100% range across statements, branches, functions and lines, and an OWASP Top-10 walkthrough with no findings — with one named blocker: the UI's composition root was wired to a no-op queue instead of the real one, so starting a campaign from the UI persisted a row but never reached the job queue. That report is explicitly superseded as a status source; it's cited here for the gate results, not as today's state. What closes the loop on "is this actually running" is the 4 August production-database read above, taken after further work landed — not the May sign-off by itself.

Stack: Next.js · TypeScript · PostgreSQL · Prisma · BullMQ + Redis · Google Places API · Companies House API · PageSpeed Insights / Lighthouse · Playwright · Hunter · OpenRouter · Resend.