Lead generation pipeline

OBdesign's clients are small businesses on Vancouver Island, and small businesses don't come to you. This is the system I built to go to them. It scrapes, scores, and drafts, so the only thing left for me is the judgment call and the send button.

It has produced most of the client work on this site. Today it holds 942 qualified leads across the Island, and I run the whole operation from one screen.

The pipeline

1. 01Scrapebusinesses for a region and niche from Google Maps — name, site, phone, reviews
2. 02Analyzevisit the site and assess platform, age signals, mobile behaviour, obvious problems. Domain registration comes over RDAP and gets cached, because a 2009 domain still serving a table layout tells you something
3. 03Discoverfind the owner's real email through the Tavily search API rather than scraping search engines, which block datacentre traffic silently — about 1.4 credits per lead against a monthly quota
4. 04Validatesuppression lists (already contacted, chains, wrong niche, asked to be removed), dedupe, and sanity checks before a lead can become an email
5. 05Draftwrite a personalized first email referencing what the analysis actually found and save it to Gmail via the API. Nothing sends automatically; I read and send every one by hand

The bulk of it is two modules: the pipeline itself, and a separate domain-status checker that does the RDAP lookups and the "is this site actually broken" verification, because that logic got fiddly enough to deserve its own module.

State lives in flat JSON files under version control: contacted, suppressed, outcomes, removed. Boring, easy to inspect, easy to diff. When the operator and the developer are the same person, that beats a database.

The dashboard

Scraping is the easy half. The hard half is deciding who to contact this morning, and that needs a human looking at real signal, fast. So the pipeline writes a self-contained dashboard: one HTML file, no server, no build step, that I open locally and work straight out of.

Every row is a lead with its score, its region, and the specific problem the analyzer found: an expired security certificate, a domain that now resolves to a parking page, a site that takes six seconds to load, a 404, no mobile viewport. Those aren't categories I typed in. They're what the analyzer observed on the live site, and they're the reason the first line of a cold email can be specific instead of generic.

The tabs across the top are the actual pipeline state: 335 ready to draft, 20 drafted, 163 contacted, and the segments underneath split leads by how they're best reached, including 316 with a phone number and no usable email.

Opening a lead gives me everything needed to make the call in one place, and then the part that matters most: outcome buttons. Replied, won, lost, bounced. That single row of buttons is why I actually know what works and what doesn't.

Tracked to real money, not opens

Because every send gets an outcome recorded, the funnel math is real money rather than marketing metrics. The numbers surprised me:

Those three numbers changed how I run the business. The bottleneck was never finding leads, it was sending. Every client the pipeline has ever produced came from a first email, so I deleted the follow-up logic instead of tuning it. I could only make that call because the outcomes were recorded.

The bug that taught me the most

The analyzer originally generated a "hook" for each lead: the specific problem with their website that the email would open with. An audit of flagged leads found that about 97% of those hooks were wrong. The analyzer was pattern-matching its way into confident claims about sites it hadn't verified, and I was one send away from emailing local business owners about problems that didn't exist.

The fix was a rule, not just code: the drafter may only claim what the analyzer actually observed, and anything inferred gets softened or cut. "No website found" still requires human confirmation, because a business with only a Facebook page and a business whose site the scraper failed to reach look identical in the data.

Every claim has to trace back to something the analyzer really saw. That's now the first thing I check in anything automated I build, and it's why nothing in this system sends on its own.

What I'd change

The flat-file state is at its limit. It's survived 942 leads because a single operator on a single machine never hits a write conflict, but the moment this runs anywhere else it needs a real database and proper migrations. The scoring weights are hand-tuned and would be better learned from the outcome data I'm already collecting, which is the next thing I want to build.