// ai postmortemsby JoshMay 13, 20265 min read

Why the Document-Chase Bot We Built for a CPA Firm Never Shipped

Eight weeks of building. Demo went great. The firm never used it. Here's why the project failed and the prompt I'd ask before starting it over.

Why the Document-Chase Bot We Built for a CPA Firm Never Shipped

Eight weeks. Three iterations. A clean architecture. A working demo that made the partner group nod.

It never shipped to a single staff member.

This is the story of why and what I'd do differently.

What we built

The firm had a chronic document-chase problem. Senior staff spent 11-14 hours a week chasing W-2s, K-1s, and 1099s. We built a system that:

  • -Generated a per-client expected-documents list from prior-year data
  • -Sent automated reminders at 48, 96, and 168 hours past expected
  • -Personalized each reminder based on prior-year communication patterns
  • -Routed responses back into the CRM with status updates

Demo went well. Partners loved the dashboard. The system was technically sound.

Why it didn't ship

The firm's CRM was Canopy. The senior staff didn't use Canopy as their source of truth. They used it for billing and for compliance, but the actual client communication happened in:

  • -Email (forwarded around the team)
  • -Phone (logged in Canopy after the fact, if at all)
  • -Text messages (the partners' personal cells)

The document-chase system pulled from Canopy. The partners didn't update Canopy in real-time. So the system saw a client as "missing docs" when the partner had already received them by text the night before.

The first time the system sent a reminder for a doc the partner already had, the partner emailed me. Three more times in week one. The system was wrong from Canopy's lag, not from a bug.

The partners stopped trusting it. The senior staff stopped using it. The system kept running but the humans stopped looking.

Root cause

I built the system on Canopy as the source of truth without verifying that Canopy WAS the source of truth.

If I had spent two days shadowing the partners before scoping, I would have seen the text messages, the phone-tag, the email forwards. I would have built either a different ingestion layer or a different trigger logic.

The fix was not to build the system. The fix was to figure out where the truth lives and build around that. I missed it because I assumed the CRM was the source of truth like it would be at a SaaS company.

What I'd do over

Two changes.

One, shadow before scoping. Two full days watching the actual workflow. Take note of every place client information lives. Build a map. Validate the map with the partners. THEN scope.

Two, build the source-of-truth question into the first proposal. "Where does the truth about a client's status currently live? If you couldn't access Canopy tomorrow, where would you go to know if Mr. Smith has filed his W-2?" The answer reveals the architecture.

What I tell new prospects now

When a CPA firm calls me about a document-chase or notice-handling automation, the first thing I ask is where the partners ACTUALLY communicate with clients. Email? Phone? Text? Canopy? Some mix?

If the answer is "all of the above, mostly text and email," the engagement scope includes an ingestion layer that pulls from those sources, not just from the CRM. That's a different project than what most firms initially scope.

The firms that say "yes, we should fix that ingestion problem first" become real engagements. The firms that say "no, just build it on Canopy and we'll fix the rest later" — I now decline politely.

The lesson that took me longer

The technical solution is rarely the bottleneck. The organizational reality — where information actually lives, who has authority, what the actual workflow looks like — is the bottleneck. A perfect technical system on a wrong assumption about reality is a perfect failure.

Two days of shadowing would have saved eight weeks of building.

postmortemcpaai consultingfailurelessons
// go deeper

Want the full guide? Check out our deep-dive page for more context, FAQs, and resources.

read the full guide
// keep reading

Related posts

// ready to ship?

Let's build yours.

Reading is the easy part. We do the work. Tell us what's broken and we'll tell you straight up whether we can help.