Three years of humans in the loop

Three years ago I wrote that abandoned carts are a people problem, and that the industry was solving it with automation because automation is what the industry reaches for. We built the version with real people in it.

Enough time has passed to grade that. Some of it held up better than I expected and one part of it I'd argue differently now.

What held up#

The conversion difference is real and it's large. A conversation that identifies and resolves an actual objection performs at a level a reminder sequence doesn't approach. This was the whole bet and it was right.

The discount stays in your pocket more often than you'd think. The automated version's only escalation is money. A person can find out that the concern was shipping time, or sizing, or whether the thing is real, and answer it. Every one of those is a sale that didn't cost margin. This is the part I most underestimated in 2018 — I framed it as a conversion story and a lot of the value is a margin story.

Nobody funded a competitor to do it. I wrote about this last year. The labor model that looked like our weakness turned out to be why we had a category to ourselves — venture-scale capital structurally can't want this business, so the well-funded companies all went to automated broadcast and left the conversational segment alone.

The byproduct was worth more than I said. Every conversation is a merchant learning why customers don't buy, in the customer's own words. I called that a byproduct in 2018. It's one of the most valuable things the product produces and we've never fully built a way to give it back to merchants.

What I'd concede#

Not every conversation needed a person. A meaningful share of recovered carts were people who were going to come back and needed a nudge. A person handled those and a template would have handled them equally well. We spent human attention on the easy half for years, because we'd made "humans" the identity of the product rather than a tool we applied selectively.

The correct architecture was always triage — automate the ones that don't need judgment, escalate the ones that do. I resisted that longer than I should have because it felt like conceding the premise. It isn't conceding the premise. The premise is that some conversations need a person, and that's still true and still rare in this category.

Quality is an operating problem forever. Software ships once and behaves consistently. People need training, guidelines, review, and someone thinking about it every week. That work never ends and it doesn't compound the way engineering does. I don't think I priced that correctly going in.

The economics don't clear everywhere. Combined with performance pricing, there are merchants where we do real work and the numbers don't work — low order values, low conversion, categories where the objection isn't answerable. We learned that one merchant at a time. Knowing in advance which merchants those are is a thing we still can't do well.

The general version#

The lesson isn't "use humans." It's that when everyone in a category reaches for the same tool by reflex, the space they've all skipped is worth looking at, and it's usually skipped for a reason that's about the builders rather than the customers.

Nobody avoided conversational recovery because it worked badly. They avoided it because it has labor in it, and labor is unfashionable and hard to raise money against. That's not a customer preference. That's a supply-side artifact, and supply-side artifacts are where small companies get to live.

I'd look for that shape again. I'd also, next time, be less precious about the mechanism and more attached to the outcome.