Five hard truths about building AI products
A designer’s view on why AI can’t be built with the SaaS playbook
This post is a version of a talk given by Molly Mahar at Building Frontier AI Products in San Francisco.
My job as an AI designer puts me in the middle of a constant tug-of-war: product teams on one side, ML teams on the other.
From day one on a project, I’m responsible for the quality of models and systems from a UX perspective. As the bridge between product teams, designers, and ML scientists and engineers, I occupy a rare vantage point: I see the tensions that arise between building great products and building great models, and I see how often they clash. That conflict is why building AI products is so hard, and why most SaaS companies underestimate the challenge.
From the outside, AI looks like a technology problem. From the inside, it’s a people problem—a set of instincts, incentives, and expectations colliding in new ways.
Over the past few years at Intercom, I’ve helped our company shift from SaaS to AI. It hasn’t been smooth: it’s required us to rethink how we design, ship, and even organize ourselves.
What we built matters, but what’s really determined our success is our team’s openness, humility, and willingness to revisit first principles. None of this was easy, but it’s necessary.
Here are five truths we’ve wrestled with—and the lessons we took from them.
Truth #1: Demos are dangerous
In most product orgs, “share early, share often” is healthy. It encourages transparency, creates alignment, and helps you catch issues before they metastasize. But in AI, shiny demos are a trap. They hide brittleness, hallucinations, and integration gaps that may make your grand vision completely infeasible. A demo isn’t just a prototype, it’s a promise about future quality. Show it too soon, and expectations spin out of control.
I’ve watched this play out in a few ways:
A team throws together a proof of concept in a few days. It’s slick enough to get people excited, so they decide to ship it—even though it’s fragile and full of gaps. Unsurprisingly, usage never materializes.
Leaders react to a shiny demo by talking about release dates and marketing campaigns, before anyone has checked whether the model can deliver. I’ve been stuck in endless conversations about how to rationalize a very early feature into the existing product system before we’ve proven its usefulness.
Each time this happens, it slows you down. Instead of building the foundations you need, you’re dragged into conversations about things that may never work. And speed is everything in AI.
We’ve had to adjust for this reality since our team was founded. For example, when we built the first alpha of Fin, we worked quietly for weeks until we were confident the essentials held together. When we finally showed it to leadership, we didn’t just show the shine—we walked through every unknown and every caveat.
Here’s what we’ve learned:
Provide AI advice and resources to teams experimenting with small AI features before they even think about demoing, so they start from a stronger place.
Keep some projects under wraps until the basics are reliable.
When you do demo, hedge carefully. In our team, we’re explicit about what’s risky, what’s untested, and what could still fail.
Above all, demo only what you’re willing to be accountable for. The moment you show something polished but incomplete, the polish itself communicates a stability your AI product doesn’t yet have.
Truth #2: Polish is a trap
Every product team struggles with balancing speed and quality. AI adds a tougher question: can we ship at all? Models can’t improve in the abstract. They need real-world exposure—fast. That makes premature polish dangerous, because every rev of the design delays the model, which delays the feedback, which delays the whole system. The cost of wasted cycles compounds fast.
This is counterintuitive for many. Polishing feels safe. It feels like you’re de-risking. But in AI, polish often hides the true risks. I’ve been in rooms where leaders want one more iteration of an interaction flow while the scientists are desperate for live data. That tension can paralyze progress if you don’t resolve it quickly.
Over time, I’ve learned to cut through debate by being smart about where I spend design effort. For example, early Fin prototypes had discernable quality gaps. Rather than spend effort on design polish, I tested my own prompt variants, ran offline evaluations, and demonstrated to scientists that my version created a better end-user experience given Fin’s positioning. They validated my work, hardened it for robustness, and shipped it as the alpha release. That trade-off—prioritizing model quality over surface polish—helped us move fast without locking into the wrong path.
Here are more tactics that can help:
Be explicit about what kind of feedback you’re giving. Is this a curious thought, a minor issue, or a major blocker? That clarity matters.
Use hybrid judgment. As a designer with an AI lens, I can sometimes arbitrate: pushing back on premature visual polish while pushing hard for system quality.
Disagree and commit. Not every debate needs to end in consensus. Sometimes the important thing is simply deciding and moving on.
Don’t polish before feasibility and user value are proven.
Debates about polish aren’t really about craft; they’re about focus. And in AI, focus can flip overnight—a breakthrough, a dead end, and suddenly your roadmap is gone.
Truth #3: Roadmaps will fail you
AI makes a mockery of traditional roadmaps. Models surprise you constantly. Sometimes you hit breakthroughs faster than you expected. Other times, something never quite works, no matter how long you hammer away. Either way, the static roadmap you carefully crafted is toast.
Intercom used to work in six-week cycles, with cross-team schedules and neat alignment. That rhythm is gone. Now we run dynamic workstreams. Teams reallocate flexibly. Plans change weekly based on whatever’s riskiest right now.
Instead of “what are we delivering this cycle?” our prioritization questions are:
Is there a big bet we haven’t made yet, that we should make this week?
What’s the biggest risk we need to de-risk this week?
Sometimes workstreams explore a new bet. Other times they double down on a proven one. The rhythm itself becomes the structure.
For ML teams, this feels natural. For product teams, it can feel like chaos. Yesterday you thought you knew your next month; today you’re not sure what you’re working on this afternoon. There are costs. Trust takes effort to rebuild as people join and leave streams. Relationships reset. Knowledge is lost. But the alternative—staying static—is worse.
Survival in AI requires ruthless, constant reprioritization. Which is why no becomes essential.
Truth #4: No is necessary
Customers are wonderful. They bring high expectations and endless ideas. But in AI, their feedback is harder to parse. Negative signals (“this doesn’t fit”) are gold. You can act on them. Positive signals (“this would be cool”) are treacherous. Because so much in AI is hypothetical, a “cool” idea may never translate into a real workflow.
This is a hard adjustment for product teams used to treating customer enthusiasm as reliable guidance. In AI, you have to dig deeper. The real question isn’t whether the solution sounds appealing—it’s whether the underlying problem is worth solving, and whether AI is the right way to solve it.
That makes saying no a survival skill. And it’s hard. Saying no to customers who threaten to churn for something shiny-but-useless is scary. Saying no to execs who ask directly for a feature is scarier. Saying no to other teams makes you feel selfish. But saying yes to everything means you never finish the bets that actually matter.
Here’s our approach:
Combine usage data with brutal, honest feedback to separate the shiny from the real.
Ask whether it’s a good business decision: is this workflow common? Is the opportunity cost justified? Are the model costs sustainable?
Say no to everything that doesn’t pass those tests.
Demands will overwhelm you. Saying no isn’t failure, it’s focus. But “no” only works if ownership makes it stick.
Truth #5: Ownership can sink you
“Who decides?” sounds like administration. In AI, it’s existential. The wrong DRI (Directly Responsible Individual) at the wrong time can derail a product completely.
We’ve tried different approaches. A PM-led model gave us speed, but marketing and release pressures sometimes overruled ML quality. An ML-led model gave us rigor, but made handoff difficult when other teams weren’t invested. A shared triad slowed us down—too many cooks.
Our current model is a hybrid approach:
a PM as overall DRI
ML and AI design leads as strong advocates.
It’s not perfect, but it gives us balance: speed of decision-making with discipline-specific guardrails.
Our advice: scope ownership as carefully as you scope features. If ownership fails, everything else—no’s, roadmaps, polish, demos—collapses. Your AI product will drift off course.
The bottom line
AI products rarely fail because the models are difficult. They fail because people are complex.
Success requires uncomfortable choices: shipping rough work, reprioritizing weekly, saying no to shiny distractions, pushing back on managers, and holding the line on quality at the right moments.
For SaaS companies pivoting to AI, the question isn’t just whether you can build with AI. It’s whether your organization can adapt to the chaos with clarity, courage, and focus. That’s the real work.
And if you can’t do that, the technology won’t save you.