Skills are a big part of our 2x journey; they allow us to build specialized capabilities for common tasks. We share these skills across our organization via a private, auto-updating plugin marketplace, created by our dedicated team (“team-2x”). When someone builds a skill to do a task in the best way possible, everyone on the team immediately benefits.

Our mission is to break down all technical work into repeatable, high-quality skills that can be run by Agents to augment our engineering work. Every improvement durably compounds into Agents getting things right in our environment, every time. A classic platform play.

As Darragh wrote in 2× – nine months later, within its first three months, our Claude Code plugins repo had become the most actively contributed-to project at Fin with 153 contributors (31% of R&D) and 267 skills.

After all this shipping, here’s what I’ve noticed separates the best skills from the worst.

Good skills:

• Do one thing really well.

• Are created using a high-quality feedback loop.

• Use scripts for the deterministic parts.

• Have evals that are regularly used and maintained.

• Use progressive discovery to avoid bloating context.

• Have guardrails. Best-effort attempts by Agents that go off-script often result in bad outcomes.

• Are cleaned up after iteration. It’s really common for skills to accumulate cruft over time, such as references to an old way of doing something – this information is usually irrelevant.

• Are regularly refreshed with new learnings. The skill itself can be instructed to do this, or periodic analysis of session data can achieve this.

• Use reference tables and prior learnings – not everything needs to be worked out from first principles every time.

• Use hooks to force skill-use when essential – for example, we never want Agents to query Snowflake without having loaded the domain-specific knowledge required to get queries right the first time.

• Use the correct plugin. Not everybody needs to use every skill, and plugins help get the right skills to the right Agents at the right time.

• Have owners.

• Are devoid of changelogs or references to the old ways of doing things.

• Are unopinionated. In the vast majority of cases, skills should not assume that specific tools or capabilities are available, beyond what is commonly used.

• Are Buzz compatible! (Buzz is our internal remote agent platform.)

Bad skills:

• Get the same things wrong every time.

• Have nobody looking at output.

• …or have non-experts evaluating output.

• Aren’t actually used.

• Are too large for Claude to actually read in one go.

• Can’t easily be invoked by keywords, etc.

• Half-ass the work – create a dashboard for humans to make decisions rather than fully delegating the work to the Agent.

If you’re interested in pointing your Agent at a small, edited, curated set of Claude Code skills/hooks that are heavily used by hundreds of engineers at Fin, we just published some.

Serious use of these will need customization for your environment.

They include:

• skill-tools — Tools for authoring and reviewing Claude Code skills. Includes the

  skill-review skill, which reviews skills against a closed 7-category quality rubric (Structural Discipline, Integrity, Test Coverage, Security, Content Quality, Convention, Cost) with structured JSON output and a determinism contract.

• security-tools — Harden GitHub Actions workflows against supply-chain and injection attacks. Includes the secure-github-actions skill (a 14-rule review checklist plus audit commands) and a hook that auto-loads it when you edit a workflow file.

• claude-code-tools — Meta-tools for running Claude Code well. permissions-analyzer vets your permission allowlist against a GREEN/YELLOW/RED safety model; tool-misses finds and fixes missing CLI tools / BSD-GNU incompatibilities; cc-cost-analysis is a framework for analyzing Claude Code usage costs from OpenTelemetry data.

• test-tools — Investigate and fix flaky tests. The fix-flaky-tests skill detects your framework and CI provider, classifies the flake, and enforces green-CI-as-the-only-verification discipline across RSpec, Jest, pytest, Go test, and more.

• code-review-tools — The thermo-nuclear-code-review skill runs an extremely strict structural and architectural review, hunting for “code judo” simplifications rather than correctness bugs or style nits. Inspired by Cursor’s plugin of the same name.

• pr-tools — Open well-formed GitHub pull requests from Claude Code and attach screenshots and recordings to them. Includes create-pr (intent-gathering, diff validation, public-repo safeguards) and attach-github-assets (upload local images/recordings to GitHub for use in PR and issue markdown).

Frontend Tech is a small frontend infrastructure team at Fin. For years, our job has been to help others build fast and safely, and the way we did that was pretty hands-on: maintaining shared code, reviewing PRs, documenting conventions, and answering dozens of the same questions a week.

We barely do any of that anymore. At least not for humans.

Instead of explaining conventions to people, we’ve encoded them into the tooling so anyone building UI, human or Agent, ends up on the right path by default. That changes how we work and also what our team is for.

Encoding our design judgment

Take Surge, our design system.

When it came to encoding our component library into skills, we had a headstart – they were already structured, named, and documented. But a component library is only half a design system. The essence of your design, your taste, lives in the details: which component to reach for, how to compose a flow, and which combinations hold up across the product and which break. That knowledge sits in designers’ heads, Figma annotations, and a variety of documentation, making it difficult for an Agent to build a cohesive picture.

Surge Intelligence doc for design system foundations

So the real work wasn’t wiring Surge into a skill, though we did do that. It was building the layer underneath: a structured, maintained knowledge base that encodes not just the components but the patterns and taste, in a form an Agent can reason over. We call it Surge Intelligence. The skills are just how you reach it; the knowledge layer is the real asset, and it compounds. A decision encoded once pays forward to everyone who builds anything similar after it.

Building this system is design work. You can’t hand it off as a documentation exercise. It means revisiting old decisions, arguing about them, and writing down why one pattern is right and another isn’t.

An example of this is the header of any given page. You could argue the simplest way to describe it is that a page has a title and an action, but it’s not that simple. You need to define everything:

• How long can the title be?

• How many actions can you have?

• Do you have a primary and a secondary action?

• Is there a limit on actions per page?

• Does the title need to follow certain content design guidelines?

Any pattern needs to be thought through in detail and encoded, ensuring usability and your taste are preserved.

The impact of a knowledge layer

We measure whether Surge Intelligence is working by comparing tasks run with and without it enabled. Without it, only 7% of the components an Agent reached for existed in our design system; with Surge Intelligence, 96% did. Code respected typechecking rules 74% of the time, up from 0% of the time. While components were placed in the correct context 68% of the time, up from 42%.

Unfortunately, that means even with the invented imports gone, about a quarter of pages still use the wrong props. But it’s early and we’re still building improvements.

The red elements show where the Agent has deviated from our design system, compared to when it has Surge Intelligence.

This is a concern because of how Agents fail. An Agent creating UI without a design system in context will default to generic patterns and custom code, drifting from your quality bar just to please you. Agents are biased to succeed, they won’t refuse a task even if they’re unable to complete it. So the design system has to be in the Agent’s hands with clear paths to success.

Beyond the visual

But Surge Intelligence is just one piece of a much larger frontend puzzle. In 2024, we began to migrate from Ember to React and committed to using AI coding Agents in the process. We built a “react” skill to ensure the code we write meets our standards and patterns, helping our engineers start building with less React experience.

Principal Engineer Pat O’Callaghan went further writing skills to evaluate existing product features that require migration, enabling product engineers to generate migration plans and verify the features once moved. Pat will share more on this soon.

In one year we’ve grown from supporting 2 engineers a week in React to up to 104.

Now, around 450 engineers have shipped React code, roughly 230 in any given month, most of whom didn’t know React before they started. They’ve taken React from 0% of the product to a third of it, writing 1.4 million lines of code, using AI infrastructure created and maintained by just eight people.

To be fair, though, that wouldn’t have been possible without investment in our overarching Agent platform and dedicated 2x Team. They’ve helped more than 90% of PRs to be written with an Agent while half are auto-approved by Agents.

So what’s the team for?

If the Agent has the entire design system, all our coding conventions, and our architectural documentation, do we even need to keep humans around to work on this stuff?

Yes. Obviously yes, and not only because I’d like to keep my job. But what we do, and how we go about it, has changed.

For years our work scaled linearly with the number of people building, and we couldn’t always keep up. Doing more meant adding more designers and opportunities for review. Now the conventions live in the environment, our leverage is completely different. By shipping one considerate skill, auto-installed for everyone, our small team can lift the velocity of the whole org.

Our frontend team is also able to serve and enable more teams than ever before. The line between engineer and designer keeps blurring – designers now prototype in code while engineers ship whole surfaces without a designer in the room. With Surge Intelligence, a designer can build something real, safely, or an engineer can ship something polished and consistent.

But this impact is only possible if we maintain and consistently improve our AI systems. This is still a work in progress.

What we’re still figuring out

One risk is the environment becoming a black box. If people can ship without understanding, you have to invest just as hard in keeping humans able to judge what the Agents produce. Your focus moves up a level: from how to do the work, to how to direct and judge it. That one’s tricky, and we haven’t solved it yet.

We have evals for the things that are straightforward to measure, but not yet a way to judge whether output is truly design-guidance compliant. Building that judging pipeline is next. We’ve encoded the obvious rules, but the part that carries our taste, like the composition decisions, the recipes, and the product knowledge, still mostly live in people. We’re eager to unlock this.

Our opportunity hasn’t changed. It’s the same as it always was: a small team that can have an outsized effect on what everyone else can build. It just means something different now: building the foundations that accelerate the company and everything built on top of them. The Agents included.

Everyone has an opinion about what AI is doing to knowledge work. Research is becoming obsolete. Data scientists are no longer needed now that anyone can query data in plain language. Everything is collapsing into one big AI insights role.

A few months ago I wrote about this, arguing AI commoditizes execution but not judgment, and that teams like mine had to move out of the comfortable middle before it disappeared underneath us. That wasn’t just my view. My leadership team and I had spent real time aligning on it and building a strategy around it. What we hadn’t done was have it affirmed by the people doing the work day-to-day. So this time, instead of adding another opinion, I asked the whole team.

I lead a 40-person research and data science (RAD) team at Fin, where we build an AI Customer Agent. A few weeks ago I sent them a short survey:

• What are you spending time on now that you weren’t a year ago?

• What’s actually changed in how you work?

• When you hear “our version of 2x,” what does that mean to you?

• Is your role evolving into something else, and if so, into what?

I expected stories about efficiency and moving faster. Instead, three themes surfaced again and again:

• The routine work has largely disappeared.

• The lines between roles are blurring.

• Impact, rather than output, was the greatest opportunity AI afforded us.

And underneath all three was something I hadn’t fully expected. We’d set the direction, but no one planned how it would play out. There was no reorg and no new job titles. People changed how they worked one by one, each following wherever AI made them most useful, and together that added up to a transformation faster than any formal restructure could have delivered.

The routine work fell away

The first change was the biggest. Many of our most common day-to-day tasks have moved to the background.

SQL, and the work that surrounds it, used to be central to our jobs. Significant time was spent on data investigation: finding the right tables, figuring out where all the data you needed lived, getting the grain right, building intermediate tables, getting the correct calculations, and debugging incorrect ones. All of this was necessary to get useful output from a SQL query. But today, this effort is largely owned by AI.

As one data scientist put it: “SQL used to be the majority of the job, and now it’s near zero.” That’s not a small change. SQL was the main thing we interviewed and tested for on the data science side of the team, and now it barely rates a mention.

Instead, we spend time building infrastructure around the SQL: the automated pipelines that gate, classify, analyze, and deliver insights. This helps teams outside of data science readily and reliably access them.

Our data visualization workflows have also changed. Creating Tableau dashboards and reports used to be a core deliverable and now they’re being replaced by HTML reports from internal tools like Claude4data and Cockpit.

Sample report from Cockpit.

The researchers feel this work from the other side. They pull their own data now, no longer waiting on a data scientist to unblock them. As one of them put it: “Exploring and analyzing customer data is now a given for any project. It’s at our fingertips, and we have every opportunity to interrogate, strengthen, and stress-test research findings without relying on data science. It would be crazy not to check whether your insights are strengthened or challenged by larger-scale customer data.” They also draw on far more evidence than they used to. A year ago, triangulating a finding mostly meant asking a colleague what they’d recently heard. Now a researcher can cross-check a hunch against real data from across the business: churn surveys, product satisfaction scores, win/loss records, and sales-call transcripts.

The most impactful part, though, is the speed of synthesis. One researcher described reading hundreds of sales calls, win/loss records, and interviews as a single body of evidence, in one working session. A year ago that was either weeks of manual reading or it simply didn’t get done. Starting from evidence that already exists, rather than running something new, used to be the fallback option. Now it’s often the smartest first step. You can build deep context on a question from existing studies, reports, and customer data before deciding whether you even need to commission fresh primary research.

So data scientists aren’t spending their days writing SQL, and researchers aren’t waiting on them for data. The slow work of gathering and synthesizing information is disappearing. The question is what rushes in to fill the space it opened, and that’s where the survey surprised me.

The roles are collapsing into each other

The most interesting takeaway from the survey was that people no longer describe themselves the way their job titles do. And the blurring runs in multiple directions.

Researchers are drifting toward strategy. “I feel less like a researcher and more like a strategist,” one wrote, because the manual work of gathering and reading evidence has diminished and the hours now go to framing the question and stress-testing the answer. Another was even more direct: “I don’t see myself primarily as a researcher anymore, if I’m honest. I’m more of a product and market strategist who focuses on analysis. I’m not sure what the title should be.” A third described research and data analysis as simply “collapsing into one.”

Meanwhile, data scientists are drifting toward engineering. They’re shipping production front-end, wiring language models into pipelines, and building internal tools that used to need a dedicated engineering team. One of them described forward-deployed data science and forward-deployed engineering as “overlapping from both sides.” Engineers can do the basic analysis that used to sit with data science, and data scientists are starting to make changes directly in our product rather than handing off. Another listed the roles he’s gradually becoming as: “part sales engineer, part value consultant, and part researcher at times.”

These changes weren’t entirely unguided. A few months earlier we’d asked the team to exit the comfortable middle, the routine work that no longer needed us, and move toward the harder, higher-judgment work. What we hadn’t done was script how each person would get there. There was no reorg, new titles, updated competency framework, or career ladder. People followed the work to where the leverage was, and the disciplines bent to meet it.

Building systems, not reports

The survey included dozens of examples, but they all shared one idea. People have stopped producing one-off work and started building things that keep working after they walk away.

One data scientist built an AI Agent that generates customer ROI decks. It started as a single pitch he wrote for one customer during a forward-deployed session, got good feedback, carried into the next deal, and then became something any account team could trigger with a click. It has now produced well over 450 decks, more than a hundred a month, and changed how the sales team pitches Fin. One system, built once, adopted far beyond the person who built it, doing commercial work every week.

Another data scientist built an entire customer feedback product inside Cockpit, our internal platform, more or less from scratch in about a month without an engineering background.This included: the backend, the React front-end, and a LLM job that clusters and summarizes customer feedback. It has fundamentally changed how the team does research, because instead of reading fragmented feedback across a dozen sources, there’s now one place to find it, group it and act on it. Interestingly, before its creation, we had been evaluating external tools for this skill but ended up building our own customized solution in less time than it usually takes to run initial evaluations on vendors.

On the commercial side, a data scientist and a researcher built a win/loss and live-deal intelligence system. It turns unstructured deal text, Gong calls, Salesforce notes, emails, and Slack threads into structured insight on why we win and lose, how competitors show up, and how live deals are progressing. The process used to be manual and infrequent. Now it runs daily, at a scale no single person could manage by hand, and it feeds a fortnightly deal briefing to our senior leadership team.

Our researchers are building similar tools. One of them ran a deep customer discovery effort, sitting down with more than 60 customers over different points in time. That part doesn’t get faster because AI exists, and it shouldn’t. Taking time to talk to the right people about the right thing is valuable. What changed is what he did with the material afterwards. Instead of writing it up once, through a single lens, as a report that starts to age the day it ships, he turned the whole set of interviews into a searchable repository that he adds to and built a skill he can query whenever a new question lands. Same high-quality research, but it now answers questions for months, from angles he never planned for when he ran it.

And some of the most important building is invisible. A few people noticed that different AI skills were each defining the same metric slightly differently, so the same question could come back with different answers depending on which one ran. Creating shared, verified infrastructure for our AI tools is essential to their accuracy, so the team is building out a semantic layer.

What 2x means for RAD

It would be easy to read all this as “the team builds systems and software now,” but it’s more than that.

Take the researcher who stood up audience messaging cohorts for marketing leadership. She pulled the raw data herself, including Gong calls, win/loss records, and recent studies, into a single picture of who the buyer is and what moves them. From there, she put an initial cohort portrait and messaging decision in front of the stakeholder to react to, and reworked it quickly in a matter of days. The value, as she put it, “was a faster decision.”

This is an example of what 2x means for my team. Or as one researcher said: “Our 2x isn’t twice the output. It’s the same hours spent a level up.“ And that “level up” looks different for different people. For some, it’s spending more time socializing insights, getting the right evidence in front of the leaders and into the rooms where decisions actually get made. For others, it’s rolling out an internal tool they’ve built and driving its adoption across the business so it genuinely improves how our customers succeed with Fin. Intelligently reinvesting our capacity is how we’ll have the greatest impact.

I find this genuinely encouraging, as the team got there on their own. The trap with any productivity gain is that the time you free up quietly fills back up with more of the same, new habits don’t get encoded. The fact that the team can see that trap, and name it, is worth more than any number of skills shipped.

The honest part

I’d be misrepresenting the survey if I made it sound like pure upside.

The velocity has a cost. One data scientist shared how it’s now harder to find time for deep focused analysis because the constant context-switching and the raised expectation of speed pull against it. She’s also spending more time verifying findings – hers and her stakeholders’ – because AI will confidently over-interpret data if you let it. Faster is not automatically better, and she’s right to say so.

There’s an identity cost too. When someone with years of craft behind them tells you they no longer know what their title should be, it’s a real and unsettling thing to sit with, and pretending otherwise helps no one.

And there’s a quieter counter-current I didn’t anticipate. One researcher described research as having “a grounding moment, coming back to basics” where the most valuable thing is cutting through the noise and helping the business focus on what matters. In a year that’s been all about acceleration and output, the instinct to slow down and ask “what’s worth knowing” might be the most strategic move of all.

Where this leaves us

These jobs aren’t going away, not from where I’m sitting. But they’re being rewritten in real time by the people doing them.

When I wrote about all this a few months ago, it was pure conviction. A thesis I believed and a vision I’d set for the team, but hadn’t yet watched play out. The survey is the closest thing I have to proof. It’s exciting to watch my team adapt to these new ways of working, designing and executing valuable projects we could have never accomplished before.

I won’t pretend we’ve arrived. We’re still working out how to keep self-serve trustworthy as more people across the company query the data directly, how to protect time for the deep questions when everything rewards speed, and how to say clearly what we’re going to stop doing. And we’re still pulled into the middle more than I’d like, because reactive work is genuinely hard to quit.

I asked the team one last thing: what they’d want people outside it to understand about this moment. I’ll leave the final word to one of them, because it’s better than mine. The next wave of value, he said, is “making what we create compound.”

A physical store has a hard ceiling – literally.

You can pack it to the rafters, but ultimately you can only fit a limited number of SKUs in a brick-and-mortar building. What ends up on the shop floor has to be curated, and knowledgeable staff do the rest: asking customers questions, learning their preferences, and providing suggestions. A limited catalog is the tax you pay for a good experience.

When commerce moved online, that limitation disappeared. Suddenly, you could host every SKU, think ASOS or Amazon. But this solution created another problem: the paradox of choice. Faced with everything, customers struggle to find anything. We presented them with a vast database and only gave them rudimentary tools, like filters and search bars, to explore it.

Seventy percent of online shopping carts are abandoned and the average online store converts at just 1.6–3% compared to brick-and-mortar’s conversion rates of up to 40%. A 3% conversion rate means 97% of people who were interested enough to visit left without buying. We’ve normalized this performance and called it a successful funnel. It’s time we ask for more.

What’s changed

The history of technology follows a pattern: great products emerge when an underlying scientific breakthrough enables a novel user interaction – one that wasn’t possible before and that feels, in retrospect, like the obvious thing to build.

Every major leap in computing tends to move in the same direction: make the interface less abstract and more human. We started with punch cards, powerful for data storage but illegible to humans. Then came the command line, giving us readable text, but you had to learn a language. Followed by the graphical user interface with its mouse and icons. And touchscreens, letting you reach through the glass and push things directly.

Each brought us a step closer to how we naturally interact with the world. Conversational AI is the next great leap forward.

And now that it’s here we can ask ourselves: what is the ideal UI and interface for ecommerce? And how can AI help us get there?

From a search bar to a sales assistant

Walk into a high-end boutique and the staff want to understand you. You mention an outdoor wedding in Spain, they ask a few smart questions, and within minutes they know more about what you need than any filter sidebar ever could. Conversational AI makes this level of service possible online and retailers are already looking for ways to provide it.

People seem to understand this intuitively. Our AI Agent Fin was initially designed for service, but we observed many of our customers using it to build their own Agents for ecommerce.

WHOOP, the fitness wearables company, is one example. During browsing, their customers had questions like: “Which membership is right for me?” or “How often do I need to charge my WHOOP?” If someone spoke to the inside sales team, they were twice as likely to convert, but there weren’t enough reps, and customers could wait more than 10 hours for a reply.

With a product launch on the line and an anticipated spike in conversations, they deployed Fin to the “Join” page – the final step before purchase. Fin resolved 84% of inbound questions, helping the team drive a 130% increase in attributable sales.

WHOOP and other retailers validated the need for an Agent that can provide personalized shopping.

What we built

We built a specific ecommerce role for Fin so it can connect to your Shopify store and answer product questions, help buyers find what they’re looking for, and handle support all in one conversation.

But to deliver a sophisticated shopping Agent, we had to rethink how AI reads product catalogs.

Traditional search – including most “AI-powered” search – flattens product data into text and loses everything that makes commerce structured. Pricing, variants, availability, and category hierarchies are all reduced to tokens.

Fin decomposes every query into semantic intent (what you mean) and hard constraints (what’s non-negotiable). When you say “I need a dress for under $200, nothing black,” the price ceiling and color exclusion are hard stops. That way, you don’t get beautiful $400 near-misses crowding out the right answer.

Its ability to reason across a product catalog beats out any attribute map. An early customer uploaded a selfie and asked what foundation color she should buy. Without the data to map pixel colors to products, Fin instead used general knowledge to identify her skin tone and undertone and search the catalog for suitable products.

Fin’s decision-making is also determined by relevant context. It tracks where you are on the site and in the buying process: exploring, narrowing, or deciding, and shifts its behavior accordingly. It knows when to show you related options, dive into detail, or close the sale.

The result is something closer to a knowledgeable sales assistant than a search bar.

What comes next

It would be an oversimplification to say conversational UI will replace shopping entirely. Technology doesn’t work that way. The new thing arrives but the old thing doesn’t immediately disappear – banks still run 50-year-old COBOL.

Conversational AI is excellent for some things, like exploration and moments of uncertainty. But traditional UI still has a place – nobody wants to fill out a delivery form through a chatbot.

The interesting design work is matching the mode to the moment, and to the shopper. Some people will always want to shop on their own.

This is the product space Fin for Ecommerce is currently exploring: how to create a perfect shopping experience that blends the best of all of the above modes.

Other more radical futures are also possible. Consumers may start to have their own personal AI Agents that shop on their behalf, meaning people won’t be on your website at all. ChatGPT and Gemini are already surfacing product results – your catalog appearing alongside competitors, a commodity in someone else’s interface.

As this type of shopping becomes more common, the retailers who own a direct, conversational relationship with their customers will have an existential advantage over those still optimizing their grid layout.

Take down the website

Early in the development of Fin for Ecommerce, someone on the team asked: what if you took down the whole ecommerce website and it was just Fin?

That’s not a joke. Almost every UI element on a storefront – product pages, collection pages, category nav, search bars, filters – exists to make a catalog self-navigable. If the default UI was conversational AI, much of that information architecture wouldn’t be necessary.

The storefront as we know it is transitional technology. Our entire interaction model was built for a world where the store couldn’t talk. Now that it can, most of what we built is about to look very, very dated.

In 2022, Intercom (now Fin) faced a reckoning. Five consecutive quarters of declining revenue growth required serious action. Our founding CEO Eoghan returned with a five-point plan, we refocused our efforts around a single product category, and underwent a reduction in force. And then, ChatGPT arrived. Our co-founder Des Traynor wondered if the emergence of AI was “adding insult to injury, or the single scenario that we can’t survive?”

All around us, SaaS companies were getting “sold off for parts to private equity.” The end seemed near.

But after a weekend of experimenting with the newly released LLM, the future appeared less bleak than initially forecast. As Des put it: “AI might actually be both the death of our company and also the best chance we have of survival.”

By Monday, we decided to go all in, and over the next three years we rebuilt almost everything we’d spent the previous eleven years building, including the name.

Des recently talked through how that happened – on The Further, Faster Podcast and the AI Revolution Show. What follows pulls the two conversations together: the entrepreneurial lessons learned along the way, what to consider when undergoing your own AI pivot, and why Des thinks we’ve grown too pessimistic about the AI-powered future.

Focus on the product in front of you

Before the AI story, there’s the unglamorous part that made it possible. On The Further, Faster Podcast, Des walks us through it and what he thinks today’s entrepreneurs should focus on.

Firstly, the job of a founder is “to do something that doesn’t make sense up until the point that it does make sense” – and the moment it makes sense, the founder phase is over and you’re running a business. Des feels it’s not good to idolize or obsess over the label of founder because it should be a temporary role; don’t cling to it rather than admitting your company has stopped working. The entrepreneurs he’d back again are the ones who “closed their business down, gave the money back, took a couple of years working in a good company, got some new ideas, and went again.”

That’s also why he’s wary of early-stage companies chasing investment capital. “A really long runway is not what you should be optimizing for. You should be optimizing for success.” Chase the valuation and you end up watching your own stock price instead of asking the only questions that matter: “are we building good software? Do people like it? Are they paying a good amount of money for it?”

You can feel the difference in the products themselves. Des refers to the experience of building Exceptional – an error-tracking tool the team developed before Intercom – as “doing your best to try and push it up a hill”. Intercom was the opposite: it “was a very easy product to bring out,” viral in the way B2B occasionally is. Early users routinely praised it, even just a few months in.

This is why it’s important to ship. Founders often delay launching because they’re afraid to find out their product doesn’t work, and a reputation as a “considerate, tasteful founder” is a comfortable disguise for that fear. After you ship, take user feedback seriously but not blindly. When they ask for 11 more features, don’t assume they’re right – “that could be their way of saying this is a bag of shit.”

Becoming an AI company takes sacrifice

The decision to go all in on AI was, oddly, the easy part. It was clear, customer service was about to be changed completely, and it was a race against the clock to be the first to market.

The hard part was everything after the decision. A transformation like this touches the whole company – the branding, the pricing, what you sell, how you build, and who you compete with. On the AI Revolution Show, Des compares it to the ship of Theseus: change enough of the parts and it’s hard to say what’s still the original boat. “Maybe the canteen is the same,” he joked.

What made that survivable was culture set deliberately, in advance. When Eoghan returned and rewrote the company values, three of them – resilience, open-mindedness, and impatience – read, in hindsight, like preparation for exactly this. So when we made radical decisions like breaking up teams or abandoning roadmaps, the ground had already been laid.

For companies going through change, Des also warns not to “be a prisoner of nostalgia.” Intercom was a longstanding, beloved brand. On the surface, sacrificing it looks like a significant risk. But he argues the opposite. “Being known isn’t always useful – it can almost be a counterweight.”

He tested this theory with a thought experiment: “If I said the coolest new payroll app is built by Workday, you’d be like, really?” The name you’ve spent a decade building can be the very thing that stops people believing you’re building something new, so you have to be willing to “shred and shed that brand legacy.”

Building and selling the AI-native way

Plenty of products call themselves AI-native. Most fail the first test: does the product work? Customer support is load-bearing – if it breaks, the business breaks – so “it works most of the time” isn’t good enough.

Clearing that bar takes more than a wrapper. We’ve gone down to the model layer with Apex, and what looks from the outside like a single product is, in Des’s words, “more like 27 systems connected together,” several revisions deep.

Looks can be deceiving though. It’s common for users to think they can recreate Fin with a frontier model, and maybe they can reach a 30 or 40% resolution rate in two minutes. “The other forty percent is going to take you three years and 60 people.” We watch prospects try to build their own, get stuck, and come back – and sometimes the only useful thing is to let them “bang their head off a brick wall” and leave the door open.

Building for AI is different from building software. We used to start with customer research, but because customers aren’t experts in what AI now makes possible, you now have to start with what’s technically reliable and take that to the market. It’s a reversal Des expects to be temporary, until the technology settles.

You also have to rethink design. The AI-native instinct isn’t to begin with “boxes and arrows and buttons.” It’s to take the least amount from the user and give back the most. The example Des provides is bug reporting. Rather than relying on users to report issues, a bug should be automatically detected so the experience goes “from seven screens to no screens.”

And the impact of all of this must be easily demonstrable. According to Des: “brand will get you the demand, but ultimately performance is what gets you the contract.” Serious buyers run bake-offs – a thousand of their own real questions across three vendors – and we lean into it. Rigour favours whoever is actually best; deliver results and the sales will follow.

What if it all goes right?

AI is going to keep moving. It’s a realization Des has had to come to terms with: “I’ve probably been more wrong than right whenever I’ve said … I think the AI is settled here.”

What does this pace of advancement mean for the wider market? The walls between products will come down – point solutions will be absorbed by whoever owns the rest of the workflow. Des thinks the systems of record will endure, and a smaller number of larger businesses will dominate.

It’s reasonable to feel apprehension towards such a mammoth change, it’s a narrative that dominates the media. And overall, the conversation about AI has become fluent in every way it might go wrong. But we’re strangely quiet about the other possibility. “No one’s asking what if it all goes right,” Des says – what it might mean for things like education or healthcare access.

The near term will hold real dislocation, and we’re not pretending otherwise. But the upside, he argues, “is potentially the biggest thing we’re ever going to see for humanity.”

We’ve already run the small version of that experiment on ourselves. We bet the company, disrupted the thing we’d built, and came out the other side stronger for it. That isn’t proof the larger bet pays off. But it’s reason enough to take the optimistic question seriously – and to keep building as though the answer is yes.

Watch or listen to the full conversations: Des Traynor on The Further, Faster Podcast and the AI Revolution Show.

Data scientist was once the sexiest job in tech. Now it’s the forward-deployed engineer. So naturally, I became both.

In 2012, Harvard Business Review gave data science that famous label. Fourteen years later, the role everyone’s talking about is the FDE. Palantir coined it, OpenAI has just launched a $4 billion deployment company around it, and The Pragmatic Engineer has covered the trend extensively. But there’s a role emerging alongside FDEs that I haven’t seen much written about.

I’m a Forward-Deployed Data Scientist at Fin (formerly Intercom), here’s what that looks like and why I believe it’s one of the most important jobs of the AI era.

The product team parallel

If you’ve worked in a product-led company, you’ll know the classic trio: a product manager, an engineer, and a data person. The PM and engineer build things. The data person helps the team understand what’s worth building, what’s happening in the product, where users are getting stuck, and what to experiment on next.

Without that analytical layer, you’re building on instinct and hoping for the best.

Forward-deployed teams work the same way. At Fin, we operate as small cross-functional units known as “swarms.” Each swarm is dedicated to a single customer engagement: a forward-deployed engineer configures Fin for the customer’s environment, a forward-deployed product manager manages the relationship and workstreams, and the forward-deployed data scientist (FDDS) owns the analytical layer. That means metrics definition, performance measurement, and the commercial story that ties it all together.

The FDE configures the product. The FDDS defines what success looks like and proves whether we got there.

Why the role exists

We’re two years into AI adoption and there’s a lot of conversation right now about whether companies are seeing ROI on their AI spend. You’ve probably heard the recently popularised phrase “tokenmaxxing” by now; companies maxing out AI usage without measuring whether it’s doing anything useful. Uber burned through its entire 2026 AI budget by April, while Amazon employees were spinning up AI agents to complete meaningless tasks just to inflate their usage stats. The backlash is real.

This is the world we work in, except on the other side of the table. Swarm engagements are high stakes, focused on our largest customers who have significant opportunities for automation. These deals can be worth seven figures, with CFOs, CEOs, and founders in the room. Nobody is signing off on that kind of spend because the demo looked good. They want to see that the AI actually works for their business, and delivers measurable value.

The FDDS role exists to do exactly that, and it works in two directions. First, we do the upfront analysis that sets Fin up to succeed: understanding the customer’s conversation mix, scoping what’s automatable, and making sure the engagement is focused on the right things. Second, we do the analysis that proves the value after Fin is live. That means tracking performance, identifying gaps, and building the commercial narrative that justifies the investment.

Without the first part, Fin might be configured for the wrong use cases and underperform. Without the second, even a great deployment won’t get past the exec sign-off. If you can’t show the value, the deal doesn’t close. Both are data problems.

What the work looks like

Understanding the customer’s world before touching the product

Every engagement starts with understanding the customer’s business: what are their goals, what do they care about, and how do we measure success? That shapes every subsequent decision.

We start with a taxonomy analysis. The customer shares a sample of their support conversations and real interactions between their customers and their support team, whether that’s human agents or an existing AI tool.

Our job is to make sense of it. What are the most common things their customers ask about? Which of those are simple and informational (someone looking for a help center article) and which require real complexity?

“I want to cancel my subscription” sounds straightforward, but it might require Fin to check what plan the customer is on, look up their billing history, apply a retention policy, and trigger a cancellation through an API. That’s a completely different challenge to “what’s your return policy?”

We break conversations down along a spectrum, from purely informational queries, through to ones requiring personalized data, all the way to those needing real actions on the customer’s behalf. We also map out the read and write operations needed to resolve each type. That gives the engineer a clear picture of what APIs and integrations to build, and it gives the PM a realistic scope for the engagement.

Without this process, targets get set in a vacuum. A customer might aim for 70% resolution rate without realising that a significant chunk of their conversations require API integrations and engineering work that hasn’t been scoped yet. The taxonomy turns an ambiguous target into something grounded: here’s what’s achievable today, here’s what needs to be built, and here’s the path to get there.

If we find that 50% of an ecommerce customer’s inbound volume is about order tracking, that becomes the focus of the proof of concept. There’s no point trying to boil the ocean in a two-week proof of concept. You find the highest-value, most achievable use case and prove it there first.

But the taxonomy doesn’t just define the POC. It also becomes the foundation of the customer’s automation roadmap. We can show them what Fin can resolve today and a clear path to an 80 or 90% automation rate. That roadmap turns a short engagement into a long-term picture of what’s possible.

Tracking performance in production

Once Fin is live, the role shifts to continuous performance analysis.

Resolution rate is often the headline metric: what percentage of conversations Fin handles end-to-end when it has the opportunity to answer. But the number alone doesn’t tell you much. You need to understand why Fin is struggling where it’s struggling. Is it a knowledge gap, like missing content in the help center? A complexity problem, where conversations need actions Fin hasn’t been set up to handle yet? A quality issue, where Fin is responding but customers aren’t satisfied?

We run this analysis throughout the engagement, flagging gaps and working with the team to close them. In one engagement, a taxonomy analysis mid-trial showed Fin was resolving 90%+ of informational queries but struggling with a specific category of action-based requests. That clarity let us focus optimization efforts on exactly the right conversations, and within weeks the overall resolution rate went from around 50% to over 65%.

We also look at quality metrics, recontact rates, and customer satisfaction to make sure we’re not just resolving conversations but resolving them well.

Proving the value

This is the part that sits furthest from traditional data science. The analysis doesn’t end at a dashboard but a contract decision.

At the close of an engagement, we present an evaluation readout: a full picture of what Fin achieved and what it means for the customer’s business. The challenge is that “value” looks different to every customer.

Some are purely focused on resolution performance. Others want a cost savings story, where Fin handled X thousand conversations that would otherwise have needed a human agent, at a saving of Y. One customer aimed to measurably improve retention by authorizing Fin to offer a discount to customers that are trying to cancel. That’s a very different success metric to everything else.

In one instance, we had a customer evaluating Fin alongside a competitor. On the surface, the competitor looked cheaper. But when we dug in, the comparison wasn’t like-for-like. The competitor charged per conversation, whether the AI performed or not. Fin charges per outcome, so you only pay when it completes the action you’ve configured it to execute. Once we built a total cost model that accounted for what the customer was really paying on the competitor’s side, the economics flipped and the customer chose Fin.

This type of analysis would be difficult for the sales team to produce independently. It requires a data scientist with commercial context and a deep understanding of performance metrics to spot discrepancies like these and explain why they matter.

Scaling what starts as bespoke

The most valuable thing about forward-deployed work is the patterns you see across customers.

My colleague Kevin O’Brien wrote about this in detail, but the short version is that we operate a loop: swarms generate signal from real customer problems, that signal gets encoded into internal tooling, and the best of that tooling eventually becomes part of the product.

The automation roadmap is a great example. What started as a one-off deliverable for a single engagement is now generated automatically, replacing hours of manual work.

Similarly, we can now easily generate performance decks. A custom-made narrative for one customer became a pipeline that created more than 450 customer-specific decks. It’s changed how our entire sales organization communicates the value of Fin.

Some tooling goes further still. Take the taxonomy analysis. It started as a manual exercise that took the better part of a day. We built it into Cockpit, our internal insights platform, so that any customer-facing team could run the same analysis without needing a data scientist in the room. And today, it’s accessible to customers directly in Fin.

This is how the work of forward-deployed data scientists compounds. Every bespoke solution makes the next engagement faster and smarter – and the best of it ends up in the hands of our customers.

Can we reclaim our title?

Fourteen years ago, Harvard Business Review called data science the sexiest job of the 21st century. Forward-deployed engineers are having their moment now. But deploying AI isn’t the same as proving it worked – and as businesses demand ROI on their deployments, someone needs to own the “did it actually work?” question. That’s a forward-deployed data scientist.

Today we’re announcing that Fin can now be used as a Service Agent on top of HubSpot and Freshworks, meaning you can use the world’s best Agent without migrating off your helpdesk.

Hubspot and Freshdesk customers can now:

• Get Fin live, integrated, and working seamlessly in less than an hour

• Delivering a 76% average resolution rate

• Across all customer channels (voice, email, chat, social and more)

• Resolving complex queries that require reading and writing to 3rd party systems

• With everything fully configurable to follow the unique policies of every individual business.

Our very visible commitment to being open

Today’s announcement is a deepening of our commitment to build an open platform that is customer first.

We’ve long known that businesses want flexibility in how they configure their customer-facing tech stack. Since the very beginning, we have built Fin as an open platform, with APIs, MCPs, CLI, and opening up access to Apex, our proprietary trained model that delivers best in class performance.

To make things easy for our customers, we have extensive public documentation of our product on our website, in our help center, and in our developer docs. We are the only Agent company in our space to do this, others hide most details behind sign-in screens, which we don’t believe is the right thing to do.

Open Agent platforms will win

We now believe our category has reached a stage where customers demand open platforms, that those who open up are more likely to win, and those who remain closed and protectionist will accelerate their demise.

We are operating in a fast changing world, and customers do not want to be locked into a single vendor or closed ecosystem. They want the ability to experiment, to swap things in and out, and move everything with ease, technically and commercially.

In an open world, the best product will win

In a world where businesses can easily swap vendors, the best product will win. We are happy to compete on that front, confident that Fin delivers the best customer experience and the highest performance.

If you are a Hubspot or Freshdesk customer, you can now have Fin integrated and live within an hour, without needing any help from us.

We’re here if you want us, but as part of our commitment to building an open platform, we’ve designed everything to be self-servable, and you can get started right here or watch a quick demo of how everything works below:

fin.ai/hubspot

fin.ai/freshdesk

For about 15 years, most websites have had the same little circle in the bottom-right corner. Sometimes blue, sometimes pulsing, mostly polite enough to stay out of the way. Click it and a panel slides up with a friendly greeting and an estimated wait time. On the other side, was your support team. We know a lot about this widget because we invented it, and the industry followed – down to the design.

But on most websites today, what’s on the other side of the circle isn’t a person, it’s an Agent, always on and ready to respond. Despite the backend evolving significantly, the look remains the same.

This is fine for support where customer behavior is established, but as Agents move into other parts of the customer lifecycle, they’ll need a different interface to encourage engagement.

Different roles deserve different front doors, and this became especially clear when we designed Fin for Sales.

AI-first, not AI-added

Almost every design assumption baked into the chat widget is about the human support team behind it.

The launcher icon is small and discreet to discourage overuse, as the reps have limited capacity. The Messenger’s UI manages expectations around response times, as teams are frequently overloaded or only available during set hours.

But when building for an Agent, especially one outside of support, none of these assumptions hold.

This means we have to approach designing differently. Instead of just taking what was and building on top of it, we look at things with fresh eyes and ask how the form factor must change when the experience is first and foremost agentic. The difference is going AI-first, not just AI-added.

Designing for agentic experiences beyond support

When we set out to design Fin for Sales, here’s what we considered:

Entry points set expectations. For many, a chat widget signals “click here when something’s broken,” thanks to more than a decade of that being its only use case. When a visitor is on a pricing page evaluating options, they may not think a widget is the best place to ask questions. The same goes for ecommerce.

Show, don’t tell. Text was always a placeholder. When you ask about pricing, the natural answer isn’t a paragraph describing a comparison – it’s a chart. When you ask where something lives, the answer is to walk you there. Cards, tables, voice, video, inline navigation – the modality should match the question. A pure-text thread will quickly become tiring.

Visibility comes with responsibility. A widget hidden in the corner has the luxury of being ignored. A composer that sits openly on a page does not. While we want to encourage interaction with the Agent, we don’t want it to be intrusive. Color, motion, copy, weight, and placement are all important. It needs to suit whatever’s around it – on every site, in every brand.

What we built

Holding to those properties led us somewhere different than another iteration on the widget. We called it the Spotlight Messenger – a floating bar at the bottom of the page, centered, and dismissable in one click. It’s the entry point used by Fin Customer Agent on fin.ai and customers like Fellow.

The bar shows Smart Suggestions: AI-generated questions written from the customer’s own site and knowledge base. It shows three questions at a time, rotating across sessions. A visitor can click one to start a conversation without typing. That’s a big change from a support widget’s usual ritual of: spot the icon, click, wait for the panel, scan the greeting, find the input, and type the first sentence.

We would have never designed this look in the past. Putting a live composer on every page of a customer’s website, making it visible by default, and surfacing questions a visitor can fire off instantly would drown a human team in five minutes. It only makes sense when the thing on the other side is instant, infinite, and never has a bad day. Spotlight is downstream of Fin, not the other way around.

The same principle that reshaped the entry point reshaped the answers. The right answer to a question often isn’t a sentence, so Fin decides how to respond based on what’s being asked, not just what to say. Ask how two plans compare and Fin lays them out as a table, with features, limits, and prices aligned so the differences are obvious at a glance. That extends to anything where structure beats prose – tiers, specs, availability.

Spotlight’s visual identity went through several iterations. We explored about a dozen aesthetic directions – glassmorphism, terminal-style monospace, custom typography systems, branded gradient washes – and rejected all of them. Distinctive AI styles tend to age fast; they look like the year they were designed. We wanted Spotlight to live on a customer’s page, not on top of it.

The treatment we landed on is deliberately quiet – a soft, translucent backdrop tint behind the composer that picks up the accent colour of the page it’s sitting on. The goal is to ensure it harmonizes with its surroundings, present but never competing. Fin’s own personality shows up most significantly in motion, small cues signal it’s thinking or streaming.

We also tested how it should interact with the surrounding webpage as users navigate. Customers asked us to close the messenger when a visitor scrolled the page, but it felt fiddly in testing – a small accidental scroll dismissed the whole conversation. Instead, we opted to close the chat window when the user clicks outside of it.

The impact

On fin.ai we ran an eight-week experiment on the new entry point against the old widget, and it produced a sharp lift in new conversations and engagement, enough to roll out to 100% of the site. Individual customers have seen larger swings – one early beta reported a 9x jump in engagement on Spotlight versus the widget Messenger they used to have.

Where this goes

We think websites are about to go through the most interesting visual change since the move to mobile. For a long time, a website was a brochure: read it, then go elsewhere to talk to a person. Then it became a brochure with a chat button: read it, and if you really need to, summon a person. The next thing is closer to a conversation that happens to have a brochure underneath.

The change is already impacting the language we use. Internally, we’ve started saying surfaces, entry points, and modalities.

Surfaces are where the Agent lives: a floating bar on a marketing page, a side panel inside a product, an embedded block on a product detail page.

Entry points are how it gets triggered: proactive, contextual, persistent, ambient.

Modalities are how it answers: text, table, voice, video, navigation.

The interesting design work for the next few years happens in the combinations – which surface, which trigger, which modality, for which job.

Spotlight is one answer for one of those combinations, the front door for a sales Agent on a relevant page. We’ll keep iterating on it. But overall, we believe the chat widget as a universal default is finished. The bulk of our efforts in the coming years will focus on defining the landscape that succeeds it.

Brian Boyle is a Principal Engineer and Daria Voronstova is a Principal Designer at Fin. They co-led the design and development of Spotlight Messenger, the entry point used by Fin Customer Agent on intercom.com, fin.ai, and customers like Fellow.

AI is breaking down the rigid boundaries between roles. PMs are taking on design and designers are running analysis. But what does that mean for specialists? I don’t think the answer is that specialists go away. I think part of their job changes from doing the work for everyone else to making sure everyone else, and every Agent, can do the work well.

Non-engineers will keep shipping code. Non-analysts will keep running analysis. We are not going backwards. Still, that work can’t be improvised against raw systems. To be most effective, this new way of working will require the people with deep knowledge to encode that expertise into tools, definitions, guardrails, and evals the whole company can use.

That’s the direction we’re taking with Ground Truth, the AI-first semantic layer we’re building at Fin. Ground Truth gives Agents approved metric definitions, SQL templates, and business context to query the warehouse and help teams throughout the business produce accurate analysis.

The problem we ran into

We rolled out Claude Code across the company a few months ago. To say it ripped through the place like wildfire is an understatement. Of the 1,400 people here, around 700 now query our warehouse on a typical day using Claude, up from 220 six months ago.

That unlocked a lot of speed. It also exposed a problem.

When we audited the analysis output, we found consistent errors from the Agent: wrong tables, wrong metric definitions, wrong inferences, and wrong answers. The worrying part was that every error we found had only been caught because a human happened to notice. We had no systematic approach for catching the errors no one was watching for.

During testing, a skill that had been scoped to a single customer was picked up on an org-wide query and returned an 80% automation rate as the number. The real number was 32%.

When we ran an internal eval across a sample of key metrics, resolution rate, one of our core business metrics, was accurate about 65-70% of the time. The Agent was often close, but close isn’t the bar if the output is going into comp, QBRs, or customer-facing analysis.

What we’re building

We’re building Ground Truth to solve this problem: an AI-first semantic layer for our data warehouse.

The core idea is simple. Instead of letting the Agent guess how to calculate resolution rate or ARR every time it is asked, we codify each definition once in a central place, written and reviewed by the domain experts who own that metric.

When someone asks a data question, the Agent searches the semantic layer first. It pulls the right definition, the right table, the right filters, the right SQL pattern, and the known traps before it touches the warehouse.

The Agent stops improvising from raw schema information and starts from the same definition an analyst would use.

How the definitions are structured

The layer is organised by business domain: Fin, helpdesk, finance, GTM, and so on.

Each domain contains two kinds of files.

The first is a _context.md file. Think of it as the five-minute briefing you’d get if you grabbed an analyst who knows that area of the warehouse and asked them what you need to know before you start querying. It maps the domain: which metrics exist, what their synonyms are, how they relate to each other, which entity filters to apply, and how to interpret common patterns.

The second kind is a metric file - one per metric. Each includes the description, formula, SQL templates at each grain, variants, and the gotchas that tend to trip people up.

For resolution rate, that includes things like:

• Never average daily rates. Always sum the numerator and denominator separately, then divide.

• Use the approved resolved conversation definition, not a status field that happens to look similar.

• Apply the right customer, paying, and channel filters before aggregating.

These files are the source of truth for Agent-facing metric definitions. They are version-controlled, owned by named domain experts, and reviewed like code.

How the Agent uses it at runtime

At runtime, the Agent does three things.

First, it searches the semantic layer by metric name, synonym, and related concept.

Second, if it finds a match, it injects the full metric definition into context: the correct table, filters, SQL template, grain, variants, and known failure modes.

Third, it generates and runs SQL from that approved definition rather than reconstructing the metric from scratch.

If there is no match, the Agent can still fall back to exploring the warehouse schema on its own. But that answer is treated with lower confidence.

More importantly, the miss is logged.

How the semantic layer keeps itself current

When the Agent cannot find a definition, the query and surrounding context are logged. A separate Agent then picks it up with the goal of creating a new metric definition or improving an existing one.

That Agent does three things.

First, it searches source tables that could plausibly answer the question. It ranks candidates by schema priority, with curated marts ahead of raw sources, and by how often each table appears in past queries on the same topic.

Second, it drafts the semantic metric file: description, formula, SQL template, variants, and gotchas inferred from related metrics and previous failure patterns.

Third, it routes the change to the right human reviewer. That might be the owner of the underlying table, the most active querier in that area, or the domain analyst.

The human then reviews, edits if required, and approves. The result is a PR into the semantic layer.

The point is that every question the system cannot answer well becomes a chance to improve the system. Accuracy compounds with usage.

Early results

We’re still early, but the results have been strong

• Accuracy on core metrics like resolution rate has moved from ~70% to 100%. How we ultimately evaluate the system is a topic in itself, which I’ll cover in a follow-up post on the eval system we built as part of Ground Truth.

• The median number of SQL queries the agent writes to land an answer dropped from six to two, because it isn’t exploring the warehouse from scratch.

• Time to answer is down about 90% for the same reason.

And, just as importantly, the system now has memory. When it fails, we can see where it failed, turn that failure into a reviewed definition, and make the next answer better.

The role shift

As more people across a company run their own analysis, analysts become part of the underlying infrastructure. The repeat questions, like “what’s our resolution rate this quarter” or “how has it trended,” get handled by the system.

Analysts spend their time on higher-leverage work: improving the system, adding guardrails to keep Agent-led analysis accurate and unbiased, and pushing into more complex forms of analysis.

I see specialists across the company are undergoing similar changes in responsibility.

Designers will not just design every experience. They’ll build and maintain systems that help others design better. Engineers will build and maintain platforms that make it safe for others to ship.

Analysts will not answer every data question. They will define what good analysis looks like, encode it into the tools, and measure whether the Agents are living up to it.

When competing in business, you have two choices. Try to win the game your competitors have chosen to play, or play a different game. In our category, which we call customer agents, our well-financed competitors have made generous use of fabricated demos, particularly when it comes to their voice technology. Voice is just extremely hard. And while we all know that the future of customer experiences will be agent-driven voice, we’re not there yet today. How many times have you called a business and spoken with a modern AI agent? The technology is not generally in the wild because it’s just not been ready. That changes today. And because we finally reached a level we’re proud of, we’re going to play a different game. No smoke or mirrors are required. We’re sharing a live demo of our tech for the world to see and it’s very, very good.

For those still reading let me give you some context on this new product and explain what makes it so special.

Yesterday we announced Apex Flash, our newest and fastest model yet, and one we built for the unique demands of low latency channels like voice. Today we’re announcing Fin Voice 2, a major upgrade to Fin Voice with over 20 new features, and our first product built on Apex Flash.

Here are three things you need to know:

First, thanks to Apex Flash, Fin Voice 2 is now the fastest, most natural Agent for phone, with higher resolution rates and customer satisfaction scores than ever before. Apex Flash is trained on millions of customer experience interactions, fine tuned for customer service, and can be configured to understand all your knowledge and follow all your policies. This means Apex Flash delivers higher resolution rates whilst being significantly faster, delivering the best of both worlds.

Fin Voice 2 is the fastest and most natural voice agent because it was built in a superior way. Most voice AI products are slow because they convert speech to text, send it to a general model, get a text answer, and then convert it back to speech. Fin Voice 2 was designed to work differently, separating the real time layer that handles speech processing, and the layer that generates answers. It is purpose built for the demands of customer service on voice.

Second, Fin Voice 2 can handle all your complex queries, for example taking actions in external systems, verifying callers’ identities, processing refunds, booking appointments, and so much more. Phone is a high stakes channel, and Fin can very naturally deal with customers in many different emotional states, adapting when their emotional state changes. Fin will clarify when needed, and confirm key details before taking action. Most of the time, Fin can resolve the query in full, and when it can’t, it seamlessly hands off to the human team, maintaining full customer context and history. Fin Voice 2 has multiple improvements to call quality, and can call proactively to follow up with people who have unresolved queries.

Third, Fin Voice 2 gives you total control, with the best tools in the industry to configure and manage how Fin behaves. We’ve shipped even more rich, detailed insights into areas like call behavior and quality, the most common topics of calls, and one-click recommendations to improve. As with everything in Fin, you can fully self-serve and then manage it all with ease, without requiring professional services. Other companies will only let you set up their voice agent under their supervision, meaning you need to contact them to change anything. With Fin, we give you everything you need.

If you’ve not yet tried the demo, go check it out. Or just wait a bit and you’ll likely end up speaking with it at your favorite brand soon.

AI and coding Agents make it much simpler, cheaper, and faster to build targeted internal solutions to go-to-market problems. Nowhere is that clearer than in sales. There, companies can gain an edge over competitors still relying on off-the-shelf software that is often bloated, expensive, and built for the last generation of SaaS.

When Nicolas Sharp, co-founder of Attio, spoke at our Fin for Sales event recently, he put words to something I’d already been seeing: a growing need for more technical roles across GTM, not just in sales, but across marketing and customer success too. He framed it through Jevons paradox. When steam engines became more efficient, coal use didn’t fall, it surged. His view is that as AI systems become more capable, demand for the people who can build, operate, and improve them will grow, not shrink.

That resonated with me because it’s close to what we’re building at Fin. We’ve set up a new engineering team dedicated to improving GTM effectiveness and efficiency. Its mission is to bring a commercial edge to the organization by applying the latest technology and techniques. It’s still early, but we’re already seeing results against our commercial targets, with adoption of these tools well beyond what we expected. As the team grows, some interesting technical and organizational dynamics are becoming clearer, and we’ve been leaning into them eagerly.

The power of the pair

We’ve had real success pairing an AI-forward generalist engineer with a domain expert who deeply understands the problem space and the organization. That has pushed me to think less in terms of “what can this team achieve” and more in terms of “what can one engineer achieve,” and increasingly, the answer is a lot more than before.

Taking this approach, we built an answer engine optimization (AEO) engine that dramatically increased Fin’s visibility in AI chatbot responses and grew bot-sourced revenue pipeline. We had similar results building an outbound engine, and now we’re applying the same model to optimize other sales workflows.

In conversations with peer companies, I’m seeing a broader pattern emerge. Engineers are working much more closely with frontline contributors and managers, and are embedded directly in the teams and workflows they’re trying to improve. Rather than building generic products, they’re creating bespoke solutions from the bottom up. This feels like one of the most important developments inside GTM right now.

Building bespoke

It’s easy to hear “build your own tools” and conclude that SaaS is dead. I don’t think that’s true, though I do think SaaS is being dramatically disrupted.

Foundational platforms like your CRM, payments infrastructure, and customer communication layer are systems of record. They’re deep, broad, and embedded in the ecosystem in ways that make them genuinely hard and unwise to rebuild. You should keep buying Salesforce. You should keep buying Stripe. You should keep buying Fin. These products encode years of hard-won capability that no small internal team should be trying to replicate.

What’s different now is the layer above them: the orchestration layer. It’s the customization that connects these systems, fills the gaps between them, and makes the job easier for the rep, the marketer, or the CSM. That used to require a vendor relationship, a six-month implementation, and a dedicated admin. Now it requires a very strong data and systems foundation, a GTM engineer, and a couple of weeks focused on a clear problem to solve.

We built a cockpit to run outbound across Outreach, our data warehouse, and our own internal infrastructure. It made it possible to turn new ideas into pipeline in a way the team could actually run. Without it, the process was extremely manual and error prone. It has already generated $1.2 million in sales pipeline, and we’re tracking toward $5 million by year-end, without any migration or the need to learn a new tool. We also built another internal tool in roughly three weeks that replaced a vendor product we had been evaluating. We found the gaps, shipped something better, and tuned it to our exact needs.

We’re seeing the same logic at other companies too. Will Jones, a GTM operations associate at Attio, is in a role that barely existed a few years ago.

What stuck with me most was how he went on to describe his job. It was less about doing the work directly, and more about directing Agents to do it well. That feels like the clearest articulation of the GTM engineer role I’ve heard so far. We’re no longer trying to find the perfect point solution in the market. We’re building the connective tissue between our foundational platforms ourselves, and for that class of problem, the advantage of building internally has been clear.

We’re seeing the same idea reshape adjacent technical functions too, though that’s a broader story in its own right.

What I’m nervous about

I’d be dishonest if I presented this as a clean story. It isn’t.

One question I keep coming back to is whether the cockpits our teams are building are durable, or whether they’re sophisticated throwaways. I genuinely don’t know whether the orchestration layer we’re investing in now becomes a lasting foundation or scaffolding around something that hasn’t been invented yet.

I’m also thinking about the staffing implications. What happens when the tools get good enough that everyone in GTM is self-serving more of their own analysis, workflows, and automation? That may be a success story for the company, but it fundamentally reshapes what my team needs to look like, how big it is, and what roles I hire for. It will probably get smaller, and the roles will become less engineering-heavy.

Build for the flex, not the forecast

My biggest concern isn’t any single one of these questions. It’s that, for all the ambiguity, excitement, and energy around what’s changing, the core principles of building and delivering software still apply.

The creative trial and error required for this transformation means you have to throw away half of what you try, maybe more. At the same time, there’s a sales team hungry for pipeline this quarter and for tools that work right now. I don’t know what the right balance looks like between exploring the next evolution of GTM software and delivering on near-term objectives that fuel the business.

What Nicolas said resonated with me here. He said Attio doesn’t try to perfectly predict what the future will look like in 18 months and then build an organization around that vision. Instead, they try to build one that’s so agile it won’t matter if their vision is 50% wrong. Build for the flex, not the forecast.

I think that’s right. I also think it’s hard to put into practice when near-term commercial pressure doesn’t go away.

Where this leaves us

GTM engineering isn’t a hypothesis anymore. The pattern is clear: embed a builder next to a domain expert, give them a fast feedback loop, and get out of the way.

Exactly where this leads is still unclear. What we’re building now may prove durable, or it may simply be a bridge to whatever comes next. Demand for GTM engineers may grow as Nicolas predicts, or the tools may get good enough that the role evolves into something else entirely.

What I do know is that the old model of buying software, plugging it in, and trying to make it all hang together is no longer the only option for the orchestration and customization that makes GTM function. The foundational platforms still matter. But the layer that connects them, shapes them around the workflow, and makes them coherent for the people doing the job is increasingly where real advantage gets built. That’s the work of GTM engineers, and it’s why this role is becoming real.

It was very much a live demo that let us walk through some of the key components of our system.

If you missed it, you can watch the recording here.

We loosely committed to answering all questions in some way, so here it is. Feel free to continue the conversation in the comments, or reach out if you have any questions about Fin, Intercom, our AI models, or anything we discussed on the webinar.

Thanks to my colleagues, Brian and Kesha, and to everyone who showed up and asked great questions. Special thanks to our guest, Claire Vo, for joining us and having Brian on her “How I AI” podcast recently.

We hope to do more events like this very soon!

Connect with us here:

X: Darragh | Brian | Kesha | Claire

LinkedIn: Darragh | Brian | Kesha | Claire

Org structure, team, and staffing

What size team works only on the agentic harnesses and how do you handle roles and responsibility? (Asked by AS, similar by EL)

How do you staff development of Fin’s “agentic coding system” (skills, Shrek, agent observability, etc)? How much is dedicated platform team(s) vs. grass roots contributions from product engineers. (Asked by NS)

Currently, a very senior core team of three IC engineers (Principal/Senior Principal), two Senior+ engineers, a Staff TPM, and a VP eng. We’d had a couple of other engineers tour in and out based on opportunity/capacity. They’re pretty autonomous on deciding what to work on. I meet with them weekly to stay close and provide input. This team has existed for about six months now. We wish we’d started sooner, but it always feels very hard to pull some of your best off other critical projects. The payback has been immense though. In addition to the team, it’s explicitly everybody’s job to contribute to the system, help encode context or skills unique to them or their team, and act on feedback loops to drive improvements in the system.

Have you restructured your org to adapt to these new processes? (Asked by YG)

Essentially no, not dramatically. We’ve formed our team-2x, and there are many local experiments with different/leaner team structures, but no dramatic re-orgs. We do, however, intend to push harder in this direction, as often making changes (for example, shrinking teams) forces valuable lessons around what works better, what breaks, how to adapt, etc. – things you can be insulated from in old world structures.

How will roles in engineering change? (Asked by KG)

At a high level, we see many adjacent roles blurring together: your best engineers already have great product judgment, and will take on more of the PM role, creating space for the PM on higher impact work, and reducing coordination overhead internally. Directionally, what is a team today should be a single person on some near-ish timeframe. In that world, engineers need to be well equipped to strongly own areas of the product/system, set great direction, and execute with high levels of independence.

Productivity metrics and impact measurement

How do you measure the actual impact beyond throughput? Does 3x increase in PR correlate to business outcomes and product value that’s relevant for your organization? (Asked by AK)

The real prize is our ability to solve customer problems, and bring our vision to life dramatically faster than before. This shows up for example in our rate of significant product launches, or in our defect backlogs burning down. You should expect revenue to be a lagging indicator; we see revenue growth to continue accelerating.

More directly to your point, it was and still is true that the number of PRs doesn't directly translate into business impact. However, it is a required pre-condition to be able to ship at least something: required, but not sufficient. At this stage, we are heavily investing in ensuring all our operations are streamlined to make sure if we want to ship something, it is agentic-first, fast, safe, with minimal human intervention.

Do your engineers know how you're measuring productivity? How do you prevent them from gaming the system or account for gamification? (Asked by AK)

Claire said this best: gamification is a culture problem, not a measurement problem. That’s the broad strokes answer. We’ve a high trust, high transparency culture with people who care about their work. Having said that, we are all human and the pressure to hit/increase this metric can be a source of stress. When we introduced tools like Buzz, which enable the offloading of work to cloud agents, initial versions didn’t attribute the work to the person who initiated it, and there was reluctance to use this system out of fear the impact wouldn’t be attributed to the person. We fixed the attribution to prevent resistance to using this power tool. Ultimately, anything egregious would become obvious quickly, energy spent trying to game is far better spent trying to win together.

What business metrics have you tracked and noticed changes in as a result of AI-assisted development? I noticed DORA-like metrics earlier – I am interested more in business metrics/OKRs etc. (Asked by SC)

Your metrics showcase eng metrics, what about customer delight or revenue? Were you able to show causal changes? (Asked by AL)

Best ones we have are product release throughput and time from idea to release, tier 0 marketing releases, or our recent Intercom 2 launch. Like I said earlier, revenue growth is accelerating for us. Another example is that we are able to say yes much more frequently, which translates into deals closing that would have been blocked, or accounts churning because we can’t support their evolving needs.

What was the target metric you started with? PR count? (Asked by AL)

Yes, merged PRs, but specifically total merged PRs divided by total number of people in the entire R&D org. This is about measuring the throughput of the entire system, not just the per engineer output. Total throughput has increased further as we’ve been growing our team too.

Have you also started measuring the change in PR size? Or ways to keep PR sizes low/discrete? (Ask by DK)

A: Yes, some details here. TL;DR; size has increased a little alongside volume of PRs. Intuitively, agents great at writing code and tests tend to get more tests as a result. Our AI approval system provides natural incentive to keep PRs small and focused, as they’re more likely to be AI approved, which enables faster flow through the system. Small focused PRs are also inherently safer, so that’s win, win. It’s early days, but we expect this to apply downward pressure on PR sizes.

Did the limit of 150 lines cause the increase in PRs per dev? (Asked by RM)

As mentioned above, there is a healthy incentive towards smaller PRs that are more likely to be AI approved and flow through more quickly. Smaller independent changes tend to be safer too.

PR workflow, review fatigue, and auto-approval

In Darragh’s article on X, you mentioned about 20% of your PRs are created and reviewed using AI. How do you maintain cognitive control, and what happens when things go wrong on these fully agent-driven PRs? (Asked by MK)

For human-initiated PRs, authors shouldn’t be blindly shipping and hoping AI allows it through. Engineers still need to have an opinion and take responsibility for the shape of the solution. In addition, our system encodes opinions about how we should solve problems, patterns to follow, etc. Our review system helps enforce this. When things truly go wrong and, for example, we have an incident, we do a deep postmortem and figure out the right corrective measures.

Curious to know how you deal/dealt with PR review fatigue. (Asked by NC)

AI review and approval is a big part of this, as well as humans having a way to encode the perspective they’d typically bring to a review into repeatable instructions/guidance for the review and approval systems.

How do you gate “Eligible for Auto Approval”? (Asked by RG)

Multiple inputs to this: PR size is one heuristic, other signals that indicate the level of risk/blast radius etc. More details here.

Do you have a classification / risk rating for PRs so you can auto-approve lower risk PRs? Do you have different agents running for this? What do your scheduled processes look like? (Asked by GF)

Yes, sub-agents with discrete tasks/responsibilities, all triggered on PR submission. Auto approval focuses most on low risk changes.

If the PR is not auto-approved, what are the next steps that take place in an agentic flow? (Asked by NF)

Today, it passes to the typical human approval flows, with some automation to notify the right people/teams in Slack.

Is everybody allowed to edit the reviewer prompts? (Asked by MB)

Yes, but this requires peer approval. In practice, owners/experts in particular parts of the system own/take responsibility for the review/approval requirements for their area of expertise.

Is it only one reviewer, or have you split it into a few different agents in parallel? (Asked by MB)

Many specialized review agents in parallel, and again, owners/experts in particular parts of the system can guide additional ways that their code is reviewed.

Skills: Creation, distribution, evals, governance

How do you distribute your skills? (Asked by IK)

A hierarchy of Claude Code plugins. Initially, we did that via a private marketplace, relying on Claude Code to auto update. We found the auto update wasn’t as reliable as we’d hoped, so we forced this using our IT MDM systems to ensure everyone has the latest version of master ~ at all times.

What skills and MCPs have you got? (Asked by GF)

LOTS. The intention is that the agent has access to all the tools a human engineer would, and that they work out of the box. Quite a bit of care and attention have gone in to achieve this. Of course, auth is required and some hooks that help make sure you resolve auth issues, rather than the agent deciding to not use the ideal tools available. We organize skills in hierarchy: our base plugin holds skills core to every person’s job, then separate plugins, many disabled by default for things like different languages/frameworks, and team/role/product area specific skills. We have hundreds of skills in total, but we also have processes for gardening and removing skills that don’t get traction, and promoting/improving skills that do.

Are the skills you’ve placed in the plugins actually reusable across multiple projects? How do you avoid them becoming too generic but still usable across several projects? (Asked by EL)

For starters, we’ve largely a monolithic architecture, so lots of general reuse. We don’t really have this problem, but do also leverage, for example, team-specific plugins where the goal isn’t general purpose. If we need to, we can promote skills that end up being generally useful from team to shared plugins.

I love the idea of constructing many small skills designed for individual parts of processes. Did you guys develop a methodology for how skills should slot in with each other? And how do you determine the architecture for how these skills interact as a whole? (Asked by JG)

Basically, not really. Emergent for now, we instrument all skills automatically so that we can spot trends in usage and effectiveness, and investigate how skills interact through session analysis.

How do you decide on narrow skills vs skills with many references, for example, a “query-snowflake” skill with many references for how to query certain table schemas where the skill.md progressively discloses which references to load, vs. individual skill workflows like “gong-call-insights” that tells Claude how to use Snowflake to query? We made a marketplace but are finding it hard to find the balance. (Asked by RC)

Agree there is a bit of exploration to find the balance. In this specific example, we’ve gotten a lot of general purpose utility by giving Claude select star and Snowflake access and that being enough to serve open ended analysis. However, it runs the risk of being good enough to be dangerous by creating a bunch of impressive but incorrect output. So alongside this, there are layers of guidance or skills that put guardrails or manners on how it will do certain things, make sure it understands some of the nuance of how data is captured, how to accurately report on ARR, for example, or more specific, like your example, task specific skills like analysing Gong calls.

How do you direct the agent to preload skills? Or does that just happen? (Asked by RC)

We use Claude Code and it puts a system reminder about every available skill into the system prompt. Unfortunately, that’s unreliable, so we use deterministic Claude Code hooks (regular scripts) that inject the context into the session based on certain conditions. For example, we have create-pr skill and we really want Claude to use it every time it creates a PR. To ensure that, we monitor Claude attempts to create a PR using gh tool, intercept it with a hook, and, if create-pr skill wasn’t loaded in the session yet, inject a prompt for Claude to load that skill.

Do you have evals that you run against the skills? How have you curated examples and set that up if so? (Asked by MK)

We have a set of evals in the same repository along with the skills. Basically like automated tests. As skills are used and new edge-cases come up, we habitually update evals with more test cases. For our PR review agent, Shrek, we do batch evaluations – we have a human-curated set of historical PRs with confirmed issues. Any time we change the agent or the prompts, we re-run evals to confirm its performance (alignment with ground truth) hasn’t degraded. Building and curating a dataset is by far the hardest and most time consuming part. As we are not experts in every repository we have to engage with experts in corresponding areas and create incentives for them to invest their time. Spending less time on reviewing PRs manually is a good incentive for them to contribute.

What’s your skill governance workflow? How are they authored, validated, reviewed? The same as code? (Asked by SS)

Yes, same as code. Then rules for how/when skills get promoted higher up the hierarchy. For example, 50 weekly active for a skill to be eligible to promote to a base plugin.

How do you avoid context explosion with so many skills? (Asked by MR)

Splitting into the hierarchy, and most plugins disabled by default, for example, team specific ones.

Agents and Shrek: Infrastructure

Can you elaborate a bit more on the infrastructure behind Shrek? Where is it running and what are the guardrails instrumented on the runtime platform? (Asked by LdAS)

Shrek is running on AWS infrastructure in isolated, short-lived containers, triggered by GitHub webhooks. Each container has read access to the codebase and the code changes diff, as well model providers’ API access (it uses both Claude and Codex). Shrek reviews are guardrailed using strict safety criteria per code repository that define what areas of the code are safe to auto-approve (if the review verdict is positive) or require an engineer in the loop.

What is Shrek built on? Is this home-rolled based just on a model, or based on something else? (Asked by IK)

Shrek is a Python app built on top of Claude SDK and the Codex APIs, with custom prompts and curated evaluation criteria for every type of review it needs to do (from basic things like PR descriptions, to company and team-specific code review and safety rules, to general logic bug finding investigations).

Which tools do you use to construct Shrek? (Asked by VD)

As above, in conjunction with a custom framework to run statistical analysis of the impact that individual prompts (and changes to those prompts) have on correctly judging a code change. Engineers use the framework to curate a dataset of PRs of various types with expected review outcomes and the framework runs simulated tests, producing statistical significance results on whether the prompt changes move the needle in terms of recall and precision.

To what extent do you use cloud environments where an engineer can start working immediately? (Asked by NG)

Engineers work on local development environments, but we invest a lot to make those work seamlessly/smoothly, have agents to help resolve issues, have the dev env in CI to automatically catch regressions, etc.

Do you have a unified solution for agent observability? How much of the agent evaluation and observability stack is in-house? Can you recommend a solution that a small company can easily adopt? (Asked by NG)

Yes, some hooks into Claude Code platform (part of what we like about Claude Code here is the extensibility,) and OTEL outputs into honeycomb.

Do you have autonomous agents that propose pull requests or any sort of changes without being explicitly asked (e.g. triggered via cron schedule, Claude routine)? (Asked by NG)

Yes, for example, our Buzz system can be triggered on schedule or by webhook.

How do you store product requirements / intents that are typically not captured in the codebase, so they can be accessed by agents? (Asked by NG)

Our agents can access docs just like humans and sometimes that’s the answer here, but there is also some discipline around capturing important context like this in the codebase documentation, and in git history.

Are you keeping your SDLC flows autonomous by creating tools/apps where engineers just interact to provide agency, or is it done in remote setups with all skills shared across the org? (Asked by AM)

A combination of local developer environments and cloud agents, skills are shared across the org (via plugin system, auto updating.) Shared skills was one of the big unlocks for us, gains compound vs. hide in silos.

Tool and model choice

Do you limit tooling? E.g. only Claude or only GPT models? If so, why or why not? (Asked by AVH)

We used to be open here, and permit every engineer/team to choose their own tools, but then got opinionated and doubled down on a single primary tool (Claude Code), which helped ensure gains compounded across the whole team, and we weren’t wasting effort trying to solve for multi-system compatibility with our skills/guidance, which can pull you to lowest common denominator and/or give you inconsistent impact across tools. So Claude Code, with bias towards Opus, and then Codex in some of our autonomous parts of the system (e.g. approval/review).

Do you only use Claude Code directly or also use non-Anthropic models? Anyone also use other tools like Cursor? Do you have Bedrock-flavour hosted models with guardrails from AWS? (Asked by SS)

We mostly use Claude Code models. We use OPEX GPT models a bit in our autonomous systems. We’re exploring Bedrock-hosted models primarily for availability reasons. Some people use Cursor, or historically did, but most of that has shifted to just vanilla vs code now. Increasingly, people aren’t in their IDEs nearly as much, if at all.

Are you migrating to GPT Codex, or introducing it in parallel to Claude? (Asked by RS)

We’re introducing it in parallel to Claude and concretely right now in our review and approval systems.

Aren’t you worried to be locked into Claude Code? (Asked by NF)

Not really worried about lock in. We love the trajectory Claude Code is on, but we also expect that other tools and systems will converge to support the platform surface area that Claude has exposed.

Are you using Cloud Agent / Dev Agents, and does the 3x number include those PRs? (Asked by KF)

We’ve started ramping up our use of cloud agents, and while it’s a small contribution right now to those PR numbers, I expect it to increase significantly as people, for example, schedule or trigger agents on queues of work.

4.6 – is that for cost? (Asked by IK)

While we don’t have any token budgets or hard restrictions on models we are constantly experimenting with different models. Cost-effectiveness is a hot topic at the moment as our usage continues to grow. Some tasks don’t require the most intelligent model assuming the prompts are in good shape. Moreover, switching to a newer model doesn’t guarantee better results. Prompts are tuned to a specific version and even switching within the same family (Opus 4.6 → Opus 4.7) might result in a degradation. We learnt this lesson while building Fin – when you ruthlessly optimize for quality it takes a lot of prompt-engineering to update the model, and it’s often the case that newer models are worse for specific tasks. With Fin, we see far more of the gains come from the architecture and system around the models, than model version changes themselves.

Security, PII, and compliance

How did you navigate DPIA or workplace monitoring laws for the prompt monitoring (as seen in the “How I AI” podcast)? (Asked by RM)

People can opt-out from session transcripts on a per-session or permanent basis, and we pseudo-anonymize session transcripts. Data collection is restricted to Claude Code, and we are not collecting personal Claude Chat sessions. All session data collected is stored in AWS with least-privilege access controls applied.

Do you conform to ISO27001 or ISO9001? How does that affect the agentic workflows and guardrails? (Asked by RM)

How does allowing an agent to approve a PR affect SOC2 compliance? (Asked by PB)

We are compliant with SOC 2, HIPAA, ISO 27001, ISO 42001, and many more. Specifically with our PR review agent, we engaged with auditors long before we even had the agent in place to ensure we have a path forward agreed. Our very first iteration was an agent based on deterministic rules approving changes to documentation/tests/etc – low risk changes. You can read more in this blog post.

Do you have to deal with PII issues when giving Claude access to certain systems, like Datadog, or have you gone through the obfuscation process everywhere? (Asked by IK)

Yes. We are lucky in a way as we already have mutual agreements with all major labs in place since we are using their LLM models inside Fin and trusting them to handle our most precious data: customer and end-user data. This enables us to feed telemetry (logs/metrics/traces) into the Claude session.

Do you have a policy for developers on what they can grant Claude access to? And do you audit that to prevent unintended access? (Asked by IK)

Yes. We have strict controls over allowed Claude marketplaces, MCPs, plugins, shell tooling, etc. There’s a massive attack vector inherent to this new developing area and we treat it accordingly. At the same time we aim for Claude to be able to do/access anything an engineer would. For example, we expose read-only access to our production infrastructure, data warehouse, etc., honoring a given engineer’s existing roles and permissions.

How are you managing access and provisioning tokens for devs and their agents? (Asked by AC)

All engineers/employees have access to Claude Code & Cowork, so far unlimited tokens.

What kind of anonymization do you do on sessions for the session analysis? (Asked by RC)

We have a best-effort obfuscation in place but of course it can’t fully anonymize the session – stripping all the data out if it would render it useless for quality analysis. We are transparent about sessions being stored (with opt-out available) and we treat sessions data in the same way we treat our customers data – there could be some PII in sessions afterall (e.g. logs). For example, we expire session data after 14 days to ensure we are compliant with GDPR data deletion requirements.

Observability, telemetry, and evals

What does your telemetry setup look like? How much visibility do you have into which skills and plugins are being used? How effective they are? (Asked by CT)

We started with the out-of-the-box OTEL monitoring that is built into Claude Code. We use Honeycomb for collecting telemetry, and starting instrumenting other flows as needed. We track telemetry events for every Claude Code Hook event, and we have significant instrumentation for SessionStart that gives us insights into any Claude Code or Plugin misconfiguration.

Have you experienced model drift, and how do you manage that? What about updates to system prompts? Do you execute a similar level of eval to any changes as you do with model changes? (Asked by GF)

We have a range of agents/prompts/skills with different levels of eval discipline depending on the use-case criticality. So far we haven’t experienced model drift in our internal dev tooling. For high-stakes agents like Shrek, virtually any change goes through evals first: changes to the agent code, system prompts, repo-specific code review skills, safety criteria, etc. We treat any change as a hypothesis that needs validation.

SDLC, codebases, and refactoring

Tell us about your SDLC? Are eng moving up front to problem discovery/framing/speccing to lessen the coordination need with PM? (Asked by AL)

Yes! We see that general trend. PM/Design incredibly focused on higher ambiguity and closer to customer and GTM.

Where does refactoring fit in? “Make the change easy, then make the easy change.” (Asked by GC)

Part of the system is encoding preferred patterns/architecture, so low level refactoring can be achieved that way. Higher level system refactoring guided by an engineer (typically in response to running into some limitation that we need to evolve our design around).

Multi-part: (1) How are you making your codebases AI-readable, especially at your scale of millions of lines across Ruby, TS etc.— have you seen the limitations of grep yet? (2) Have you found particular codebases have been harder to bridge to being agentic-first development? (3) How have you been measuring whether a codebase change actually improves agent accuracy? (Asked by OT)

(1) Honestly, not doing much here, some in directory overview documentation, some top level repo documentation to help explain how things are laid out. (2) Yes, a good example of this has been seeing agents/models far more effective writing high quality react code vs ember code. (3) Using a system of evals.

Do you have a monorepo / multiple applications which all share structure so that skills translate across projects? (Asked by RM)

We have a small number of very large repos - e.g. our rails app, a front end application repo, a python repo where much of our machine learning lives etc. We also have a general philosophy of making boring technology decisions, so there is a lot of consistency across all our code bases. So much of the skills that are focused on those primary repos become useful beyond, e.g. we have other rails apps, react apps.

Cost, token spend, and ROI

Were you able to get to an ROI calculation per agent (tokens + cloud costs vs. value)? (Asked by AC)

Yes, kind of. So far, we look at fully loaded cost per PR, and as token spend goes up, and PR throughput follows, fully loaded cost drops dramatically.

How do you manage spend of your tokens across the org? Did you consider leveraging open source tooling? (Asked by AK)

Right now we monitor, but don’t limit. As spend has ramped up we spot systematic improvements that yield cost savings. For example, we realized playwright was defaulting to screenshots that were needlessly high resolution, incurring unnecessary token usage, optimising this saved a projected $300k annual costs without any loss in performance. We aren’t using any specific open source tool here.

How much of developer AI spend ends up being spent on AI skill development? (Asked by RG)

We would have to check more deeply, but by ~PR count it’s ~5%.

API usage billing here – what about the bundled team costs or the enterprise plans? Huge discounts there. Not material? (Asked by IK)

At our scale, API based billing was the only path available to us, some discounts at scale, but not massive. Costs definitely require oversight in the medium term.

Roles, juniors, design, and cognitive load

How do you see the effectiveness of different seniorities when using AI tools? How are juniors growing now that there’s a push to outsource all coding to Claude? (Asked by AK)

The general trend is that some of our highest judgement most experienced people are seeing the highest leverage, however some junior people bring wonderful levels of curiosity, and hunger to learn, and near unbounded levels of energy, and much less hard wiring to the old way and are really thriving. I think we’ll bias towards AI-pilled people with experience who can likely own discrete areas of the product/system, but I’m rooting for the juniors too.

How has this transition affected design? Has design Xx’d output also? (Asked by DB)

Yes, designers are shipping, designers are contributing to the system, mindset of a designer, and generalisable to everyone, is for the things I do, that are often repetitious - is there a way I can encode that into the system so that it can be automated, and similar thinking for places where historically work would block on their input. Trend is they work on higher ambiguity work, and are perhaps pulled in async on other things. Easier/faster to build real prototypes/explore multiple directions in parallel. Now it is expected that designers can/do fix papercuts or small bugs, rather than cut issues, or badger their engineer peers to do it.

Have you found any issues with cognitive debt as Addy Osmani defined it: comprehension debt is the growing gap between how much code exists in your system and how much of it any human being genuinely understands. When we wrote code by hand we would find we couldn’t remember writing it 3 months later; now with AI writing the code we really don’t form a lot of memory about the code and over time have a widening gap of our understanding of the codebase. (Asked by RC)

We think about it conceptually, but haven’t seen major evidence of it being a problem for us. One counter balance to it being hard to keep up with all the change, is that it’s easier than ever to absorb unfamiliar context just in time, and our systems also encourage richer context being captured in our code base and in our git history that adds richness to the context that’s available at our fingertips.

Are you struggling with way more context switching? (Multiple projects/things to think deeply about, etc.) If not, how? We are. 😂 (Asked by L M-C)

Yes! We are all managers now, multi-tasking is often the default, it can be exhausting, we need to still be deliberate at taking focus time for deeper thought. AI does buy more time for us, frees us up from so much of the toil - our approach is that directionally that our agents should be able to do all aspects of the job a senior engineer does, and most obvious places to start are taking the least interesting work off our plates so we can focus our precious brain cycles on the hardest/highest leverage ones.

Culture, training, mindset, and cautionary tales

You mentioned about “pulling up the floor” so that all your staff embrace AI into their BAU. What cultural, training and awareness practices have you implemented (that are beginning to show returns) that have staff both engage and utilise AI – especially staff outside of the classic technical dev roles? (Asked by BMcA)

Couple of things. Early on we wanted to make sure everyone was having a real ‘aha’ moment with respect to how powerful AI tools, for us specifically Claude Code, can be. We had an internal competitive hackathon where people aimed to build a very simplified version of Fin, and compete against some secret LLM judges for the highest performance. Everyone, no matter what their role, had to participate. It got us all tackling problems harder than most of us would have believed we could tackle in a very compressed time window, and exposed us all to how powerful the tools are. At a lower level, yes, plenty of training, plenty of show and tell, people who’ve done interesting stuff demo’ing or documenting it, to inspire others, things like no-IDE weeks, to bootstrap people into an agent first mindset.

What about your situation before you engaged in this effort — systems, processes, people – was crucial to enabling it? (Asked by CS)

Culture that was focused on shipping, and systems/processes optimized around that (e.g. CI/CD).

How hard was it to let code quality slide at the beginning and what was the mindset you needed to have there? I guess that’s the scariest thing for teams that are high performing with high quality bars. Was there any time you weren’t sure you were on the right path? And what was the turning point? (Asked by SA)

Honestly, our take at the time was that much of the measurable drop in quality was acceptable as it was often within reasonable encapsulation boundaries - i.e. a class that’s internally a little messier or more verbose than we might like, or tests that are less precise/targeted than we’d like, but overall working as intended. We also believed in the eventual exit velocity, that as we improved the guidance and context to our agents we could steer them towards higher levels of quality and adherence to our preferred approaches, and that in doing so we could also target and improve legacy areas of lower quality in scalable and autonomous ways.

What’s the biggest mistake / cautionary tale that we should avoid? 🙂 (Asked by DW)

Dragging our heels, not pushing hard enough, being slow to invest in a team to own the holistic system, being slow to pick an opinionated approach, being slow to make it clear how our jobs are/need to change, and the evolving expectations on us all, under forecasting token spend (even though we all thought we were adding generous padding) and now dealing with all the reactive financial planning work.

Tell us about a failure of applying AI in your organization. What did you learn? (Asked by AK)

I think resolving open-ended investigations such as in incident response quickly and accurately is still WIP for us. More often than not, even with a good deal of context about our environment, our agents still confidently identify plausible suggestions that are red herrings. More work to do here! More meta but when we started the effort, our approach was to be maximally permissive, supporting everybody to try any/all tooling. We weren’t opinionated enough early enough, there were gains but mostly in isolation, individuals figuring out personal leverage but rarely compounding across the whole team. It took us too long to centralize on a platform. after choosing claude code, it became immediately obvious that the impact was in improving that platform, and seeing every gain automatically compound across the whole organization.

What are the 1 or 2 things that teams should focus on to get 80% of the value? Fin has a massive R&D team so I am curious about how smaller companies can learn from you. (Asked by MK)

Decide what you want to see, measure it, dedicate appropriate capacity to it, approach it systematically, smaller teams, or teams starting now likely have better off the shelf options for some parts of the system, tools are improving at a rapid rate, so maybe especially for smaller teams they can catch up quickly.

You must have had some scary stuff or near misses happen during the adoption and learning process — tell us! (Asked by SS)

Danny tweeted about this war-story, our biggest WTF so far was when an agent got quite confused, and ended up pushing hundreds of thousands of changes to our Rails monolith, which initially slowed CI down substantially, which escalated to completely breaking our ability to use the repo at all. The good folks at Github ended up having to do surgery on our repo so that we could get back shipping. Agents really will try anything to work around blockers, and sometimes get things terribly wrong.

Another type of nearmiss is in how easily Claude Code will innocently exfiltrate internal info - we realised this early and locked it down - e.g. private gists with problem spec md files - you need to understand and support the kind of workflows your team needs and make sure the path of least resistance isn’t a bad path.

Beyond engineering: Bottlenecks and other teams

Can you also share details how the Sales, CS, and Marketing teams are utilizing AI? Are they using the same infra you built? Do they serve themselves? (Asked by AC)

We built something called “Claude4Data,” built on same CC system, more non-engineers than engineers active Claude Code users. Expanding approach to Claude for sales, general approach is role specific skills and agents that share same data/tool access as human team members, skills that automate common tasks for that team. Also similar to R&D building our the general systems around these teams, e.g. automatic review/feedback on customer interactions, just in time enablement etc. Expect us to share more in the year ahead.

When R&D output 3x’s, the bottleneck tends to move – generally out of engineering and into decision-making, PM throughput, or the narrative layer for the rest of the business. Where did it land for you, and what did the communication design have to become to absorb and benefit from this 3x increase? (Asked by IG)

Claire answered this excellently. Check out what she had to say in the recording.

What’s next?

Why isn’t this a product yet? 😉 (Asked by CS)

It would be a fun one to productize, but we’ve got to focus! I expect there are / will be great products in this space and that e.g. Claude Code is already eating into some of the areas we had to build (e.g. session analysis like Brian talked about). I would add a cautionary note here that the things we are doing are a direct continuation of the culture we’ve been building for over a decade, it’s hard to productize culture, and some of these systems will be very opinionated/over-fitted to our culture. Even with off the shelf tools, you need a very strong internal push, business goals alignment, and willingness to take risks to make progress here.

That’s it! Thanks for reading this far. Thanks for all the questions, and your time/attention on the webinar. If there’s demand we’ll likely do more of these. Always happy to talk 1-1 with our customers who are looking for advice or insight into how to do similar things within their organizations.

When a frontier lab releases a new model, you might assume it’s gotten better at everything. Reasoning, coding, Ancient Greek, customer service, all enhanced in one fell swoop. And as this model moves horizontally, mastering disciplines, it will gobble up all the AI products built atop it.

But we’ve seen evidence that isn’t true.

Before we launched Apex – our own model, trained specifically for customer service – we ran Fin in production on Claude Sonnet 4.0. We tested later releases, Sonnet 4.5, 4.6, and Opus 4.5, and none outperformed Sonnet 4.0 on our RAG customer service task.

Turns out, the latest aren’t always the greatest – at everything.

As Fergal, our Chief AI Officer, puts it: “I think there will be some tasks that you can saturate in terms of intelligence and extra general intelligence doesn’t really move the needle for those tasks.” That means, some AI products, built with specific judgment and expertise can outperform the general models, even if they rely upon them.

Fergal recently joined The Chief AI Officer Show to talk about what three years of building AI products teaches you about models, pricing, and why Agent adoption is slower than it should be.

Unseen effort

Two years ago, many AI businesses heard the same criticism: “you’re just wrapping the model.” Cynics assumed that the journey from ChatGPT to Fin was easy – that what we were building was just a thin layer on top.

Fergal thinks that was “a total misunderstanding… People did not understand how much work you have to do around the model in order to deliver a product that was valuable.” He believes our investment in AI was “deeper and broader than any new entrant,” requiring a significant portion of R&D effort.

Today, people talk about “the harness around the model.” That framing is also flawed.

An AI model isn’t something you build a harness around, but a component, one building block in a larger engineered system. For Fin, the general models showed up at defined points in its architecture to deliver specific constrained functionality. Everything else was underlying infrastructure.

That will change over time. Models will take on more. But as they do, the pressure to specialize – rather than rely on horizontal general intelligence – increases alongside it. “You want to make sure that you’re continuing to add a big thick layer of value between the model and the set of business problems that you’re addressing.”

Why the frontier model isn’t always right

When trying to explain a model’s general intelligence vs. ability on specific tasks, and why AI products can survive outside of frontier labs, Fergal provides a thought experiment.

You’re hiring a retail assistant in a clothing store and you have two choices: a Nobel Prize winning physicist with a really high IQ or a retail assistant with 20 years of expertise.

Initially, you might want to go for the physicist – they’re smarter. But in reality, you only need so much general intelligence for the task. You also need emotional intelligence, sales experience, and personality. That’s true in the real world and Fergal believes it’ll be true for models.

This is one of the reasons he and his team began building our own specialized model, Apex.

The other was the significant improvement in open-weight model performance. Once they began to approach frontier models, it opened a new strategic option: take a strong open-weight model, apply a significant reinforcement learning program, and teach it the exact tradeoffs your domain requires. Initially, Fergal just expected Apex to be cheaper to run, but during training, he soon realized “it’s going to be better by the time we’re done.”

The long-term play for application-layer companies is vertical integration: build or fine-tune models optimized for your specific domain, rather than relying on horizontal models carrying capability you don’t need.

The gamble of outcome-based pricing

When we launched Fin with outcome-based pricing at $0.99 per resolution, we were pricing at a loss. The cost to serve a single resolution (running GPT-4 with dedicated capacity) was around $1.50 at the time.

The decision to price that way came down to two bets Fergal owned: that inference costs would fall, and that resolution rates would rise. Both felt like genuine risks but he “was very confident in them.”

However, the pricing model wasn’t primarily a unit economics decision. The more important reason was alignment. Outcome-based pricing makes every part of the organization – sales, customer success, R&D – want exactly what the customer wants: a resolution that actually works. “That’s not to be underestimated.”

Three years later, outcome-based pricing has become the expectation for high-value AI products. The market caught up.

Adoption is slower than the technology deserves

Fin now automates 83% of our own customer support volume and we’re not an outlier – we have customers with even higher automation rates. The technology works. And yet customer service adoption across the industry has been slow.

According to Fergal: “To my mind we should have seen much more radical adoption of Agents in customer service than we actually have.”

The reason is structural. Many businesses view customer service as a cost centre and are therefore slow to adopt new technology. Unlike coding Agents, there’s no product-led growth dynamic pulling people in – no engineer spinning up a trial, no bottom-up pressure from people who’ve already seen it work.

The skepticism is understandable, even if it’s expensive. When we tell people that 83% of our support volume is fully automated, they often don’t believe it. “In the back of their head they’d be like, it’s probably 30%.” We’ve had to learn to show the distribution of automation rates across our customer base to make the numbers credible – even internally.

AI is underhyped, Fergal argues as people are pattern-matching to previous hype cycles and find it hard to take genuine capability seriously. The diffusion curve is playing out but slower than the technology warrants.

To hear the full conversation and Fergal’s thoughts on evaluating models, the company’s initial reaction to ChatGPT, and if society is prepared for the impacts of AI, tune in here.

In B2B AI, there are two users:

1. The customer configuring the system.

2. The end-user asking a question and getting a resolution.

It’s tempting to collapse these groups together when designing, but the first user, the customer-builder, needs different evidence to understand and trust the system they are helping to create.

Most AI systems are black boxes by default: inputs go in, answers come out, and the system in between is hard to inspect. A lot of trust work focuses on making the answer more transparent – with citations, visible sources, input context, status signals, and clear admittances of uncertainty. That matters. But it mostly serves the end-user.

Customer-builders need visibility one layer deeper: “Why did this happen, and what can I do about it?”

Consequently, you must build two glass boxes.

When there’s only one visible thing, that’s what gets blamed

In AI products, the final end-user answer is what both the customer-builder and end-user see and judge. But a bad answer can come from many places: missing content or knowledge, incorrect config, e.g., guidance or guardrails, or the model itself. If those layers are invisible, every failure collapses into the same explanation: “the AI is broken.”

Here’s a story from the trenches back when I was on the Fin Content team. A prospective customer-builder was trying out Fin for the first time. As part of the setup, they entered their website URL into the content page and saw hundreds of pages come back as “enabled.”

What the content page looked like before. No visibility, simply “enabled” / “not enabled”. That didn’t communicate the nuance of website imports, which are the most popular type for new customers trying out Fin for the first time.

To them, that meant the system was ready, so they used the Preview panel to test Fin. They asked a product question they already knew the answer to and Fin got it wrong. From the outside, it looked like a hallucination, since the correct answer was on their site inside an FAQ accordion.

The actual cause was upstream: the page had been detected, but the content inside the FAQ accordion’s collapsed section hadn’t been successfully read, so the answer never made it into Fin’s knowledge base and it responded with incomplete information.

Two trust failures in the same moment, for two different parties. Had this reached an end-user, they would have seen a wrong answer with no signal it was wrong. The customer-builder almost concluded “Fin is not good” because the import surface said “synced” and gave them nowhere else to look.

Fin doesn’t usually fail in isolation, so the fix is rarely at the model layer. Most issues are caused during set-up – wrong content, configuration, or guidance. So the product team must build a glass box around these processes.

In this instance, that means allowing the customer-builder to see what has been read, what was excluded, what failed, and why. Otherwise, “Fin is broken” is the only conclusion the customer can draw, because Fin is the only part of the system they can see.

What goes under each glass box

When we rebuilt the content import experience to expose failure modes and enable troubleshooting, we built a glass box for that part of the experience.

Glass boxes build trust. But “show everything” isn’t a strategy. Revealing too much creates noise and false confidence. Too little leaves people guessing. So the real work is judgement: what goes in the black box, what goes under glass box one for the customer-builder, what goes under glass box two for the end-user.

The principle I keep coming back to is one I borrowed from data science. Early in my DS career I used to put two decimal places on high-level trend reports for product marketing because it felt more rigorous. Actually… for product marketing, what mattered was the trend. The decimals were noise dressed as precision. The lesson learned: detail to the extent it’s actionable for the audience. Anything beyond that is distraction, regardless of how true it is.

The same principle applies to glass boxes, with one twist: actionability isn’t the same for everyone. We must ask: what can each audience do with what they’re seeing?

• Glass box one: the customer-builder. Their action is “iterate the configuration,” so the surface shows what’s accessible to Fin and what’s configurable; what content was available and retrieved; what guidance shaped the response; what task is being executed. Anything they can change, they need to see. Anything they can’t act on stays out.

• Glass box two: the end-user. Their action is “decide whether their query is resolved,” so the surface shows what helps them make that decision: the answer and sources that they can verify, should they want to. Anything beyond it is two-decimal-places of noise.

A small example: one team thought Fin had become inconsistent because PDF source links appeared in some answers and not others. Initially, the model was deemed flaky, but the real cause was a guidance rule that needed iteration. The team could see and fix it because the Preview tool showed the one configuration choice that was producing this behaviour.

The harder version of this judgment is the one that goes the other way.

Sometimes the right call is to leave detail out because it confuses more than it helps. The website import surface is an example we lived through. Underneath it, website imports can fail for a long list of low-level technical reasons: different kinds of network issues, blocking, parsing problems, etc. We could have surfaced every one but chose not to, as most customers can’t act on the raw error type even when they see it. We collapsed the long tail into a small number of high-level categories, each tied to a fix the customer could actually make. The glass box got smaller.

Remember: visibility isn’t a virtue. It’s a tool, and like any tool, it can make things worse when it’s the wrong size for the job.

Once you accept ‘detail to the extent it’s actionable,’ how and where you present it matters just as much

A glass box is only useful if it sits next to the thing it’s about. Configuration on one page and logs on another means the customer has to reconstruct context every time they look. Where possible, don’t make them travel or hold things in working memory to find the evidence for a decision they’re making right now.

Example 1: the Content page after. Beyond “enabled”, the page tells you what failed, what’s live, and once the fixes are addressed, the user can take an action to re-sync.

Example 2: the Preview panel. It sits inside the same pages where Fin is configured. It works as a glass box in two ways: it lets customers test a new configuration in the place they just set it, and it surfaces every event Fin was configured around so the customer can trace the root cause of an answer.

Beyond Fin

If you’re developing AI for builders – people who configure it for their own end-users – you are building two glass boxes at once. One you control end-to-end, and one your customer assembles in your interface.

The second one is the part most teams underbuild. It’s also the part that decides whether your customer can trust your product, and confidently put their name on it.

Sometimes you see a corporate announcement that’s so obvious and so late, it’s almost an admission of failure. And I’m open to feedback on how I should have pulled the trigger on this change much sooner. But the tech industry is so littered with awkward and arguably unsuccessful name changes that I thought it prudent to procrastinate on this one.

But finally, today, we’re changing the name of our company to Fin. The name Intercom will still live on as the name of our customer service software platform that increasingly the best brands are choosing as their primary help desk; we just launched a complete rebuild, Intercom 2, and we’re dramatically increasing our investment in it. But the company behind it will be named after our leading customer agent platform, Fin. All of our 1,400 employees are now employed by Fin. You’re reading a post by the CEO of Fin.

In this moment of great change, for our industry, and for all economic players, the winners of yesteryear have largely been left holding on to their former positions that brought them so much glory. Yet, as I’ve written before, the only path to success in the future is through destroying your past. We’ve done this with our culture, our pricing model, our product lineup, and our investment priorities. But the company name stayed the same.

However, when we started the service agent category three years ago, we promoted our new technology with the name Fin, with Intercom very much in the background, and usually not even mentioned at all. The simple reason is that humans are very good at mapping meaning onto the world and then updating that map rarely, with a very strong bias for the things that are new and shiny. We couldn’t possibly wake up every morning questioning what a chair was and how it works. But when a new type of drone or artificially intelligent software product reaches our attention, we do our best to understand it and add it to the repertoire of things we understand about the world.

This is true in all aspects of life, including with companies and their brand names and the positions they hold in the market. I believe this so thoroughly that I actually think that the relative success of the newcomers in our category, despite the fact that we have provably superior technology, is a result of the fact that they have no baggage. They don’t need to convince anyone of their new position in the market, because they never had an old one.

So, today we say, “baggage be gone.” Fin is clearly our future, and the future of this new customer agent category. It’s driving our renaissance and is about to be the largest part of our business. We thank sincerely everyone who supported Intercom in these past 15 years. And again, the product and its name will very much live on. But our company is now finally, fully reborn and its name is Fin.

AI will always fill in the gaps. Whatever you don’t specify, it decides – toward the average of everything it’s trained on. A generic purple gradient or repetitive sentence structure (hello ‘it’s not X, it’s Y’) or corners that always seem slightly-too-rounded.

The interface encourages short, simple input. Most AI tools have a prompt bar only about three lines tall. And while that level of detail might work for quick fixes, it will leave much to be desired when you’re building something more meaningful.

As Michelle says: “Every blank you leave is a decision you’ve handed over.”

When it comes to getting worthwhile performance out of your Agents or AI systems you need diligence upfront. For us, that looks like providing Claude with 300 detailed skills, hooks that enforce how it works inside our codebase, and read-only access to production data.

The goal is simple: the less Claude has to guess, the better the output.

This week, we recap stories from across our team on how they collaborate with AI. Including: the workflows and org structures that deliver best results, the experiments they’re learning from, and how they’re feeling about the changing nature of work.

When you don’t have the words

Over the past few months, Róisín has been building Nudge – an app that helps busy friends stay in touch.

The frontend came easily. She’s a designer; wireframes are her bread and butter and Claude turned these into working prototypes. With her experience, she could identify an issue: “This button is too far from that input” or “The error message doesn’t explain what went wrong.” And then she and Claude would debate the best solution and action it.

The backend is a different story.

Claude had to take the lead.

“The terminology alone was like Klingon to me.”

Róisín started every session by reminding Claude to do everything in the browser rather than the terminal and that she was a complete beginner. It walked her through procedures step-by-step, explaining as it went, and worked hard to detangle screenshots of console errors. Over time, she’s learned what error codes mean and how to debug simple things on her own.

But Claude can only do so much.

Nudge’s pending invite status kept throwing 400s and after debugging it three times, Claude suggested cutting it. The feature wasn’t essential for beta testing and it was delaying the valuable insights she’d gain from shipping to users. It was the right call.

Learning what to cut, when to stop iterating, and what’s good enough is something Róisín is learning, and in her own words: “You learn it by shipping.”

Read more about Róisín and Nudge here.

Three lines isn’t a brief

Most coding Agents have a mode called “dangerously accept all permissions.” Michelle argues the bigger risk is the posture most people bring to AI: dangerously accept all opinions.

AI presents recommendations confidently, making you feel like it knows best. It doesn’t – it knows the average of its training data. In most cases, that’s not enough.

Her piece walks you through three practical tools for building the muscle of better direction:

• Plan mode: Separates thinking from doing. Map out intent before anything gets built, then push beyond the build plan to the product plan: what should this feel like, what would make someone use it a second time?

• Skills: Small bundles of instructions that give AI better judgment in specific areas. Worth bookmarking: Superpowers for brainstorming, Impeccable’s Shape skill for structured discovery interviews, Get Shit Done for spec-driven planning.

• Living docs: Brief.md, design.md, architecture.md saved in your project folder. The AI references them every session. Your prompts get lighter because it already knows where you’re coming from.

According to Michelle: “The building is the easy part. The hard part is knowing what to build and being detailed and opinionated enough about it so what gets built is actually yours.”

Get all of Michelle’s tips here.

The treadmill that punishes mastery

With the rise of AI tools, designers are now being asked to run two scorecards simultaneously – technical fluency and craft – and it’s a tough balancing act. Worse still, according to John, the demands on each are ramping up.

As the AI space gets more competitive, new tools and models come to market almost everyday, making it hard to achieve true mastery. The external pressure doesn’t help. The AI corner of design Twitter and every internal Slack channel runs on a constant drip of “look what I shipped this weekend.” Any designer who pauses to think feels like they’re falling behind.

“At the same time, you’re still expected to be a world-class designer. Now that anyone can ship, the bar is only going up – for craft, for taste, for the thousand small decisions that make products feel great.”

John fears the designers attempting to excel in both disciplines risk running themselves into the ground. But he also thinks a reprieve could be on the horizon: “There’s probably a correction of some sort coming. The pace isn’t sustainable.”

In the end, “the designers who come out of this strongest won’t be the ones who ran hardest on either axis. They’ll be the ones who stayed clear-headed enough, and rested enough, to do the judgement work that compounds.”

Read his full piece here.

Exit the comfortable middle

Karen leads a 40-person Research and Data Science (RAD) organization at Intercom. Her conclusion from the last few months of working with AI isn’t what you might expect.

“AI commoditises execution. It does not commoditise judgment. And for research and data science teams, that’s the difference between being relevant and being replaced.”

The work most at risk to AI automation, she argues, isn’t the obvious tactical work. But the comfortable middle – the incremental analysis that feels strategic, a dashboard update that is visible but never changes a decision, or simple research exercises. All of this can now be handled by stakeholders with the help of AI.

So how can researchers and data scientists maintain their relevancy?

According to Karen: they can move in three directions:

• Down: Encode business context as infrastructure to make self-serve trustworthy.

• Up: Move from delivering insights to building decision systems.

• Back: Lean into the human edge, the slow, formative work that doesn’t exist in any dataset yet.

Read her full analysis here.

Write it all down

We’ve written a lot about our 2x initiative but no one has spoken about it more than Brian.

Last week, he sat down with Akash Bajwa to talk through what building AI-first looks like inside a 15-year-old Rails monolith.

Getting Claude to perform reliably within such a code base meant building a harness around it – with linters, standardised patterns, and explicit written guidance about the right way to do things. Brian actually thinks the process “mirrors human onboarding”: you show Claude what good looks like, which parts of the code represent the old way, and how to get things done. The difference is you have to write it all down.

How is the harness working?

Brian highlights two metrics: quality of code and cost per PR.

Over the past few weeks, code quality at Intercom is going up, after an initial period of expected decline. Brian credits the auto-approval process for this as it encourages engineers to produce good code on the first pass. PRs that are small, single-purpose, use feature flags, and stay out of dangerous code paths get approved without a human reviewer. Kesha and Niamh recently wrote about this process.

When it comes to costs, Brian encourages companies to avoid becoming fixated on tokenspend. What matters more is the fully loaded cost of a PR including salaries, management overhead, review time, and rework. When you measure that, the economics of AI-assisted engineering look very different. “We have an awful and ever-growing bill,” he says, “and yet the cost per change is dropping. That’s exactly what you want.”

The interview also covers hiring for openness over experience, why committing to Claude Code is less of a risk than staying agnostic, and how org structure has changed.

Read the full interview here. Or Brian appeared on the On Rails podcast and the HangarDX podcast to talk about our use of Claude Code, so tune in if you want to go deeper.

Our final thought

When anyone can ship, the real difference maker is your point of view and your ability to communicate it to your AI. Getting great results requires great documentation, down to every detail.

The idea that software is developing at a pace never seen before is now more than trite. But implications of it are still being reckoned with. One interesting result is the breakneck pace at which categories are evolving. It used to be the case that software categories would mature for a decade before there were opportunities to specialize them for specific verticals, but that timeline just won’t make sense anymore. This Fin announcement is an interesting example of this.

Today we’re announcing Fin for Ecommerce, by far the world’s best shopping assistant agent. The best real life, in-store experiences almost always involve interactions with knowledgeable, helpful experts who close the gap between your problem and their solution. But online you’re left alone, navigating menus, dropdowns, and filters that represent inventory, trying to find products by trial and error, and unsure what’s even available. This is a big reason why 70% of carts are abandoned. Customers are unsure what to buy and have no help.

Fin for Ecommerce fixes this. The world’s first expert shopping assistant, working across the full customer journey, that can move seamlessly from sales to service, delivering perfect customer experiences and driving conversion and revenue.

It works to understand customer intent and has deep knowledge of real time inventory and price changes. It handles abstract questions and guides the customer to the right product. It manages difficult product and service questions, and moves customers towards conversion, adding products to the cart. After purchase, it knows everything about the order, can chat about order status and problems, and can take action to resolve customer change requests. Best of all, you can connect to your store and have it working with customers literally in minutes.

This all runs on the Fin architecture that serves over 12k customers and many millions of conversations per week, and on Apex, the world’s first and best customer agent model. It’s driving epic early results already, with significant lift in average order value (AOV) and higher cart conversions.

What surprised us most about Fin for Ecommerce is how quickly it delivers high-quality support with minimal, non-technical setup

Fin for Ecommerce is already driving meaningful revenue, with 10% of conversations converting to orders averaging 20% above our store AOV. It’s doing the work of a sales and support team combined.

The handoff between support and sales is so smooth I can’t tell the difference without checking the filters. Fin talks policy, sells products, and references our mattress break-in period all in one conversation.

This represents a further demonstration of our philosophically unique approach to this category. Our competitors are still mired in their consultingware approach, where they must hack together, customer by customer, brittle bespoke solutions. While we’ll always customize our offerings for our customers who need it, you can go from zero to one (with one being the best performing in the market) faster with Fin than anything else out there. If you’d like to see the latest and greatest example of this, I encourage all ecommerce companies to sign up for Fin for Ecommerce today.

Today, people use Agents to do tasks – Claude Code can quickly fix that bug that’s been annoying your customers, or OpenClaw can optimize your calendar each day. But increasingly, Agents are also making decisions on our behalf.

If you’re a SaaS business, this means you’re no longer just selling to a CTO or founder, but the Agents they’re using. Ensuring your product is discoverable, configurable, and trustworthy to an Agent is the next product surface worth designing for and prioritizing.

Agents build what they can’t buy

At Intercom, one of our engineering philosophies is to run less software. Don’t build what someone else has already built better.

Agents should work the same way. But they often can’t, because SaaS wasn’t built for them. And as a result, they end up preferring to build instead of buy.

To sign up for most software products, you open a browser, fill in a form, verify an email, and navigate a configuration UI. Sometimes you even have to get on a call with the sales team. That’s fine for humans, but Agents can’t do any of it (yet). So when an Agent needs feature flags, it probably won’t sign up for LaunchDarkly, it’ll build its own feature flag system from scratch.

Agents love to solve problems on their own and they always try their hardest; they’re not always going to ask you to help them navigate a sign-up flow.

The implications of this for providers are best articulated by Aaron Levie, CEO of Box:

“The platforms that are easier for agents to adopt, and solve the agent (and user’s) problems the best, will get ahead far faster than those that don’t. Agents won’t be going to your webinar or seeing your ad; they’re just going to use the best tool for the job, and you’ll want it to be yours.”

We’re already seeing the first wave of providers prioritizing agentic accessibility. When combined with strong brand recognition, they become an Agent’s go-to solution. For example, when setting up payments, Claude almost always picks Stripe. If you need website hosting, it heads to Vercel.

Right now, the financial impacts of Agents’ software preferences are limited. After all, developers are the largest users of coding Agents and they tend to have their own preferences for software and can steer an Agent elsewhere, especially if they’re working in an existing tech stack.

But over time, as Agents become more capable and non-technical users adopt them to set up and run their businesses, an Agent’s choice of software quickly becomes the market’s default choice. Providers hoping to get a leg up in this environment must make SaaS directly consumable by Agents and not assume that an OpenClaw-style browser and email access will provide a quick fix.

What Agent-first means

Since the launch of ChatGPT, there’s been considerable chatter about Answer Engine Optimization (AEO). It’s the LLM counterpart to Search Engine Optimization (SEO). In brief, you optimize your website and content so it will be understood and cited by an LLM. By doing so, you bring organic traffic and potential customers to you. If the LLMs can’t find you, neither can consumers, and it will only get worse as LLMs take on more search volume.

In a similar way, a Command Line Interface (CLI) is the Agent’s equivalent of easy-to-navigate and appealing UI. It allows Agents to understand and use your product. The better your CLI, the more likely it is that an Agent will choose your product when building. Some providers opt to use a Model Context Protocol (MCP), which functions like a wrapper around an API, but CLIs can take on tasks with multiple steps and help Agents get more done.

Creating a CLI is step one of preparing for the Agent-first future, but step two is having a product that can function out-of-the-box. Not “easy to sign up for” – actually deployable by a non-human without any engineering help.

That’s a much larger product issue, not just an interface one. You can’t wrap a CLI around a product that needs hand-tuning. Agents will hit a wall and look for something else or deliver a disappointing experience to users. This grants a considerable advantage to SaaS providers that are already self-configurable and places pressure upon those that are not.

What we built

I’d been watching the rise of agentic software preferences for a while – talking about it on Slack, posting about it, generally making noise. Jordan eventually told me to stop complaining and actually build something. So we did.

The Fin CLI is built for Agents and can be launched by a single prompt. It signs up to Intercom and Fin, configures the Messenger, pulls in your help content, and sets it live. It also returns contextual feedback to the Agent at each step – what to check, what to set up next, and where to go if something isn’t working.

The vision is this: someone asks Claude to build them a product, then says, “I want to talk to my customers and handle support automatically.” Claude says “yes,” and gets Fin up and running all on its own.

AI Agents orchestrating AI Agents. That’s where this goes.

Check out our CLI at fin.ai/cli

Shipping fast is our priority

We’ve long held a belief that might sound counterintuitive: speed is not the enemy of safety. It’s a prerequisite for it. Accumulating code creates risk. Shipping small batches minimizes it. The faster you ship, the smaller each change is, and the easier it is to catch problems, and roll back when something goes wrong as the context is still fresh in your head.

Today, over 93% of our pull requests (PRs) across our two main codebases are Agent-driven. And over 19% are auto-approved with no human reviewer in the loop.

This post is about that second number, and why we think it makes us safer. Most people hear “AI is approving our pull requests” and think that’s reckless. We think the data tells a different story.

The 2x effort

Last year, our CTO Darragh Curran set an explicit goal: double the productivity of our entire R&D organization within 12 months. Because the faster we can build and ship, the faster our customers get the capabilities they need.

Nine months later, we did it. The results have been significant across the board, but the number that matters most for this post: downtime from breaking code changes dropped 35%, even as our deployments doubled. Shipping faster made us safer. As we modernize how we build and ship software, we systematically surface bottlenecks and tackle them. One of the biggest we’ve found? PR review.

The PR review problem

Humans simply don’t have the time or mental capacity to properly review the volume of AI-generated code we’re now producing.

When an AI Agent can produce a working implementation in minutes, waiting hours or days for a human to review it is an impedance mismatch. The production line is moving faster than the quality gate can keep up.

When that happens, one of two things follows: either the queue backs up and velocity drops, or, more dangerously, humans start rubber-stamping. Glancing at a diff, skimming the description, clicking approve.

Some companies are drifting into this failure mode silently. We chose to confront it head-on and built a rigorous solution.

PR review, done properly, is a genuinely complicated process. A good reviewer assesses the quality of the problem statement based on the PR description. They confirm the changeset actually matches the stated intent. They review the code against best practices, look for logical issues, apply their personal product context to validate the changes make sense, and check for performance issues, safety concerns, and more. No single human reviewer realistically has experience to properly cover every one of these dimensions on every PR, especially not under time pressure. And as we’ll see in the data, the baseline we’ve been comparing against, human review, was weaker than most of us assumed.

So we asked ourselves: what if we could do better?

How our PR review Agent works

Our PR review Agent doesn’t treat code review as a single task. It decomposes it into separate sub-jobs, each handled by an independent sub-Agent. One assesses the quality of the problem description. Another checks whether the diff actually aligns with the stated intent. Another reviews for safety concerns. Another checks for logical correctness. Another reviews against best practices and known anti-patterns. And so on.

The result is that every PR is reviewed as if a dozen of our most tenured and knowledgeable engineers were all looking at it simultaneously, each bringing their own specialist lens. In the past, getting that breadth of review on a single PR was impossible. Now it’s the default.

A human reviewer typically focuses on the actual code changes, the diff. Our Agent goes deeper. It traces execution paths, following the implications of a change through the codebase. This is something humans rarely had time to do, even when they wanted to.

A real example:

While testing our new PR review Agent on a set of historical PRs, we found it flagging a one-line text copy change as incorrect. On the surface, it looked completely harmless, just a text update. We assumed it was a mistake, but it wasn’t. Our Agent caught that the new copy contradicted an existing validation mechanism elsewhere in the codebase. No human reviewer would have realistically found this unless they happened to have written that validation code very recently. Our Agent catches this kind of thing consistently, every time, because it’s always tracing execution.

The review isn’t generic either. It’s grounded in Intercom-specific guidance that our engineers have built and continue to refine, encoding the same context, standards, and product knowledge they’d apply if they were reviewing the PR themselves. When the Agent reviews a PR, engineers flag whether the review comments were helpful or not, and that feedback continuously sharpens the guidance. It’s a flywheel: the more our engineers invest in teaching the system how to think about our codebase, the better every subsequent review gets.

Automated approval is also never forced. Any engineer can request a human review on any change, at any time. The system is a tool, not a mandate. At Intercom, shipping code doesn’t end at merge. The engineer who ships a change is expected to watch it go live, monitor its behaviour in production, and be ready to roll back if something isn’t right. AI approval doesn’t change that. The human who ships the code remains accountable for the outcome.

Not just faster; safer

The naive take on AI-approved PRs is that it’s just a rubber-stamp LLM call so that humans don’t have to bother. A convenience feature. That misses what’s actually happening.

Our Agent is strict. It won’t approve large PRs. If a change is too big, too complex, or too broad in scope, it flags it and requires it to be broken down. Creating a direct, positive incentive for engineers to ship smaller, more incremental, well-scoped changes.

This matters enormously for safety. Small changes are easier to review, easier to test, easier to understand, and, critically, easier to roll back when something goes wrong. This is the same principle that has always underpinned our shipping culture, but now the PR review Agent actively enforces it.

It’s tempting to look at a goal like “>50% AI-approved PRs” and worry that we’re optimizing for the metric rather than the outcome. We see it differently. Our goal was to remove a bottleneck that, if left unchecked, risked pushing engineers towards rubber-stamping reviews under time pressure.

On the surface, our PR review Agent is only a solution to “humans don’t have time to review PRs.” But what it really is, at its core, is a safety mechanism. It ensures we continue shipping fast, in small increments, maintaining and increasing our heartbeat, even as the volume of AI-generated code grows.

The data

We didn’t assume AI review would be good enough, we actively ran an experiment.

Our hypothesis was that AI review could match or outperform human review quality, measured by the outcomes that actually matter: were the changes correct? Did they cause problems in production? How quickly were they reviewed and approved?

We started with a controlled pilot of over 100 PRs through the AI approval pipeline. The results: zero reverts of AI-approved PRs, and a 6–16x improvement in time-to-approval at the 75th percentile. Since then, the system has scaled significantly. In the first four weeks of broader rollout, 497 PRs went fully autonomous, with Claude writing the code and our AI approval system reviewing, approving, and shipping to production.

Beyond the approval pipeline itself, we also looked more broadly at how AI-authored code performs in production compared to human-authored code. AI-authored backend code had a revert rate of 0.53%, compared to 5.39% for human-authored. On the frontend, it was 0.22% versus 2.00%.

AI-authored code, reviewed and approved through our automated pipeline, is being reverted at a fraction of the rate of human-authored, human-approved code. We don’t expect that to hold at zero forever, but the data so far tells us that the quality bar our Agent holds is at least as high as the one humans were holding, and in many cases higher.

And perhaps the most important perspective shift: those product changes that did cause outages in the past? They were all reviewed and approved by humans. Human review is not a guarantee of safety. It never was. It’s a useful heuristic, but one with real limitations that we’ve been quietly accepting for decades.

Staying compliant while we innovate

Everything we’ve described in this post, the sub-Agent architecture, the traceability, the labelling, the data, none of it was built just to make the system fast. It was built to make the system auditable. That was a design constraint from day one.

Every AI-approved PR is labelled, logged, and queryable. The review comments, the approval decision, the test results, the merge event: all recorded. The evidence an auditor expects to see is the same whether a human or an AI approved the change. The “who” may change, but the “what” doesn’t.

We engaged our auditors, Schellman, early, before we scaled. We proactively worked with them to confirm that our automated review processes and the evidence they produce meet the requirements of our compliance frameworks, including SOC 2, HIPAA, ISO 27001, ISO 42001, and AIUC-1, among others. We think AI-driven change management can meet and exceed the standards that human-driven processes set, and we want to help prove that.

We see this as a feature of building the system the right way, not an extra hurdle we had to clear. When you build for safety, compliance follows.

What’s next

You can only go so far with PR review as a safety mechanism, no matter how good the reviewer is, human or AI. Only in production do you discover the unknown unknowns. The majority of Intercom’s largest outages weren’t even caused by changes to product code at all. They were infrastructure issues, unanticipated customer usage patterns, or third-party outages. PR review, whether human or AI, was never going to catch those. That’s why, in parallel, we’re also working on an Agent that proactively diagnoses issues in production. We’ll share more on this soon.

Speed has always been at the core of how we build, not in spite of safety, but because of it. And we’re getting even faster with AI. It’s easy to assume that AI-approved PRs would lead to a drop in quality and safety but our data proves otherwise. We’re just getting stronger.