B2B "AI Products" That Are Really Services in a Trenchcoat
Say no to services
A lot of B2B AI products aren’t actually Products, and I think the market doesn’t realise the consequences of this yet.
In the early days of SaaS, there were startups that would say, “Yes, of course we can deliver that feature”, because they’d fork the database just for you. A sophisticated buyer would be really suspicious:
“How you are you going to maintain this? Will we get the same updates as everyone else?”
You worry that once the contract is signed, you’ll be abandoned. You thought you bought a product, but you got consultingware in a trenchcoat!
This is happening all over B2B AI at the moment! But today it’s not the databases that are being forked, it’s the prompts. And most people don’t realise the problem!
And because prompts seem so small, so innocuous, buyers don’t realise that a forked prompt is a significant piece of technical debt. Whenever you want to change the underlying model, you need to validate the prompt is still as high quality as it was - and frequently re-architect it.
The issue is that, while many startups are building ‘product’ by having Forward Deployed Engineers craft a custom prompt for each customer during the evaluation period, the customer isn’t going to get that same attention every time a model needs to change. The economics just don’t work.
So, either the application falls behind new models and techniques, missing out on quality that way; or the vendor will YOLO change the model, and the actual quality of the prompt will fall, in a way that’s initially invisible, but eventually is really disappointing.
With Fin we’ve been really opinionated that we’re building a product from day one. I’ve been really opinionated that we shouldn’t diverge our prompts unless we absolutely have to. There’s been costs to this. We have to invest more time and care upfront for each thing we bring to market.
But this standardisation has a second huge benefit: it has meant that we can improve one unified product across all our customers. The standardisation enables us to apply scientific principles, of ab-testing, and optimisation, across a large customer base using a unified AI product.
Further, every time Fin makes a mistake that we fix, the product improves for every customer. This doesn’t happen if every customer has a unique prompt, handcrafted for them. If you are a customer, you don’t want that!
Obviously we’re opinionated here. Our receipts are Fin’s constantly increasing resolution rate - increasing even for our earliest cohorts - and the >7000 customers, from large enterprises, to small self-serve customers, who get the best-in-class resolution rates Fin provides.
It’s a lot more work to do things in a standardised way, but we’re proud that they all customers from the standardisation and scientific process of continuous improvement that we apply.
I think the period the industry is in now, of hacking products per customer, won’t last.