Handling inveitable failures in a startup
We have had a few interesting issues recently
1. We were helping one of our customers fill out some tax forms, and Skyvern failed to interact with the "yes" and "no" radio buttons because clicking the labels didn't work -- very unusual for websites
2. We were helping another customer fill out some permit applications, and Skyvern failed to find the correct address and ran into an issue
This got us to an interesting series of conversations with our customers -- what kind of success / failure rate should be expected from AI agents?
Obviously, every customer wants 100% success rate for everything, but if you read the fine print for any software product (ie AWS), they charge an arm and a leg for anything above 99.999% uptime / success.
AI Web agents introduce another class of possible software issues -- Skyvern uptime isn't enough.. the underlying website must also be up, functional and without error.
How can we guarantee correctness to our customers?
Should we promise 99% uptime, but then add small text at the bottom indicating that it's only uptime as dictated by us?
Should we
Well --> It turned out to be pretty straightforward.
We just talked to the customer to better understand their risk profile (+ expectations around successes and failures)
And in talking to the customer, we built trust. This is what customers are really looking for when they're trying to get to a specific tier of uptime
That being said, we haven't turned this into a formula just yet
How do you build trust with your customers?