Great question Rohan. I think keeping business impact as the north star is critical here. With foundation models evolving so quickly, it's easy for teams to get trapped in an endless cycle of chasing higher accuracy.
In many cases, an 80% solution that solves a real customer problem today is more valuable than spending months trying to reach 99% accuracy. I'd rather validate that users find value in the workflow, then iterate on the model, than perfect the model before proving the use case.
For example, if an AI assistant can successfully handle 80% of support queries and significantly reduce response times, that's already a meaningful business outcome. The remaining 20% can be routed to humans while the product continues to learn. Value discovery comes from proving that customers care about the outcome, not from maximizing benchmark scores.
Great question Rohan. I think keeping business impact as the north star is critical here. With foundation models evolving so quickly, it's easy for teams to get trapped in an endless cycle of chasing higher accuracy.
In many cases, an 80% solution that solves a real customer problem today is more valuable than spending months trying to reach 99% accuracy. I'd rather validate that users find value in the workflow, then iterate on the model, than perfect the model before proving the use case.
For example, if an AI assistant can successfully handle 80% of support queries and significantly reduce response times, that's already a meaningful business outcome. The remaining 20% can be routed to humans while the product continues to learn. Value discovery comes from proving that customers care about the outcome, not from maximizing benchmark scores.