The Rise of the AI Scientist

“If you feel that a product is hard to eval, or you feel like, ‘I don’t even know how to eval this,’ it’s a strong smell that your product isn’t good.” — Hamel Husain, on AI product design An AI data agent tells you last quarter’s net revenue. It doesn’t show the metric definition, source tables, filters, query, intermediate calculations, or assumptions. You can’t trust the answer without asking a data scientist to reproduce it. Hamel Husain argues that the eval problem is evidence of bad product design. If the user can’t inspect the work well enough to decide whether the answer is right, anot