The Quant Who Evaluates AI on the Problems That Broke the Last Generation of Models
By Oliver Brett, PhD — 2026-01-01
Oliver Brett has a PhD in applied mathematics and fourteen years of experience in quantitative research at two leading quantitative hedge funds. He has spent his career at the intersection of statistical modeling and financial markets — building the kinds of models that most people assume are infallible until the market regime changes and they aren't. He came to IXO because he sees AI entering quantitative finance with the same overconfidence that has preceded every major quant failure.
The category error in financial AI
"There's a category error that keeps appearing in financial AI," he says. "People build systems that perform well on the data they have, demonstrate impressive backtest results, and conclude that the system is robust. The quant failures of the last thirty years tell you exactly why that conclusion is wrong — and yet the pattern keeps repeating."
Brett's IXO work focuses on quantitative model evaluation — testing AI financial modeling tools against the standards applied by experienced quants. His annotations address model specification appropriateness, assumption clarity, overfitting risk, out-of-sample validation methodology, and whether the model's confidence intervals are calibrated to reflect genuine uncertainty rather than in-sample fit.
The overfitting detection problem
"The overfitting detection problem is the one I focus on most. AI financial models can be trained with enough regularization to avoid obvious overfitting while still being subtly overfit to regimes that no longer exist. Detecting that requires someone who can read a model's residual structure and understand what it's telling you about the data generating process."
Beyond surface sentiment
He also evaluates AI natural language processing tools used in financial contexts. "The information content of earnings call language is real but it's also extensively gamed. Management teams know what analysts are listening for. A model trained on surface sentiment misses the signal that comes from changes in linguistic patterns — increased hedging, decreased specificity, shifts in forward-looking statement density. That's the part that requires a practitioner to annotate correctly."