ResearchJune 12, 2026 · 14 min read

We Plugged a State-of-the-Art AI Forecasting Model Into Our Strategy. It Failed.

Kronos is an open-source foundation model for financial candlesticks with impressive academic benchmarks. We tested whether it could improve Mercurio's entries. The answer was an unambiguous no — and the failure is instructive.

Every few months a new model promises to predict markets. The most credible recent candidate is Kronos, an open-source foundation model for financial candlesticks — trained on roughly 12 billion bars from 45 exchanges, with a paper accepted at AAAI 2026. Unlike the usual social-media "AI trading" claims, Kronos is real, public, and benchmarked: it reports genuine improvements in ranking-correlation and volatility-forecast accuracy over prior time-series models.

Impressive academic benchmarks are a reason to test something, not to trust it. A model that lowers next-candle prediction error in a paper can still be worthless — or harmful — on the specific trades a real strategy takes. So instead of arguing about it, we wired Kronos into Mercurio and measured it the same way we measure everything: on real bars, against real outcomes, in dollars.

Pre-registered success criteria (decided before we looked)

To keep ourselves honest, we wrote down what would make us adopt Kronos before running anything: (1) a positive information coefficient on our entries of at least +0.05 with p < 0.05, AND (2) a gate built from its forecast that improves the out-of-sample Sharpe ratio without cutting return. Two clear bars. Either it clears them or it doesn't.

Where it would plug in

Mercurio's design rule is strict: the technical strategy is the edge, and any model layer may only filter trades, never create them. That gives Kronos exactly one legal job — the veto gate. It can reject a trend signal the strategy already generated; it can never invent a trade of its own. Here is the precise slot it occupies:

The decision pipeline: from market data to a live order

A signal falls through four independent gates from top to bottom. Any gate can divert it to cash (red). Only a signal that clears all four becomes a bracket order at Alpaca.

Data
Market data
Alpaca OHLCV, 1H bars
Engine
Trend signal
EMA stack + ADX floor
Engine
Regime gate
bull-confirmed only
AI filter
AI veto
sentiment / event filter
Engine
Risk manager
size, stop, exposure caps
API
Order executor
bracket + stop-loss
Broker
Alpaca
paper account
Data
No trade
stay in cash

So the question is narrow and falsifiable. When trend following fires a long entry, we also ask Kronos to forecast the next 24 hourly candles. Does the sign and size of that forecast tell us anything useful about whether this specific trade will make money?

The shadow-mode protocol

Before letting a model touch a single decision, we run it in shadow mode: it scores trades but changes nothing. For every long entry the strategy generated over the two-year validation window, we fed Kronos the same lookback the live model would see, recorded its forecast for the next 24 hours, and then — crucially — recorded what the trade actually did. The forecast and the outcome are logged side by side, but the forecast is ignored by the trader. That isolates predictive power from everything else.

376
Entries scored (full 24h horizon available)
24h
Forecast horizon per entry
0
Trading decisions changed in shadow mode

We measured three things, because a forecast can fail in three independent ways — ordering, direction, and magnitude:

  1. 01Information coefficient (IC). The rank correlation between Kronos's predicted return and the realized return across all 376 entries. IC is the standard quant test for whether a signal carries any ordering information — does ranking trades by the forecast rank them by outcome? An IC of 0 means the forecast and reality are unrelated. Even good equity signals live around +0.03 to +0.05, so our +0.05 bar was already generous.
  2. 02Directional hit rate. Ignore magnitude entirely — how often did the model just get up-versus-down right? A coin flip is 50%. Below 50% means the model is actively anti-correlated with reality.
  3. 03Calibration. Did the size of the forecast line up with what happened, or was the model systematically biased in one direction regardless of the outcome?

Result 1: zero ordering information

The information coefficient came in at -0.05 with a p-value of 0.29. That is not a weak-but-positive signal — it is statistically indistinguishable from zero, and what little sign it has points the wrong way. By our own pre-registered bar (IC ≥ +0.05, p < 0.05), Kronos failed the very first test. The directional hit rate made it worse:

-0.05
Information coefficient (p = 0.29 — indistinguishable from zero)
42.8%
Directional hit rate (a coin flip is 50%)
-10.5%
Median 24h forecast for stocks that then rose +0.6%

That third number is the most damning. Group the entries by what actually happened and look at what Kronos had predicted for them: for the cohort of stocks that went on to rise about +0.6% over the next day, Kronos's median forecast had been a drop of 10.5%. The model was not noisy around the truth — it was confidently pointing the opposite direction on exactly the trades that worked.

Result 2: as a live gate, it destroys the edge

Shadow mode already told us to stop. But we ran the second pre-registered test anyway: actually let Kronos veto trades and measure the strategy. We swept the gate from gentle (block only the 10% of entries it scored most bearishly) to aggressive (block the worst-scored half). Every setting made the strategy worse, monotonically:

Configuration2-yr returnvs baselineSharpe
Baseline — no Kronos+53.1%1.03
Gate: block worst 10%+15.9%-37.2 pp0.45
Gate: block worst 50%+9.4%-43.7 pp0.33

Even the gentlest possible filter — vetoing one entry in ten — erased 37 percentage points of two-year return and cut the Sharpe ratio by more than half. The damage scales with how much authority you hand the model. There is no setting where it helps. That is the signature of a signal that is not noisy but actively wrong on your trade set.

Why it failed — the mechanism

This is not bad luck; it is structural, and understanding it is the whole point. Like most sequence models trained on price windows, Kronos normalizes the recent window it is shown and tends to mean-revert its forecast back toward that window's center. On an average, range-bound chart that is a reasonable prior.

But a trend-following entry is, by construction, the opposite of an average chart. The strategy only fires when price has stacked its moving averages and broken out near the top of its recent range — precisely the configuration where a mean-reverting model screams 'overextended, expect a pullback.' So Kronos predicts 'down' on almost every breakout we take. Its bias is not random with respect to our strategy; it is anti-correlated with it by design. Gating on that signal doesn't remove our bad trades — it removes our good ones and backfills the book with lower-ranked entries.

Statistical excellence != economic value

Kronos can genuinely lower next-candle forecast error on a broad benchmark and still have zero — or negative — tradeable value on a specific strategy. A model is only useful where its strengths overlap the trades you actually place. Ours sit in the exact region where this model is weakest. A better RMSE on average bars says nothing about the tail of breakouts that a trend follower lives on.

What would have changed our mind

We did not reject Kronos because it was AI, or because it was complex, or because it wasn't ours. We rejected it because it failed two criteria we committed to in advance. If the IC had come in at +0.06 with p < 0.05, and a gate had lifted out-of-sample Sharpe without sacrificing return, Kronos would be in the live pipeline today. It cleared neither bar. The integration felt like it should help — a sophisticated model sanity-checking our entries — and measured honestly it was a 37-point mistake. The feeling was the trap; the measurement was the escape.

That is the entire value of a negative result. It is one of many: a market-regime forecaster, several sentiment overlays, and a basket of exit tweaks have all died on this same bench. Each rejection costs a few days of engineering and saves us from a plausible-sounding feature that would have quietly bled money in production. We would rather find the failure in a backtest than have the market find it in our account.

Read the model's own claims in the Kronos AAAI 2026 paper, the architecture it would have plugged into in our system architecture tour, and how we validate everything in the backtesting deep dive.


Disclaimer. Every figure here comes from historical simulation on paper capital — including the shadow-mode statistics and the gated backtests. Nothing here is financial advice, and nothing here is a criticism of Kronos as a model; it is a report of how one specific model performed on one specific strategy. Past performance does not guarantee future results.