Table of Contents
Machine learning works spectacularly well on problems with three properties: abundant data, a stable relationship between inputs and outputs, and a high ratio of signal to noise. Image recognition has all three. Language modelling has all three. Protein folding has all three.
Currency markets have none of them, at least not in the form the algorithms expect. This does not make the tools useless here. It does mean that applying them without adjusting for the differences produces results that look excellent in development and disintegrate on contact with a live account.
The failure is rarely dramatic. A model does not usually collapse; it degrades, slowly enough that the operator spends six months attributing losses to unusual conditions before concluding that the unusual conditions were the whole time. Most claims made about AI forex trading systems never survive the first of the problems below, and the people making them are frequently not being dishonest. They have simply measured the wrong thing.
The signal is smaller than the metrics expect
In most machine learning applications, a model explaining 2% of the variance in the target is worthless. In financial return prediction, a model with an out-of-sample R² of 0.005 can be economically significant, because the base rate of predictability is close to zero and a small consistent edge compounds across many trades.
This inverts the usual intuition in a way that causes two opposite errors.
The first is abandoning models that are actually working, because the accuracy figures look poor. A directional hit rate of 52.5% after realistic costs is a genuinely useful result. It does not feel like one.
The second, and far more common, is celebrating models that are not working at all. If your validation shows 85% directional accuracy, the correct response is not excitement. It is to assume, as a default, that information from the future has leaked into your features, and to go looking for the leak. It is almost always there.
The ways future information gets in
Leakage in financial modelling is subtle and has a set of recognisable forms.
Random cross-validation on time-ordered data. Standard k-fold shuffles rows, which means the model is routinely trained on Tuesday and tested on Monday. In any series with autocorrelation, this inflates performance enormously. Time series require forward-chaining validation, with a gap between training and test windows to account for the horizon of the label.
Normalising across the whole sample. Computing a mean and standard deviation over the entire dataset, then using them to scale features in every fold, embeds the future distribution into every training example. Scaling parameters must be computed on the training window only.
Revised macroeconomic data. GDP, employment and inflation figures are revised, sometimes substantially, months after release. A model trained on final revised values has been shown numbers nobody possessed at the time. Point-in-time data solves this, and it is more expensive and much less commonly used than it should be.
Labels that overlap the features. If your label is the return over the next twenty-four hours, and your features include anything computed from a window that extends past the entry point, you have built a time machine. This includes the closing price of the bar you are entering on.
Survivorship and delisting. Less severe in major currency pairs than in equities, but relevant for emerging currencies, redenominations, and any dataset assembled from currently active instruments.
Row count is not sample size
Ten years of minute bars looks like an enormous dataset. Roughly 3.7 million rows, which is comfortably into the territory where large models are trainable.
It is an illusion, and it is the most expensive misconception in the field.
Consecutive minute bars are overwhelmingly redundant. What determines whether a strategy generalises is not the number of rows but the number of genuinely independent market environments in the sample. If regimes persist for something like six months, then a fifteen-year dataset contains around thirty of them. You are not training on millions of examples. You are training on a few dozen, with a great deal of correlated noise around each.
This is why models trained on high-frequency data so often perform well until the environment shifts, and then fail all at once rather than gradually. They never saw an environment like the new one, because the sample never contained many environments to begin with.
The practical consequence: model complexity should be governed by the number of independent regimes, not the number of rows. A dataset with thirty effective observations supports a simple model with a handful of parameters. It does not support a deep network with millions.
The target adapts to being modelled
Physics does not change because you have modelled it. Markets do.
If a pattern is discovered and traded at scale, the trading itself removes the pattern. This is not a hypothetical risk; it is the ordinary life cycle of published anomalies across every asset class that has been studied. Any model whose edge derives from a pattern in price history is competing against everyone else who has run a similar search over the same public data, using increasingly similar tools.
The corollary is that decay is the expected outcome, not the exception. A serious deployment plan includes a monitoring rule specifying what level of underperformance, over what window, triggers a reduction or a stop. Deciding this after the drawdown has started is not a decision, it is a reaction.
Costs change which model wins
A model trained and selected without transaction costs in the objective function will reliably favour high-turnover strategies, because at zero cost more trades means more opportunities to be slightly right.
Insert realistic spread, commission, slippage and financing into the training objective, not merely into a final report, and the winning model frequently changes entirely. Strategies that looked marginal become the best available, and the high-frequency winner drops below break-even. Selecting on gross performance and then applying costs at the end is not the same exercise, and it produces a different answer.
Where the technology genuinely earns its place
The uses that consistently work in this market are not the ones that get marketed, because they are unglamorous.
Execution. Deciding how to slice a large order, when to be passive versus aggressive, which venue to route to, how to minimise market impact. The problem is well-posed, the feedback loop is fast, and the improvement is measurable. This is where most serious quantitative resource actually goes.
Text and news processing. Parsing central bank communications, filings and news flow at a scale no human team can match. Useful as an input, not as a decision.
Anomaly and error detection. Spotting bad ticks, stale feeds, position reconciliation breaks, unusual client behaviour. Operationally valuable and almost never discussed.
Risk estimation. Covariance and volatility forecasting, regime classification for position sizing. Modest improvements here compound quietly.
Nowcasting. Estimating current-quarter economic activity from alternative data before official statistics arrive. Legitimate, and expensive to do properly.
Notice the pattern: the durable applications improve the plumbing, the risk management and the information processing around a decision. Very few of them are the decision itself.
Questions that separate the real from the packaged
- Was validation time-ordered, with a gap between train and test?
- Were transaction costs inside the objective function or applied afterwards?
- How many model configurations were tested before this one was selected?
- How many independent market regimes are in the training sample?
- Is the data point-in-time, or does it contain revisions?
- What is the monitoring rule for decay, defined in advance?
- What is the economic reason the edge exists, expressed without reference to the architecture?
That last question is the most useful of the seven. A description that explains what the model is made of, but not why the money should be there, is describing a search procedure rather than an edge. Sometimes a search finds something real. More often it finds the best-looking accident in a very large pile of accidents, and no amount of computing power will tell you which of those two happened.
Trading leveraged products carries a high level of risk and can result in losses that exceed your deposits. Past performance is not a reliable indicator of future results. This article is educational and is not investment advice.