This started as a question I asked without expecting a complicated answer: if mathematics describes physics so well, why does it seem to describe markets so badly?
I am not qualified to settle that. But I read enough to understand why the question is harder than it looks, and I want to write down what I found while it is still fresh.
Where mathematics clearly helps
Some things about markets are genuinely quantitative, and treating them as such works.
- Diversification. If you hold two assets whose returns are not perfectly correlated, the variance of the combined portfolio is lower than the weighted average of the individual variances. That is not an opinion. It falls directly out of the algebra of variance.
- Expected value. If you can estimate the probabilities and the payoffs, you can compare two decisions on a common scale, even when one of them feels more exciting.
- Position sizing. How much of your capital you put into one idea has a mathematically describable effect on your risk of ruin.
None of this predicts prices. All of it changes decisions.
The random walk idea
One influential model treats price changes as approximately random and independent — a random walk. Tomorrow's change carries no reliable information from today's.
If that were exactly true, then patterns in a price chart would be no more meaningful than shapes in clouds. Our brains are extremely good at finding structure in noise, which is precisely the problem.
I generated random walks in Python and looked at them without labels. Several of them had 'clear trends', 'support levels', and what looked like a head-and-shoulders pattern. They were generated by a random number generator. That experiment did more to change my thinking than anything I read.
If I cannot tell a random series from a real one by eye, then 'it looks like a pattern' is not evidence.
Where the mathematics gets uncomfortable
The convenient version of the model assumes returns are normally distributed. Real returns are not. Extreme days happen far more often than a normal distribution predicts — the tails are fatter.
This matters enormously, because a model that assigns a probability of roughly never to an event that happens once a decade will underestimate risk exactly when it counts. The mathematics is not wrong; the assumption fed into it is.
There is also the reflexivity problem. A model of a planet does not change the planet. A model of a market, used widely enough, changes the market. The thing being measured reacts to the measurement.
My provisional answer
Mathematics helps with structure, not with prophecy. It is very good at telling you how risk combines, what a decision is worth on average, and how confident you are entitled to be. It is much worse at telling you what happens next.
That is a less satisfying answer than I wanted when I started. It is also, I think, the correct one, and I would rather hold a correct uncomfortable answer than a tidy wrong one.
What I want to investigate next
- How much of observed market behaviour survives when you correct for multiple testing and survivorship bias.
- Whether the fat tails are better described by a specific distribution or simply by admitting we do not know.
- What practitioners actually do with these models, as opposed to what textbooks say they do.