Random Thoughts on Predicting Randomness

Awhile back, I pondered the true randomness of a dice throw, the difficulty of generating random numbers, and concluded that while the markets may appear to be random, they are in fact predictable.  Perfectly Reasonable Deviations has some thoughts on the randomness of nature and the universe, and the potential for "predicting everything" with quantum computers.  Indeed, if one could predict everything, one should easily profit from each nuance of the market.  He points out that Einstein had similar beliefs about the predictability of the universe.  Einstein stated "God does not play dice," meaning nothing is random (including dice).  As a case in point, predicting earthquakes apparently just got a little easier.

One has to wonder, though, if like that weird quantum world, the very act of observing (or predicting and profiting from) something alters the outcome, which has important implications for quantum cryptography.  In the famous two slit experiment with photons, the very act of observing the quantum world (photons) changes the result.  Predicting the stock market and acting accordingly also alters the outcome.  It was once hypothesized the quantum world was so weird that photons could "know" in advance they were going to be observed and presumably act accordingly.  This was recently proven false, giving speculators everywhere hope that similarly baffling things like the stock market are not secretly plotting against them and aware of their every move.

As Nobel Prize winner Myron Scholes points out, predictability is transient and alterable:

My belief is that because the system is now more stable, we'll make it less stable through more leverage, more risk taking.

There are some baffling problems that arise, however, if the stock market is predictable and the very act of prediction (and profiting) alters the outcome:

1. Profiting from the prediction game implies that you must not only a) correctly predict the future if you had not acted, but also b) correctly predict the future if you do act.  Luckily, in some cases, a) and b) are essentially the same because your acts have a trivial impact.  For predatory trading systems and firms with a lot of capital, however, they may be quite different.
2. Assuming you can deal with 1), it's probably safe to say others can too.  If others can predict the same thing you are predicting, they are changing the outcome as well.  You are all of a sudden in a predictive "arms race" with them, trying to alter the future before they do.

I don't think any of this is incompatible with my favorite theory of how the market behaves - the Adaptive Market Hypothesis (AMH).  In fact, the AMH implies that there are people who can and do solve problems 1) and 2) at a profit - and there always will be.  With quantum computers now being demonstrated live ahead of achedule (some scientists are skeptical), the potential to predict the universe grows each day.  However, if 2) is indeed true, the one thing that may stop us from predicting things accurately is ourselves.

Comments

[rod.]

Hi Alan!

This is a very interesting topic. I remember a cool .ppt presentation by Doyne Farmer, which addressed the same ideas. I will try to find it and write a post about is as well...

Thanks for your email. I am trying to find the time to respond, because you have good ideas, and I would like to think about them a while before rushing in a dumb reply, eh eh

Great to have you back to blogging!

regards
rod.

[Alan J]

Hi Rod!

Thanks, and it's good to be back. There is no thing such as a dumb reply, just conversation. I will look forward to hearing from you and seeing your post.

Alan

[rod.]

I found Doyne Farmer's presentation (about 15 MB big) at:

http://www.santafe.edu/~jdf/ulamLecture2.ppt

I find it inspiring and really interesting. I don't know whether you are acquainted with Farmer's work, but he's quite a colored character: studied physics, was one of the founders of chaos theory and complex systems, built a shoe computer to beat roulette, started Prediction Company to predict financial markets, then became a researcher at the legendary Santa Fe Institute and started the field of "econophysics". Overall, a maverick. Given that he's aware of the meaning of prediction in physics and other sciences, he offers a rather interesting hollistic perspective on what prediction is, what it means, and what it tells us.

Prediction itself is a concept quite difficult to define. Probably because it means different things to different people. Nobody said that the world is supposed to be easy to grasp, right? eh eh

[Alan J]

Thanks, Rod. This is definately in my "to read" pile.