CASTrader I, The good, the bad, and the ugly - Part I, the good

The first incarnation of CASTrader is complete, and the results are in, and like a good spaghetti western, they can be divided into the good (this post), the bad (Part II) and the ugly (Part III).  I'll discuss the improvements required for CASTrader II in Part IV.  Now, on to the good:

Click the graph for an annotated trade simulation of CASTrader vs. the Dow Jones from 1915 to 2006.  I can only characterize the overall performance as very good, despite a long list of caveats I will discuss in Part II and the ugly period I'll discuss in Part III.  Three simulations are plotted.  The two dark lines start just before the crash of 1929, and the third (yellow) starts in 1915.  The purple line is the Dow Jones Industrial Average.  Each simulation starts with 1000 randomly created traders.  Many of the annotations on the graph speak for themselves, but I'll run through the history of the third (yellow) simulation:

• 1915-1920.  This period is characterized by full diversity.  Traders completely disagree with one another, and there is essentially no net trade.
• 1920-1929.  A net (mostly long) long trade emerges from the aggregate of traders and keeps up with the market, even though it is only partially exposed to the market.
• 1929-1934.  Good performance and adaptation to some of the largest market gyrations ever.
• 1934-1967.  CASTrader keeps up with the market with low exposure to it.  It could use some improvement, but it's encouraging.  The performance here is probably actually better than it looks due to improper accounting of interest rates (more on this later).
• 1967-1983.  The ugly - more on this in Part III.
• 1983-2006.  The absurd.  CASTrader starts beating the market at an astonishing apparent rate, primarily due to some elite traders finally surpassing all their brethren and dominating the trading with leveraged trades that seem to be amazingly accurate.  Be sure to read Part II to see why this performance is not to be trusted.  Despite those substantial caveats, the elite traders appear to beat the market with amazingly low drawdown and volatility, something that flies in the face of the Efficient Market Hypothesis, yet fully consistent with the Adaptive Market Hypothesis.  The elite trader starts with 1/1000th of the total capital, and typically ends up with 50-70% of it.

Now for a summary of the good points:

• CASTrader is finding patterns.  Despite the caveats I'll discuss in Part II, there is no doubt that CASTrader is finding regular patterns.  The pattern finding algorithm is unconventional, and to my knowledge is not used by any other trading method or published research.  The algorithm even performed well on predicting "random" stock returns generated by the conventional (not cryptographic) VB.Net random number generator. Given the difficulties of generating truly random (unpredictable) numbers (numbers may not need to be random to be unpredictable), it is absurd to think that a collection of humanity with behavioral biases will produce a market characterized by a random walk, or even an unpredictable one.  (Aside: cryptography requires the generation of true random numbers (or at least unpredictable ones) to encrypt information, and people go to great lengths to try to generate them - many think they cannot be generated on classical computers, but others disagree.  In a sense cracking the market is like cracking a cipher - trying to find patterns in seeming random numbers) There are patterns to be found, and CASTrader is capable of finding some of them.  It's impossible to even describe in a meaningful way what the patterns are.  Examining the net trades of CASTrader provides no clues - the minute you think it's going with the momentum, it goes contrarian and vice-versa.  Sometimes it increases the stakes after losing, and sometimes it scales way back, and ditto for winning  Other than checking that the returns are not due to a programming error, I have not delved into what it is finding or what it has "learned," because it would be an exercise in futility.
• Patterns seem to be growing.  Whatever these patterns are, there is no question that they are more prevalent in the recent past than the distant past, even after discounting the emergence effects observed in CASTrader.  Despite the recent rise of the Super Quants, and laments about the lack of alpha, these patterns persist, and even grow.  Why?  I have no idea other than the fact that more people are trading, derivatives have exploded, and perhaps things are accelerating, creating opportunities.  It's counterintuitive that these things would lead to less randomness, but it's fully consistent with the Adaptive Market Hypothesis, in my opinion.
• Performance from 1915 to 1983 may be better than it looks.  Most of this period was characterized by little or no aggregate leverage and only partial exposure to the market.  Furthermore, the traders do not earn interest on the cash they are typically sitting on.  More importantly, though, the period 1920 to 1983 is characterized by a period of partial diversity.  Since the traders are sitting on cash a lot, there is opportunity to reap the benefits of partial diversity by adding more diverse traders or even trading other, less correlated markets in tandem with the Dow.
• CASTrader did this on price alone.  CASTrader I only uses price history as input.  This leaves plenty of room to improve performance by adding other exploitable information such as volume, economic data, interest rates, fundamentals, news, individual stock selection, etc.
• No data snooping.  No traders or algorithms were ever discarded and then carried forward to an "out-of-sample" test.  The whole test is "out-of-sample."  It's sink or swim for CASTrader from the beginning.  Furthermore, none of the algorithms were selected from a pool of algorithms known to do well in the markets - they are all completely novel and random.  There is an equal likelihood that a trader will be buy-and-hold or always short the market than some other kind of novel pattern trader/detector.  That said, it takes CASTrader 5-6 years of data to begin to "emerge."  In effect, CASTrader is creating it's own learning/observation period before creating a net trade and going live - a good, natural thing unrelated to data snooping in my opinion.
• CASTrader did it without evolution.  Although there is emergence in CASTrader, there is no evolution involved to produce it, because I haven't gotten around to coding it yet.  Evolution can potentially improve results even more by allowing the elite traders to "mate" and produce evolved offspring with even better abilities.  Right now, they are stuck with the basic properties they had in 1915 and must survive intact trading the market until 2006.  It will be fascinating to see how much evolution can improve the learning and hence results, but  I won't be totally disappointed if it shows mild or no improvement.
• CASTrader did this on the Dow.  One would think that the Dow would be one of the toughest places to find patterns in, but CASTrader did it.  Most research consistently finds higher alpha in smaller caps, leading me to believe CASTrader would as well.  That CASTrader finds patterns in the Dow is encouraging.

Despite all the good, there is a lot of bad (Part II) and even ugly (Part III) to temper the enthusiasm, because much of the gains may be illusory.  CASTrader is definately not ready to go live with my full bankroll, despite the apparent good.  It's just not worth the risk.  In Part IV, I'll discuss what can be done.  With any luck, CASTrader II will fit the bill to go live with a full bankroll.  Unfortunately, it may be months away.  Between now and then, I may take CASTrader live with a very small bankroll I can afford to lose to see how it performs on paper vs. real money.