CASTrader Blog

GStock

I just found out about GStock. It is billed on their website as follows: "GStock is a Virtual Supercomputer formed by many computers calculating a huge amount of investment stratagies with one goal - To pick the best profit making strategy for each stock." 

They continue: "With its humongous computing power, GStock tests over a billion investment strategies PER STOCK. Strategies are tested on years of trading data, and eventually produce the best investment strategy PER STOCK. Notice that it is not one single strategy that is applied to all stocks as done by lame investment tools and stock picking services which try to convince you as if one trading strategy for all stock is sufficient .. as if, technology companies stocks behave like oil companies stocks. Companies, markets, core investors and operations are different - using the same strategy for all stock just doesn't make sense. That is why we find the best strategy that fits like a glove for every single stock. and it works like a charm!"

An interview with the co-founder, Oren Rossen, reveals: "We have a donor who is extremely wealthy and who believes that the next big thing is supercomputing based on grid or distributed computing. He is willing to pore large sums of cash on a project that would both have a technological edge and be commercially viable. However, we don't have a clue what can be done commercially with so much computing power."

A press release says: "A recent test of GStock’s massive computing system over a two and a half year period yielded an average 5.1% return per trade over a 53-day period, with 21,000 of the 30,000 trades yielding profits - a 70% success rate."  This is astonishing if there are no caveats.

{digital}alchemy is skeptical: "I wouldn't put too much stock (pun intended) in the buy and sell signals, but GStock is an interesting idea. It's got a nice user interface so it's a useful site if only for that."  They point out that your can download past trading alerts.

I have no opinion on the merits of this concept one way or another except that it's a cool idea, and needless to say, I'd love to know what these guys are doing, exactly.

Thomas Cover's Universal Portfolio - Part II, Further Reading

In Part I, I briefly discussed Thomas Cover and the Universal Portfolio.  There's lots of detailed reading material on the Universal Portfolio Method, including in Cover's publication list.  Cover has a book on Information Theory that contains a chapter on Information Theory and the stock market (that chapter from the first edition appears to be online here).  Ernest Chan provides an easy to read description.  Further reading:

• Universal Portfolios With and Without Transaction Costs (1997).  For the universal algorithm of Cover (Cover, 1991), we provide a simple analysis which naturally extends to the case of a fixed percentage transaction cost (commission), answering a question raised in (Cover, 1991; Helmbold et al., 1996; Cover and Ordentlich, 1996a; Cover and Ordentlich, 1996b; Ordentlich and Cover, 1996; Cover, 1996). In addition, we present a simple randomized implementation that is significantly faster in practice.
• Can we learn to Beat the Best Stock? (2003).  A novel algorithm for actively trading stocks is presented. While traditional universal algorithms (and technical trading heuristics) attempt to predict winners or trends, our approach relies on predictable statistical relations between all pairs of stocks in the market. Our empirical results on historical markets provide strong evidence that this type of technical trading can “beat the market” and moreover, can beat the best stock in the market. In doing so we utilize a new idea for smoothing critical parameters in the context of expert learning. 
• Universal Portfolios and the Hindsight Bias Issue: A New Perspective?    As this paper involves the application of the Universal Portfolio concepts of Cover ([1][2]) to the Hindsight Bias issue, we begin by introducing each of these individually, beginning with the latter. The Hindsight Bias problem, often also referred to as the ’Data Mining’ problem, occurs when first, many possible models to describe an aspect of the past are posited, and then the bestfitting of these is selected a posteriori (i.e., after the fact). When a model thus selected is then applied as a model for the future, the results tend to be disappointing...
• Efficient Algorithms for Universal Portfolios (2002).  There has been much work on Cover's Universal algorithm, which is competitive with the best constant rebalanced portfolio de- termined in hindsight (Cover, 1991, Helmbold et al, 1998, Blum and Kalai, 1999, Foster and Vohra, 1999, Vovk, 1998, Cover and Ordentlich, 1996a, Cover, 1996c). While this algorithm has good performance guarantees, all known implementations are exponential in the number of stocks, restricting the number of stocks used in experiments (Helmbold et al, 1998, Cover and Ordentlich, 1996a, Ordentlich and Cover, 1996b, Cover, 1996c, Blum and Kalai, 1999). We present an effcient implementation of the Universal algorithm that is based on non-uniform random walks that are rapidly mixing (Applegate and Kannan, 1991, Lovasz and Simonovits, 1992, Frieze and Kannan, 1999). This same implementation also works for nonfinancial applications of the Universal algorithm, such as data compression (Cover, 1996c) and language modeling (Chen et al, 1999).
• A statistical view of universal portfolios (2005)  In this thesis, we provide a statistical view of universal portfolios in order to develop a clearer understanding of their performance on actual financial data sequences. By recasting the analysis of a universal portfolio in statistical terms;with a special emphasis on statistical estimation;we are able to resolve a long standing and false perception of a disconnect between information theory and empirical finance.

There's even a method that claims to outperform the Universal Portfolio:

• Switching Portfolios.  Recently, there has been work on on-line portfolio selection algorithms which are competitive with the best constant rebalanced portfolio determined in hindsight [2, 6, 3]. By their nature, these algorithms employ the assumption that high yield returns can be achieved using a fixed asset allocation strategy. However, stock markets are far from being stationary and in many cases the return of a constant rebalanced portfolio is much smaller than the return of an ad-hoc investment strategy that adapts to changes in the market. In this paper we present an efficient portfolio selection algorithm that is able to track a changing market. We also describe a simple extension of the algorithm for the case of a general transaction cost, including a fixed percentage transaction cost which was recently investigated [1]. We provide a simple analysis of the competitiveness of the algorithm and check its performance on real stock data from the New York Stock Exchange accumulated during a 22-year period.  On this data, our algorithm outperforms all other algorithms mentioned above both with and without transaction costs.
• Nonparametric Kernel-Based Sequential Investment Strategies (2006). The purpose of this paper is to introduce sequential investment strategies that guarantee an optimal rate of growth of the capital, under minimal assumptions on the behavior of the market. The new strategies are analyzed both theoretically and empirically. The theoretical results show that the asymptotic rate of growth matches the optimal one that one could achieve with a full knowledge of the statistical properties of the underlying process generating the market, under the only assumption that the market is stationary and ergodic. The empirical results show that the performance of the proposed investment strategies measured on past NYSE and currency exchange data is solid, and sometimes even spectacular. (See also a related slideshow)

A general slideshow on information theory and investing.

Thomas Cover's Universal Portfolio - Part I, Introduction

Another trading methodology similar to Shannon's Method (aka Shannon's Demon) is the Universal Portfolio method, advocated by Thomas Cover, a professor at Stanford.  Interestingly, he was a Shannon Award winner (the highest award in information theory) along with Elwyn Berlekamp.  Cover's methodology is implemented by Mountain View Analytics, which he is involved with.  The Cover method is probably best described in an old (2000) Stanford news article:

To create a universal portfolio, the investor buys very small amounts of every stock in a market -- no small task in itself. The New York Stock Exchange, for example, lists 3,025 companies. In essence, the universal investor mimics the buy order of a sea of investors using all possible "constant rebalanced" strategies, in which the amount of money invested in each stock is adjusted each day to achieve a fixed proportion.

...

"Imagine we have, for a simple example, two stocks," Cover explains. "A good constant rebalanced portfolio might invest, say, one-fourth in one stock and three-fourths in the other. At the end of the day, the wealth you have in each stock would not be exactly one-fourth, three-fourths because the prices of the stocks change, so you would do the necessary buying and selling to restore it to one-fourth, three-fourths."

Cover's universal portfolio algorithm invests uniformly in all constant rebalanced portfolio strategies. The result is a strategy that is nearly optimal. Cover has shown, for any sequence of stock market outcomes, that this mixture of investments has as high a compound growth rate in the long run as the best constant rebalanced portfolio. Over time, the best strategy (that is, the best constant rebalanced portfolio) fights its way to the top of the fiscal food chain.

...

One aspect of information theory is data compression. "The beauty of it is, the mathematics of growth-rate-optimal investment turns out to be parallel to the mathematics for optimal data compression," Cover says. Thus universal investment algorithms are a counterpart to the universal data compression algorithms used to compress voice, fax and computer files.

...

"This is an automatic investment algorithm in the stock market," Cover says. "The portfolio rides the stocks and lives off the fluctuations. It essentially puts a little bit of money on every possible rebalanced investment algorithm, and the surviving algorithms -- the ones that made most of the money -- make enough so that your money grows at the same rate as if you had used the best algorithm to start with."

The algorithm is "somewhat ponderous," Cover says. "The performance of the algorithm, although good relative to the best portfolio in hindsight, is still slow in responding in an absolute sense. It sometimes requires hundreds of days before the initial conditions wash out, leaving the 'fittest' rebalanced portfolio dominating the performance. It's guiding thinking, but no one's making money off it yet."

The Universal Portfolio has quite a bit in common with the Shannon Method:

• It's a Constantly Rebalanced Portfolio (CRP), meaning it it is an extension of Kelly's optimal long term growth rate formula. 
• The CRP thrives on volatility.  In fact, Thomas Cover coined the term "volatility pumping" to describe the way it makes money.  I think "volatility harvesting" is a good description as well.  In a crude sense, it's turning beta into alpha.

The Universal Portfolio is a lot more complicated than the Shannon Method, but I gather by what Cover says, it is proven to outperform it.

In Part II, I'll list some additional detailed reading on the Universal Portfolio.

Postscript academic paper viewing

I try to keep this blog on topic, but this trick is too good to pass up.  Several key academic finance papers are only available in Adobe postcript as far as I can tell.  Am I the only Windows user that finds it incredibly annoying not to have a built in Postscript viewer in Windows?  Postscript is over 20 years old for Pete's sake.  Google will convert Postscript files to text if they show up in search, but it often mangles the file to the point of being unreadable.  You can buy Adobe Acrobat to view the files, or you can get some open source software to view and print Postscript files, but the easiest method of all in my opinion is to use an online Postscript to Adobe PDF file converter.  A few seconds and you're done, and no software to install.  One wonders why Google can't do this.

Elwyn Berlekamp

Elwyn Berlekamp can probably be considered a Super Quant, given that his purchase of a majority interest in Axcom eventually became the Medallion Fund run by Jim Simons and appears to have developed at least part of the trading strategy there. Berlekamp had Claude Shannon as an advisor, and is known for his work in information theory and combinatorial game theory.  Berlekamp also reviewed William Poundstone's Fortune's Formula for American Scientist (see my review here) and was John Kelly's research assistant.  In the review, he stated:

No one who has made a legitimate fortune in the markets believes the efficient-market hypothesis. And conversely, no one who believes the efficient-market hypothesis has ever made a large fortune investing in the financial markets, unless she began with a moderately large fortune. Of the stories presented in Fortune's Formula, the case of Ed Thorp presents the greatest challenge to the efficient-market hypothesis. Poundstone devotes only a single paragraph to the even stronger cases of Ken Griffin, D. E. Shaw and Jim Simons, presumably because financial wizards as successful as these have always been unwilling to discuss their formulas in public.

According to Wikipedia:

Berlekamp began information-theoretic studies of commodity and financial futures. In 1989, Berlekamp purchased a majority interest in Axcom. After the firm's futures trading algorithms were rewritten, Axcom's flagship fund had a return (in 1990) of 55%, net of all management fees and transaction costs.

According to Berlekamp's home page, the fund had quite a run between 1990 and 2001, delivering what look to be about 46% annual returns (before fees) at a very consistent level after Berlekamp sold out, with continuous tweaks to the algorithms.  Berlekamp states:

This fund has almost no correlation with any of the stock indices. Although the volume of trading is quite high, the volatility is surprisingly low. When the return/risk ratio of this fund is quantified by Sharp ratios, this fund is seen to have achieved an enviable sequence of record-breaking successes.

There's some speculation that Berlekamp's and thus Simon's algorithms may employ the Kelly Formula, but I haven't found anything to confirm.  Since Simons runs what has been called the most successful hedge fund ever, I'm surprised that Poundstone (Fortune's Formula) never at least mentioned the Kelly-Berlekamp-Simons connection.

CASTrader II Architecture

CASTrader I was a single program in VB.NET.  CASTrader II will be a distributed computing system in VB.NET.  It became apparent to me that distributed computing will solve a lot of problems, such as:

• Code complexity - CASTrader I is getting complex and unwieldy.
• Continuous operation - CASTrader II will run 24/7, so some fault tolerance is needed and distributed computing provides a means to do it.
• Modification - CASTrader will receive continuous improvement.  It's much easier to modify a piece at a time rather than a whole system for each little change.
• Power - It's pretty clear to me CASTrader will need to run on more than one computer, because it will eventually consume all the processor cycles it can get.  (I may borrow spare processor cycles from friends, family, and who knows?)
• Modularity - I can add new features (traders) without stopping the system.

I've never made a big distributed system before, but I think I know the basics.  Some challenges will be making the system robust in the event of failure of a program or entire computer.  Another related challenge will be figuring out how fine of pieces to slice the program up into.  Should I make a run parallel dark markets for purposes of robustness?  For CASTrader II, things will likely be simple, because I can add things like that later.  The important thing is to get it up and running and optimize/bulletproof it later.  That said, I'll write the code with these potential additions in mind.

I don't know if VB.NET is the best choice for developing such a system, but it's what I know.  I'm an old FORTRAN programmer who used to program on big iron like the IBM 3090, which was IBM's fastest "super" computer at the time and cost several million dollars.  Somehow, I never learned C/C++, but I'm glad at least some people don't think it's all it's cracked up to be.  From what I understand, VB is about as fast as C++ on the .NET platform.  I'd like to try something off the wall like OCaml, which is supposed to be very fast, but I don't have the time to learn it.  With the distributed programming model, there's no reason I can't add components programmed in many languages later.  Besides, .NET has two excellent features I'll use extensively:

1. .Net Remoting - an easy way to let programs interact, and hopefully it's fast, or my plans are somewhat screwed.  It's an excellent concept, though - you can work directly with the object of another program, so there is no need to write a messaging protocol.  It's made possible by...
2. Object Serialization - Finally, a way to save and restore data easily without writing the I/O code to do it in a language I use.  I'd be hard-pressed to ever switch to a language that does not support this feature.  Since CASTrader will use many objects that need to have their state saved, I don't have to write the custom state-saving code for every @#$%@#& object I create.  It's an ingenious and major time-saving invention.

Since I (finally) learned object oriented programming a few years ago, the only thing that I'm aware of that I'd like to have is multiple inheritance in VB, although there are some workarounds.

It's going to be fun developing CASTrader II, and I'm looking forward to having a distributed computing system working 24/7 to hopefuly extract money from the markets.

How to develop a predatory trading system

In a previous post, I discussed the difference between two types of market trading systems: predator and prey-type.  The key difference between the two types as I defined them is that predatory trading systems induce changes in the market, and prey-type trading systems react to the history of the market.  In other words, prey-type trading systems hope to profit from past price patterns, and predatory trading systems hope to profit from price patterns they create. Prey-type systems hope for infinite liquidity, predatory hope for finite liquidity.  Just like in the real world, both predatory and prey-type trading systems can "win," but predatory type systems, to the extent they exist at all, are harder to develop for the reasons I previously explained.

How would one develop a predatory trading system?  <pure speculation ahead> First, you must have sufficient cash to move the markets and induce the changes in the market.  Second, you would have to develop a theory of the behavior you induce and would need to put real money on the line in a real market to know if your theory works.  Maybe you get your theories from the field of behavioral finance.  You probably can't just dip your toe in, because after all, you must induce changes in the market.  You can't likely test thousands of theories either, you have to test a select few, because this isn't fake money that can be lost.  On the bright side, there's probably less chance of experiencing the pitfalls of datasnooping than with prey-type systems for those developing them.

Third, backtesting, if used at all, probably can't simply rely on the typical price bars, tick and volume data. You would need to have all the market microstructure data.  With such data (see also here), you could make an informed guess with regard to your initial impact on the markets by examining liquidity in the limit order book, but from there on, it's increasingly guesswork.  You would then need to know how the microstructure of the market works and how liquidity reacts to your actions.  Predatory backtests, if used at all, would be based on the assumption of finite, morphing liquidity.  This means that a predatory system must adapt and evolve over time to changing liquidity and price patterns, because if they don't, pretty soon the prey is going to be them.

It doesn't sound easy, but perhaps the rewards for predatory trading systems are rich. 

Traders: Are you a predator or prey?

Is your trading system acting like a predator or prey in the markets?  This is an exploration of two hypothetical trading system types.  This isn't about who wins and who loses, necessarily.  In this thought exercise, both predator and prey can "win."  I'm guessing that chances are, even if your system is a winning one, it is a "prey-type" system.  CASTrader as it stands now, is a prey-type system.

What is a prey-type trading system?  A prey-type system is a system that is developed purely based on historical market patterns and perfected assuming market conditions will not be altered by your actions.  You are reacting to market history, hopefully profiting from the patterns the market hands you.  If you are working with a small amount of capital, you can happily graze on the profits you find.  However, if you overgraze with a bigger bankroll, you may find the very patterns you hoped to profit from are gone.  Either they never existed in the first place (due to datasnooping), or you've eliminated them.  The pitfalls of backtesting are many, but it is the only logical way to test out a prey-type system.  Most testing software I am aware of calculates profits assuming market conditions will not be altered.  A prey-type system generates profits from existing unclaimed price patterns.

Ok, what is a predatory trading system.  A predatory trading system would be a system that aims to profit from market changes that it induces in the market.   On the Great plains of North America, indigenous peoples once guided buffalo herds towards what's known as a buffalo jump.  (At left is the appropriately-named "Head-Smashed-In" buffalo jump - image courtesy of Wikipedia).  In other words, the buffalo were induced off the cliff by the actions of their predators preying on the behavioral biases of the buffalo.  Sane buffalo obviously do not jump off cliffs on their own.  A predatory trading system profits from induced price patterns.  In other words, it makes a market-moving trade that induces sufficient favorable liquidity to get out with a profit.  If the market is a chaotic system, this might be the butterfly effect that you can predict, and thus profit from(Update: PRD has more on Chaos and the financial markets) .  Unlike a prey-type system, a predatory system cannot be backtested, at least in the standard way.  Conventional backtesting systems that I am aware of do not attempt to measure how your system alters the market.  Prey-type backtests are based on the assumption of infinite liquidity, which isn't a bad assumption for small traders.  In fact, a perfect backtest could not possibly be performed on a predator system because you cannot possibly know apriori the exact consequences of your actions.

Who are the predators and who do they prey on, exactly?  Well, to wring out the last drops of cliche from the predator/prey analogy: the weak, oblivious, or simply unlucky prey.  Prey-type systems are numerous and diverse - predators wouldn't capture them all, so some survive, even thrive.  Others simply see their profits vanish, greatly reduced, or worse: go negative.  It's certainly happened before on a grand scale in other markets with disastrous results for the predators.  Naked shorting is predatory, in my opinion.  Predator systems at the day-in/day-out level however, if they exist at all, are probably rare, but well-funded.  It's pure speculation on my part that such systems exist and are profitable in the stock market, as I am not aware of any.   Then again, I don't hang out on Wall Street much.  Perhaps market-makers know, since they have the ability to alter the bid-ask, and I'm pretty sure they are well-aware of the many nuances of what happens when they do while they provide liquidity for profit.  Maybe the Super Quants have used high frequency predatory algorithms.  When a quantitative fund can account for 10% of all the transactions on the NASDAQ in a single day, would you guess they are using prey-type or predator algorithms?  Maybe some traders simply do it in their heads by looking at the order book.  Are you predator or prey?  If you are prey, are you being hunted and are you nimble enough to escape?

Update: Move the Markets offers more perspective in You actually do move the markets.

Randomness, Part III - So, is the stock market random?

In Part I, I discussed how a dice throw isn't as random as you might think, and pondered where randomness really comes from.  In the previous Part II, I looked at why randomness is rare (if it exists at all), and asked if the stock market is truly unpredictable or random. This Part III attempts to answer that.

Well, a proponent of the Efficient Market Hypothesis (EMH) would have you believe the stock market is unpredictable.  Actually, as I understand the theory, any predictability is arbitraged away by people who do it for no reward whatsoever.  This creates a paradox, called the Stiglitz Paradox that I wrote about before:

If a market were informationally efficient, i.e., all relevant information is reflected in market prices, then no single agent would have sufficient incentive to acquire the information on which prices are based.

So if EMH is right, these arbitrageurs do it for free just to tork the rest of us off.  So in theory anyway, the market was once predictable, but no longer.  Don't look for any EMH proponent to pinpoint that point in time where the market crossed over discontinuously from predictability to unpredictability and every arbitrage that ever will be needed to keep the market efficient was invented.  Was it in 1776? 1902? 1929? 1960?  There are no answers that I'm aware of, but it doesn't really matter because most EMH believers think the market is a random walk.  You need not look for patterns because there are none left.  We are before that point in time where the dice throw of the market becomes unpredictable.  Arbitrageurs have manufactured randomness from predictability long ago, completely depleting alpha (probably after the first shares traded hands), and have been confounding us with random price movement beta ever since.  Capital is allocated with perfect efficiency, all the time, and all that's left to show for it is pure unpredictable randomness.

Again, for those left a little puzzled by the logic of the EMH, the Adaptive Market Hypothesis (AMH) comes to the rescue.  In an AMH world, everything has not crossed over fully into randomness (or at least unpredictability) and it probably never will.  AMH doesn't say it is easy to predict, however.  Arbitageurs and savvy investors thankfully get paid excess returns to wring inefficiencies out of the market, but they miss stuff, leaving some scraps for the next person.  This dice throw, in other words, is predictable, if you make an effort at it.  CASTrader is my effort at it.

Elections, politics and the markets weekend linkfest

A Merrill Lynch Analysis finds gridlock good for the market and examines it's impact on sectors (via The Kirk Report).  Even more to the point, CXO Advisory examines a paper that finds, among other things, that "stock index volatility is lower when Congress is not in session," and "annualized returns are therefore 2%-6% higher when Congress is out of session."

Financial Sense has some detailed statistics on elections and market performance.

Barry Ritholtz examines some studies and data.

Automated political risk analysis - "Researchers at Harvard and Ohio State University have been working for the past 6 or so years on an automated Political Crisis Early Warning System that anticipates the worst case scenario --internal armed conflict."

Econoblog looks at how election shakeups affect the economy.

Election prediction markets and how they react to political blunders.

What really wins elections: "An uncanny correlation exists between social mood trends and the success or failure of incumbent politicians, and this relationship doesn't end at The White House. Social mood affects everything from the elections for U.S. Congress and governors' offices to your city council and school board. "

Quant Investor does a pre-election analysis.

CXO Advisory asks "is there a significant positive correlation between company campaign contributions and stock returns?"  Among their conclusions: "The number of candidates a company supports is a significant positive indicator of the company's future stock returns. A one standard deviation increase in the number of candidates supported indicates a 0.5% increase in monthly returns. This effect is larger than the value premium and the size effect, and is about as large as the momentum effect."  Their bottom line conclusion: "In summary, investors might want lean toward companies that contribute to lots of political candidates (especially Democratic candidates for the House of Representatives)."

The Presidential Year Cycle: "In summary, there appears to be some connection between the presidential term cycle and stock market performance, with Year 3 the best and Year 1 the worst."