CASTrader Blog

Super Quants Update

Awhile back, I profiled a few of the Super Quants.  I've collected a few articles on what they're up to lately.

James Simons/Renaissance. Simon's firm is hiring a programmer for a starting salary of $125K-$250K, even as "tantalizing tidbits" emerge from a lawsuit against former employees.  They are also launching a $25B managed futures fund, which Mebane Faber deconstructs in superb detail.  Some wonder if Simons is running out of alpha in other funds, but still manages to leap from #26 to #6 in assets. (Peak alpha of $30 billion, anyone?).  Some wonder if Simons, Shaw and others are just glorified market makers.  Meanwhile, Simons donates to the Hillary campaign and puts some cash in a buyout fund.  Simons says he was lucky and offers inspiration and a lot of tidbits on how he did it (from the article, I infer that Simons basically started in his late 30s, giving hope to old quants everywhere). Before they were Super Quants: Me and Jim Simons - groovy, but no comment.

D.E. Shaw.  Shaw aquires insurer James River, will invest $1B in India (more here), and contemplates a private equity fund (read the article for good commentary and why this may be the trend of the future), while Lehman aquires a 20% stake in Shaw, and Shaw places #4 in assets.

Ken Griffin/Citadel.  Grave dancer or white knight? (Citadel buys Sowood assets - more here)  Griffin: aiming to be a titan.  In Will a Hedge Fund become the next Goldman Sachs (meanwhile luring employees from them), lots of detail on Griffin are provided, including how Citadel has become the leading options market maker, bought Amaranth assets, and is "arrogant, unapologetic and extremely smart."  More on Citadel's plans here, with indications they are planning to go public.  Meanwhile, he opposes new taxes on himself (a statement for which he was skewered by Bill Gross), and maxes out donations to Obama.

Meanwhile Elwyn Berlekamp solves a problem about hats.

Machines in the Market Linkfest & Blogs

Day Trading on Steroids: He's seated amid 24 computer screens, each of which flashes hundreds of numbers. Vos points out the one that matters: a line chart that shows he's made more than $5,000 after rapidly buying and selling millions of shares of stock. ... "I saw the world of computers and trading converging," Cummings said. "So the idea was really to just create a robot that traded like a pit trader." ... Cummings doesn't think his idea is unique. Instead, he was uniquely positioned to execute it -- he could code software with a trader's mind. ...Rizzello said he thinks Tradebot is one of the Nasdaq's top 10 traders, by volume.

Alea, a new to me blog, is all over Tradebots, and it's successor, BATS.  More from Alea:

Market Effects of Autodealing
High Frequency Automated Trading
Algorithmic Arms Race
Algorithmic Arms Race [2]

Chips with everything for traders:  This particular program, he says, is actually a modified version of a previous one he wrote for the Norwegian ministry of defence that could be used to track down nuclear submarines in the murky waters of the fjords...."One has some notion of what kind of sounds a sub can make and we try to spot this sound among an ensemble of signals," he says. "This works on the same principle - it is very much like eavesdropping on one specific conversation in a noisy crowded room. But I'm not going to tell you any more than that."...Computerised trading of one kind or another now dominates London's financial markets. At least 50 per cent of the trades made in FTSE100 stocks originate from so-called "black-box" trading systems, or trading algorithms. Some investment banks put the figure nearer 60 per cent.

The smartest (or nuttiest) futurist on Earth: Kurzweil says he is applying Simons-like quantitative analysis to take advantage of market inefficiencies. And he's confident that, just as he trained computers to recognize patterns in human speech or the sound of a violin, he can do the same with currency fluctuations and stock-ownership trends. The ultimate goal is to create the first fully artificially intelligent quant fund - a black box that can learn to monitor itself and adjust. Although he started the company back in 1999, the fund has only been trading for about a year....How's he doing? Kurzweil won't say, citing SEC rules, nor will his investors. "I view Ray as one of the best pattern-recognition people in the world," says Khosla, when asked why he put money into FatKat. "I am a happy investor in Ray's company. A very happy investor."

Blogs with a quant/machine trading focus:

Tales from a Trading Desk

Dinosaur Technology and Trading

Magmasystems Blog

Coding the Markets

Chris Donnan: Programming - Brooklyn Style (you gotta love that title)

Last, but not least, another new to me blog, Zero Beta, takes a skeptical view of machines in the market:  "By handing over power and money to machines left unchecked we may be setting ourselves up for an ugly scenario.  What if they are THAT good?  What if they can learn? A mild result would be a lack of interest in the financial market system from the ordinary investor.  If expected returns go negative – much like a casino, you will see much less investment and much more hording of money.  There will always be the gamblers but those with the best machine will always win – I believe they are called the house at the Borgata."

Update:  C++ Trader links a Bloomerg article on AI and trading and has some thoughts.

Computer-guided stock trading reaches levels undreamed of in sci-fi: "In his cubicle overlooking the trading floor, Kearns, 44, consults with Lehman Brothers traders as Ph.D.s tap away at secret software. The programs they’re writing are designed to sift through billions of trades and spot subtle patterns in world markets.  Kearns, a computer scientist who has a doctorate from Harvard University, says the code is part of a dream he’s been chasing for more than two decades: to imbue computers with artificial intelligence, or AI."

E*Trade Automation, Taxes, and the Pursuit of Happiness

<Rant>So I've been struggling with the 67,000 page US tax code, spending my fair share of the estimated 6 billion hours lost to that colossally complicated waste of time.  This year, I figure the economic value to the economy that was lost by my having to comply with the beast is equal to 50% of my taxes paid.  That is to say that the value of the time I spent on taxes and required record keeping, rather than more productive endeavors is equal to 50% of the money I send to Uncle Sam.  I would gladly pay, say 20% more tax this year if I could just fill out a postcard (PDF), because it would leave me free to pursue other things like life, liberty, and the pursuit of happiness, which the good old USA is supposed to be about.  Uncle Sam threw away the memo that simplicity is an ingredient of happiness. I long for the simple days when my filing system could be this easy.   Other countries have adopted a simple tax and brought in more revenue.  The reason why is clear to me;  I have no idea why this is a partisan issue.

Speaking of record keeping and happiness, keeping records as a trader is not fun.  Basically, to comply with the tax code, some people recommend you keep trade confirmations forever.   I admit that I'm completely ignorant of whether this situation would be simplified if I elected trader status - that's one part of the tax code I have not explored yet.  I elected an electronic records option with E*Trade long ago to avoid the mountain of dead trees.  Unfortunately, though, I discovered E*Trade is deleting my records older than 7 years, so now I must download them to protect them and comply with record keeping requirements.  E*Trade believes I should do this one at a time, which would cost me an estimated 24 hours of my life - or more.  The way their website functions, it's almost as if it's intended to be that way.  They save a lot of money by not mailing confirmations, but they eliminate (almost) every known method of downloading these things in bulk.  A telephone call to customer service confirmed that it is impossible to get these things in bulk but downloading every trade confirmation "was very good advice."

I have recently rediscovered, however, the joys of automation.  In a couple of hours, I cobbled together a total kludge of a program in VB.NET to automated E*TRADE in my browser.  Basically, the program sends keystrokes (via the trusty SendKeys command) to Mozilla Firefox, simulating button presses on the E*TRADE website, telling Firefox to download and print each confirmation to Microsoft Document Imaging Format (MDI) rather than the PDF.  GetMainWindowTitle allows a check of the title bar of Firefox to ensure things are going as expected on the E*TRADE website.  A FileSystemWatcher makes sure the file was downloaded and even gets the name. This even allows the program to wait for E*TRADE to respond - it's pretty slow now.  I can even screen scrape the website to match the trade confirmations to the more descriptive links on E*Trade.  I was unfamiliar with MDI format, but just happened to try it out for this purpose.  I was pleasantly surprised to find that I can even screen scrape the trade confirmations (convert the trade confirmation images back to text) using Microsoft Document Imaging should I so desire!  I may do this to verify the downloads and create an index to them, with amounts, shares and symbols.

I've eaten breakfast, showered, and typed this blog post while the program has downloaded some 250 confirmations, and I've reached the break-even point of my efforts (hours coding vs. hours I would have spent) with only a few easily fixed glitches so far.    As far as advice for rookie programmers, number one on my list would be learn how to automate, and use it whenever you can.  For me, using programming to get around a fubar system or save myself a ton of tedious work always puts a big smile on my face.

It even occured to me that I could automate CASTrader trading via the same technique with a few hours of code and some very detailed bulletproofing (screen scraping each screen to verify the expected response).  This isn't a fail-safe programming technique without some very careful programming - basically, I'd probably have to babysit it.  Or, I could just switch to the Interactive Brokers API.  I wonder if they allow automatic downloads of electronic trade confirmations?</rant>

Update:  an excellent article with some discussion on screen scraping, it's drawbacks, and alternatives.

The two stock market machines

Benjamin Graham said, in perhaps his most famous words:

"In the short run, the market is a voting machine but in the long run it is a weighing machine."

Those words are true genius in the way they concisely define the way the financial markets operate.  The voting machine is behavioral finance, high frequency finance, money flows, demographic trends, statistical arbitrage and trading on the news, among other things, all rolled up into one.  The weighing machine is the pure essence of the reason financial markets exist: to accurately assay the worth of financial securities, or in a nutshell, to assess the present worth of discounted future cash flows.  There is a waiting queue for the weighing machine, but eventually, everything is assessed it's true worth.

Graham's understudy Warren Buffett built a fortune by mastering a focus on the weighing machine part of the market, accurately weighing assets prior to their turn at the weighing machine.     Graham and Buffett have largely ignored the voting machine part, except where it presented them with golden opportunities to rearrange their holdings.  Buffett is the undisputed master of the weighing machine.  The Super Quants, on the other hand, tend to pay a lot more or sometimes all of their attention to making money off the voting machine part of the market.  By all accounts, many of the Super Quants produce returns rivaling Buffett, albeit over a shorter history.  Bringing up the rear are the Efficient Market Hypothesis believers seem to think that there is no voting machine (other than a random walk) and the weighing machine is 100% functional, a concept which Buffett has long thought absurd, and no doubt the Super Quants do as well.

CASTrader development thus far has been squarely focused on the voting machine aspect.  It's not that I don't believe in the Graham and Warren Buffett way - quite the contrary.  Most of my investing successes so far can be attributed directly to Buffett's teachings.  More importantly, some of my investing blunders can be attributed to not heeding his teachings. The bulk of the rest of investment successes can be attributed to variations on the Shannon Method, perhaps the simplest way imaginable to milk some profit from the voting machine.  The Buffett stockpicking method is conceptually easy, but implementation is harder, in my opinion.  One of the hardest parts is the patience involved in finding suitable investment candidates.  Quantitative Buffett stock screeners I have seen pick many stocks Buffett wouldn't touch and often discard those he would, in my opinion, but they aren't a bad place to start.  There's quite a lot of judgement involved that simply cannot be replicated by a machine - at least not yet or anytime soon.  Despite what Buffett says, even very few humans ever achieve the same success by emulating him, though many try very hard.  Chalk it up to Buffett's personality, perhaps (note: contrary to the article, many think Buffett is ISFJ).

Anyone trying to make money off the voting machine should be well-versed in the concepts of the weighing machine, in my opinion.  To the extent I can, I want to program Buffett into CASTrader.

GStock

I just found out about GStock. It is billed on their website as follows: "GStock is a Virtual Supercomputer formed by many computers calculating a huge amount of investment stratagies with one goal - To pick the best profit making strategy for each stock." 

They continue: "With its humongous computing power, GStock tests over a billion investment strategies PER STOCK. Strategies are tested on years of trading data, and eventually produce the best investment strategy PER STOCK. Notice that it is not one single strategy that is applied to all stocks as done by lame investment tools and stock picking services which try to convince you as if one trading strategy for all stock is sufficient .. as if, technology companies stocks behave like oil companies stocks. Companies, markets, core investors and operations are different - using the same strategy for all stock just doesn't make sense. That is why we find the best strategy that fits like a glove for every single stock. and it works like a charm!"

An interview with the co-founder, Oren Rossen, reveals: "We have a donor who is extremely wealthy and who believes that the next big thing is supercomputing based on grid or distributed computing. He is willing to pore large sums of cash on a project that would both have a technological edge and be commercially viable. However, we don't have a clue what can be done commercially with so much computing power."

A press release says: "A recent test of GStock’s massive computing system over a two and a half year period yielded an average 5.1% return per trade over a 53-day period, with 21,000 of the 30,000 trades yielding profits - a 70% success rate."  This is astonishing if there are no caveats.

{digital}alchemy is skeptical: "I wouldn't put too much stock (pun intended) in the buy and sell signals, but GStock is an interesting idea. It's got a nice user interface so it's a useful site if only for that."  They point out that your can download past trading alerts.

I have no opinion on the merits of this concept one way or another except that it's a cool idea, and needless to say, I'd love to know what these guys are doing, exactly.

Thomas Cover's Universal Portfolio - Part I, Introduction

Another trading methodology similar to Shannon's Method (aka Shannon's Demon) is the Universal Portfolio method, advocated by Thomas Cover, a professor at Stanford.  Interestingly, he was a Shannon Award winner (the highest award in information theory) along with Elwyn Berlekamp.  Cover's methodology is implemented by Mountain View Analytics, which he is involved with.  The Cover method is probably best described in an old (2000) Stanford news article:

To create a universal portfolio, the investor buys very small amounts of every stock in a market -- no small task in itself. The New York Stock Exchange, for example, lists 3,025 companies. In essence, the universal investor mimics the buy order of a sea of investors using all possible "constant rebalanced" strategies, in which the amount of money invested in each stock is adjusted each day to achieve a fixed proportion.

...

"Imagine we have, for a simple example, two stocks," Cover explains. "A good constant rebalanced portfolio might invest, say, one-fourth in one stock and three-fourths in the other. At the end of the day, the wealth you have in each stock would not be exactly one-fourth, three-fourths because the prices of the stocks change, so you would do the necessary buying and selling to restore it to one-fourth, three-fourths."

Cover's universal portfolio algorithm invests uniformly in all constant rebalanced portfolio strategies. The result is a strategy that is nearly optimal. Cover has shown, for any sequence of stock market outcomes, that this mixture of investments has as high a compound growth rate in the long run as the best constant rebalanced portfolio. Over time, the best strategy (that is, the best constant rebalanced portfolio) fights its way to the top of the fiscal food chain.

...

One aspect of information theory is data compression. "The beauty of it is, the mathematics of growth-rate-optimal investment turns out to be parallel to the mathematics for optimal data compression," Cover says. Thus universal investment algorithms are a counterpart to the universal data compression algorithms used to compress voice, fax and computer files.

...

"This is an automatic investment algorithm in the stock market," Cover says. "The portfolio rides the stocks and lives off the fluctuations. It essentially puts a little bit of money on every possible rebalanced investment algorithm, and the surviving algorithms -- the ones that made most of the money -- make enough so that your money grows at the same rate as if you had used the best algorithm to start with."

The algorithm is "somewhat ponderous," Cover says. "The performance of the algorithm, although good relative to the best portfolio in hindsight, is still slow in responding in an absolute sense. It sometimes requires hundreds of days before the initial conditions wash out, leaving the 'fittest' rebalanced portfolio dominating the performance. It's guiding thinking, but no one's making money off it yet."

The Universal Portfolio has quite a bit in common with the Shannon Method:

• It's a Constantly Rebalanced Portfolio (CRP), meaning it it is an extension of Kelly's optimal long term growth rate formula. 
• The CRP thrives on volatility.  In fact, Thomas Cover coined the term "volatility pumping" to describe the way it makes money.  I think "volatility harvesting" is a good description as well.  In a crude sense, it's turning beta into alpha.

The Universal Portfolio is a lot more complicated than the Shannon Method, but I gather by what Cover says, it is proven to outperform it.

In Part II, I'll list some additional detailed reading on the Universal Portfolio.

Elwyn Berlekamp

Elwyn Berlekamp can probably be considered a Super Quant, given that his purchase of a majority interest in Axcom eventually became the Medallion Fund run by Jim Simons and appears to have developed at least part of the trading strategy there. Berlekamp had Claude Shannon as an advisor, and is known for his work in information theory and combinatorial game theory.  Berlekamp also reviewed William Poundstone's Fortune's Formula for American Scientist (see my review here) and was John Kelly's research assistant.  In the review, he stated:

No one who has made a legitimate fortune in the markets believes the efficient-market hypothesis. And conversely, no one who believes the efficient-market hypothesis has ever made a large fortune investing in the financial markets, unless she began with a moderately large fortune. Of the stories presented in Fortune's Formula, the case of Ed Thorp presents the greatest challenge to the efficient-market hypothesis. Poundstone devotes only a single paragraph to the even stronger cases of Ken Griffin, D. E. Shaw and Jim Simons, presumably because financial wizards as successful as these have always been unwilling to discuss their formulas in public.

According to Wikipedia:

Berlekamp began information-theoretic studies of commodity and financial futures. In 1989, Berlekamp purchased a majority interest in Axcom. After the firm's futures trading algorithms were rewritten, Axcom's flagship fund had a return (in 1990) of 55%, net of all management fees and transaction costs.

According to Berlekamp's home page, the fund had quite a run between 1990 and 2001, delivering what look to be about 46% annual returns (before fees) at a very consistent level after Berlekamp sold out, with continuous tweaks to the algorithms.  Berlekamp states:

This fund has almost no correlation with any of the stock indices. Although the volume of trading is quite high, the volatility is surprisingly low. When the return/risk ratio of this fund is quantified by Sharp ratios, this fund is seen to have achieved an enviable sequence of record-breaking successes.

There's some speculation that Berlekamp's and thus Simon's algorithms may employ the Kelly Formula, but I haven't found anything to confirm.  Since Simons runs what has been called the most successful hedge fund ever, I'm surprised that Poundstone (Fortune's Formula) never at least mentioned the Kelly-Berlekamp-Simons connection.

Machine learning and machines in the market linkfest

A Markov Chain Analysis of Blackjack Strategy.

Naive Bayes classifier vs. logistic regression

Machine learning videos

Portable alpha - rise of the machine:

Automated algorithm-based investment strategies have been gaining ground with institutional and high-net-worth investors over the past few months. But do they pose a real threat to traditional fund managers? Gareth Gore reports

What if you built a machine to predict hit movies?

Four years ago, when McCready was working with a similar version of the program at a firm in Barcelona, he ran thirty just-released albums, chosen at random, through his system. One stood out. The computer said that nine of the fourteen songs on the album had clear hit potential—which was unheard of. Nobody in his group knew much about the artist or had even listened to the record before, but the numbers said the album was going to be big, and McCready and his crew were of the belief that numbers do not lie. “Right around that time, a local newspaper came by and asked us what we were doing,” McCready said. “We explained the hit-prediction thing, and that we were really turned on to a record by this artist called Norah Jones.” The record was “Come Away with Me.” It went on to sell twenty million copies and win eight Grammy awards.

Soldier of fortunes:

I scarcely dare ask, but is the computer in the building? "Yes," he says. "It's a big one." He leads me through a locked door into what I can best describe as a large, noisy cupboard. Banks of superpowered servers are churning through calculations of unimaginable complexity. Schroeder and I can hardly hear each other above the whirring of the fans trying to keep them cool. On average, this machine buys and sells $10m every hour. On the floor is a row of plugs - just ordinary plugs. I keep far away from them.

The Pope issues a warning on artificial intelligence:

Pope Benedict told scientists on Saturday that by believing only in "artificial intelligence" and technology they risked the fate of the mythical Icarus, whose home-made wings melted when he flew too close to the sun.

"Contemporary life gives pride of place to an artificial intelligence ever more enslaved to experimental tecnhiques, thereby forgetting that all science should safeguard mankind and promote his tendency to authentic goodness," the Pope said.

Emergence is coupled to scope, not level

Is the Increased Speed of Direct Market Data Feeds Worth the Price?

So for a simple, albeit not light-speed fast, aggregated data feed, you now need ticker plants, feed handlers, normalizers, stream processors and time-series databases. Oy. And by the way, none of this is cheap. Including buying the data and transporting it (which I didn't even mention), you are looking at each feed costing upwards of six figures. Multiply that by a dozen and, along with the expanding number of markets, suddenly you are talking real money.

The Super Quants

Perfectly Reasonable Deviations highlighted an interesting, if old article on D.E. Shaw, so I thought I'd profile three of the super quants of Wall Street for a little inspiration, including Shaw.

Dubbed King Quant by Fortune, Shaw is a billionaire, and made the Forbes 400 list in 2006.  He created a company called D.E. Shaw dubbed by Fortune as  "the most intriguing and mysterious force on Wall Street."  The company employed Jeff Bezos before he founded Amazon, now employs 1000, and supposedly rejects 1 in 500 job applicants.  It is very secretive, but is known to employ statistical arbitrage and more recently, some good old bargain-hunting methods.  Interestingly, he is the son of an efficient markets pioneer (Update: Wikipedia may be wrong, as it may have been his grandfather, not his father - see comments).  His firm, like LTCM had some problems in 1998, losing a bundle ($375 million writedown for BankAmerica, according to Wikipedia). He implied fairly recently that the growth of hedge funds is making alpha harder to find. David Shaw is apparently no longer involved in the day-today management of money, rather he is now  working on algorithms to fight the war on cancer and other diseases.  DE Shaw now manages $23 billion in assets and is making a push into traditional asset management.

Jim Simons is a true math nerd, being a cryptanalyst, mathematical physicist, and even has his own mathematical theory which has use in string theory, a mind-blowing theory of everything. He is ranked as the 64th richest person in America at $4.5 Billion.  He founded the secretive Renaissance Technologies (which was once online, but apparently no longer) that according to Wikipedia:

Renaissance uses computer-based models to predict price changes in easily-traded financial instruments. These models are based on analyzing as much data as can be gathered, then looking for non-random movements to make predictions.

Renaissance employs more than 60 top scientific specialists, including mathematicians, physicists, astrophysicists and statisticians, from countries as diverse as Japan and Cuba ^[3]who review market data to find statistical relationships that predict the price movements of commodities, currencies and stocks. Its $5 billion Medallion Fund has averaged 35% annual returns, after fees, since 1989, and is considered in the industry to be the most successful hedge fund,^[4] yielding returns ten percentage points higher than investors Bruce Kovner, George Soros, Paul Tudor Jones, Louis Bacon, Mark Kingdon or Monroe Trout.

More description of their methodologies is here. He wants to launch a fund that could handle $100 Billion, giving hope that alpha is still to be found in mass quantities.

Ken Griffin is the youngster of the group and started trading in his dorm room at Harvard and eventually started Citadel Investment Group, with $12 Billion under management in 2005.  According to this Bloomberg article:

Citadel employs 72 Ph.D.s, including former mathematics professors and astrophysicists. They are the heart of the firm's Quantitative Research Group, which develops proprietary mathematical models to support traders. They staff so-called Ph.D. Row, the south side of the 36th floor, an area dominated by erasable floor-to-ceiling white boards full of complex math formulas.

...

As more money pours into the industry, performance has dropped, and Citadel is no exception. While Citadel's annual returns, which have averaged 15 percent net of fees since the beginning of 2001, are among the best compared with his firm's peers, they're half the 31 percent Griffin produced from 1991 to 2000. His offshore fund returned about 2 percent in the first quarter.

Citadel was one of the rare buyers in the LTCM crisis, returning 35% that year.  More from the article:

Citadel's four meteorologists sit in front of imaging and mapping computers in an alcove off the trading floor. On a typical day, they might analyze snowpack and rainfall density in the U.S. Northwest to see how a recent increase in precipitation in the region will provide a boost to hydropower producers and, in turn, cut demand for other types of electricity producers.

Citadel has also started dabbling in pollution rights and catastrophe, or ``cat,'' bonds. Reinsurers, such as Swiss Reinsurance, issue these securities to hedge some of the risk of paying claims in catastrophic losses. The bonds pay high interest rates, though investors may lose their principal and interest payments if a storm generates losses at or above a set amount.

Griffin isn't afraid to go into areas untested by other hedge funds. The firm now has a 100-person team that makes markets in equity options for retail brokers such as Ameritrade Holding Corp. and E*Trade Financial Corp.

...

Citadel plans to start making markets in equities in the next few months, says Matthew Andresen, former CEO of electronic trading network Island ECN Inc., who was hired last year to head up the market-making business.

Citadel apparently gives all new hires two books: Hardball: Are You Playing to Play or Playing to Win? and Good to Great.

Update: Much more in another Perfectly Reasonable Deviations post.

Update: Elwyn Berlekamp and Thomas Cover, a possible members of the Super Quant club.

Update: Why are information theorists so successful in the markets? and Super Quants Update.