A close friend and professor of mine once told me that the academy will warp your senses, and if you spend too much time there you lose touch with reality. There are many esoteric research topics and subjects in the academy. Often it is difficult to understand how they are at all relevant to society. Contemporary critical analyses of centuries-old philosophers that will never be read by more than a handful of people are published almost daily. And while the first few years studying at the academy pay off handsomely, most students who pursue a PhD will make less money than had they entered the industry. Depending on the subject matter, after a few years there is more money to be had through industry success than academic research.
This is an interesting phenomenon. Most professors become so specialized in a subset of an area of knowledge that they cannot simply leave and enter the industry. Skills (or lack thereof) with people, sales, teamwork, and project execution, can quickly overshadow more detailed research abilities. While there is the rare genius in each field who does contribute to a breakthrough invention, this is far from guaranteed, and does not often offer riches. Biomedical researchers who try to create breakthrough medicine tend to make less money than a family physician. For most of these research-oriented academic types money is far from their true goal. The opportunity to learn and have a high level of comprehension in a specific field can be intoxicating.
The field of financial economics has a particularly strong dichotomy between academia and industry. I have heard traders claim that academia is entirely detached from reality, and I have heard academics claim most of the investments and trading industry is wrong. This is confusing and unsettling for most students of finance or economics. It is difficult to reconcile these two views. In my experience most students defer to what they view as ‘reality.’ For a student who is ultimately interested in working for an investments firm, it isn’t particularly useful to disagree with everything their manager or mentor teaches. I learned that lesson the hard way. Citing academic journals that suggest an entire multi-billion dollar subset of a firm is ‘wrong’ does not win friends in the industry.
The true answer as to why this difference between academics and businesspeople is nuanced and a function of the different approaches to the scientific method that these two groups embrace. This is a more pleasing answer than claiming that either all professors are ‘out of touch’ or all successful hedge funds are ‘lucky and stupid.’ Instead, I will focus on how each group searches for information and approaches what they view as truth.
The scientific method is an incredibly important discovery in the academy. Interestingly, the scientific method cannot be used to prove itself. Our contemporary idea of the scientific method has been refined over centuries. David Hume is a key figure who contributed to the refinement of scientific method in the 18th century. An epistemologist,he wrote in length on logical positivism, which is the human attempt to identify a cause or phenomenon and ‘prove’ that the cause exists. The short answer to this problem is that it is impossible to prove anything, because logic cannot prove its own existence. All this means is that we cannot prove anything with 100% certainty. This is why, to simplify the field, our scientific method uses contemporary statistics to attach significance levels to our observations. In finance, the burden of proof is especially high.
The goal of a financial economist in writing a paper is to formulate a question that can be tested using the scientific method. This requires a strong thesis, an empirical test, and theory that can bring the theory of economics and social sciences together with the results of the empirical test. Once this is completed often other researchers will attempt to replicate the test, create a different test to account for other factors, or prove the test incorrect. If this all is completed and the various conclusions still support the initial thesis, there will often be a level of agreement on a the likely existence of a phenomenon. This process is necessary to come to a strong conclusion. The burden of proof is extremely high. In addition, since the social sciences rarely allow for controlled test environments, it becomes incredibly difficult to prove a theory over space and time.
The field of financial economics began to grow exponentially in the 1960s and 1970s. It was at this time that Eugene Fama put forward his paper on the efficient market hypothesis. This paper created an incredibly robust and enticing framework for understanding how financial markets operate. His central theory is that financial markets are efficient and function by incorporating new information. It suggests that all information that currently exists is already incorporated into the price of a stock and it will only fluctuate due to new, non-forecastable, information. Most financial economists are not stupid. They are aware that some hedge funds continue to make money above and beyond the market. While some academics did remain headstrong that this was only because they had access to insider information or more robust infrastructure, it slowly became accepted that perhaps they did have superior skills in predicting the future. Despite this, the theory of efficient markets was still incredibly impressive. It allowed a generally accepted understanding of how financial markets work as a base assumption. Exceptions had a very large burden of proof. They had to prove that they could, on average, be smarter than everyone else in the market.
Now let’s consider a group of 100 educated traders in the 1970s, all of whom worked at large banks with great infrastructure. Each one attempted to day trade based on macro-economic events. First and foremost, these traders are likely making a lot of money. So even if the theory of efficient markets was entirely true, they wouldn’t just quit their job. However, explaining the difference between traders and academics is more complex.
Consider a financial economist who wanted to put this group of traders to the test. He wants to study if these traders are using strategies that allow them to beat the market, achieving greater returns than should be expected. The first serious issue in any set of data is survivorship bias. This means that the bad traders were likely fired while the good traders kept their jobs and kept trading. As a result any set of data will likely seem better than it is, since the good traders remained in the data set while the bad traders were removed. Secondly, due to the high volatility in financial markets, even if all the traders had no true skill, some would perform very well and make lots of money (and some would fail immediately and be fired). So even a trader who has done very well might be lucky. Even if he is incredibly lucky, in a world with massive amounts of traders some would be expected to perform phenomenally over decades through pure luck. The third note of interest is that there is no way to distinguish between various strategy subsets. While all these traders might be considered ‘macro-economic’ traders, that is an incredibly vague category. A fourth note of interest is that even if a trader seems to beat the market, insider trading violations occur quite often. So if a researcher does find some weak evidence to suggest some traders can beat the market in the long run, he likely cannot prove that it is without the help of insider trading. And to complicate the matter further most traders in high infrastructure environments often receive information that is not illegal, but is better than what others receive. For example, Warren Buffett is able to have meetings with the executives of a firm before he invests.
While those are a few of the primary errors, the empirical issues continue to stack on top of each other. For example, imagine that one trader out of a data sample of thousands discovered a short-term trading strategy that allows some risk-adjusted excess returns for eight months, at which point the strategy stopped working since the overall market caught up with the trader and corrected the inefficiency. Strange little idiosyncrasies such as this would not be captured through research. Another example is a trading strategy called technical analysis. It is so disagreeable to contemporary finance that it is not taught in the academy. In essence it violates market efficiency entirely.
Technical analysis traders search for statistical trends or patterns in the market. The goal is to trade entirely free of any level of market information, theory, or the reality of firms and economies. This is different than quantitative hedge funds, which despite trading on algorithms and programming are built upon the premise of bringing efficiency to the market. A quantitative hedge fund might bid on the difference between convertible debt and a stock, or make mass amounts of arbitrage transactions across markets or on index baskets. A technical trader would instead attempt to predict the future through past information. This violates efficiency since it requires that there be predictable patterns in a market. Up until relatively recently this strategy did not have credibility in financial literature. Now there is evidence that intraday resistance levels might have some merit. The primary theory is that while in the long-run this phenomenon wouldn’t exist, during intraday trading market participants often sell or buy stocks for reasons unrelated to new information. For example if the 501st largest stock in the US shows signs of becoming the 500th largest stock, its price will increase since ETFs and mutual funds tracking the S&P 500 index will need to hold this stock. A technical trader might look at the largest 501-510th stocks and depending on the probability of them passing the threshold, make intraday bets on the firms. This is a quick and simple example I’ve created just for consideration, but it does introduce the possibility that there are short-term fluctuations that are due to institutional cash flows as opposed to true fundamental information. (http://www.ny.frb.org/research/epr/00v06n2/0007osle.pdf)
Before empirical papers did support some areas of technical analysis, there was no true academic proof that it existed. In fact, while some of the earlier literature did not consider very short-term trading, it did not find any proof of the potential for technical analysis in the long-run. As a result it would have been incorrect to say any empirical evidence that followed the scientific method in financial economics provided evidence for technical analysis. In addition, technical analysis has a weak theoretical argument. This is where the dichotomy between traders and academics is most apparent Some traders were making money on technical analysis before the academic body found any evidence it was a true phenomenon. The traders did not have to pass a peer reviewed journal or follow the scientific method. They only needed to make money. The end result seems to be that each group had a salient point. While the traders who followed technical analysis were not without merit, many of them were following particularly awful strategies. For example some theories that look for long-term patterns (such as ‘head and shoulders’) in stock prices fail spectacularly. Conversely, many academics who strongly believe in the efficiency of markets focused too heavily on searching for abstract solutions, and could not accept minor violations of the theory. It was not until a few researchers at the New York Federal Reserve, who are likely more in tune with industry practices, looked into the phenomenon that they found some evidence that technical trading could work in some instances due to institutional discrepancies.
The primary conclusion is that the burden of proof required for academics is much higher than traders. This is logical. When formulating and building knowledge in a field, it is important to demand well articulated proof for causal factors. When trying to make money as a trader, it is important to make money as a trader using cutting edge strategies that others have not yet developed (and have not yet proved). As an example that runs more in the favor of academics, the large majority of investments firms that actively manage mutual funds have lost money after accounting for expenses over the past four decades. The literature on actively managed mutual funds is grim, with the consensus being that the strategy is usually unsuccessful. It now seems as though the evidence has grown large condemning active management, slowly leading institutional and individual investors into more passive ETFs and mutual funds. However, it might still take another decade to see if the industry of mutual funds does move away from active management due to the academic conclusion.
For a young professional or student, it is important to remain open to both sides of the debate. For the first couple years I studied finance I was enamored by the efficient market hypothesis. I thought it was such a beautiful and wonderful theory, and argued against those who thought they could beat the market. I have also had other friends who believed in active trading and technical analysis could lead to easy money. A more reasonable framework demands consideration of both sides. It is important to build a foundation with the academic literature to understand the empirics and theory behind financial markets. However, the academy does not teach students about the financial industry, which is incredibly complicated and often diverges from what is taught in finance courses. It is important to understand where the debate is located. My personal advice is to rely more heavily on the academic literature, while accepting that the world is too large and complex for there to be a research paper that tests every phenomenon from every different perspective. Most importantly though, be skeptical. If you believe there is an inefficiency or predictability in the market you are taking the stand that you see something everyone else is missing.
Thank you for reading, my posts have been less frequent due to a particularly large project I am working on for this blog. In case you were worried.
Side note: Research alluded to available upon request.