Skip to content
January 25, 2016 / schoolthought

Bubble? Seattle Housing Prices

It’s common knowledge that Seattle housing prices are increasing. I frequently hear that Seattle prices are currently in a bubble, and buyers should hold off. I also hear that houses are selling very quickly and for cash. On the other hand, it’s folk wisdom that demand is being driven by the tech sector. So if that’s true, then maybe it’s not a bubble. Or, maybe house prices are going up for good reason, but it’s also a bubble, but no one knows when the bubble will pop.

Any good model needs to have criteria for acceptance and falsifiability. A bubble is a nebulous term in finance, which appears to be a real phenomena, but is also extremely hard to measure. Robert Shiller’s careful econometric study of mean reversion in equity prices and valuations made a strong claim that there is some predictability in asset markets (when detrending growth and inflation). He attributes this in large part to ‘irrational exuberance,’ and the psychology of investors. This is hard to measure. When a new home buyer in Seattle is going out today to buy a home, is it based on rational expectations? Or is it because everyone else is buying homes, and he wants to get in before it’s too late? Shiller measured this as well, and found that people’s expectations of housing price increases (last in 2003) were much more optimistic than reality.

What is extremely interesting, is when controlling for inflation Shiller finds that based on his housing prices index, prices don’t increase that much. Since 1990 housing prices have only increased about 28% in real terms, which is orders of magnitude less than equity markets. Real estate though has a duality, as it generates wealth over time, either in the form of rental or consumption. Seattle is also often compared to San Francisco, and it’s true that the real-estate market has increased massively in San Francisco due to the tech boom. However, San Francisco is also surrounded by water and has extremely restrictive construction regulation both in the city and down near the Menlo Park region. In the short-run all cities are static, but Seattle can continue to expand, build, and create new neighborhoods.  

Shillers_chart

Case-Shiller Home Price Index Chart taken from: multpl.com/case-shiller-home-price-index-inflation-adjusted

 

While housing prices on average don’t seem to consistently increase, not dramatically at least, over the long-run, they do vary. Based on Zillow Seattle data to 2006, a one standard-deviation movement would be on average $40,000.

From here we know a few things:

1.) Seattle prices have been increasing for nearly six years straight, even when controlling for inflation.

2.) There is no strong evidence indicating that real-estate assets always increase in real value, as unlike equity there is no cash-flow or (measured) risk-premium

3.) Real-estate markets have high transaction costs, are very illiquid, and often rely on cyclical factors such as cost of credit.

Zillow1

Zillow Data (zillow.com/seattle-wa/home-values/)

Shiller1

Case-Shiller data from FRED (series: SEXRNSA)

 

Above is a graph of the Seattle Case-Shiller house price index, as well as the same index that I adjusted using the Seattle-CPI measure of inflation. Just eyeballing the chart there are two obvious hypothesis. The first is that Seattle housing prices are consistently increasing, probably due to the tech boom and it being an awesome city, with an exception for the housing-crisis. The second is that housing prices generally don’t increase, but do follow a mean-reverting random walk. We see the same thing using Zillow data, which only goes back to 2006, but has a better methodology using real micro-data.

To start I will just let the time-series speak for itself. I chose an AR(1) to model this data as a simple Gaussian distribution. Some other specification might be better, but the general point here is to show how the inflation-adjusted series is a stationary process. The question these models raise is whether the time-series process captures the dynamics of Seattle real-estate, or if there is some additional structural knowledge about the economy that means the recent increase isn’t just cyclical.

 

Shiller4Zillow3

 

One explanation could be the leverage cycle. John Geanakoplos argues that there is a cycle of leverage driving asset price variation in addition to interest rates. If this were true, increases and variation in the Seattle housing market could be explained in part due to cyclical variation in credit. It’s hard to pin down the mechanism of action here. In retrospect the housing crisis came down to sub-prime mortgages. Asset prices due to a leverage cycle could be due to very small changes in the preferences of investors and homebuyers. As a starting point, I have plotted the R2 values of regressions of Seattle housing prices on the Bank of America US High Yield Spread. Using a high yield spread might show if something is going on as it proxies for credit-market risk, but won’t help in pinpointing causality. On this chart a positive lag represents a lag. So a lag of 6 means we are using a 6 month old Credit Spread to explain the current house price. A negative lag means using data from the ‘future’ to explain current house prices. Using the Zillow data, we see that the future of the high yield data explains around 50% of the current price. We know this was the case during the housing-crisis, so the core question is whether it is also cyclical on a lower scale, or only happens rarely during a crisis. If it only happens during a crisis, these charts are just picking up a structural issue, and have no use in understanding the dynamics of housing or the future. But otherwise, it could mean increasing house prices are able to predict increased risk in credit markets. This data keeps the question alive, but isn’t sufficient to answer it.

Shiller2

 

Zillow2

To finish, I decided to throw the high yield data into a vector autoregression of Seattle housing prices. These aren’t much different from the auto-regressive models, but I was interested in seeing if there were any joint-dynamics. A shock to the high yield series does lower average house prices by a few thousand dollars, but nothing massive. It would be interesting though to see what the dynamics are like if there were a better measure of leverage, ease-of-credit, and liquidity in the Seattle market. If Shiller’s research is correct, these variables could explain most of the movement in the house prices, which would be essentially mean-reverting.

Zillow4

Vector Auto-regression charts

Zillow5

Shock to high-yield series, measured on Seattle housing prices

What would be really interesting is if Seattle house prices do have an upwards trend, and by explaining the factors that are mean-reverting it would be easier to search for variables that predict price increases. This could be related to more traditional supply and demand factors. For example, if there are areas that are exceptionally nice in Seattle, living there could be a signal to others as to your value. In this case social-signalling and agglomeration could result in a few areas that are explained poorly by a model that captures the mean-reverting pricing factors. This would require a more granular view of subsets of the housing-market. My ultimate prediction is that Seattle house prices, in real terms, will decrease over the coming decade as more houses are built and new firms open offices in cheaper locations creeping the city away from the center. Bellevue has become a beautiful area, but before Microsoft it wasn’t particularly noteworthy. In addition, certain subsets of the market will increase massively due to specific economic factors. Whether those subsets are predictable, I am not sure yet.

 

Code I used to generate charts: Github

Shiller’s paper: https://www.nber.org/papers/w13553

Geanakopolos’s paper: https://www.nber.org/chapters/c11786

Washington Employment Data: https://fortress.wa.gov/esd/employmentdata/reports-publications/economic-reports/monthly-employment-report
FRED CPI Series: https://research.stlouisfed.org/fred2/series/CUURA423SAH
Zillow Seattle Data: zillow.com/seattle-wa/home-values/
FRED Case-Shiller series: https://research.stlouisfed.org/fred2/series/SEXRNSA/downloaddata
FRED Bank of America High Yield Debt Series: https://research.stlouisfed.org/fred2/series/BAMLH0A0HYM2

 

 

 

 

https://www.nber.org/papers/w13553.pdf

January 12, 2016 / schoolthought

The ‘right side of history’ with some ending thoughts on a scientific method to evaluate refugee policies.

 

Lately I have been reading Karl Popper’s work on the scientific method in Social Sciences as well as following the German refugee crisis. These are just some of my thoughts on the dangers of thinking you are on the ‘right side of history,’ no matter how right it feels, which come straight out of Karl Popper’s essays. This isn’t a well vetted post, just my trying to sort out what’s been going on in my head.

We all want to be on the right side of history. Our study of the world seems to show a clear trend towards a better world, at least in developed nations. The 20th century stands out to me as both the most optimistic and most efficiently barbaric periods in human history. It was an experimental testing ground not only academically, but also for ideas of human collectivization and government. We take for granted knowledge we gained from these tests as having always been self-evident. In the U.S. we learned that testing diseases on humans without their consent is not worth the cost. This seems really obvious now, but at the time it was often rationalized as being worth it for the greater good. It’s easy to look back and think of these scientists as trivially evil, but they honestly believed what they were doing was right. For example, the Tuskegee syphilis experiment is a black mark in our history in the US. From 1932 to 1972 the U.S. Public Health Service purposefully didn’t treat syphilis in rural African-American men in order to study the disease. Our scientific knowledge in health gained from these experiments. The cost was a profound damage towards trust in government and social cohesion, not to mention individual suffering. It’s challenging to not think of this cost as being obvious at the time. Our view of morality feels mostly invariant to time, but it’s really not.

It is true that many ethical philosophies would have claimed this was unethical prior to the experiment (it wasn’t a controlled experiment, but it was an experiment in that we look back on it and study the outcomes). Of course, many could have claimed it was ethical as well. It wasn’t until all these experiments were done and the results measured that it was decided this type of experiment is unethical, as the negative outcomes outweigh the positive outcomes. Unethical experiments, race based legislation, eugenics, mandatory minimum sentencing, and many more experiments that we now view as unethical took place. Given the information at the time it was not self-evident these were unethical policies, it was only through experiment it became self-evident.

The right side of history could be classified as learning from these experiments, and preventing them from reoccurring. In my experience though it is typically used as a justification for a prediction, based on the view that the ethical choice is self-evident. Arguing that a policy or course of action is on the “right side of history” is claiming that before we even measure the outcome, it is clear that it is and will always be the correct choice. This is a clear distinction between predicting a policy or course of action will be on the “right side of history,” and setting criteria to evaluate the prediction at a future point.

Before the 20th century this same belief system that professed to always be on ‘the right side of history’ existed, but instead with an emphasis on God. Karl Popper wrote “Sinners against God are replaced by ‘criminals who vainly resist the march of history.” Whether it is a prophecy from God, or a progressive ‘march of history,’ these beliefs that social progress, ethics, or morality, are self-evident and have finally revealed themselves have historically failed. Christianity offered to solve the flaws of society, yet the Christian today follows a much different set of rules and values than Christians of the past. It seems then that the solution wasn’t clear. If the solution was clear, we wouldn’t have needed to keep learning throughout time, as the answers would have already been provided. Marx and Engel’s work, building on Hegel’s ethics, unveiled a utopia with promises similar to religion: If only you can follow this text, and follow it properly, the failures of society and humanity will disappear. Yet the failures didn’t disappear, and the dream of a set of rules that leads us to utopia and cures us of our flaws failed once more.

My favorite example focuses on the Soviet Union. The foundation of the Soviet Union found heavy support among the left academic elites in England. Journalist Malcolm Muggeridge was initially fascinated by the promises of communism. He wasn’t the only one. The British Fabian society consisted of the modern elites, including George Bernard Shaw (and the founders of where I studied, the LSE). He wrote that when stationed in Moscow as a journalist, he would play a game with other journalists to see who could convince the public intellectuals visiting from England the most outrageous stories. They would routinely convince visitors that the shortage of milk was only due to its being allocated most heavily to nursing mothers, thus making its shortage a good thing. The public intellectuals often believed this stories, because they deeply believed that Communism was the future of society.

George Orwell had a similar story. He essentially wrote the book for the ‘Left Book Club’ and Communist party. The first half of his book was well received by communists and those on the far left. The second half of his book did not prescribe communism as the answer. As Orwell documented the suffering of the working poor in England, his view was that the often one-dimensional and economically illiterate policies of the communist party could not solve these problems. It seems obvious to us now, but at the time he was viewed by many as being on the wrong side of history, and not supporting the cause. Orwell was still cautiously supportive of many policies associated with Socialism. Prior to great social programs, it was hard to completely reject an untested set of policies that pointed out that we ought to take some of the resources spent on the luxuries of the ultra-rich, and instead use them to provide healthcare to poor orphans. After testing various levels of socialism, we integrated some of the best and stand-alone policies.

Martin Malia, author of The Soviet Tragedy, points out that many academics in the 1960s and 70s attempted to explain the Soviet system “as the product of popular action, and hence as democratically legitimate.” This was necessary to preserve belief in communism. In this view Stalin’s rule was an aberration following legitimate rule, and the system could return to a “democratic and humane socialism.” This was the most widely adopted view, and as a result focused on what went wrong in the Soviet Union with Stalin, and how the system could return to its optimistic outlook. The prevailing view now would be that there is no optimistic outlook. Stalin himself wasn’t an aberration, but the mass murder was communism itself. As a result, it would be both ignorant and unethical to advocate for communist rule, as we now know the result.

Popper’s solution was to constantly try new policy experiments, mark our criteria beforehand, and continuously evaluate them. This would also mean we would spend more time analyzing the dynamics of social progress over the past 50 years opposed to trying to outright solve the problems of human ethics. This would create a more efficient way to evaluate our choices. It wouldn’t eliminate the need to make hard choices, as we still need to make uncertain choices. Admitting more refugees into Germany is a hard choice, and the historical evidence can simultaneously support the view that the cultural clash will damage Germany over the long-run, as well as the view that assimilation works and by taking in asylum seekers we are improving the world for everyone. However, we can decide now on the criteria to decide if the choice to admit refugees was correct or not. To start, all those who are in favor of admitting refugees need to set a series of empirical criteria that need to be met to have falsified their prediction. How bad would crime, rape, welfare-burdens, and other key metrics have to be for people like Merkel to admit they made a mistake? Conversely, those who don’t want refugees need to set their own criteria as well for their prediction on the failures of accepting refugees. How bad do they expect it to get, and if it doesn’t become that bad will they admit they were falsified? We could also formulate policy clauses that trigger based on the outcomes of certain predictions. As a very simple example, Germany could accept some refugees permanently, but many others on a potentially temporary permit. Over the next five years Germany could gather data and evaluate how well the program has worked, and as a result choose how many refugees to retain or send back. I offer that only as a toy example, but the greater point being even in the face of extremely challenging and unclear examples, we could create dynamic policies that we update based on pre-defined outcomes.

It’s impossible to agree on all policies, there will always be fierce disagreements. What we can do is try to agree on the right outcomes to measure beforehand. The less emotional and more methodological our debates become, the less prone we are ourselves to confirming our own biases and viewing those who disagree with us as idiots. The recent sexual assaults in Cologne are a big deal. The question now should be what systems can we set up now so as new information unfolds, we agree on what course of action we need to take, instead of having large emotional reactions.

 

January 8, 2016 / schoolthought

My Time as an Intellectual Monk. And the Year of the Signal:

Just under two years ago I started working at the Federal Reserve in Financial Research. My year at the LSE prior to the Fed had given me an intense year to begin thinking mathematically for essentially the first time in my life.  After the LSE I wanted more academic training before I forever left the world of academia. At the Fed I’ve studied everything that interests me, both during work and outside of work, without obsessing how I will accurately signal my knowledge for my next job. I simply wanted to focus on becoming smarter. I knew that everything I was learning in the world of computational finance, scientific inference, statistics, programming, and math, was valued in other jobs. As a result I worked on plenty of personal projects, completed lots of segments of Coursera courses that were useful, and began to enjoy my new abilities to efficiently learn new disciplines. These two years have been great. Now it’s time to start shamelessly signaling. It is challenging to find a balance between developing my own skills on my terms, and also remembering to find credible ways to prove what I know to others. Time to restore balance and abandon life as San Francisco Economic Monk, and enter the world of Business.

For my own sake, here is a list of everything I’ve done and taught myself at the Fed (both in and out of work) over the past two years.

1.) Math:

 When I began working at the Fed I had a weak foundation in mathematics. It appears to me that the majority of Economics and private sector quantitative work sticks to linear models. The Financial Economics team I am on at the Fed is one of the more mathematically intensive teams at the Fed, since yield curves and asset pricing exists in the world of Stochastic Differential Equations (SDEs), continuous time, and filtering.

image06[1]

A Non-Homogeneous ODE. Useful in Asset Pricing.

I began with multivariable integrals. This was a topic I had briefly covered before, and needed to be better at in order to not struggle with probability distributions. I primarily used Stewart’s textbook, which doesn’t go into proofs but focuses instead on how to compute problems. I spent about 100-120 hours just doing integrals by hand this way after work. I then moved on to ordinary differential equations (ODEs), because this was the technique used to model the yield curve (technically the yield curve is modeled with SDEs, but they reduce to ODEs with a few theorems).

My first goal was to learn how to solve this 

image07[1]

From An arbitrage-free generalized Nelson-Siegel term structure model. 

For the first six months of living in San Francisco, before Alison joined me, every weekend I would wake up and then walk to the park near my place and lie on the grass doing math for a few hours. Eventually I decided I wanted to be even better at math, and began studying the proofs underlying calculus. I’m not sure if this was a good use of time. I had hoped it would give me a richer ability to follow practical applications, which I’m not sure that it did. At least it gave me a small peak into the beauty of math, which I hadn’t appreciated fully.

Otherwise I continued to become better at Matrix Algebra, both because I used it at work and in its practical usage it’s really not that hard. I spent about three months teaching myself the intro to SDEs and brownian motion in 2016, since I wanted to have a better understanding of the models that we work with at the Fed.

2.) The Paper:

At the Fed my boss Jens Christensen put me as a co-author on a (working) paper called The TIPS Liquidity Premium. This is an incredible paper that attempts to filter out the liquidity premium on Treasury Inflation Protected Securities issued by the U.S. Government. When we succeed, we will have the least biased measure of the U.S. real rate in the field. This is an interesting question because the TIPS market has historically not been as traded as normal Treasuries. This means if you buy a TIPS, you are assuming some risk in the form of fear that in the future if you want to sell it immediately you may not be able to find a buyer. And if you desperately need to sell it, you can only find a buyer by lowering your price below its true value. As a result, because of this fear, the buyer of a TIPS from the U.S. Government will require a slightly better price to compensate them for this risk. For example, today the 5-year constant maturity TIPS is trading at a yield of 39 basis points, and the 5-year Treasury at 176 basis points. The difference is considered to be break-even inflation, which is what inflation expectation has to be for the asset prices to be in equilibrium with one another. The TIPS is here representing the real rate of intertemporal substitution in the US economy.

I rewrote the computational heart of the model from R to C++ to improve the speed. This has been my biggest project over the past two years, and the one where I have learned the most. I began working at the Fed barely able to code in R. I had a project that required me to take a heavy filtering model in a language I barely knew, and rewrite it in a language I didn’t know at all, at a point where I had barely even learned how a loop works. I started with ‘Hello World’ in C++, and ended with a few hundred lines of dense mathematical code in C++ that I compiled into an R package. In between was a year of nights and weekends of constant frustration with occasional profound excitement.

image03[1]

The ODE pricing equations for our paper, which I can now solve.

3.)  More Asset Pricing:

Since I worked on an asset pricing team, I spent a lot of time outside work trying to learn the foundations of the field. I worked a lot out of John Cochrane’s book on Asset Pricing, and did about four weeks of his Coursera course. This took a long time since I had to spend a few months first learning more time series and stochastic processes to follow along. I also read the seminal work on return predictability in equities, and some in bond prices, by guys like Campbell, Shiller, Fama, and Cochrane. I studied yield curve modeling from my bosses paper and the book Yield Curve Modeling and Forecasting by Diebold and Rudebusch.

I have mixed feelings about this field. Financial markets are extremely interesting, and the years I have spent working in investments, studying economic theory, financial institutions, political economy, and pairing this with working in academic financial economics, I have developed a robust and deep knowledge of asset markets and investments that is more advanced than almost all who work in private sector investments research. While many in the field are better at the coding and math than I am, my time studying and working at the Fed has given me an advanced understanding of financial market reasoning and theory.

What does shock me is how rich the literature on predictability is, yet how little investments ‘professionals’ know about these papers. If your goal is to predict asset prices and beat the market, shouldn’t you have at least a great base knowledge of the academic literature, what works and doesn’t work, and where the current research is in this field? This isn’t necessary just for quant funds, but even for endowments or fundamental funds. For example, if I worked at an endowment or fundamental fund I would (and could) recreate the main valuation signals from Shiller’s papers. Even if investing in individual stocks, wouldn’t it be useful to have a program that uses time-series techniques to report on historical and mean-reverting processes of market valuations, and estimates the parameters to give a context within which to talk about individual companies or stocks?  This would be even more useful for endowments, which invest across all asset classes.

image10[1]

A chart from Parsimonious Yield Curve Modeling by Nelson and Siegel. 

4.) Econometrics and Time-Series:

I have spent a reasonable amount of time studying Jim Hamilton’s PhD textbook on Time-Series. I began with the foundations of ARIMA models, and eventually moved on to the Kalman Filter, which we use in our model at the Fed. I’ve also continued to study experimental design, by finishing reading and understanding the book Mostly Harmless Econometrics by Angrist and Pischke.  I combined this with studying Asset Pricing to code up a toy-model to forecast the price of a short-VIX ETF.

I chose the short-VIX for a few reasons. The first is that the VIX is based off of option prices for the S&P500, meaning it is a mean-reverting process. The second is that the VIX is an asset that increases as expected variance in the future price goes (this is correlated with market fear, but not the same thing). This means that it is similar to insurance, which requires the buyer pays a risk premium. As a result the short-VIX receives a risk premium for selling market insurance. Lastly, selling volatility insurance is extremely risky for investments funds, as it means when the market does poorly they will have to pay out ‘insurance’ to other funds: In short, when the market does poorly, they will do the worst. I hypothesize that for this reason the risk premium received for selling volatility insurance is very high.  Using this thesis, I coded up a relatively simple rolling ARIMA model to forecast the future price of short-VIX, which I’ll post to my website eventually. I’ve recently read more on Options, Futures, and Other Derivatives by Hull. It would be a fun challenge to recreate this model, but instead of using the SVXY ETF, to build it from scratch using the S&P500 options values. I might work on this with my brother at some point, as we want to build some investments strategies together.

I have also studied Bayesian Computation with R by Jim Albert. I only spent a few weeks on that book, but it was great to finally code up the basics behind Bayesian prior and posterior modeling. I hope to have an opportunity someday to learn more about pure Bayesian statistics. At the moment, I’m investing my efforts into machine learning algorithms, which leads me into my next category.

image09[1]

The steps associated with the Kalman Filter. I think it’s so neat the same mathematical models we use for satellites and Rocket Science is also used in asset pricing. Filtering information from noise isn’t field specific.

5.) Data Science and Machine Learning:

I want to stay on the West Coast, and I really want to live in Seattle. The most booming industry over here is in data science. A poorly defined field that requires people who know statistics, computer science, and scientific inference. It sounds like a great fit for me, as I have been using a mix of the scientific method, causal inference, programming, and math, for the past three years. What I have learned is that the biggest challenge is that economists are never as good at coding as a software engineer and not as good at statistics as a Statistician. In general, we are better at asking the right questions and setting up good scientific practices and research design. Because ultimately equations and code are tools used to answer questions, but can’t solve anything by themselves.

For the past five months I have been studying data science tools. At the Fed we rebuilt our data infrastructure to use Python to query PostgreSQL databases. On my own time I have read a lot of machine learning textbooks, and academic papers on models (i.e. Random Forests). And recently I competed in a Kaggle data science competition, landing in the 40th percentile. Nothing to brag about, but not awful for it being my first competition, in a new programming language, using new models. I am confident I will do much better in my next one, as I spent about 100 hours struggling through lots of easy issues.

I am now doing a Stanford Coursera course on Machine Learning for a certificate. I’m annoyed because it’s very easy so far, as I’ve already taught myself most of these foundational topics. Unfortunately, I have no way to credibly signal it yet, so I have to play the game. As I finish this course and clean up my code from the Kaggle competition, I will have a credible signal that I can do Machine Learning. It still feels like I’m new at it, but with my background at the Fed I underestimate how fast I pick this stuff up compared to others.

image02[1]

A standard classification problem in machine learning. What is the right way to find the dividing lines in a high-dimensional space?

6.) Coding:

I could barely code at all when I started at the Fed. I’ve learned an incredible amount over the past two years. I’m proficient in R now, and able to write fast code within the realm of data cleaning, analysis, and modeling, without having to google too much. I spent a lot of time coding in C++ using mathematical libraries, and learning how to address inefficiencies in R by creating new functions in C++. Again, this isn’t traditional development coding, but coding as a tool for quantitative scientific work.

Despite spending on average five hours coding a day over the past two years, I have done it all by figuring things out as I go, and spending most of my time working with data and models. Over the past half year, I have been coding in Python. I’m still not as efficient as I am in R, but I have been learning far more about what it means to be a true programmer since I started working in Python. At work we have been tackling problems that involve rebuilding production code, which has given me a chance to learn how to write stable automated code outside of simply modelling.

I’ve spent a few weeks studying basic algorithms, and learning best practices when programming. If I want a job in data science, I will need to spend at least 3-5 months studying data structures and algorithms. This will also help me in the long run, as currently most of what I know I have just picked up and never properly studied.

image08[1]

Some pricing equations I coded in C++

7.)  History and Journalism (and Russia):

I have written a few blog posts on this topic, and it’s a field I find fascinating. This year I read Homage to Catalonia and The Road to Wiggan Pier by George Orwell, as well as his published journals and letters. I also read Chronicles of Wasted Time by Malcom Muggeridge’s, who was a British journalist, as well as a spy during WWII. Both were men documenting the political environment of Britain and Europe during the 20th century. They both wrote heavily about Russia and the Soviet Union, as well as the misguided communist sympathizers on the left. It’s incredible how wrong the left was on the success of communism, while all the intellectuals at Oxford and Cambridge opined on how progressive it was as a state model. It’s easy to look back as Muggeridge and Orwell having simply been smart men, but at the time they were widely hated by both nationalists and communist groups. Many intellectuals had disdain for their work. Despite this, they abstained from excitement over ideologies and movements for excitements sake, and instead worked intensely to travel the world and base their worldview off of the empirical reality. Muggeridge saw first hand the starvation of the Russian people, and no rationalization by the intellectual left would change his view.

My interest in these two writers interestingly lined up with my reading of War and Peace and Anna Karenina by Leo Tolstoy. The first gave an incredible account on the will of the Russian People. Anna Karenina devoted a large amount of time to the failing structure of agriculture in 19th century Russia.  Having moved from serfdom, to sharecropping, and then to collectivization under communist rule. A failure that would pave the path for the starvation of untold millions, and the rise of totalitarianism.

This journalist and novel account of communism and Russia resulted in my talking to a professor of Development, who specializes in Russian history, and recommended I read The Soviet Tragedy: A History of Socialism In Russia, 1917-1991 by Martin Malia, which I have not yet finished.

image05[1]

Audrey Hepburn in the film adaptation of War and Peace (I still need to watch this)

8.) Syria:

The tragedy in Syria has no end in sight. I have followed it intensely for the past few years, keeping track of the different groups fighting within the country. As I have read the news, I have also paid close attention to the social media space of information. On reddit there is a subreddit called SyrianCivilWar, which posts twitter accounts, youtube videos, and primary source blog posts, tracking battles, rumors, and information. Having followed this for some time, it’s incredible how much more complicated the reality is than what is presented in the news. To an extent this is understandable, as the news must condense information. The issue is that the complexity of the sites, the reasons for fighting, the massacres, and the conditions for peace, are highly dimensional. There is no clear policy to solve this issue.

The foreign policy status quo has been something as follows: Identify the bad guy. Make sure he’s not making new alliances with other big bad guys (e.g. Russia or Iran). Find guys who are fighting the bad guys (call them moderates). Give moderates guns. Bomb bad guys. Threaten big bad guys to stay away.

This strategy worked to an extent during the Cold War. By that I mean it drained the Soviet Union of tons of resources (e.g. Afghanistan), which some argue resulted in the collapse of the Soviet Union. But it has yielded no obvious dividends in the 21st century. Each time our status quo analysis has missed a highly dimensional complexity, involving clashing value systems, cultures, and our own desire to completely solve the situation. In this case Assad is the ‘bad guy’ working with Russia ‘the badder guy.’ And the amount of civilian murder Assad has presided over absolutely means he has secured himself his place in hell, if such a place exists. On the other hand, Assad is the only hope for a secular government in Syria, and still has the popular support of the people under his control. Without Assad, it is almost certain an Islamist government would take power. Russia realizes this, and also realizes this would mean either more terrorist attacks within their country, or a new Western puppet government in Syria (less likely). Iran realizes a destabilized Syria, with an Islamist government, would threaten their security as well. Surprisingly, they aren’t evil countries who scheme together, but are instead looking out for their regional interests.

The answers are not obvious, but neither are they simple. Imposing a no fly zone for Russia, bombing Assad and ousting him out of power, and putting some US troops on the ground, will literally not solve a single issue.

image00[1]

An info-graphic I made a few years ago.

9.) Philosophy of Science:
This is the ace in my sleeve. It is the intellectual foundation of how I hope to be better at modeling the world than my competition. Scientific inference within the face of massive uncertainty is hardly taught at all within academia. It’s not taught at all in engineering disciplines, which instead exploit the logical rules of the universe. Programming follows logical rules, but doesn’t rely on inference. Within Economics it is taught more than the STEM fields, but even within Economics it is not formally studied. It is instead taught by osmosis through reading the great academic papers of the past. This is usually not enough, which explains why most academic research is shitty regressions with zero citations.

Not surprisingly, studying the connection between making scientific inferences on the world and combining it with statistics is extremely challenging. It is incredibly common for a researcher to pose a question, gather data, and fit a model to the data, only to present it to a roomful of scientists who all aren’t convinced of its accuracy. I’ve seen it happen all the time at the Fed. But on what basis do these scientists not find it compelling? They can’t prove the model is wrong, but they have a hunch that there is something missing. This hunch is based on their own empirical experiences, where they know you can make a model sing however you want if you torture it enough.

This happens more often in quantitative consulting and analysis. It is easy to ask a question, guess what the answer could be, find data and make it match your guess, and then turn it all into a neat story to sell your model. If those you present to don’t have a great background and intuition, they are likely going to look at your t-statistic and think “golly-gee, what’s significant! The model must be true.” When, the truth is, everything that has happened is probably just a step above randomness.

This year I have been reading Against Method by Paul Feyerband, Conjectures and Refutations , The Poverty of Historicism, and The Logic of Scientific Discovery by Karl Popper, and Error and the Growth of Experimental Knowledge by Deborah Mayo. I have also followed Andrew Gelman, Scott Adams, and Scott Alexander’s blogs, who cover the scientific method, statistics, and empiricism. Mostly Harmless Econometrics also focuses on issues of Causal Inference, which I was first exposed to at the LSE.

image04[1]

Karl Popper: The King of the Philosophy of Science

10.) Success:

I have taken my goals of the pursuit of knowledge and rigorous skills very seriously these past three years. And I have been lucky to have been accepted to the LSE and then the Fed. Now I’m preparing to leave the world of academia, and work in the private sector (although at the Fed we did a lot of faster paced policy work). I want to live in Seattle again, and for much of this year despaired that it couldn’t happen. That the jobs I want are on the East Coast, that there aren’t the right jobs in Seattle, or that I’m not properly qualified to get jobs in Tech, as they don’t get many applicants from the LSE or the Fed. It was too easy to let myself become discouraged, and I’m working on ending that style of thinking. Finding a fulfilling job in the right city isn’t easy, but it’s not impossible either. After reading How to Fail at Almost Everything and Still Win Big by Scott Adams (author of Dilbert) I decided to form a new outlook. I’m usually not the type to read self-help books. But Scott Adams has the right mix of cynicism and optimism that appeals to me, without being too heavy on the ‘self-help’ garbage.

He writes a lot on our own psychology, and how we often form negative outlooks based on our own guesses of the odds of success, despite the fact that we rarely get the odds right. I assumed I couldn’t get a great job I want in Seattle, because I already ‘knew’ that all the jobs I want are on the East Coast. It’s certainly true the easiest and highest paying jobs for me to find would be on the East Coast. However, it’s also true that Seattle is a pretty big city, and I don’t know everything that exists. I’m working on forming a more positive and optimistic attitude. There is no reason I’m unable to find a great job in Seattle. I might have to work harder to find the right team, and network harder, but that’s within my abilities.

image01[1]

A great Dilbert comic by Scott Adams (from Dilbert.com)

11.) Final Thoughts:
I spent just over the first half of my twenties trying to learn as much as I possibly could. It was rough graduating with a Finance degree into a crippled job market in Seattle. My greatest fear was not being able to learn more in the fields I found most interesting. I had a deep admiration for analytical scientists and economists who were able to not only come up with clever questions, but to implement their own answers using a mix of statistics, modeling, and programming. While I will continue to improve for the rest of my life, my time at the LSE, the Federal Reserve, and my own auto-didactic ways have taken me to a rigorous foundation.

Now I need to not only refine these skills, but learn how to market and signal them in order to build my own career progression in the private sector. I’ll succeed at this too.

October 7, 2015 / schoolthought

A few thoughts on Syria

Tolstoy wrote War and Peace as a retort to all the armchair European analysts of his time, who wrote and discussed Napoleon’s conquests, and his failure in the Russian winter in the year 1812. Their analysis always sounded reasonable, and explained all the events by attributing clear decision making and brilliant analysis to Napoleon and his enemies. Tolstoy’s problem with this was that he saw all the intellectuals of his day as just observing all that happened, and then crafting a nice story as an explanation. A grand success from the French was due to Napoleon’s sheer brilliance. A failure was due to a tragic mistake, or a stroke of misfortune. The messy complexities of actual reality were smoothed over. Tolstoy saw it differently, and even suggested that Napoleon wasn’t particularly great. He viewed Russia’s success as a fluke. In his view the entire course of history is chaotic, and since we miss so much of what truly happens, we convince ourselves the course of history is set by the grand commanders and pivotal moments.

I sometimes wonder if we make this same mistake with Syria. We have our major players: The Assad regime, ISIS, the Free Syrian Army (FSA), Al-Nusra (al-Qaeda), the Kurds, Turkey, Iraq, Iran, the US, and now Russia. We then look to each player and ascribe strategic intentions, and use that in our analysis. What is interesting though is there seem to be two separate areas of information, with only slight overlap, on Syria. The first is the standard journalism information, and the second is from non-professionals scanning and interpreting the vast amount data posted by combatants and civilians on social media. The subreddit /r/syriancivilwar aggregates the most interesting data, which is usually far more personal and noisy than the stories on NPR or the NYtimes. On the other hand, they are often videos of fighting or discussion, or personal thoughts. The more I read and watch the more confusing Syria becomes. Things rarely become clearer, and it’s obvious they aren’t that clear for people on the ground either.

What is the Free Syrian Army? Is it really a moderate group of soldiers working together against oppression? To me it seems like a term we have assigned to groups of rebels each fighting to protect their cities from outside threats. What is the al-Nusra front, and what is their goal in Syria? The Institute for the Study of War argues that smaller opposition groups might ally themselves with larger groups (like al-Nusra) as larger outside threats approach. Part of the problem is when we speak about Syria, we assume there is a semblance of organization and communication that makes belonging to a faction simple and systematic. I think an important question is to understand what drives Syrian civilians to affiliate themselves with a group. My impression is that if you are a man, and not in a major city, you need to form a militia to protect your town or village, since without any formal protection you are essentially living in anarchy. These are then characterized as ‘opposition’ groups, and somewhere along the way they are clumped together with the FSA, or ignored.

I worry that the analysis and stories that we hear on Syria misunderstand the entire country. While there are concrete issues with Russia involved, in terms of understanding the civil war itself, it is closer to anarchy than anything else. And if that’s the case, I think trying to ‘arm moderates’ or support the right side won’t work, since there is no clear side or organization (although there are wrong sides, namely ISIS and al-Qaeda). I think the only end to the war will come when some person or group gains overwhelming power, and can create an authoritarian government.

http://www.understandingwar.org/sites/default/files/Syrian%20Opposition%20Guide_3.pdf

October 5, 2015 / schoolthought

Scientific Failures

I.)

For us humans it is important to not fixate on the chaos of cause and effect. If we see someone eat a plant and get sick, we shouldn’t eat it. If someone seems to get sick when they are cold, we should stay warm. This kept our ancestors alive. It is also why only up until a couple centuries ago Europeans and American settlers thought tomatoes were poison, and why people tell you to stay warm so you don’t ‘catch cold.’

It seems likely that humans didn’t evolve to be scientists. We evolved to survive, and our most basic model is a simple iterative cost vs. benefit analysis. This has resulted in incredible discoveries. Native Americans used tea from birch trees since pre-history, which contained vitamin C, to prevent scurvy. Making that connection probably took a while, and involved some luck, but it was extraordinary. On the other hand, in Europe the earliest record of finding the solution to scurvy was a British explorer recommending orange and lime juice in 1593. Despite this, there were tons of competing theories, most of which didn’t work. In the early 18th century over a hundred thousand men in the British navy died due to scurvy, and the Navy doctors wouldn’t suggest limes as they originally did not confirm to their theories of disease. Then, finally, in 1753 a British physician James Lind conducted a clinical trial that more or less settled the issue.

Looking back it seems obvious that they should have solved it sooner, and if they had a more developed philosophy of science they would have. But there were hilariously challenging confounding factors to work through. Fresh citrus cured scurvy, but juice that had been exposed to copper tubing and light didn’t. Fresh meat contained vitamin C as well, but salted meat did not. Improved nutrition in general prevented scurvy. If you were trying to figure this out you might notice that citrus juice is not helping, but it’s going to be a few centuries before the periodic table of elements is even invented, so you have no conception about how all substances are in fact composed of many smaller molecules and vitamins.

Then you theorize that you need fresh produce, but fresh meat prevents or cures scurvy in your crew. So that is no longer a convincing argument. And over thousands of shipping trips, someone might eat one of many different foods with vitamin C and be cured, and now you have a whole set of anything they did or ate in the past few days as a potential solution. So everyone starts developing folk theories about how to prevent scurvy. And what is really comical, is that the new scientific view formed following the germ theory of disease suggested that scurvy was caused by bacteria in tainted or old meat. So if you were a scientist invested in the germ theory of disease, you might not be too keen on evidence that seems to go against your scientific argument.

II.)

Science journalism seems to be growing in popularity. On my daily commute I listen to NPR and hear the newest social science research. The papers on popular or controversial issues are quickly distilled and find themselves on major journalism publications, such as the New York Times.  There have even been new platforms, like Vox, which claim to take a scientific and analytical approach to the news. The articles are usually about inequality, gender or race, labor economics, and a whole bunch of sci-fi space junk.  The articles usually play up the authority of the scientists, and take their findings at face value.

I know a lot of smart people who read and share these articles on Facebook, LinkedIn, and even talk about it and mention it at work. Pointing out methodological flaws or telling people you don’t believe them when they talk about interesting research they heard isn’t something you should do if you enjoy having friends.

The abuse of research-design and statistical methods is what lets most awful research take on a veneer of authority.  Fishing for significance is the most common error, as p-values are the bread-and-butter of modern statistical inference. If we have a 5% p-value that tells us our parameter is 8.5 under the null-hypothesis, it means that if we were to assume that the parameter we are estimating was 0 (null), there would only be a 5% probability that it is 8.5 or greater.  One major problem with this is explained by Andrew Gelman, who wrote a great paper that touches on one of the main issues here,  Statistical Significance is not itself Statistically Significant. The point here is basically that a p-value moving from 4.9% to 5.1% isn’t actually a significant movement, even though a 5% p-value is often viewed as ‘proof’ of scientific existent in the peer-review process.
There are additional statistical issues with significance. For example, the assumption is usually that hypothesis we are testing our parameter against is ‘no effect’ (i.e. zero). But this depends on the circumstances, and is not always true. Then there are also concerns about the size of the parameter.  If you wanted to measure differences in height within the US, and cut the country into two equal halves, the difference in height would be statistically significant. After all, we are dealing with the population, so as long as the two averages aren’t equal, they are statistically significant by definition (our standard errors are zero).

Both of those issues are the most commonly cited when criticizing modern science, but in my view they are derived from a more insidious issue. When fitting a model there are usually thousands of plausible specifications to choose from. It’s easy to test tons of model variations until p-value sticks out, and then create a great story on why this is the optimal model.  For example, there is a new paper out claiming an anti-depressant, Paxil, can cause increased risk of suicide in teenagers. This paper uses the same data that was used in the drug trial that concluded Paxil was safe, but comes to a separate conclusion that Paxil has side effects that were ignored in the original study. The original paper argued Paxil was safe, and the statistical evidenced in the original paper did not suggest it caused an increase in suicide risk. It’s not surprising this is hard to measure, as suicide is very rare, which means you might only have a few cases of patients reporting suicidal thoughts, and probably zero patients who commit suicide.

The data was from 1994 to 1998, and focused on 275 adolescents with major depression that had lasted at least eight weeks. There was a double-blind treatment with paroxetine (Paxil), imipramine, or a placebo. The study had an eight week randomized control trial, and was then followed by a six month continuation phase.  Similar to most antidepressant trials, the main outcome variable is a survey called HAM-D, which indexes depression from 0 (none) to 52 (extremely suicidal due to depression).  There are many assumptions on how to interpret this, which you can read in the paper if you’re interested.

The criticisms in this paper appear to be somewhat justified. The original paper made a series of choices when recording and reporting the data, each of which would be plausible on its own, but when combined suggests that—whether by luck or design—their data was presented in a way that slightly understates adverse effects.  There were two points the new paper makes that I found most compelling: The first was that the original study only reported a negative effect if it was above 5% of the sample, but went on to create very specific categorizations. For example, anxiousness, nervousness, and agitation could each only reach 4% of the sample, but could be argued that are different words for the same symptom. The second was that the original authors made access to their data and documentation extremely challenging. For such important research this is unacceptable, and should be required with publication.

The main and most popular finding in this paper had to do with adolescents being at higher suicide risk than originally thought, so let’s look into this: Using their new methodology, they found that five patients dropped out due to suicidality, whereas the original paper had that metric as zero. This new methodology also had 3 patients drop out due to suicidality in the placebo groups, which were also originally zero. Based off patient documentation they also noted that there were 11 suicidal patients during the acute phase and taper compared to 5 suicidal patients in the original study (although this first number is including the taper phase, which the original study didn’t include). Throughout the entire study one patient unsuccessfully attempted suicide.

The difference in the two papers can be explained by the garden of forking paths, sometimes also called researchers degrees of freedom. It’s a concept of how many different ways a researcher can compare the same data to achieve the desired results. In these two papers, the authors of the original paper would benefit more from supporting this medicine, as they were employed by the drug company. In the second paper, they would benefit more from finding a severe flaw to support their argument and get a great publication (and to their credit, they admit this in their paper).

Based off the replications main analysis, their biggest complaint is that the first paper understates suicidality, as well as other minor issues.  But this paper doesn’t find the smoking gun they claim. The truth seems to be one of differences in coding. Imagine two Psychiatrists who each meet with the same 80 severely depressed patients over eight weeks. At the end one says “I think about five of them were low-risk suicidal” (because remember, high-risk suicide requires being committed). The second says “I disagree, I think 11 of them were suicidal.” They then sit and compare notes, and it turns out they look for slightly different signals and indicators. One of them is really conservative and documents anything that could be perceived as suicidal, and the other takes a pragmatic approach.

The statistical power here, the likelihood to detect an effect when there is an effect to be tested, is very low for rare events. If one in a thousand users of Paxil kills themselves due to the drug, this study wouldn’t even have a high chance to detect this result.  Not to mention this study took place about 20 years ago. Since then there have been millions of adolescents who have taken Paxil. While that data might be harder to find, and isn’t a randomized study, it has a sample size of millions. I do not think quibbles over classifications over a few people out of a sub-sample of 80 from 20 years ago should hold that much weight – although I could be wrong as I haven’t worked in this field.

In each case though there are many reasonable choices that could result in slight benefits either for or against the drug’s safety. This gets at the reason I was skeptical of both papers strong claims towards safety or danger, the truth is I don’t think they know to the extent they claim. Both research papers are important, as it gives us a reasonable profile of the risks and benefits of Paxil. But when arguing on the margins of an extra few people being suicidal, it’s hard to take it seriously, as the variance of the research design itself is much bigger than the change in effect.

III.)

Karl Popper argues that our reasons for coming up with a hypothesis or question exist outside of a scientific framework and are unimportant, but once they appear they must be tested rigorously and properly. As a strict philosophy of science this makes sense, since human curiosity is capricious. Unfortunately for the pure philosophy of science, most academic and private sector research has a clear benefit towards proving their hypothesis correct. When someone asks a question, there is usually an answer they either want to be true, or one they think is true and they want to try and prove their intuition is correct. There is a famously bad paper on whether women are more likely to wear pink or read when they are fertile. Why did they ask that question? I’m guessing that their thought process went something like “Women wear red and pink to embrace their femininity, since society views them as feminine colors. I bet when women are most fertile they subconsciously act more feminine to attract the attention of males. I should explore this!”

There are a few problems immediately. The most obvious one is the researcher clearly wants the answer to support the hypothesis; otherwise there is no fun quirky research paper that gets published and widespread science journalism acclaim. The second is it will justify their brilliant intuition and earn them respect and advance their career. Then the third is that there are many different ways to measure this hypothesis, both in the original question and the model specification. The same scientific question could be achieved by examining the level of skin women show, cleavage, makeup, time spent talking to men, and so on. They would all try to measure the same phenomena. Once any of those are chosen, there are many different ways to set up the research design, collect data, and fit a model to the data. It’s so easy to support your hypothesis when you have such wide freedom and you only need to find one that supports your hypothesis and ignore the rest.

None of this is reassuring for the scientific method. There aren’t clear rules on how to set up the right design outside of a randomized experiment. In this instance the question and data do not seem rooted in a robust method. Part of this is also because I view the subtleties of human behavior as usually hard to tease out from the daily noise and complexity of our world.

If all these scientific issues are known—as I certainly didn’t come up with them—why do they persist?  I think it is because even though philosophers of science and some statisticians are extremely interested in them, most other academics don’t appreciate the complexity of reality.  Trying to understand all the chaos that we can’t understand is strange, but it is necessary to have a measure of our uncertainty, which is the heart of Debora Mayo’s seminal research on the philosophy of error statistics. I recently watched a youtube video of a 9/11 ‘truth’ conference, created by an organization of engineers. The presenters were mathematicians, engineers, and other PhDs and academics. They created computational simulations of the towers crashing, presented chemical experiments showing reactions between steel beams and thermite, and generally had a deep and impressive knowledge of structural physics. I know very little about their fields, but I know they are wrong. The world is full of emergent properties on a scale we probably can’t comprehend.  Even if they are much better at mathematical models than I am, my conception of omitted variable bias is better, even though all I’m doing is appealing to the complexity of the world. Even brilliant men make this mistake.  Alan Turing in 1950 made the following claim:

I assume that the reader is familiar with the idea of extra-sensory perception, and the meaning of the four items of it, viz. telepathy, clairvoyance, precognition and psycho-kinesis. These disturbing phenomena seem to deny all our usual scientific ideas. How we should like to discredit them! Unfortunately the statistical evidence, at least for telepathy, is overwhelming.

Turing was a defining genius in human history that focused on math, computers and cryptography, which are inherently logical structures that are fully founded in their base properties. Alan Turing bought into the poor research design on telepathy that found statistical significance, and felt there was no choice but to accept it then as scientific fact.

Linus Pauling founded quantum chemistry and molecular biology, and won the Nobel prize in chemistry. He later claimed vitamin C could cure cancer based on a reasonable hypothesis, and nothing could change his mind. He was convinced it was the case. If you think he was just crazy in his old age, and then you need to explain why despite being completely refuted, it’s still common knowledge that vitamin C cures colds (although these days its Zinc, based on new bad research).

IV.)

This all ties back to modelling. It becomes easy to let the strange and unpredictable emergent properties and chaos of the world drop out. Since we can’t observe them, and we don’t know how they bias our model, it is difficult to understand how our model of the world is wrong. The randomization can do a great job fixing this, but is usually impossible to implement. By conceptualizing the world through science experiments we have made incredible progress. If we were able to send our knowledge on the scientific method back to 16th century Britain, but no additional knowledge, they would probably have been able to set up a series of tests on different boats with clever use of controls, and find a solution to scurvy within a year.

I think if we were able to similarly only receive knowledge on the scientific method from 1,000 years from now, we could also make a leap in progress in understanding how to set up and learn from research designs on issues from drug research to microeconomics. That is the optimistic view. The pessimistic view is that we already know far more about the proper use of the scientific method than is used in academic research, even at the highest levels of research, since the truth often does not line up with passing a drug trial, being published, or getting tenure.

September 20, 2015 / schoolthought

It’s time to change how we view theoretical models (ft. Anna Karenina).

“But you see we manage our land without such extreme measures,” said he, smiling: “Levin and I and this gentleman.”

He indicated the other landowner.

“Yes, the thing’s done at Mihail Petrovitch’s, but ask him how it’s done. Do you call that a rational system?” said the landowner, obviously rather proud of the word “rational.”

 –Anna Karenina, Leo Tolstoy

I.)

Petrovitch is a wealthy landowner in early 19th century Russia, in the novel Anna Karenina. He’s a side character and exists mostly for one of the main characters—Levin—to argue with and explain his new idea on farming and labor. Tolstoy subtly crafts Petrovitch’s arguments to be stale, but with the right mix of economic terminology. Rationality means something important in their discussions, but isn’t well defined. In the book, Levin operates a farm that employs hundreds of peasants shortly after serfdom had been abolished. Like other farmers, he isn’t turning a profit. The peasants don’t have an incentive to work hard, it’s a simple principle-agent problem. And Levin crafts an idea to have them share in the profits with him to give them motivation. This was when economic reasoning was beginning to offer solutions in the abstract theory, but before much of it had been actually tested.  Levin called his system rational farming, and it develops into his life’s work to implement it and teach it to the Russian economy.

II.)

Economists have always prided themselves on being scientifically more rigorous and rational than other academic fields. The field has grown without a rigorous empirical scientific method. Many of the greats, such as Samuelson, Wicksell, and Keynes, were viewed as mostly theoretical. It made sense at the time, and they were far smarter than I am, but following another half century of scientific knowledge, it’s time to realize that economics needs to be grounded completely in empirics. This doesn’t mean we have to start over though, it’s possible to reformulate their arguments within an empirical and testable framework. I won’t try to do that in this post, but I want to explain why it’s important. For example, it’s easy to assume an abstract and true model of the world, and then impose what rationality means. But it is challenging to rigorously explain what it means to call someone else who disagrees with you irrational.

In economic models it’s uncommon to consider that each individual is using his own different model of the world. It is more standard to have a single model that defines the action space, and have players within that world. Even in probability games, the players still are interacting within a shared structure.  As a famous example, there is the prisoners’ dilemma.  While originally viewed as theoretical, these models are absolutely empirical. This model is built on empirical observations ranging back before economics was a discipline. The concept of human betrayal is built into the written history of our species. Hobbes modeled our political system similar to a prisoner’s dilemma. Except in his model it was one where we were all caught in a bad government, with an optimal solution of cooperating.

That people are self-interested, and that people want more rather than less, are predictions based on observations and individual conceptions of history. Hobbes was a well-read philosopher, and as he studied history he saw that there was an equilibrium where even stable dictatorships were preferable to some idealistic democratic dream. This seemed to be a constant throughout different places and events. This reasoning was the type of prediction a computer can’t handle, and is based on our incomplete data of things we have seen and read throughout our lives. As well as the way we combine our empirical views with our own personal experiences to generate a prediction.

Throughout these observations, we have seen patterns that let us consistently look for a few key variables that are always present. Within this perspective, an economist is just someone who uses the scientific method when studying people, but who starts with a prior that a few variables regarding self-interest explain most of the variation. The reasoning behind these models distinguished economics as its own discipline. It wasn’t until later that economists went back and formalized these models in terms of rigorous assumptions and math.

III.)

The prisoners’ dilemma isn’t a theoretical model that needs to be specifically tested in a controlled environment to see if it’s true or not. In fact, the controlled environment will add experiment noise, seeing as how it will just appear to be a silly game. There are no rules for how ‘serious’ the punishments need to be for the game to be valid, but I think it’s reasonable to hypothesize that they should be way worse than simply gaining or losing $20. The way the model was originally created was by looking for a common structure among all historical events and human interaction. Following Ostrom’s work on how these models interact within society, in any given laboratory test of the model it will probably miss other important parameters unique to the time, location, and circumstance.

What would add to the field is an emphasis on studying what causes additional considerations and predicting them. Here a laboratory test could be useful, if it were standardized across different countries. Even if it’s only an approximation to a serious game, the differences between sample populations might still be meaningful. More importantly, the view that a formal test using error statistics and a computer is always the optimal tool for empirical analysis is wrong. Currently computers are not even close to being able to pour over disparate historical texts to search for a common structure of human interaction in certain situations.

Economics as a discipline is built on centuries of empirical research showing that simple measures of self-interest can predict people’s actions. But instead of letting sociologists handle the rest (and usually mess it up), economists should instead start studying the residuals. An empirical achievement would probably involve slightly less effort on extremely complicated mathematical equilibrium refinements (and I’m not just saying that because I’m jealous that I don’t understand them), and instead try to estimate how simple measures of self-interest in the form of interaction in tested games vary throughout different regions and cultures.  These could be used to help predict policy success.

IV.)

If we step away from the world of testing models, we can look at how terms like ‘rational’ are casually misused by Nobel Prize winning economists. Krugman is a good example here of how even smart economists often have a tenuous grip on the rigors of economic scientific inference (I also get to be like my academic role model John Cochrane, and bash on Krugman). Or they somehow think the rules don’t apply when not formally making a model for an academic paper, which is ridiculous, since the entire point of scientific inference isn’t just to publish, but to actually meaningfully understand the world.  Krugman’s scientific formalization of the world starts by imposing preferences on others. Of course, since he is not other people, he needs to try to infer their preferences and their own model of how their actions would affect the world, and then imagine how he would have acted given their preferences but using his model of the world. Imagine a conversation between Paul Krugman and a conservative voter:

Krugman:  I was looking at your economic information. It says that you work as a janitor for a local school system, and make $28,000 a year. The Republicans don’t work as hard to fund education as Democrats. They also believe and will support high quality health insurance for you, and are in favor of unions that could help you negotiate higher wages.

Voter: I don’t need a hand-out from anyone. I know a janitor doesn’t make much money, but it’s my job and I work for what I earn. I don’t know as much about funding, but the school looks alright. My wife and I work hard to pay for our health insurance, and so far have not had any issues. But we really don’t like the condescending attitude most Democrats have about how we choose to live. We are part of a Christian community, and we think a great society needs to follow the right values.

Krugman: By ‘right values’ do you mean you want to defund Planned Parenthood and force the poor to suffer from high unemployment and low opportunities because the Republicans will shoot down any economic policy that tries to lower inequality?

Voter: We work hard and are proud of our values. It’s not our job to fund welfare programs and abortion clinics. If liberals want those, fine, they can buy them.

We know Krugman views these voters are part of, as he calls them, the “irrational right.” To come to this conclusion he tries to infer what they care about and what they want from their country, then he thinks to himself “If these are the things I wanted, how would I go about getting them?”, then he looks at how they go about trying to get them. If these two things don’t match, he calls them irrational. It’s not just Krugman, it’s how most economists use the word.  He is making the implicit statement here that he knows what they want, and knows what they should do to get it better than they know themselves. And he could be right, after all he knows more about economics and policy than most people, but it would be hard to know.

It gets weird when we think about how we would test his hypothesis that they are irrational. We could split the world into two timelines, and then simulate ones where their policies go through and where Krugman’s go through. Imagine it is a game show, and Krugman and the voter are each in separate rooms. Then we can have the results of any variable—no matter how intangible or vague—quantified and given to our players. After pouring over the results, they both meet. They each say “Hah! Looks like I was right!” Krugman looked at all the policies he thought they would care about. But the voter looked at other things, like the excitement of celebrating an election victory with his community, the excitement and feeling of accomplishment he shared with his family, and the belief he had that the world would be better for his children.

This is an area I’m still trying to work out myself. It’s very hard to properly be scientific when talking and disagreeing with someone else. When starting it’s possible to share information and try to understand the evidence and model being used by whoever you disagree with. But if you two still disagree it is not clear how to resolve the issue scientifically (if that is even possible). From all this follows that rationality is at its core about scientific inference and how it is used to infer what people want given their preferences. And even this is unsatisfactory, as preferences are based on our model of the world, so it’s not accurate to consider them as separate or conditional. At its best it’s a useful term because it lets us embed scientific inference when we talk, and still represents an important idea about disagreement on models and conceptions of the world. But at its worse, it is a term that is associated with our over-confidence in our ability to predict how others view the world.

V.)

In Anna Karenina, Levin’s rational farming system didn’t work how it was supposed to. He incentivized the peasants, taught them how profit-sharing works, and tried to inspire an excitement at the prospect of mutual riches. But the system was too different. They had no generational or cultural history of entrepreneurship. Their conception of work and life was one that had developed throughout serfdom, and was antagonistic at its very core. They were fundamentally suspicious of land-owners, uneducated, and viewed every elaborate new plan as one that was zero-sum.

He guessed that if they were rational, they would want to participate in his rational-farming system. But his prediction of the peasants’ model was wrong. He didn’t know what they wanted or how they would use their prior knowledge of the world to interpret his economic view of the world.  Unfortunately, Russian agriculture after serfdom wasn’t able to bring an entrepreneurial system to the peasants. By the end of the 19th century it was the worst agricultural system throughout all of Europe. And by the 1930s the centrally planned Soviet model of agriculture resulted in the deaths of around 6 million people.

August 26, 2015 / schoolthought

Looking Under the Hood of the OLS Model (Mathy)

The best way for me to learn something is to force myself to write it up, and then post it here. Unlike some of my other posts, I didn’t write this to be an accessible and easily digestible post. But if you already have a solid understanding of how OLS works, and want to see how it’s derived, this might be interesting for you.
I wrote it for the following reasons:

1.) While I use linear regressions all the time, I had never derived the analytical solution myself.

2.) I wanted to show how to solve it in R to using an optimizer and coding it myselff.

3.) I wanted to see how to solve OLS with maximum likelihood, instead of just using the closed form solution.

4.) I wanted more Latex experience.

 

The post itself is a pdf file. I need to eventually move this blog and host it myself, so that I can add plugins so Latex shows up here!

 

PDF File:

OLS_Final

%d bloggers like this: