Machine Learning and the Future of Investing: A Q&A With Economist Mehmet Caner

How can hedge funds tap into the power of machine learning to make smarter investment decisions? How can economists integrate machine learning with key financial theories? Is it better to invest in a few stocks, or many? These are just a few of the topics Mehmet Caner, Thurman-Raytheon Distinguished Professor of Economics in the Poole College of Management, is committed to digging into through ongoing research. 

His paper “Sharpe Ratio Analysis in High Dimensions: Residual-Based Nodewise Regression in Factor Models” integrates Sharpe ratios and machine learning, while his paper “Generalized Linear Models with Structured Sparsity Estimators” proposes a new machine learning method. Both are printed in the Journal of Econometrics, a top journal in the discipline.

Caner recently lent his expertise to a panel discussion about the impact of big data and AI in economics and finance at Indiana University Bloomington, as well as a machine learning finance conference at Erasmus University Rotterdam. To learn more, we sat down with Caner to discuss the power of machine learning and the practical implications of his research.

What is deep learning and how can it be leveraged for investing?

Deep learning is a kind of machine learning that uses artificial neural networks to solve problems in a human-like way. The difference is that it has the ability to process large amounts of data — so it can solve problems that humans couldn’t easily solve before. It is used in a number of different applications — like voice recognition and image processing — as well as investing.

Thinking back to the late 1900s, there were a couple of economics Nobel Prize-winning ideas that made important contributions in evaluating risks and returns in the stock market. However, these concepts are not heavily used today because these theories could not understand the correlation and covariance between different stocks — and especially not when the numbers of stocks are incredibly large. As human beings, it is difficult for us to tell stories about how Amazon stock is related to Google stock — much less how 300 different stocks are related to one another. These correlations between stocks ought to determine your portfolio advice — but without machine learning, it’s not always clear what these correlations are.

How does deep learning differ from other models used in financial networks?

Let’s say you are trying to determine what will happen in the stock market tomorrow. You have all these macrovariables, but the relationship between them is unknown. A lot of other models try to find linear relationships — where if inflation goes up a certain way, or interest rates go up a certain way, there’s a one-to-one relationship between the stocks. However, this is not how those relationships usually work in the real world. More likely, there is a nonlinear relationship, or a more vague relationship, that exists between them. 

The power of deep learning is that it can approximate these unknown relationships much, much better than other models through a network of several hidden layers. Twenty years ago, we had shallow neural networks in economics that used just one hidden layer — and that didn’t work very well. Increasing the number of layers approximates the relationship much better in real life, which is a huge advancement.

What are the major contributions of your research?

Through my research, we have been able to leverage machine learning and build a formula to figure those relationships out — and then feed those into financial markets to get better results. Specifically, we combine machine learning with existing financial theories such as the Markowitz Portfolio Theory. Deep learning discovers relationships between different variables, and we can then apply those relationships to Markowitz Portfolio Theory to determine which stocks to invest in. What we are not doing is simply inputting data and letting deep learning dictate whether or not to buy or sell certain stocks. I do not recommend that approach. Rather, we are using deep learning to tell a story that informs our intermediate steps.

In the paper I recently put forward, we show that you can make a lot of money in NASDAQ through deep learning-based portfolio advice. We are also employing this formula in the portfolio formation for two hedge funds in the Istanbul Stock Exchange (Borsa İstanbul Türkiye) with great success.

What are some of the gaps in literature that your research addresses?

Most of the gaps are between academic thinking and the actual practice of things. For example, if you’re going to give portfolio advice about rebalancing an investment portfolio, the question is when, or how? In real life, hedge fund managers usually tell you when to rebalance — or you get horrible returns and you go into panic mode and then rebalance. But is there a scientific, better way of doing these things? Another thing is that in real life, you often have constraints on your portfolio — you cannot buy more than 10% of a certain stock in your portfolio because of the hedge fund managers or government regulations. So, how do you navigate that?

These are some of the questions that are coming from the practice side that have not been answered very well by finance professionals. Usually, the finance literature focuses on the more theoretical aspects of it, such as the efficiency of the markets or whether there are certain factors affecting the returns. However, in practice, I think that’s less important. For that reason, my research marries finance theory with the high-level mathematics of machine learning to discover practical insights for investing. 

What are some of the practical implications of your research and what is the future of investing?

One of the biggest advantages of this research is that it helps us marry machine learning with low signal-to-noise ratio environments. In other words, when the signal is low and there’s lots of noise, deep learning is unaffected. This is great, because in real life, financial markets are often complex and noisy. Compared to other models, deep learning is very useful for providing better portfolio advice and asset pricing — especially for forming large portfolios.

That said, one question that is often asked in academia and in practice is whether we should invest in a small number of stocks or a large number of stocks. There’s some literature that’s emerging from Switzerland about this, and some top finance leaders are suggesting that what we’ve seen in the last 60-70 years in finance is wrong — that it may be better to invest in a few stocks than several. Some of the evidence they have found for this is super interesting. Machine learning also suggests that it may be better to invest in fewer stocks, not more. This is useful, because it means less transaction costs — and if you’re a research analyst, you will be able to focus on five stocks rather than hundreds. So, there are advantages to a smaller portfolio so long as you can keep the risk minimal. You don’t want to buy something that’s going to tank from $10 to $1 tomorrow.