Data science can bring tremendous value to the world of private equity (PE). From investment sourcing to due diligence and analyzing post-investment data assets, the range of challenges is matched by the rich data and potential for enormous impact. 

This explains why such projects are a favorite among Tribe AI’s community of data scientists and ML engineers, and I am not immune. I was particularly excited to sit down with legendary data scientist and Tribe AI advisor Drew Conway, head of data science at Two Sigma Investments, to talk all things data and investing, including:

• How PE investors and PE-backed companies can use data to build a competitive advantage
• Why PE is one of the most exciting places to apply data science

Here are some paraphrased highlights from my recent fireside chat with Conway on AI Exchange. 

In private equity, there’s a huge gap between leveraging data science and being data-driven 

Drew Conway: Often we think that traditional, well-established PE firms aren’t using data, but that’s just not true. They’re all leveraging data. Real estate and PE in general are data-rich. Firms know that and use it—but for most, their use comes post hoc. They’ve used their traditional approach—relying on intuition, their networks, and boots on the ground—to identify an opportunity and to diligence that opportunity. It’s only then that they begin to use data. 

Not only is this approach ripe for confirmation bias, but it’s simply not a data-driven approach. Data isn’t helping them identify new opportunities or evaluate whether one opportunity is better than another. This is where most of the opportunity lies in private investing and where firms like Two Sigma have found tremendous value.

Data science can broaden the investing funnel and surface the most attractive opportunities

DC: When it comes to data-driven investing, the value is about widening the funnel. The worst thing that can happen to an investor is to miss an opportunity. But when you widen the funnel, then you also need to be able to assess how good that opportunity is relative to your strategy. That’s where data science can be incredibly powerful. 

There are broad top-down analyses and data science tools that you can build to inform investors’ decisions at the strategic level. And there are much more bottom-up analyses that you can do to help inform their decisions at a tactical level.

The thing they all share is that there’s always a human in the loop—it’s really about augmenting investor decision making with data. Market selection is an example; it’s really about helping investment teams focus their time on the areas that will have the highest impact. 

Public data holds a huge opportunity for forecasting

DC: Investment firms heavily leverage public data. For example, labor market data can help them understand why certain labor markets are growing. Consumer behavior and demographic data like U.S. Census data can help them understand how consumers interact with products. Combining all these data sources together provides a unique view of the market.

Forecasting is a great example. We use data to give investors a long-term view of where the market is going. When you invest in a building, you have to hold it for several years. That means our forecasting model has to account for that range of time, which is incredibly challenging.

The most exciting part is that it’s been proven that you can use data science to scale an investment strategy once you’ve interrogated its bona fides with data. For example, maybe that’s investing in a certain kind of property asset. Data science allows you to rapidly and at scale identify where similar opportunities are in the market. So it’s not just on-off deals in the pipeline anymore. This scaling is what gives newer, data-driven players an advantage over established players.

Data science can impact all aspects of private equity

DC: Data science isn’t just influential in investment decisions; it can have a massive impact on PE-backed companies and how they are run. Like private equity firms, but unlike startups, PE-backed companies are often more mature and therefore have quite a bit of data. This data can prove extremely valuable when thinking about where the best markets are for them to open locations, how to think about marketing and ways to reach their customers, dynamically price their products based on user behavior, or even better understand who their customers are. We’ve seen companies generate huge returns and drive real impact by turning their existing data into insights and optimizing operations. 

If data science doesn’t drive business value, it doesn’t matter

DC: There’s so much that data can tell an investor, but ultimately if it doesn’t move the needle, then it’s a slide in a pitch deck and nothing more. The most important thing you can do to ensure a return on your investment in data science is to apply data science to the right problems and understand how insights would actually influence the investing process. Simply put, designing data science solutions is like designing products: It starts with understanding the user. 

In private investing, decisions are made in a discretionary way—humans are sitting down and deciding whether to invest in a company. So all the work data scientists do has to have a front-end interface that can be understood by investors, rather than just technologists. This is especially different from the public markets, where models are interacting with the market directly. Data-enabled private equity investing remains human-in-the-loop, so the key to success is ensuring that your data scientists are deeply integrated with investors and marrying their data expertise with the deep domain expertise of the investors themselves. 

Private equity investing is exciting for data scientists

DC: What I love to think about is how data can be used to understand human decision making at scale. Why do groups of people make choices? If that’s something that you have even an inkling of interest in, there’s truly no better industry to be a data scientist. There are so many different types of questions both internally—Why would we value a business one way versus the market?—and externally to understand why certain assets behave in the way they do. All of it is driven by human behavior. 

Watch Jaclyn Rice Nelson and Drew Conway in the full fireside chat, “Using Data to Drive Private Equity: Lessons, Trends, and Opportunities for Data Scientists.”