Introduction

One of the most important lessons from In Pursuit of the Perfect Portfolio is that one size does not fit all, and that different people need different approaches to managing their assets at different points in their lives. But choosing the perfect portfolio also depends on the particular environment in which we find ourselves. In the same way that we dress differently in the summer than in the winter (at least those of us who live in New England!), we need to invest differently during bull markets than bear markets. In short, we need to be adaptive.

To understand why and how, it’s useful to consider the evolutionary path that financial markets have taken over the last three quarters of a century. This is what I do in a recent publication in the Financial Analysts Journal (Lo, Andrew W. 2021. “The Financial System Red in Tooth and Claw: 75 Years of Co-Evolving Markets and Technology,” Financial Analysts Journal 77, 5–33, https://doi.org/10.1080/0015198X.2021.1929030), and I’d like to provide a short summary of that article here and highlight the themes most relevant to the Perfect Portfolio.

The starting point is the observation that financial markets form a complex ecosystem in which technological innovation interacts with shifting business conditions and a growing population of financial stakeholders. Using the lens of the Adaptive Markets Hypothesis—the principles of evolutionary biology and ecology applied to finance—we can clearly identify eight discrete financial “eras” in which unique combinations of economic need and technological advances gave rise to new products, services, and financial institutions. By understanding the underlying drivers and resulting dynamics of these eras, we can begin to develop a deeper appreciation for the origins of financial innovation and its great promise for our future.

The Evolution of Technology and Finance

How did Homo sapiens come to dominate its environment so completely? The answer is technology: agricultural, biomedical, manufacturing, transportation, computational, telecommunications, etc. Underlying each of these technological advances is a common denominator: financial technology. The reason is simple: even the best ideas can only be put into practice with financing. Here, “financing” does not refer just to money, but also the particular terms under which money is provided to stakeholders to foster innovation.

As Robert C. Merton observed, individual institutions change much more quickly than the core functions of finance. Merton enumerates six such functions:

1. To provide ways to transfer economic resources through time, across borders, and among industries.
2. To provide ways of managing risk.
3. To provide ways of clearing and settling payments to facilitate trade.
4. To provide a mechanism for the pooling of resources and for the subdividing of ownership in various enterprises.
5. To provide price information to help coordinate decentralized decision making in various sectors of the economy.
6. To provide ways of dealing with the incentive problems created when one party to a transaction has information that the other party does not or when one party acts as agent for another.

By focusing on functions rather than institutions, we can develop a deeper understanding of how financial technology co-evolves with other technologies, and can even anticipate to some degree how institutions will adapt as the environment changes.

Timetable of Financial Evolution

Expanding on Merton’s insight, I classify financial innovation over time into six practical categories of implementation based on their broad function: communication, computation, security, commercial use, accessibility, and cost. Communication under this schema is the transmission of financial information, while computation is the transformation of financial information, and security is the protection of that information. Similarly, taking into account a financial innovation’s interaction with the human environment, commercial use is the application of an innovation in a business setting, accessibility is a widening of its potential user base, and cost is a reduction in price.

Looking through the evolutionary lens reveals that different financial environments will benefit different sets of stakeholders. They can be considered analogous to species within the financial ecosystem. For example, an inflationary environment favors debtors, while a deflationary environment favors creditors. High transactions costs will favor large institutional investors and financial intermediaries over retail investors and ordinary consumers. When interest rates become less attractive than stock market returns, that will shift the influence of banking relative to trading. And two groups—academics and regulators—will often respond to a rapidly changing financial environment with innovations of their own.

We can use Merton’s six functions of the financial system, the six corresponding conceptual categories of implementation, and the intellectual history of academic finance to organize the long postwar period of financial innovation into a sequence of eight distinct periods—analogous to major eras in the timetable of evolution—each with its particular defining characteristics (see the chart below). By viewing the natural history of the financial industry in a sequential narrative, the functional perspective of finance and how it relates to the Adaptive Markets Hypothesis comes into sharper focus. Also, by recognizing how environmental changes can help or harm a given species, we can use the Adaptive Markets framework to increase our chances of survival in the face of extinction-level events.

The first era, dating from the establishment of the Bretton Woods monetary system in 1944, we call the “Classical Financial Era.” The Bretton Woods agreement established the U.S. dollar, backed by gold, as the basis for the global postwar economy, which was dominated by the United States unlike any other era before or since.

The Classical Era ends with Harry Markowitz’s first definitive paper on modern portfolio theory in 1952. Although it would take time for Markowitz’s ideas to be digested by the financial community, it introduced the theoretical basis for portfolio diversification into risk management, at a time when the American economy was beginning to diversify from the heavy industry with which it used to become a superpower. For this reason, it seems appropriate to characterize this next period as the “Portfolio Era”.

The Portfolio Era ends with William Sharpe’s first paper in 1964 on the Capital Asset Pricing Model. This new financial model was applied to the first technology boom, which John Brooks called “the Go-Go Years,” but also to the bust that followed. Overall, this era can be called the “Alpha Beta Era.”

The Alpha Beta Era comes to an end with the opening of the Chicago Board Options Exchange in 1973, and shortly thereafter, Fischer Black and Myron Scholes’ publication of their paper on option pricing, and Robert C. Merton’s near-simultaneous development of these ideas. This tumultuous period can only be named the “Derivatives Era,” for the explosive growth of these instruments that followed.

The Derivatives Era ends with a technological development, the advent of the first Bloomberg terminal, at the end of 1982. This phase in financial history was characterized by the adoption of consumer electronics in financial practice, from specialized terminals to home spreadsheets to program trading. The “Automation Era” captures the essence of these years.

The Automation Era ends with another technological milestone, the development of the World Wide Web in 1989 by Sir Timothy Berners-Lee, then a young computer scientist working at a CERN research facility outside Geneva, Switzerland. This event coincides with major geopolitical changes that led to the reintegration of the global economy, hence the “Financial Globalization Era.”

The next dividing line is the year 2000, which may appear arbitrary but many computer professionals were worried about the systemic failures that might have taken place due to obsolescent code on December 31, 1999, the infamous “Y2K problem.” It’s also a convenient milestone for the establishment of high frequency trading in the global markets. For this reason, the following period is named the “Algorithmic Trading Era.”

The final era begins with the self-publication of the Bitcoin protocol by the mysterious figure known as “Satoshi Nakamoto” in 2009. While still in their infancy, cryptocurrencies and their related technologies are an entirely new evolutionary development in financial history, and they have certainly contributed to the spirit of recent financial developments. This brings us to the present day, and the “Digital Assets Era.”

2010 to the Present: The Digital Assets Era (Bitcoin to Coinbase)

Let’s examine the most recent era in more detail. Following the financial crisis, the U.S. led the long recovery among the advanced economies through financial sector reforms, bailouts of systemically important institutions, and the largest overhaul of the financial regulatory system since the 1930s, culminating in the Dodd-Frank Wall Street Reform and Consumer Protection Act of 2010. This era also saw expansive monetary policy, including several rounds of quantitative easing. Despite fears of inflation, deflationary pressures dominated the recovery. With lower yields and lower amounts of leverage available to hedge funds, their returns suffered, hastening the already sizable flow of capital from actively managed financial products and services to passive index products, and causing even greater consolidation across all categories of financial institutions.

Smartphones became popular and then nearly ubiquitous in the U.S. Another technology boom began, centered on applications for these phones. Many previously overlooked services were transformed through these “apps,” such as taxi and ridesharing services and short-term housing rentals. Not surprisingly, banking, stock trading, and financial advisory apps quickly followed, allowing financial services to become portable, accessible, and available at low cost to anyone with a phone.

The rise of social media led to a corresponding rise in misinformation and deliberate disinformation. Rumors, gossip, and “memes” transmitted through alternate forms of media became commonplace, fueling the rise of nativist and populist politics around the world. This had an exact parallel in the financial sector, with the rise of meme stocks and meme trading strategies transmitted on message boards, trading apps, and through short videos shared for public consumption. New financial apps allowed consumers to make financially sophisticated trades at little or no cost, but at high risk to themselves. This trend culminated in January 2021 in the meteoric rise of GameStop that started the year at $20 per share, and reached a peak of $483, thanks to the coordinated efforts of participants on the WallStreetBets channel of the popular Internet forum Reddit.

Cryptocurrencies, developed in the wake of Bitcoin, were also beneficiaries of this movement, as their idiosyncratic features resonated with a populist audience. While there’s still considerable skepticism regarding the long-term viability of these “assets,” their proponents were validated by the successful initial public offering of Coinbase, the largest U.S. cryptocurrencies exchange platform, with a market capitalization of $60 billion as of late September 2021. That same month witnessed the historic moment when the first sovereign entity, El Salvador, declared Bitcoin as legal tender. However, investors need to carefully consider the extraordinary volatility of Bitcoin and other cryptocurrencies, often multiples of the volatility of the S&P 500.

The Digital Assets Era also saw another shift in the fitness landscape of the financial ecosystem. The ability to store and process massive datasets has changed the relative value of training in quantitative finance versus computer science. Data scientists—software engineers with strong data wrangling skills and experience with Python, TensorFlow, and “deep learning” networks—have begun displacing traditional finance quants during this era.

The decidedly non-technological trend of ESG and impact investing also blossomed during this era, as well as the study of the financial implications of climate change. ESG began as a grassroots movement arising largely from Millennials, who led the charge for greater meaning from their investments during the previous era. Given the enormous amount of investable wealth represented by this demographic, the financial industry has listened carefully to their concerns, producing an array of products and services to satisfy these demands.

We end with the global COVID-19 pandemic. March 2020 saw a spectacular drop in the financial markets, and perhaps a narrowly averted financial catastrophe as liquidity problems reached U.S. Treasury bills, one of the most liquid of markets in the financial ecosystem. The world is emerging from its pandemic-induced recession, after frantic social and technological shifts to maintain its economic functions while biomedical researchers develop vaccines and treatments. There will undoubtedly be more financial adaptations to this new status quo, which likely marks the dawn of a new financial era. From the perspective of the Adaptive Markets Hypothesis, the selective pressures imposed by the 18-month lockdown imply that less profitable businesses have been eliminated, leaving stronger and more adaptable businesses in its wake.

Lessons Learned

So what can be learned from the evolutionary rhythms of financial innovation? There are several interesting clusters of developments, each following a shift in the financial environment, defining a characteristic financial ecology of innovation. For example, the needs of World War II almost simultaneously generated the first digital computers and the postwar global financial order of Bretton Woods. These two developments interacted with each other to produce a generation of financial stability and slow but steady innovation. Likewise, the impact of the transistor and the integrated circuit had accelerating effects on the cost and efficiency of computation and communication. The circle of innovation began to feed on itself, with advances in computing leading to advances in communication, making financial services more accessible and less costly to the financial consumer which could in turn be used to fund further innovation. Ultimately, financial services that were only available to a select few thirty years ago have become today’s latest viral freemium app.

A parallel trend is accessibility. The credit card today is increasingly used as a substitute for cash. In the 1930s, however, a credit card was a luxury service available only to elite travelers to purchase airline tickets. The automated teller machine allows people to have access to banking services around the clock, when in recent memory, it was almost impossible to withdraw money from an account except during banking hours on selected days of the week. And now, we have Apple Pay, Google Pay, PayPal, Venmo, and Zelle. The combination of low cost and high accessibility has created an unusual and unprecedented financial environment.

Behind these developments has been the scholarship of academic finance. Modern portfolio theory and the CAPM are two cornerstones of this edifice, leading to a better understanding of performance attribution and the risk-reward tradeoff. However, these would exist in a largely irrelevant theoretical vacuum if it weren’t for the extensive empirical studies of real-world financial data that’s been the hallmark of financial analysis from its beginning. The collection, measurement, and statistical study of this data has led in a remarkably straight line to financial innovations such as option pricing, index funds, and cryptocurrencies. It continues to drive financial innovation as academic researchers try to understand the mysteries of liquidity, volatility, systemic risk, and market structure.

Future Finance

So what does the future hold for financial analysis? As the plumbing of the financial system converges with other systems of information technology, we should naturally expect the materials and methods from those related fields to impinge on financial academic research, for example, in statistical methods for the analysis of Big Data, in network analysis techniques for the world’s increasingly fragmented financial markets, and in cryptographic methods, to create structures for the analysis of sensitive or proprietary financial data.

Two recent innovations appear to have the potential to shift the financial landscape as much as the earlier computing and communications revolution. The first is in the category of security, the use of cryptographic principles in finance, including the secure distributed ledger technology known as blockchain. These innovations have expanded the range of financial products to include not only new forms of currency and other financial instruments, but also to items like “non-fungible token” works of art, which attempt to create scarcity in digital goods in an age of perfect digital reproducibility.

The second innovation is perhaps the most important of all: the mathematical field of artificial neural networks, emulating the behavior of biological nervous systems. Neural networks were a road not taken decades ago when conventional digital technologies were developed due to their difficulty in implementation and prevailing theoretical beliefs about their lack of capacity. As it turns out, these networks, along with other machine learning techniques, may turn out to be a forerunner of the next wave of general-purpose technologies, fulfilling the goals of the decades-long research program of artificial intelligence. Already these networks have been able to master games that had been resistant to older artificial intelligence techniques, such as the classic board game Go. Even more impressively, prototype language models using deep neural networks have been able to compose lengthy pieces of text on impromptu topics that are indistinguishable from those written by humans. As a general-purpose technology, it’s possible that these neural networks will reshape the financial markets as deeply as electricity or the computer have done in decades past.

Perhaps in due course, an AI algorithm will finally be able to construct the Perfect Portfolio for each of us!