Abstract

The current dialogue on artificial intelligence (AI)—in the media, in academia, in industry, in government commissions, in online forums, and around dinner tables—often seems untethered to reality. The discussion tends to focus on whether one should be on the side of hype or hysteria—AI will either solve humanity’s most pressing problems and usher in a new era of plenty or it will destroy or enslave the human species. Whereas previous eras of rapid technological development were accompanied by such dialogue, the current extreme nature of the hype and hysteria seems unprecedented and unproductive.

The phrase “AI” arose in the 1950s, and while the phrase was provocative and exciting, the action over the ensuing decades in computer science was elsewhere—in the development of hardware, languages, networks, search engines, human-computer interaction, and eventually data collection and machine learning. The phrase “machine learning” (ML) was coined by an AI researcher [1], but it was eventually adopted by researchers in many other fields, including operations research, control theory, and statistics, who brought with them a range of experience and applications in engineering and science. Machine learning thus served as an intellectual bridge for a data-intensive era—catalyzing the formation of cross-disciplinary connections and (critically) connecting mathematically inclined researchers from various backgrounds with the computer scientists who were building computing systems and networks of ever-increasing scale.

And then came large-language models (LLMs). The ideas and architectures underlying LLMs were fully in the ML tradition—using gradient-based methods to adjust parameters in large-scale predictive systems—with the key novelty that the data was human language, in truly massive quantities. The outputs of LLMs were (strikingly) fluent language, giving the appearance of a human-like entity. This triggered the return to prominence of the phrase “AI.”

What should be the driving aspirations of a technology? Whereas technological developments are often fortuitous and bottom-up, it helps to have some clear high-level aspirations—for researchers, educators, funders, government, industry, and society. The problem with “AI” as the aspiration for our most advanced information technology is that it mainly evokes the 1950s goal of building “thinking machines” that compete with humans, and not much else. Perhaps recognizing the inadequacy of this goal, some people have felt it necessary to coin an amped-up term—“artificial general intelligence” (AGI)—whereby machines would surpass human cognitive abilities—whatever that might mean.

In promoting such goals, with their vagueness and their whiff of Frankenstein, we’re neglecting the fact that humans are social animals, and that their intelligence is partly social in origin. We’re also treating the social consequences of technology as an afterthought, which is unacceptable given the massive effect that current technology is poised to have on society.

The remainder of this article will flesh out a perspective—via descriptions of research results and challenges—that brings social concepts into closer juxtaposition with data-rich information technology. This perspective comes into view when one steps back from the individualist, cognitive perspective of classical AI. It involves revisiting an old idea—that complex systems based on simple local interactions can give rise to emergent, systems-level behavior that is intelligent (or at least interesting). This idea needs revisiting in the data-rich, machine-learning era, where the idea of “local interactions” can be taken in entirely new directions. Such interactions can reflect global and local knowledge, they can be adaptive, strategic, and contextual, and they can involve the exchange of data and models. Local nodes can also be intelligent (in a cognitive sense), augmenting the space of possible global outcomes.

The recent success stories of machine learning motivate our perspective, but we must remember that these success stories have mostly involved centralization of data and the building of large models that aim to act like a single coherent agent. In the kind of dynamic, highly distributed systems that we believe are our likely future, data will be increasingly partial, noisy, biased, local, contextual, strategic, and generally incoherent. This is a feature and not a bug! And it leads us to a second key point: the embrace and management of uncertainty is necessary for intelligent behavior in the real world. Indeed, humans—as individuals and in groups—are remarkable for their ability to navigate in a world in which uncertainties abound. Psychologists debate how good humans are relative to normative standards such as Bayes’ theorem [see, e.g., 2]. My point is not to engage in that debate but to highlight human skill relative to current technology. Thus, in developing a vision for AI technology, we need to recognize that intelligence is not just about knowing things, and giving answers that are “correct,” but knowing how to act when one’s knowledge is partial, and knowing how to interact with other individuals whose knowledge is partial.

The two themes of “social” and “uncertainty” are closely related, a point that has been emphasized in the field of finance [3]. On the one hand, social environments create various kinds of uncertainty, including information asymmetry (some actors are better informed than others, in ways that depend on the problem at hand) and strategic obfuscation (am I being told the truth and how would I know?). On the other hand, social environments permit cooperation and the sharing of information, mitigating uncertainty for everyone, and thereby improving decision-making and enabling long-term planning. In short, real-world intelligence is as much a social, communications, economic, and cultural concept as a cognitive concept.

Given this background, let’s take a second look at the LLM. Whereas an LLM may seem like a single “entity” that is human-like, it is equally well understood as a “collectivist” artifact. Indeed, in interacting with an LLM, one is interacting implicitly with a vast number of humans who have contributed micro-level data, opinions, and linguistic constructions to the LLM via the Internet and other media. When these human contributions agree in various ways, the LLM is able to promote that agreement into abstractions that are useful and that strengthen the illusion of personhood. But, while the analogy of an LLM to a person seems irresistible, an analogy of an LLM to a culture is equally valid. Cultures are repositories of narratives, opinions, and abstractions. Cultures have personalities.

The study of multiple agents in computer science is by no means new, and researchers in fields such as multi-agent ML, human-computer interaction, and algorithmic game theory will recognize that it is their work that is being promoted here, as are the perspectives of the social sciences broadly speaking. Moreover, there are antecedents of our arguments in the study of “collective intelligence” [4, 5, 6]. That literature has two main branches, one focused on qualitative, experimental work with groups, and the other focused on artificial collectives with designed agent utilities. Our focus is different. We want to uncover quantitative design principles for emerging real-world ML-based systems in which many of the participants are human and many are non-human. The goals and utilities of the humans are to be understood and respected, not designed. The principles should be simultaneously computational, economic, and inferential.

Of particular importance, we view economics both as a source of distributed algorithms (“mechanisms”) and also as a social science. We argue that to build AI systems that enhance the human condition, while operating in our social world, we need to make full use of economic ideas such as markets, incentives, information design, trade, prices, contracts, and externalities. In particular, markets, which are collectives that are based on certain patterns of interactions and information flows, can be creative, robust, and scalable. They can operate over long spans of time, and they do more than just generate text and images—they make real-world decisions, create value, spread value among multiple participants, and hedge against risk. They are, in a word, intelligent. Indeed, they were intelligent even before the advent of computing, and it seems likely that they can become even more intelligent—in hitherto unexplored ways—if we view economics through the lens of modern ML (and vice versa). Moreover, social intelligence can be naturally aligned with human welfare.

In summary, our goal is to connect economic ideas—in particular, the fields of mechanism design [7] and information design [8]—with the data-analytic and large-scale perspectives of modern ML. We want designers to speak not only the language of learning algorithms but also be able to consider the overall social system in which ML algorithms are embedded and to treat social welfare as a first-class citizen in their design. Developing a collectivist perspective on information technology that connects algorithmic and statistical concepts to economic and social concepts can be just as exciting intellectually as AGI, and at least as promising for the future of the species.

LLMs can give the impression that they are quantifying uncertainty and reasoning inductively. But this is in part an illusion, arising from the fact that the humans who produced the data on which the learning algorithms are based employed terminology and arguments associated with reasoning under uncertainty. Real-world management of uncertainty needs a more solid foundation.

Uncertainty is a broad concept, encompassing the kinds of sampling-based and probabilistic assertions one finds in statistics, the epistemic considerations of the sciences, and the information asymmetries of economics. Moreover, real-world uncertainty is shaped by phenomena such as the provenance of data and the freshness of data. A doctor relying on medical data that is several years old should be more uncertain about a conclusion than someone relying on fresh data, all else being equal. Such judgments should trade off with respect to other sources of uncertainty. LLMs are currently far from humans with respect to this kind of real-world uncertainty management. Even once known technical issues are mitigated in LLMs, such as systematic overconfidence [9], and even once basic statistical concepts are more fully incorporated in LLM design [10], LLMs will continue to fail to be trustworthy for high-stakes decision-making until the management of uncertainty becomes a first-class citizen in the LLM research agenda.

A broad understanding of uncertainty in human intelligence goes beyond cognitive science into social science. Indeed, collectives play an essential role in changing the world in ways that shape and mitigate uncertainty. An individual human foraging for fruit may or may not succeed in finding fruit on any given day, but once a collective creates a market for produce, the uncertainty regarding finding fruit drops significantly. A human can then depend on fruit being available and build on that certainty, perhaps opening a pastry shop.

Collectives also help to define which uncertainties an individual should focus on and how uncertainties impact the choice of actions. Consider an example from behavioral psychology—a two-choice maze experiment in which a mouse learns that there is more food on the left branch than on the right branch in a ratio of 2/1. The mouse now needs to decide which branch to take on the next trial. A mouse that is decision-theoretically optimal would visit the left branch with probability one, maximizing its chances of finding food. Real mice (and humans in analogous situations) often do something different, known as “probability matching”—they visit the left branch twice as often as the right branch.

Why should that be? Consider a setting in which there are many mice. Here, if each mouse goes left with probability one, then the food on the right is an unexploited resource. A probability-matching algorithm rectifies this—each mouse independently chooses to go left versus right with a ratio of 2/1 and the result is high overall social welfare. An appealing explanation for its emergence is that it is an equilibrium that is found via evolutionary forces [11].

The overall message is that collectives both frame problems involving scarcity and uncertainty and help to solve those problems. If our technology is to help rather than hinder our adaptive decision-making on a large scale, it needs to be designed with these (collectivist) principles in mind.

Current LLMs are an outgrowth of the modern Internet, but they take a limited view of the Internet as a huge collection of text and images. A broader view recognizes that the Internet is a place where interactions occur, where creative collectivist activity such as Wikipedia has taken place, and where markets have arisen. Many of these markets have created real social value, but many are also defective along one or more dimensions. They are defective in particular in their inability to reward creators, to value data as an economic good, to create trust, and to disincentivize socially harmful behavior. A constructive way to think about this issue is to recognize that little thought has been given to microeconomics and mechanism design in building out the Internet. An exception is advertising markets, which have created revenue but which have been a mixed bag with respect to social welfare. In this section, we will highlight other markets that exist on the Internet, and suggest how they could be improved using data.

3.1 Recommendation systems

Recommendation systems are a classical example of ML systems that are collectivist. In one instantiation of a recommendation system, one considers a graph which links customers on one side and products on the other. Purchases are represented by edges between a customer and the products that they purchase, and graph-theoretic methods exploit similarity patterns in the graph to make targeted recommendations.

Although recommendation systems do bring ML closer to microeconomic considerations, they are limited as microeconomic entities—in particular, no money changes hands. Good recommendations may lead customers to make purchases, but conceptually this is just a way to make an existing market for physical goods more efficient. There is no strong need for consideration of incentives.

Let us consider a market that is currently in the midst of technology-driven change—that of recorded music. In the days of yore recorded music was a physical good but it has become a virtual good. For virtual goods, the lack of economic mechanism design in recommendation systems is problematic, leading to an impoverished reward system for creators, who wield little market power. Let us consider an alternative. In Figure 1, we depict the design of a three-way market for recorded music. This market is the architecture underlying the company United Masters, where the author is a board member and consulted on the market design. At one vertex are musicians, who supply songs, and at a second vertex are listeners. The musicians and listeners are linked by a classical ML-based recommendation system. Critically, there is a third vertex, which are brands. Brands often make use of music in products and outreach and they need well-chosen music which fits their image and connects well with the demographic they cater to. Additional recommendation systems, also powered by trained ML models, provide these connections. Moreover, critically, the overall design incorporates incentives. When a brand needs a song, they are supplied with a song from a particular artist (using an ML model), and the artist is paid in that moment. Audience reaction is measured. Other brands can see that reaction, and if it happens to be aligned with a demographic they’re also interested in, then they are incentized to reach out to the artist and partner with them. It is worth noting that United Masters has signed over 1.5 million musicians to date, and their music is used by brands such as the NBA, Bose, and State Farm. This is an instance of a collectivist AI system that has created jobs.

Note the difference with the classic online business model for recorded music, where musicians upload their music to the cloud, and it is streamed to listeners for free. Money is made by the platform, via subscriptions or by advertising, but there is no direct connection between producer and consumer, and there is a weak incentive for the platform to send money back to the musicians. Indeed, there’s a strong incentive for the platform to use generative AI tools to replace the musicians!

Different ways of conceiving of market dynamics for a learning-based online service can have rather different outcomes in terms of social welfare.

3.2 Data markets

Let us now consider a different problem domain that features a three-component market, in this case taking the form of a layered structure [12]. In Figure 2,

a user is shown interacting with a platform, which provides a service and receives a payment in return (concretely, let us suppose that the platforms provide access to credit for a fee). We imagine that the platform is able to learn from the data that it obtains from the user as part of providing the service, and thereby improve the service.

Thus far we have a market where data plays an informational role, but data is not a transacted good. Both the user and the platform are incentivized to engage in this market, depending on the details of the service and the fee. But in many such situations, the platform does not make enough revenue from the fees it collects to realize a profit [13]. Thus, it turns to a set of third-party data buyers. These data buyers wish to acquire data for their own purposes (such as carrying out market research). The platform acts as a supplier, and in this case, the data is the transacted good. It will be priced according to its value to the data buyer and other factors. Again, the incentives are aligned for the platform and the buyer.

But now consider the overall system and ask whether the user is incentivized to participate in this layered market. A new issue has arisen—the user has lost control over their privacy in this market. Whereas before the platform could be held to a contract so that the user has a guarantee that their private data are being used in a limited way—and they are receiving a service and therefore are presumably willing to incur some privacy loss—now the user is told that third-party data buyers are acquiring his or her data and all bets are off. No new service accrues to the user in exchange. The privacy loss can be unbounded and substantial.

Thus, the user is likely to walk away, and the design needs to be elaborated for the market to function. Let us imagine, for example, that platforms decide to provide a formal guarantee of privacy when sending data to third-party data buyers. Concretely, this would mean that noise is added to the data at a level that is contractually specified (and can be audited). Although the noise level could be subject to government regulation, let’s instead leave the choice in the hands of the platforms. A platform is incentivized to provide a nontrivial level of noise, because users will shop among platforms to find one that provides a desirable level of privacy in conjunction with an effective service. Moreover, such a platform, by accruing more users, will collect more data and can make further improvements to its service. On the other hand, data buyers are averse to noise and will presumably pay less for data from the platforms that provide a stronger privacy guarantee. Thus, there is a conflict for the platforms. The way to understand how the conflict will play out is to model the overall system as a game (it is a generalized Stackelberg game) and find its equilibria. This requires specifying utility functions for the various players.

In this scenario, both the platforms and the data buyers will likely be ML systems, learning from the data that they receive. Moreover, data is an endogeneous part of an overall system in which learning, data, and human preferences all interact. Understanding how the overall system will behave—and in particular whether it will work at all—requires a collectivist perspective that combines ML with economics.

The phrase “computational thinking” captures the broad applicability of the abstractions and the modular design strategies that are the fruit of several decades of experience with programming computers [14]. But this style of thinking is not the only way to conceive of algorithm design, and when we aim to design systems that operate in social environments and that treat uncertainty quantification with the reverence it deserves, there are two complementary thinking styles that come into play. We refer to these styles—which are also the fruit of decades of experience—as “inferential thinking” and “economic thinking.”

4.1 Computation and inference in database design

Let us introduce “inferential thinking” in a stylized database problem. Consider a database in a bank in which the rows correspond to clients and the columns correspond to financial data associated with each client. The bank or others (e.g., auditors) may wish to perform various operations on the data, from simple calculations such as finding the average balance across clients, or more elaborate computations involving spotting unusual transactions. The bank may also wish to provide a privacy guarantee to clients in response to such queries and thus may incorporate randomization; see Figure 3(a). It will also be necessary to track provenance, provide interfaces for clients, and provide long-term storage of data. In short, a great deal of computational thinking will need to go into the deployment of such a database.

What might we mean by “inferential thinking” in such an example? On the one hand, the database can implement statistical operations, such as computing a standard deviation or a linear regression. Such computations are in the realm of what a statistician would call “descriptive” but they are not necessarily examples of “inferential thinking.” To clarify the distinction, consider a different database in which the rows are patients in a hospital and the columns are vital signs for the patients, as well as indicators of treatment and responses to treatments. Now, at query time, we might like to ask how likely a particular patient is to respond favorably to a particular treatment. In contrast to the banking example, we are probably not interested in patients who were in the original dataset—we’re interested in “new” patients who come from the same population as the original patients; see Figure 3(b). Indeed, the original patients may be dead and gone, but the data is still valuable. “Inferential thinking” refers to the design and analysis of algorithms that can extract this value. It requires consideration of the underlying population, the set of possible queries, and the extent to which the sampling operator \( S\) can be designed or is already fixed. It involves methods for checking whether the assumptions made in the design are reasonable post hoc. Although the result of such design methodology is a set of algorithms, the thinking behind the design and analysis goes beyond “computational thinking” in its focus on entities that haven’t been seen before. For such entities the goal is not only to make a prediction, but also to provide a measure of uncertainty regarding that prediction.

More generally, inferential thinking involves characterizations of populations as something more than a set of entities, by attempting to delineate underlying mechanisms by which data might arise. Such efforts often fall under the topic of causal inference, where a key concept is the “what if” question—what if the database were different in some way from the data we collected? What if a patient had been given the treatment rather than the control? Can estimates of population-level treatment effects be obtained given the way the population was sampled? These issues are subtle; see, e.g., [22].

Of course, in real-world problems there is generally a need to combine these two thinking styles. In Figure 3(c), we depict a setup in which the goal is inference for a new patient, but where we also want to protect the privacy of the individuals in the original dataset, perhaps because their offspring will have an interest in ensuring their own privacy.

4.2 Inference and incentives

The issue of bias in AI systems is an issue of vast importance. A full treatment of bias goes well beyond our scope, but we briefly sketch how the collectivist perspective can illuminate some aspects of the (very complex) notion of bias.

Collectivist AI systems will likely consist of many heterogeneous participants interacting with their neighbors in a dynamic network. The participants may be human or non-human (or both) and the interactions may involve data flows and other signals. The participants will have access to global and local computing resources. Some knowledge is shared and some data is available to all. But much knowledge and data will not be shared because of strategic interests of the participants and, in particular, their desire to obtain value from their particular knowledge or data. Thus, information asymmetries will abound and hubs of expertise will arise. The network that links the participants should be conceived of as not merely a set of data flows and queries to knowledge sources, but also as contractual interactions that are based on mechanism design.

Moreover, heterogeneity will arise not only from differences in the types of participants and their goals, but also from the fact that data and knowledge are often local, contextual, and fleeting. Solving one’s local problem may use outside resources, but they will be combined with local resources, and, critically, the answers provided by the outside resource will need to be vetted locally. All sources of knowledge will have biases, and it may be necessary to mitigate those biases, or impose one’s own biases, for any particular local problem-solving activity. One way of achieving such bias mitigation is via the methodology of prediction-powered inference [21], where assessments of uncertainty obtained from global foundation models are adjusted based on local ground-truth measurements which may be possessed only by the local agent. Such local knowledge may involve measurements which were not available in the training of a foundation model or may simply reflect a particular (desirable) bias of the local agent. The underlying methodology is an inferential one, involving the correction of bias in confidence intervals.

In addition to heterogeneous goals and data in collectivist AI systems, there will be heterogeneity in agent capabilities, and many new roles can be expected to arise, akin to those that have arisen in previous eras of technology—auditors, brokers, aggregators, sellers, buyers, artists, forecasters, insurers, and explorers. These roles will give rise to personalized services, efficiencies of scale, and appropriate touch points at which regulatory control can be exerted to mitigate biases that are proscribed by legal or ethical considerations.

How should academia help shape and support a collectivist perspective on AI? This is not just a question for computer science, economics, and statistics. (And indeed the terms “computational,” “economic,” and “inferential” have been chosen so as to be inclusive of a range of fields in the mathematical, physical, biological, social, and behavioral sciences).

Academia has already given birth to fields that can be viewed as pairwise blends of the thinking styles that we have identified. As depicted in Figure 4, the field of machine learning blends computation and inference, econometrics blends economics and inference, and algorithmic game theory blends computation and economics. There are conferences, communities, and curricula devoted to these blends.

But it is the tripartite blend that we have emphasized—and academia has not yet caught on. Indeed, while the pairwise blends are the likely progenitors of the tripartite blend, to date their mission statements have been overly limited. For example, algorithmic game theory makes significant use of these economic ideas, but it makes little use of data and inference. Econometrics makes little use of the recent wave of progress in large-scale machine learning. Machine learning has made relatively little usage of economics, especially incentive-theoretic ideas and concepts of information asymmetry. These are, of course, rough characterizations, with notable counterexamples. But they are indicative of significant gaps, missed connections, lack of relevance of academic work to industry, and an impoverished education for an emerging AI workforce.

As we have alluded to earlier, Figure 4 should be viewed as a hub that brings in other academic disciplines, via existing links or new links. For example, economics brings connections to social sciences and public policy, computer science brings connections to cognitive science, and statistics brings connections to biology and medicine. These engender further connections to the humanities. For all this connectivity to yield productive dialog, however, some commonality in language is needed. Our argument is that “computational thinking,” or “AI” as currently practiced, does not provide a sufficiently rich language. The tripartite blend that we have presented—which academically is a middle kingdom between engineering and the humanities—is more likely to create meaningful engagement and provide a starting place for forward-looking curricula.

We have argued that computational, economic, and inferential concepts bring complementary strengths to the problem of envisioning the future of information technology. Bringing all three perspectives to the table helps in discussing complex issues that involve the interaction of technology with individuals and society. For example, issues such as fairness, privacy, ownership, alignment, and transparency—which are often reduced to black-and-white distinctions—can be given more subtle consideration using our tripartite blend.

Of the many disciplines that we have not discussed explicitly, the disciplines of cognitive science, social psychology, and the humanities should be recognized for providing additional perspectives that are essential in shaping human-centric technology. To take but one example where these perspectives link to our discussion, consider behavioral economics, which links cognitive and social psychology with economics, aiming to overcome a perceived overreliance on unrealistic assumptions of rationality in classical economics. Here too, however, existing concepts need revisiting in the data-centric era. Recalling that humans provide the data on which AI artifacts such as LLMs are based, we see that behavioral data is in some sense already baked into learning-based systems. Blending the massive-but-messy information in such data with the more traditional, experiment-driven insights coming from behavioral economics is an important agenda item for future research.

Finally, for AI to grow into a mature engineering discipline, it will need far more than blends of existing concepts (and far more than just more data and more compute). In this regard, it’s useful to learn from the history of the fields of chemical engineering and electrical engineering, which arose to bring the complex phenomena of chemical reactions and electromagnetism under control. This was achieved by developing modular, transparent design concepts that were appropriate for the phenomena. The modularity allowed large systems to be designed piecemeal, allowed system failures to be diagnosed and repaired, and allowed multiple stakeholders to participate in the evolution and regulation of systems. We are far from such modular design concepts in the current stage of development of AI. Moreover, chemical engineering and electrical engineering had at their foundations Schr
specialChar{34}odinger’s equation and Maxwell’s equations, solid foundations that could guide the development of simplifying modular approximations in the face of exceedingly complex phenomena.

For AI, we certainly have exceedingly complex phenomena—cognitive, social, commercial, and scientific—but we do not have the equivalent of Maxwell’s equations as a guide. We are winging it. Going forward, we therefore need the very best of our overarching, hard-won general scientific and humanistic principles—including rationality, experimentation, dialog, openness, cooperation, skepticism, creative freedom, empathy, and humility—as daily companions on the journey ahead.

I would like to acknowledge helpful discussions with Anastasios Angelopoulos, Francis Bach, Stephen Bates, David Blei, Alireza Fallah, Nika Haghtalab, Guido Imbens, Meena Jagadeesan, Barbara Rosario, Ion Stoica, Steve Stoute, Hal Varian, Rakesh Vohra, Serena Wang, and Tijana Zrnic. This work was funded by the European Union, ERC-2022-SYG-OCEAN-101071601. Views and opinions expressed are however those of the author(s) only and do not necessarily reflect those of the European Union or the European Research Council Executive Agency. Neither the European Union nor the granting authority can be held responsible for them. I also wish to acknowledge funding by the Chair “Markets and Learning,” supported by Air Liquide, BNP PARIBAS ASSET MANAGEMENT Europe, EDF, Orange and SNCF, sponsors of the Inria Foundation.