Research Agenda | AGI Social Contract

Anthology

Summit

Anthology

Summit

Forging A New AGI Social Contract

Deric Cheng

April 10th, 2025

This is the introductory piece for a series of essays written by AI economists, policy researchers, and political thinkers on the topic of a new AGI Social Contract. These articles will be published over the next few months on our Substack and website, and eventually consolidated into an anthology.

Current contributors include: Anton Korinek, Deger Turan (CEO of Metaculus), Steve Omohundro, Iason Gabriel (DeepMind), Julian Jacobs (DeepMind), Sam Manning (GovAI), Dylan Hadfield (MIT), Seth Lazar (ANU), Peter Salib (UH), Colleen McKenzie (AOI), Philip Tomei (AOI), Dean Ball (Hyperdimensional), Justin Bullock (ARI), Anna Yelizarova (Windfall Trust).

Forging a New AGI Social Contract

A Research Agenda: Key Questions For a New Social Contract

Labor Market Transformations

Taxation Policy

Spending & Redistribution Policy

Governance and Public Ownership

Economic Justice and Inequality

Global Coordination and Power Concentration

Legal Rights

A Research Agenda: Key Questions For a New Social Contract

We’ll share a few dozen outstanding theoretical and practical questions, point to existing work in this direction, and identify gaps that we see that should be filled.

Note that, for many of these questions, the currently available resources do not grapple with the full implications of AGI systems. Instead, many of them focus more broadly on technological progress or on current generative AI technologies. We believe that there is space for researchers to build on top of existing literature by analyzing the likely consequences in a world with transformative AI.

For all of these questions, there is still outstanding research and societal consensus needed in order to enact policy. Many of these may need real-world economic impacts to shift the Overton window and build such a mandate. We believe that excellent research today will accelerate the pace at which public opinion shifts, and lead to a greater chance that we arrive at a flourishing society for all humans – not merely those who will hold the reins of capital.

Summary of Research Questions

Labor Market Transformations

Which jobs and sectors are most at risk of displacement by AGI?

What jobs and sectors would be most resilient to AGI technologies?

What kinds of new jobs or economic roles might emerge during an AGI transition?

How should education and training systems be reformed to help displaced workers transition into new roles?

How will major economic indicators (e.g. AI diffusion, inflation, growth) play out in different scenarios?

How might open-source vs. proprietary approaches to frontier AI systems affect their economic impacts?

Taxation Policy

Could a substantial shift of income from human labor to capital create a potential collapse of governmental tax revenues?

What are viable strategies to tax increasingly profitable AI labs or companies?

How should international corporate tax policy be redesigned to ensure that countries receive a fair share of profits created by AGI?

Spending & Redistribution Policy

What will be the social responsibilities of governments with regards to redistribution in a new economic era?

What are the most effective methods of government infrastructural investment in order to stabilize and grow economies during an AI transition?

What are the most effective infrastructural investments to improve the quality of life and societal outcomes during an AI transition?

What are the pros and cons of plausible safety net structures to protect workers from short-term labor displacement?

What are plausible, nuanced alternatives to UBI as a redistribution mechanism?

Governance and Public Ownership

What policies should counteract the potential concentration of intellectual property among AI corporations?

Should AI systems be regulated as a public utility, or treated as a universally accessible public good?

Should AI systems face more stringent accountability standards?

Should governments have partial or majority ownership over AI systems and labs?

Economic Justice and Inequality

How should our concept of economic justice be redefined in a society where human labor is no longer the primary source of value creation?

What governance measures could prevent the emergence of an extreme wealth gap where a small elite of capital owners capture most of the economy’s gains?

What types of predistributive policies or market incentive mechanisms would preemptively spread value from AI systems without redistribution?

What alternative corporate structures or ownership models could align AGI technologies with equitable outcomes?

Global Coordination and Power Concentration

What mechanisms could ensure that AI systems do not exacerbate global wealth disparities between nations?

What are the geopolitical implications if economic power from AI becomes concentrated in leading nations?

What are the socioeconomic consequences if a small oligopoly of AI corporations controls the means of production?

Legal Rights

Should advanced AI systems have any form of legal economic personhood akin to corporate personhood?

What types of legal rights (e.g. property & asset ownership, forming contracts, making transactions) should AI systems be given?

Should agentic AI systems eventually be deemed to be sentient or conscious, will they qualify for moral patienthood?

What types of decision-making roles will society determine to be off-limits to AI systems?

How will AI liability work? Who will be responsible for mistakes or negative consequences caused by AI systems?

Labor Market Transformations

Which jobs and sectors are most at risk of displacement by AGI, and how quickly will these transformations occur?

What jobs and sectors would be most resilient to AGI technologies? What does this imply about the future of human labor?

What kinds of new jobs or economic roles might emerge during an AGI transition, and will they be sufficient to offset the jobs lost to automation?

How should education and training systems be reformed to help displaced workers transition into new roles in an AI-driven labor market?

How will major economic indicators (e.g. AI diffusion, inflation, growth) play out in different scenarios of AI capabilities development? What are the major observable economic variables that may predict these outcomes?

How might open-source vs. proprietary approaches to frontier AI systems affect their economic impacts?

Taxation Policy

Could a substantial shift of income from human labor to capital create a potential collapse of governmental tax revenues?

What are viable strategies to tax increasingly profitable AI labs or companies to ensure they contribute a fair share to society? What are their benefits and externalities? (e.g. automation taxes, robot taxes, windfall taxes, capital gains tax reform, corporate income taxes, alternative minimum taxes, wealth taxes, digital service taxes)

How should international corporate tax policy be redesigned to ensure that countries receive a fair share of the profits created by AGI corporations, especially if such profits occur at the expense of their labor base?

Spending & Redistribution Policy

What are the social responsibilities of governments with regards to redistribution in an economic era with skyrocketing wealth inequality and a substantial decrease in labor’s share of income?

What are the most effective methods of government infrastructural investment (e.g. industry subsidies, tax breaks, public works) in order to stabilize and grow economies during an AI transition?

What are the most effective infrastructural investments to improve the quality of life and societal outcomes (e.g. public healthcare, education, housing) for the citizenry during an AI transition?

What are the pros and cons of plausible safety net structures to protect workers from short-term labor displacement from AI systems (e.g. unemployment benefits, reskilling funds)?

What are plausible, nuanced alternatives to UBI as a redistribution mechanism (e.g. universal basic services, negative income taxes, tax credit expansions, AI dividend funds)?

Governance and Public Ownership

What policies should counteract the potential concentration of intellectual property and new patents among AI corporations? Should IP ownership change when AI systems develop the majority of new research?

Should AI systems be regulated as a public utility, or treated as a universally accessible public good (e.g. pricing restrictions, antitrust regulation, equitable access, common ownership)? How should

Should AI systems be regulated as a public utility, or treated as a universally accessible public good (e.g. pricing restrictions, antitrust regulation, equitable access, common ownership)?

Should AI systems face more stringent accountability standards (e.g. external audits, transparency requirements, insurance & liability, procurement standards)?

Should governments have partial or majority ownership over AI systems and labs in order to ensure AGI is developed with the public interest in mind? How would this happen (e.g. board representation, sovereign wealth funds, soft nationalization, shareholding)?

Economic Justice and Inequality

How should our concept of economic justice be redefined in a society where human labor is no longer the primary source of value creation?

What governance measures could prevent the emergence of an extreme wealth gap where a small elite of capital owners capture most of the economy’s gains?

What types of predistributive policies or market incentive mechanisms (e.g. advanced market commitments, unions, profit-sharing with data creators) would preemptively spread value from AI systems without redistribution?

What alternative corporate structures or ownership models (e.g. stakeholder capitalism, public-benefit corporations, cooperatives) could align AGI technologies with equitable outcomes?

Global Coordination and Power Concentration

What mechanisms could ensure that AI systems do not exacerbate global wealth disparities between nations and include less-advantaged countries in the gains from AI systems? (e.g. global corporate taxation reform, international UBI, open-source, data dividends)

What are the geopolitical implications if economic power from AI becomes concentrated in leading nations such as the US and China? What role would international agreements play in such a scenario?

What are the socioeconomic consequences if a small oligopoly of AI corporations controls the means of production for a substantial percentage of GDP? What policies could resist such power concentration?

Legal Rights

Should advanced AI systems have any form of legal economic personhood akin to corporate personhood? What would the threshold for such AI systems look like?

What types of legal rights (e.g. property & asset ownership, forming contracts, making transactions) should AI systems be given? What are the economic implications?

What types of decision-making roles (e.g. judicial roles, legislative roles) will society determine to be off-limits to AI systems, even if they can perform at a higher level than humans?

How will AI liability work? Who will be responsible for mistakes or negative consequences caused by AI systems: the model developers, the corporations fine-tuning and deploying the models, or the users of said models?