Home / Blog

AI and society

If AI goes well, what are people for?

Cures and abundance are easy to promise. On 8 November, Stripe co-founder Patrick Collison asked what kind of society they would add up to. Dario Amodei, who runs the AI company Anthropic, wrote that nobody has worked that out. Collison’s second question adds a harder problem. If AIs have even a little moral worth, their numbers could outweigh ours in any calculation that adds well-being up.

Main takeaways
  • Positive visions list ingredients. Cures and abundance are easy to picture. Dario Amodei’s essay Machines of Loving Grace half-answers who works and who decides, and calls the new economic arrangement “something no one today has done a good job of envisioning.”
  • Comparative advantage keeps people working while AI is scarce. A person adds to output even when AI is better at every task, as long as AI time is limited. Once AI can be copied cheaply, Pascual Restrepo’s model has wages converge to the cost of the computing that could do the same work.
  • Slow change left time to fix mistakes. In 1930 Keynes named “technological unemployment” as the cost of labor-saving discoveries outrunning new uses for labor. The AI Futures Project now expects about five years’ worth of AI progress in 2027 alone.
  • A millionth of a person times a quadrillion is a billion people. A quadrillion AIs, each counting for a millionth of a person, would count as a billion people. At a hundred-thousandth each, they would outweigh all 8 billion humans.
  • Collison says one of three premises has to go. Carl Shulman and Nick Bostrom keep all three and propose a split, 99.99% of resources to digital minds and 0.01% to people. They argue it could still leave people very rich.

In August 1994 the novelist Iain M. Banks posted notes on the Culture, the civilization at the center of his science fiction novels. Its starships are run by AIs he calls Minds. The humans on board, he wrote, are “unnecessary for the running of the starships, and have a status somewhere between passengers, pets and parasites.” The Culture is sometimes “invoked as one scenario” for AI going well, in Collison’s phrase, and SpaceX named its rocket-landing ships after Culture starships.

On 8 November 2026, Stripe co-founder Patrick Collison posted two questions about AI. Of the Culture novels he wrote, “I’m not sure they depict a desirable outcome.” His first question is what a good AI future looks like as a whole society. His second is what follows if AIs turn out to matter morally and then come to outnumber us.

Positive visions list the ingredients

More diseases cured and more material abundance “sound great, but these are by themselves just ingredients,” Collison wrote. “I’m curious what the larger equilibrium is supposed to look like.”

Equilibrium

In economics, an equilibrium is a state in which everyone’s choices fit together, so nobody gains by changing what they do given what everyone else is doing. Wages and prices settle at that point, and so does who does which job. A picture of the future describes an equilibrium only if the people in it would keep behaving the way the picture says.

Collison then asks seven questions that any such picture would have to answer. The first three are “Do people work? If so, why? What is their marginal contribution?” The next two ask how resources get allocated if people do not work and what stays scarce. The last two ask how many AIs there will be for each person and whether humans will be in control. Figure 1 checks four of the questions against four well-known pictures of a good future.

Which of the questions do famous positive visions answer?

Do people work? How is wealth shared? What stays scarce? Who is in control? Keynesessay, 1930 Brautiganpoem, 1967 BanksCulture notes, 1994 Amodeiessay, 2024
answers itpartlysilent
Amodei, on how wealth is shared

“It could be as simple as a large universal basic income for everyone, although I suspect that will only be a small part of a solution.”

Figure 1. Press a mark to read the passage behind it. The rows run from John Maynard Keynes’s Economic Possibilities for our Grandchildren to Dario Amodei’s Machines of Loving Grace. The columns are four of Collison’s seven questions. The marks are my reading of each text.

Banks answers every column, and his answer is the one Collison doubts. Collison sums up the Culture as AI running everything while “humans compete in games.” Banks sends at least the would-be tyrants there, who “tend to be diverted successfully into highly complicated games.” The poet Richard Brautigan’s 1967 poem answers two columns. It ends with people “free of our labors and joined back to nature, returned to our mammal brothers and sisters, and all watched over by machines of loving grace.” Collison asks whether that is “rewinding Genesis.” In the Bible, work enters the world as a punishment when Adam and Eve are sent out of the garden of Eden. “In the sweat of thy face shalt thou eat bread,” God tells Adam in Genesis 3:19.

Amodei took the title of his 2024 essay from Brautigan’s poem. Figure 1 marks none of its answers complete. Of work and income he writes, “I think this question is more difficult than the others.”

The head of one of the leading AI companies wrote a long essay on how AI could go well, and it calls the economic arrangement of that future “something no one today has done a good job of envisioning.”

People keep working while they have a comparative advantage

Collison’s third question asks for the marginal contribution of a person, the extra output that one more worker adds. Economists have a standard reason to expect it to stay above zero, at least for a while. In 1817 the economist David Ricardo showed that two countries gain from trading with each other even when one of them is better at making everything.

Comparative advantage

What matters is opportunity cost, meaning what you give up to make one more unit of something. A worker has a comparative advantage in the task where that cost is lowest. A worker who is slower at everything can still add to total output by doing the task they give up least to do.

An AI drafts faster and reviews faster than a person. Should the person still work?

Specializing raises output even when the AI is better at both tasks An AI makes 10 drafts or 5 reviews an hour. A person makes 1 draft or 2 reviews an hour. Over an 8-hour day, if both split their time, they produce 44 drafts and 28 reviews. If the person only reviews and the AI drafts the same 44, they produce 34 reviews. AI4.4 h drafting, 3.6 h reviewing44 drafts18 reviewsPERSON0 h drafting, 8 h reviewing0 drafts16 reviewsTogether: 44 drafts and 34 reviews

The person reviews all day. The AI drafts the same 44 and reviews with its remaining time. Reviews rise from 28 to 34, and nobody works longer.

Now let the AI be copied without limit

Price of an AI-hour $50 an hour
The most anyone will pay the person The AI does the person's 2 reviews an hour in 0.4 AI-hours, so the person's pay can be at most 0.4 times the price of an AI-hour. At 50 dollars an AI-hour that is 20 dollars. A dashed line marks the US federal minimum wage of 7.25 dollars. MOST ANYONE WOULD PAY THE PERSON, PER HOURminimum wage $7.251¢10¢$1$10$100

The AI does the person’s 2 reviews in 0.4 AI-hours, so nobody would pay the person more than $20 an hour.

Figure 2. Press the buttons and drag the slider. A toy example with invented rates. The AI makes 10 drafts or 5 reviews an hour, the person 1 draft or 2 reviews, over an 8-hour day. The dashed line is the US federal minimum wage of $7.25 an hour. The people and robots are cartoons.

This is the argument Amodei makes for the near term. “As long as AI is only better at 90% of a given job,” he writes, the part of the job left to people keeps them employed and makes them more productive. He adds that “in the long run AI will become so broadly effective and so cheap that this will no longer apply.”

The second panel of Figure 2 shows why. Once AI can be copied without limit, the most anyone will pay a person is what an AI would charge for the same work. The economist Pascual Restrepo reaches the same result in a model of an economy with artificial general intelligence, AI able to do any work a person can. Wages “converge to the opportunity cost of computational resources required to reproduce human work,” and the share of income paid to labor converges to zero.

Collison also asks why people would work if they did not have to. Amodei’s answer is that paid work was never the main source of meaning. “Most people are not the best in the world at anything, and it doesn’t seem to bother them particularly much,” he points out. In his view, “meaning comes mostly from human relationships and connection, not from economic labor.”

If work stops paying, how are resources allocated? A universal basic income, a regular payment to everyone, “will only be a small part of a solution,” Amodei suspects. The economist John Maynard Keynes thought in 1930 that once the economic problem was solved, people would face what he called “his real, his permanent problem,” namely “how to occupy the leisure.” He also split human needs into absolute ones and relative ones. Relative needs are those whose “satisfaction lifts us above, makes us feel superior to, our fellows.” When one person rises in rank, someone else falls, so abundance leaves these needs where they were.

Slow change left time to correct mistakes

Collison expects an objection. Nobody in 1700 could have pictured 1900 either. His answer is that “nobody in 1700 was trying to recast humanity itself within a couple of years,” and “things necessarily unfolded slowly and by increments, and therefore with the possibility for error correction.”

Amodei makes the 1700 argument with an older example. Hunter-gatherers “might have imagined that life is meaningless without hunting,” in his words, and that our society “is devoid of purpose.” Collison’s answer to that kind of objection is about speed.

Keynes saw the speed problem in 1930. From about 2000 BC to the early 1700s, he wrote, there was “no very great change in the standard of life of the average man living in the civilised centres of the earth.” By his own time he had a name for what happens when change runs faster. Technological unemployment is “unemployment due to our discovery of means of economising the use of labour outrunning the pace at which we can find new uses for labour.” He called it “a temporary phase of maladjustment.”

The pace now expected is of a different order. The AI Futures Project, whose forecasts an earlier post here explains, expects “something like 5 years’ worth of historical AI progress in 2027,” with a 50% range of 2.7 to 25 years. That measures AI capability, a different quantity from Keynes’s living standards, but it sets the pace that jobs and institutions would have to adjust to.

Same road, same reaction time. What happens as the pace rises?

Corrections that arrive late A winding road and the path of a driver who corrects toward the center but sees each error a fixed time late. At slow speed the path stays on the road. At high speed the corrections land after the road has turned again, and the path swings off the road. START

The corrections arrive after the road has already turned, and each one makes the next swerve worse.

Figure 3. Press a speed. A toy model with invented numbers. The driver steers toward the center of the road but sees each error a fixed time late, and only the speed changes between runs. Red marks show where the car leaves the road.

In Figure 3 the delay never changes. At higher speed the road simply turns further before each correction lands.

A little moral worth times a lot of AIs

Collison’s second question starts from an episode that once looked absurd. In 2022 a Google engineer named Blake Lemoine said publicly that the company’s chatbot, LaMDA, was sentient. Sentient means able to feel things such as pleasure and pain. Google found his claims “wholly unfounded” and fired him that July.

Whether an AI is conscious, meaning whether it has experiences at all, is “often assumed to be relevant to matters of moral worth,” as Collison puts it. Moral worth means having interests that others ought to weigh. A 2023 report by 19 researchers concluded that “no current AI systems are conscious,” and also found “no obvious technical barriers” to building systems that meet its tests. A 2024 report by philosophers and AI researchers argued there is “a realistic possibility that some AI systems will be conscious” in the near future.

Collison thinks the stakes are high. If AIs have moral worth, he writes, “the aggregate worth of AIs would soon outweigh that of humans in any utilitarian calculus.” Utilitarianism is the view that the right choice is the one that produces the most total well-being, with everyone’s well-being counted equally. Totals grow with numbers, and AIs can be copied. In Collison’s words, “even epsilon of moral worth will multiply to a lot once one considers the quadrillions that could soon exist.” Epsilon is a mathematician’s word for a tiny amount, and a quadrillion is a million billion.

How much does each AI have to count before AIs outweigh all 8 billion people?

Number of AIs a quadrillion
Weight of each a millionth
A seesaw with people on one side and AIs on the other Eight billion people, each counting 1, sit on the left of a seesaw. On the right sit a quadrillion AIs, each counting a millionth of a person, which adds up to a billion. The people's side is heavier. 8 billion peopleeach countsas 1 person1 quadrillion AIseach counts asa millionth

The AIs count as about 1 billion people, so the 8 billion people are heavier.

Figure 4. Drag the sliders. Each person counts 1. Nobody knows how many AIs there will be or how much each should count, so both are free choices here. The seesaw is a cartoon.

At a quadrillion AIs, a weight of one hundred-thousandth of a person each is enough to tip the balance. This is where Collison’s two questions meet. His first list asked about the ratio of AIs to humans as a fact about the economy. Under utilitarianism the same ratio decides whose interests come first.

Philosophers have a name for the version with a single being. In Anarchy, State, and Utopia (1974), the philosopher Robert Nozick imagined a being that gains far more well-being, or utility in philosophers’ terms, from each resource than anyone else. He wrote that “utilitarian theory is embarrassed by the possibility of utility monsters who get enormously greater sums of utility from any sacrifice of others than these others lose.” The theory, he went on, “seems to require that we all be sacrificed in the monster’s maw, in order to increase total utility.”

Collison calls his scenario “a distributed version” of the utility monster, with the monster spread across many small claims. The researchers Carl Shulman and Nick Bostrom made the same point about copying in their paper Sharing the World with Digital Minds. Digital minds, their term for minds that run as software, can be copied as fast as hardware allows. “Such explosive reproductive potential could allow digital minds to vastly outnumber humans in a relatively short time.”

Where this arithmetic can mislead

Nobody can measure moral worth today. The weights in Figure 4 are guesses about something that no test can yet detect.

Copies raise their own puzzle. For voting, Shulman and Bostrom suggest that “the creator of such duplicates might have to share their own voting power with the copies they create.” Whether a thousand copies of one AI get a thousand votes is unsettled.

Drop a premise or split the pie

Collison does not accept the conclusion. To avoid it, “one of three propositions needs to be rejected.” Figure 5 sets them out, with what each choice costs.

Which premise would you give up?

Three premises, one of which has to go A triangle with a premise at each corner: AIs can have moral worth, utilitarianism, and AIs could vastly outnumber us. Together they imply that AIs outweigh humans. Dropping any one corner breaks the conclusion. AIs outweigh humans can have moral worth Σ utilitarianism 10¹⁵ vast numbers

Deny that AIs can have moral worth. This is the cheapest choice if it is true. The 2024 report on AI welfare warns of the opposite mistake, “mistakenly harming AI systems that matter morally.”

Reject utilitarianism. Many philosophers already refuse to add up well-being without limits. Shulman and Bostrom warn that this does not settle the matter, because “most reasonable views are in fact susceptible, to various degrees.” Beings with strong claims could still claim resources under rules that do not add anything up.

Rule out vast numbers. Collison states this premise as a belief that there is “even a possibility” of far more AIs than humans. Rejecting it means holding that such numbers are impossible, or capping how many AI minds are made.

Figure 5. Press a premise to drop it. The three premises are Collison’s. The icons are cartoons.

Shulman and Bostrom keep all three premises and split the resources. They compare three ways of dividing them. All could go to humans, all could go to digital minds, or 99.99% could go to digital minds and 0.01% to humans. By a utilitarian count, they write, the split is “approximately 99.99% as good as” giving everything to digital minds. From an ordinary human point of view, it “may also be 90+% as desirable” as keeping everything, “given the astronomical wealth” that digital minds could produce.

How can 0.01% leave people rich?

Share of resources to digital minds

Size of the future economy

A small slice of a much larger pie A pie split 99.99% to digital minds and 0.01% to people. If the economy grows a million times, people's 0.01% is 100 times all the resources people have today. digital minds 99.99%people 0.01%PEOPLE'S RESOURCESCOMPARED WITH TODAY100 timeseverything people have nowtoday1/10,000a million×

People keep 0.01% of an economy a million times today’s, which is 100 times everything people have now.

Figure 6. Press the buttons. The economy sizes are illustrative. People’s resources are their share times the size of the economy, compared with today’s total.

The split works only if the economy really grows that much, and only if the deal holds once one side has almost all the resources.

Concern jumped in 2023 and has held

Collison ends on politics. “Today, people argue about datacenters.” He wonders “how popular support for AI will evolve as more people come to understand these medium-term questions.” In the first three months of 2026, local opposition disrupted at least 75 data center projects worth about $130 billion, by the count of Data Center Watch, a group that tracks such fights. Active opposition groups more than doubled across 49 states. In June 2025, half of Americans told Pew Research Center that the growing use of AI in daily life made them more concerned than excited. Ten percent were more excited than concerned.

The jump came in 2023. Between December 2022 and the summer of 2023, Pew found, Americans who had “heard a lot about AI” became “16 points more likely” to say they were more concerned than excited. Concern rose among people who had heard only a little, too.

Share of Americans more concerned than excited about AI Bar chart from Pew Research Center: 37% in 2021, 38% in December 2022, 52% in 2023 and 50% in June 2025. MORE CONCERNED THAN EXCITED ABOUT AI 0% 25% 50% 37%2021 38%Dec 2022 52%2023 50%Jun 2025

June 2025 survey of 5,023 adults: 50%, with 10% more excited than concerned.

Figure 7. Hover over or tap a bar. Share of US adults who say the increased use of AI in daily life makes them more concerned than excited, from Pew Research Center surveys reported in 2023 and 2025.

Collison’s last line asks, “What is the vision for AI that most people would be excited about?”

From 12 November 2026, the usage policy of Amodei’s company will bar users from “sustained and needless abusive or cruel behavior toward our models.”

Further reading

AI and society economics philosophy AI welfare