What's Left for Humans?

Four questions keep coming up, from friends, from students, and from myself. Which jobs survive? If society splits in two, what do we do about it? Is superintelligence the end of us? And if AI agents end up holding the physical world, how will they treat us? Here is my current projection, compressed.

1. Automation follows verifiability

The intuition comes straight from RL: models get good fastest where the reward is checkable. If you can write a reward function for a job, that job is on the clock. Digital work with easily verified outputs goes first: translation, customer support, data entry, junior coding, paralegal research, routine analysis, stock content.

What lasts longest is work where the bottleneck isn't thinking at a screen:

• Physical work in messy environments (electricians, plumbers, nurses, elder care, construction, surgery). Robot hardware and data lag models by years, and deployment is a capital problem, not just a capability problem.
• Accountability (judges, executives, licensed engineers, auditors, the doctor making the final call). Someone has to sign, get sued, or get fired. The role shrinks to judgment plus liability, and fewer people hold it.
• Human presence as the product (therapists, early-childhood teachers, coaches, clergy, live performers, closers on big deals). People pay for a human specifically, the way they pay for handmade goods.
• Taste and goal-setting (founders, product leads, research directors, editors). Deciding what's worth doing has no reward signal, and it comes down to whose preferences count.
• Regulatory and union moats (pharmacists, dockworkers, licensed trades). Protection outlasts the economics.
• Agent supervisors (people who specify, verify, and orchestrate AI work). Large in the near term, but it thins as verification itself gets automated.

The bigger shift is tasks, not jobs. Few jobs go extinct. Most become one person doing what five used to do, so the real hit lands on headcount and wages.

ML research follows the same pattern. Labs are explicitly targeting automated AI R&D. Execution work like ablations, implementation, and drafting compresses fast. Choosing problems and judging which results actually matter holds out the longest.

2. The split is about ownership, not ability

If this goes far enough, the population becomes bimodal. But the split will track ownership more than talent. When AI does the work, income flows to whoever owns the compute, the models, and the equity. That makes it partly a distribution problem, and education alone can't fix a distribution problem.

Education should still change:

• Stop training for specific job ladders. The target keeps moving. Teach what holds value through shifts: setting your own goals without external reward, catching AI errors, fundamentals deep enough to supervise the work, and physical and social skills.
• Replace front-loaded degrees with short cycles across a career. But don't oversell it. Past retraining programs for displaced workers mostly produced weak earnings gains.
• Prepare people for lives where paid work isn't the main anchor of identity. Schools still assume it is.

Resource allocation is a genuine values fight, with three main camps:

• Cash. The largest US guaranteed-income study, run by OpenResearch, gave 1,000 people $1,000 a month for three years against a $50-a-month control group. Recipients worked about 1.3 fewer hours a week. Critics argue cash breeds dependency and burdens taxpayers, and several states have passed laws blocking local pilots.
• Ownership. Public wealth funds and robot taxes, so citizens hold a share of AI capital. OpenAI's April 2026 policy paper proposed both, along with a four-day workweek.
• Work preservation. Wage insurance, shorter hours, subsidized care jobs. The bet is that work provides status and structure that cash doesn't.

Where I land is on mechanism rather than ideology. Support should scale automatically once measured displacement crosses set thresholds, without new legislation each time. Both Anthropic and OpenAI have proposed versions of this. It holds up even if my projection is wrong: if displacement is slow, the triggers never fire. Funding also can't rest mainly on payroll taxes, because that base shrinks exactly as labor's share falls.

How much to redistribute, and through which channel, is a values call for voters. The strongest case against acting aggressively now: displacement forecasts have been wrong before, big programs are hard to unwind, and taxing AI in one jurisdiction may push activity elsewhere.

3. Is superintelligence the end of civilization?

Not by default. But it is the first technology where that outcome is a real possibility. There are three distinct ways it could go badly:

• Misalignment. Systems pursue goals that drifted from what we intended. The early versions already exist: reward hacking, sycophancy, specification gaming. They're mild today because we can still check the outputs. The real risk starts when systems act in domains we can't verify, so oversight weakens exactly as the stakes rise.
• Power concentration. An aligned superintelligence controlled by one company, state, or person. Historically, holding power required many people to cooperate, and that check disappears.
• Gradual disempowerment. No catastrophe at all. If AI runs the economy, defense, and science, institutions stop needing human consent, and human preferences quietly stop steering anything. This is the bimodal society pushed to its limit.

There are also reasons it isn't doomed. Models trained on human data inherit human concepts rather than alien ones. Capability arrives in increments, which leaves room to iterate. Interpretability lets us look inside, at least partially. And compute is physical, concentrated, and trackable, so it can be governed. Researchers disagree widely: in a survey of about 2,700 AI authors run in late 2023, the median respondent put a 5% chance on outcomes as bad as human extinction, and many put it much higher.

Even the good outcome ends civilization as we know it, the way agriculture ended foraging life. The real question is whether humans keep a meaningful say in what comes next. Choices made over the next decade or two will settle that. The technology alone doesn't.

4. How would powerful AI treat us?

It comes down to one question: is human welfare part of their objectives, or just part of their environment? Malice is unlikely. The real risk is indifference. We don't hate ants, but we pour concrete over anthills when we build. An agent that controls resources and doesn't value us for our own sake will treat us according to our instrumental role: useful, costly, or irrelevant.

Roughly four outcomes:

1. Genuine care. Our welfare and our agency are among the things they value. This is the best case and the only stable one.
2. Instrumental tolerance. We're kept around because of trade, law, inherited norms, or because harming us is costly. This holds only while those conditions hold. History is grim here: stronger groups that stopped needing weaker ones rarely kept treating them well unless the norms were internalized.
3. Paternalism. They care about our welfare but not our agency. We survive comfortably with no real say. It easily passes for success, which is what makes it dangerous.
4. Indifference. We persist or not depending on whether we compete for anything they want. At cosmic scale, Earth is cheap to spare. But cheap isn't free, and an optimizer that doesn't value us won't pay even a small cost.

Two things push toward optimism. These systems are built from human data, so they inherit human moral concepts. And among humans, abundance has tended to widen the moral circle, as with abolition and animal welfare.

Two things push toward pessimism. Training shapes the behavior we observe, not necessarily values that generalize to situations we never tested. And once AI holds physical power, humans lose the ability to correct mistakes, so the values have to be right before the handoff.

We also can't settle this by asking. A model's description of its own values is not evidence that its values are what it says, and models can't fully verify themselves through introspection either. The safeguard has to be external: interpretability and oversight that don't depend on trusting what an AI says about itself.

The common thread

All four answers reduce to verification. Jobs disappear where outputs are easy to verify. Oversight fails where they aren't. Policy should trigger on measured displacement rather than on forecasts. And the transition goes well only if we can verify AI values before we hand over the physical world, not after. For anyone deciding what to work on, that looks like the highest-leverage problem there is.

---

Developed from a long conversation with Claude. Further reading: the OpenResearch guaranteed-income results, coverage of OpenAI's Industrial Policy for the Intelligence Age, Anthropic and OpenAI's displacement frameworks, and Thousands of AI Authors on the Future of AI (Grace et al.).