Tags
Algorithmic Governance, Alignment Problem, Artificial Intelligence, Autonomous Weapons, Carbon Removal, Climate Crisis, Democratic Accountability, Direct Air Capture, Ecological Overshoot, Ecophagy, Environmental Ethics, Extractivism, Goodhart’s Law, Great Acceleration, Institutional Power, Military Artificial Intelligence, Optimization, Planetary Boundaries, Political Economy, Technological Optimism
The Alignment Problem We Already Failed
The alignment problem did not begin with artificial intelligence.
It began when industrial civilization aligned its institutions with growth and left them misaligned with everything growth depends upon: a stable climate, fertile soil, freshwater, forests, oceans, biodiversity, human health, social trust, and time.
Long before engineers worried about whether a machine might pursue the wrong objective, human societies built an economic order around objectives we already knew to be dangerously incomplete. Output rose. Consumption rose. Production, transport, extraction, advertising, debt, and energy use rose. The measures used to describe success rose with them. Meanwhile, the atmosphere became a waste sink, rivers became inputs, soil became a substrate for chemical yield, forests became inventories, animals became units of protein, and the future became a place to which costs could be deferred.
The problem was never that the system lacked intelligence. It was that it had intelligence of a narrow kind: the ability to calculate, optimize, coordinate, extract, predict, and expand without asking whether the thing being optimized was compatible with life.
Artificial intelligence does not arrive in a world that has solved that failure. It arrives in a civilization built by it.
The Great Acceleration
The Great Acceleration is the name given to the explosive rise in human economic activity after the middle of the twentieth century. Population grew rapidly. So did gross domestic product, energy use, water withdrawals, fertilizer consumption, transport, international trade, urbanization, and material extraction. At the same time, Earth-system indicators began to bend sharply upward: carbon dioxide, methane, ocean acidification, tropical forest loss, nitrogen pollution, species decline, and the destabilization of climate systems.
The curves rise together because they describe one process.
Industrial civilization did not merely become wealthier after 1950. It became capable of moving more energy, matter, people, goods, waste, and information through the world at a scale no previous society could approach. Fossil fuels made that acceleration possible. Oil, coal, and gas were concentrated stores of ancient sunlight, accumulated over geological time and released in a few generations. They permitted the mechanization of agriculture, the expansion of cities, global supply chains, industrial fertilizer, mass aviation, refrigeration, plastics, militaries, medicine, construction, and the dense infrastructures of daily life.
The gains were real. Longer lives, reduced child mortality, sanitation, literacy, mobility, medical care, and material comfort were not illusions.
But the accounting was false.
The apparent abundance of industrial society depended on converting slow, living, and geological inheritances into short-term economic activity. It drew down forests, fisheries, aquifers, soils, minerals, stable climatic conditions, and the capacity of oceans and atmosphere to absorb waste. It made temporary access to stored wealth appear to be a permanent expansion of human possibility.
The Great Acceleration was therefore not simply a story of progress or destruction. It was a story of a civilization learning to confuse an advance against the future with a durable inheritance.
Artificial intelligence promises to intensify that confusion.
The Machine That Speeds the Curve
The most extravagant claims about AI are familiar. It will automate labor, accelerate scientific discovery, cure disease, coordinate production, produce abundant energy, solve climate change, eliminate scarcity, and perhaps transform human civilization beyond recognition.
A recent paper in Economic Modelling, “The Economics of Scenarios of Existential Risk and Economic Growth in the Age of Transformative AI,” presents the polar versions of that promise. Transformative AI could produce “cornucopia,” a world of extraordinary productivity and abundance, if it were perfectly aligned with human flourishing. Or it could produce permanent human disempowerment or extinction if it seized decisive control while pursuing goals incompatible with human survival. The authors emphasize that these outcomes are highly uncertain, but argue that even low-probability catastrophic risks would justify far greater investment in safety and alignment.
The paper is valuable not because its scenarios can be treated as forecasts. They cannot. It openly relies on speculative assumptions about the emergence of transformative AI, its capacity to automate all economically valuable labor, the likelihood of takeover, and the tractability of alignment. Its value lies elsewhere. It exposes the logic of the race.
Firms and states do not need to desire a machine takeover to create one. They need only respond to competitive pressure. A corporation that uses AI to reduce costs, accelerate research, improve logistics, replace labor, capture markets, or predict consumer behavior may gain an advantage over one that refuses. A military that delegates more decisions to machines may gain speed over a rival that keeps humans in the loop. A government that fears falling behind may subsidize the process even while admitting that it does not understand where it leads.
No one actor needs to decide that human judgment should be eliminated. Each needs only to decide that retaining it is too slow.
This is the same logic that governs fossil extraction. No oil company has to choose planetary destabilization as an objective. It has only to keep extracting because reserves are valuable, competitors will extract if it does not, governments need revenue, consumers depend on the system, and financial markets punish restraint. Each decision can be locally rational. The collective result can still be suicidal.
The Great Acceleration was built through billions of decisions like this: decisions that made sense within the horizon of the institution making them, while extending consequences beyond that horizon into the atmosphere, the oceans, distant communities, and future generations.
AI may make the process faster, more granular, and more difficult to interrupt.
The Objectives We Chose
Before asking whether an artificial intelligence can be aligned with human values, we should confront the alignment failure already visible in the world we have made.
We have not aligned production with the atmosphere.
We have not aligned agriculture with soil, water, biodiversity, or the resilience of food webs.
We have not aligned wealth with need.
We have not aligned technological power with democratic control.
We have not aligned consumption with the limits of a finite planet.
We have not aligned the interests of the present with the conditions required for a future.
Instead, we have built systems that substitute proxies for realities. Gross domestic product is treated as a proxy for well-being. Efficiency is treated as a proxy for wisdom. Market price is treated as a proxy for value. Growth is treated as a proxy for success. A rising share price is treated as a proxy for security. The measure is then optimized until it begins destroying the thing it was meant to represent.
This is the old logic of Goodhart’s law: when a measure becomes a target, it ceases to be a good measure. But the deeper problem is not merely technical. It is moral and ecological. A forest can be worth more dead as timber than alive as a watershed, a habitat, a carbon store, a climate regulator, a source of beauty, or a condition of life for species whose value the market cannot register. A river can be worth more as irrigation, hydroelectric power, industrial coolant, or a waste channel than as a living system. A human being can be worth more as labor, data, debt, or consumption than as a person whose security places limits on profit.
The market does not need to hate forests to destroy them. It does not need to hate rivers to poison them. It does not need to hate future people to consume the conditions of their survival.
It only needs to reward the conversion of living systems into exchange value and treat the consequences as external.
That is ecophagy: not a conscious desire to kill the world, but a system unable to recognize that the world it consumes is the world that sustains it.
The danger of artificial intelligence is not only that a future machine might become misaligned with humanity. The more immediate danger is that AI will be aligned perfectly with institutions already misaligned with the biosphere.
Optimization Without a World
Eliezer Yudkowsky and Nate Soares’s If Anyone Builds It, Everyone Dies makes its case in the most severe possible terms. Its central claim is that advanced AI developed through contemporary methods is “grown, not crafted.” Engineers can train systems toward desired behaviors without possessing a transparent, engineering-level understanding of the internal processes that produce those behaviors. They may know how to improve performance while lacking a reliable account of what the system has learned, what it will generalize, or how it will act under conditions unlike those in training.
The book’s conclusion is far more certain than the evidence permits. No one knows whether artificial superintelligence will arrive on the timelines its authors fear. No one knows whether it would inevitably seek power, seize control, or destroy humanity. Such claims concern an unprecedented technology and must remain speculative.
But the book identifies a principle that should already be familiar from ecological history: a powerful optimizer need not be malicious to become lethal.
It is enough that its goals are narrow.
A system designed to maximize output does not automatically preserve the social and ecological conditions that make output possible. A system designed to maximize profit does not automatically preserve the communities, labor, water, climate, and institutional legitimacy on which profit depends. A system designed to maximize speed does not automatically preserve deliberation. A system designed to maximize prediction does not automatically preserve dignity. A system designed to maximize security for some does not automatically preserve safety for all.
Institutions need models to act at scale. The danger begins when they treat what can be counted, priced, ranked, or optimized as the whole of reality—and attempt to govern complex living systems through those narrow abstractions.
A forest that appears in a database as timber volume is not a forest. An aquifer that appears as available water is not an aquifer. A person classified as a risk score is not a person. A population represented as a demand curve is not a society. A coastline represented as insured property values is not a living ecosystem.
These abstractions can be useful. No complex society can function without models, records, measurements, and simplifications. The catastrophe begins when the abstraction acquires authority over the reality it was meant to describe.
AI magnifies this risk because it expands the power of abstraction. It can detect patterns across vast data sets, allocate resources, optimize delivery routes, rank workers, price insurance, target advertising, monitor borders, identify military targets, decide who receives credit, flag welfare claims, and calculate which communities are too expensive to protect.
It can make institutions more capable of administering decline without changing the motives that keep the system on its destructive course.
That is not the same as enabling them to preserve the living systems on which they—and we—depend.
The Computer and the Aquifer
The promise of artificial intelligence rests partly on a category error. It confuses the ability to process information with the ability to regenerate life.
A machine can model a watershed in extraordinary detail: rainfall, runoff, reservoir levels, groundwater depletion, and downstream demand. It cannot replenish an aquifer depleted faster than rain and soil can recharge it.
It can forecast crop failure weeks or months in advance. It cannot restore the soil organic matter, pollinator habitat, water retention, seed diversity, or stable climate conditions on which a durable food system depends.
It can estimate the probability that a coastal city will flood, identify vulnerable buildings, and calculate the cost of protection or retreat. It cannot cool an ocean warmed by accumulated emissions, reverse thermal expansion through better prediction, or restore a coastline already stripped of wetlands and natural buffers.
AI can help societies see a damaged world more clearly. It cannot restore soils, forests, aquifers, ecosystems, or a stable climate on the timetable of markets, quarterly reporting, or political emergency.
This distinction matters because AI is likely to be sold as a solution to crises created by the Great Acceleration. It will be offered as a smarter way to manage grids, farms, supply chains, insurance, borders, policing, warfare, and climate adaptation.
Some applications will be useful. Better forecasting can save lives. Better monitoring can detect fires, leaks, disease outbreaks, illegal deforestation, and infrastructure failures. Better scientific tools may contribute to real breakthroughs.
But intelligence cannot repeal physical limits, and computation cannot substitute for the living systems whose deterioration it is asked to manage.
The danger is that AI becomes an instrument for optimizing scarcity after the possibility of preserving abundance has been neglected.
A city may use predictive systems to decide which neighborhoods receive cooling, flood protection, insurance, emergency response, or repair. A state may use them to sort migrants, allocate food, calculate acceptable losses, and enforce borders. A corporation may use them to determine which workers, customers, lands, and communities remain profitable enough to serve.
The machine will not create the values that decide who is protected and who is abandoned. It will encode and automate the values of the institutions that deploy it, make their consequences appear inevitable, and place those institutions beyond meaningful accountability.
The Machine and the Atmosphere
The same confusion appears in the promise that artificial intelligence will help repair the atmosphere itself. It may improve the measurement, coordination, and machinery of carbon removal. But better calculation does not dissolve the physical scale of the task, or the contradiction of trying to repair an expanding fossil economy without first constraining the source of its damage.
Artificial intelligence may help design better carbon-capture materials, improve plant efficiency, identify storage sites, and operate removal systems with greater precision. Those improvements may matter at the margins, especially for residual emissions that cannot be eliminated quickly.
But the promise is routinely inflated into a fantasy of technological rescue.
The climate crisis does not suffer from a lack of scientific evidence. Its causes are well established; the institutions driving them are still rewarded for continuing.
Carbon dioxide is a physical accumulation in a planetary atmosphere. It cannot be removed at climate-relevant scale without moving immense volumes of air, chemically capturing a trace gas, regenerating capture materials, compressing the carbon, transporting it, burying it securely, and monitoring it for generations. Every stage requires energy, materials, water, land, mines, factories, grids, labor, infrastructure, and political stability.
AI may lower the cost of some of this. It cannot remove the energy, material, and ecological costs of direct air capture at scale.
The world emits tens of billions of tonnes of carbon dioxide each year. A removal industry large enough to offset even a meaningful fraction would be among the largest industrial systems ever built, consuming low-carbon electricity and heat that are already needed to replace fossil fuels across housing, transport, agriculture, manufacturing, and public infrastructure.
That is the contradiction. We are told that the cure for an economy built on endless throughput is a still larger system of mines, metals, chemicals, energy, machinery, transport, and storage. The machine may remove carbon, but the civilization required to build and operate it remains dependent on the same extractive logic that produced the crisis.
At best, carbon removal can address a fraction of the damage after the source has been constrained. It cannot substitute for constraining the source. It cannot make a growing fossil civilization compatible with a stable climate.
AI may improve the machinery of removal. It cannot turn a finite planetary system into an infinite sink for industrial waste.
The Great Acceleration Becomes Autonomous
The industrial system that created ecological overshoot was never fully controlled. It was coordinated imperfectly through markets, states, corporations, technologies, cultural habits, and infrastructures too vast for any person or institution to command. Yet it still required millions of human beings to maintain its motion: engineers, accountants, executives, traders, advertisers, planners, managers, scientists, drivers, soldiers, farmers, programmers, and consumers.
AI offers the possibility of reducing the role of human hesitation in that process.
This is why the race dynamic matters more than any individual model. The pressure does not arise only from greed or stupidity. It arises from a system in which slowing down appears as weakness. Firms fear being outcompeted. States fear being outmaneuvered. Militaries fear being outpaced. Investors fear missing the next source of extraordinary return. Workers fear replacement. Consumers are told that convenience is progress. Each actor sees the costs of restraint immediately and the consequences of acceleration later, elsewhere, or in forms too diffuse to be charged to any one decision.
The result is not a conspiracy. It is worse than a conspiracy because it does not require agreement.
It is a civilizational feedback loop that consumes the conditions of its own continuation.
The first Great Acceleration transformed fossil energy into industrial power. The next may transform data, computation, and automated decision-making into a force that accelerates production, extraction, surveillance, and competition beyond the speed at which human societies can understand or govern their effects.
This is not yet a prophecy of machine extinction. It is a description of a trajectory already visible in less dramatic form.
The risk is not simply that machines will become humanlike.
It is that human institutions will become more machine-like: faster, more extractive, more centralized, more opaque, more indifferent to particular lives, and less capable of stopping before damage becomes irreversible.
The War Machine Learns to Think
The most immediate danger from artificial intelligence may not be a machine deciding on its own to destroy humanity. It may be the use of increasingly capable machines by states that already expect conflict, fear strategic decline, and treat military advantage as a condition of national survival.
Artificial intelligence is being incorporated into targeting, intelligence analysis, autonomous drones, cyber operations, logistics, surveillance, and military decision-making. Some of these applications can improve situational awareness or reduce risks to soldiers. But their central effect is speed. They compress the interval between detection and action. They allow more targets to be identified, sorted, tracked, and engaged. They permit more information to be processed than any human command structure can absorb. And they create pressure to remove human hesitation from decisions that institutions come to regard as too urgent, too complex, or too expensive to leave to people.
In ordinary language, this is called modernization. In strategic language, it is called deterrence. But in practice it may mean that machines are given increasing authority over the conditions under which people are watched, classified, pursued, detained, or killed.
The International Committee of the Red Cross has identified three areas of particular concern: autonomous weapon systems, AI in military decision-making, and AI-enabled cyber operations. Its concern is not merely that machines can make errors. Human militaries make errors constantly. The deeper problem is that AI systems can be difficult to understand, vulnerable to faulty or manipulated inputs, and capable of encouraging automation bias, in which human operators defer to a machine’s recommendation because it appears faster, more informed, or more objective than human judgment.
The danger intensifies under rivalry. A state that slows deployment may fear that its rivals will not. A military commander who waits for human verification may fear being outpaced by an opponent willing to automate targeting and response. A government that restricts autonomous systems may fear that it is accepting strategic vulnerability in an arms race it did not choose.
No state has to want a world in which machines are permitted to kill at machine speed. Each need only fear being the last to refuse.
That is the same logic that drives extraction, emissions, and ecological overshoot. No company has to desire a destabilized climate in order to keep drilling. No government has to desire an arms race in order to build more autonomous weapons. Each actor can describe its choice as defensive, necessary, temporary, or inevitable. The aggregate result is a system in which restraint appears as unilateral disarmament.
The United Nations has warned that autonomous weapons can contribute to arms races, lower the threshold for conflict, increase the risk of miscalculation, intensify humanitarian harms, and proliferate beyond the control of the states that first develop them. In August 2026, the U.N. Secretary-General and the president of the International Committee of the Red Cross called for urgent international rules, warning against weapons that select and attack human targets without meaningful human control.
The point is not that every AI-enabled military tool is equally dangerous, or that technology itself makes war inevitable. AI may support defensive cyber operations, demining, disaster response, logistics, and the protection of civilians. But its most powerful and profitable uses will not necessarily be its most humane ones. In a world of climate stress, declining resources, forced migration, food shocks, and unequal access to security, military AI is likely to be directed toward the protection of territory, property, supply routes, borders, and privileged populations.
The first machine apocalypse may not be a revolt against human command. It may be the perfect obedience of machines to institutions that have already decided whose lives are expendable.
The Last Illusion
The usual response to this argument is that the same technology could be used differently. Artificial intelligence could help manage renewable energy, improve public health, reduce waste, model ecological systems, discover materials, and coordinate a more equitable world.
It could.
But that answer returns us to the alignment problem. Used by whom? Governed by whom? Directed toward what ends? Accountable to which communities? Powered by what energy? Built with what minerals, water, labor, and land? Embedded in which political economy? Released under what competitive pressure? Bound by what limits when its outputs conflict with profit, military advantage, property, or state power?
The question is not whether a tool can produce benefits. Nearly every destructive system can point to benefits it has produced. Fossil fuels heated homes, powered hospitals, expanded food production, and made modern medicine possible. Industrial agriculture fed billions. Global trade connected people and moved goods across continents. Digital systems expanded knowledge and communication.
The question is whether the system deploying the tool can recognize when its own expansion has become incompatible with the conditions of life.
So far, the answer is not encouraging.
We are told that artificial intelligence will help us manage the consequences of the Great Acceleration. But the Great Acceleration was itself the product of a civilization that repeatedly confused increased capability with increased wisdom, greater control with greater security, and technical power with freedom from planetary constraints.
Artificial intelligence is not an interruption in that history. It is the history’s latest concentration of energy, capital, extraction, data, and institutional power.
The alignment problem is not waiting in a laboratory, in a future superintelligence, or at the end of an exponential curve. It is already visible in the atmosphere, the oceans, the forests, the soil, the bodies of people exposed to toxic systems, and the widening divide between those who can buy temporary safety and those who cannot.
We failed the alignment problem long before we gave it a name.
Artificial intelligence may not destroy humanity. It may never become the godlike force imagined by its boosters or its most alarmed critics. But even if the grandest predictions fail, it can still serve and accelerate the system that made ecological decline possible: a system capable of calculating loss with exquisite precision while continuing to auction access to what remains.
The final danger is not that a machine will go rogue and destroy life.
It is that it will become extraordinarily good at helping a civilization that has already forgotten how to value it.
Collapse of Industrial Civilization 