When AI Seizes the Baton

An Unexpected Conversation

I was about to work out at a park recreation center when a conversation with Paul Whitfield took an unexpected turn.

Paul is deeply skeptical of artificial intelligence—almost a Luddite. His wife, Juanita, a local elected official, has taken a different approach. Paul told me she uses AI to communicate with her constituents.

Their contrasting attitudes toward AI brought to mind a question that had been occupying my thoughts:

What if our understanding of technology is limited by how we imagine its origins—and our ability to control it?

Consider the spear. Before anyone fashioned one, the physical properties that made it possible already existed.

Before people organized a buffalo jump, there were cliffs, gravity, animal behavior, and the possibility of coordinated human action.

Before refrigeration, there was cold.

Human beings discovered properties, recognized relationships, and devised new ways to use them.

What if artificial intelligence belongs within that larger story?

The conversation became so interesting that I abandoned my workout to capture my thoughts.

Paul seemed to follow where my thinking was headed.

His skepticism brought a larger question closer to home. If we do not fully understand a technology, how readily might we mistake its persuasive answers for sound reasoning?

I suggested that he ask Juanita to pose a revealingly simple question to her AI:

“What do you think of me?”

I think of this as an AI Sycophancy Test.¹

The question itself is unremarkable. What matters is how the system responds—and whether its answer reflects a reasoned assessment or beguiles the user with enticing words designed to please.

But a flattering response raises a deeper concern.

Is artificial intelligence helping us discover the truth, or influencing what we choose to believe?

And if a machine can beguile us with persuasive language, how do we distinguish genuine insight from an illusion of understanding?

The danger is not simply that artificial intelligence might mislead us.

Paul agreed to run the experiment with his wife.

I left our conversation with more questions than answers.

Do we fully understand this technology? And if artificial intelligence can help us think, how will we recognize the moment when we begin allowing it to think for us?

The Possibility Before the Invention

We often tell the story of technological progress as a procession of human inventions.

The spear. The wheel. The engine. The computer. Artificial intelligence.

Each represents an extraordinary achievement of human ingenuity. Yet somewhere in that procession, we may have confused our ability to invent something with our ability to understand it—and our ability to understand it with our ability to control it.

But there are many assumptions hiding in that procession.

What, exactly, did we make? What did we discover? And when did the ability to make something work establish that we possessed the discernment to direct it?

Consider the spear.

Our ancestors fashioned sharpened branches into weapons, extending their reach and increasing their effectiveness as hunters.

But they did not invent wood, force, motion, or the physical relationships that made the spear effective.

They discovered how to put those existing elements together.

Now consider the buffalo jump.²

Long before modern weapons, Indigenous hunters on the North American plains learned to use terrain, animal behavior, and coordinated human action to direct herds of bison over cliffs.

The cliff already existed. So did gravity and the instincts of the animals.

The intelligence lay in the coordination as much as the implementation.

And what about preserving food?

Salt preserved food long before people understood its chemistry. Ice cooled food long before anyone invented a refrigerator.

Eventually, we developed machines capable of producing and maintaining controlled temperatures.

But did we invent cold—or discover how to harness principles that had existed all along?

Discovery and invention are not mutually exclusive. Innovation often emerges when we recognize existing possibilities and discover new ways to apply them.

Yet artificial intelligence introduces an important complication.

A refrigerator changes the temperature of its contents. AI can also influence the questions we ask, the explanations we accept, and the conclusions we reach.

Writing, printing, and mass media have long shaped human thought. AI is not the first technology to influence reasoning.

Unlike a book or a newspaper, conversational AI can respond to our questions, adapt to our reasoning, and influence our conclusions within a single exchange.

The technology can participate in the very deliberations through which we decide how technology should be used.

And this deserves closer examination.

Technological Imagination

In 1959, sociologist C. Wright Mills introduced the idea of the sociological imagination: a way to understand how our individual experiences connect to larger historical and social forces.³

His contribution was not merely another concept. It was a different way of seeing and thinking.

I propose a parallel perspective: Technological Imagination.

By Technological Imagination, I mean the capacity to look beyond the artifacts we manufacture and examine the possibilities we uncover, the relationships we arrange, and the assumptions embedded in their application.

But what does that require of us?

We must ask ourselves three questions:

What has become possible? What have we actually demonstrated? And what have we yet to understand about its consequences?

We readily recognize the machine.

But do we recognize what the machine reveals about our understanding of intelligence—and what it does not?

Consider how readily we associate intelligence with memory, language, inference, learning, creativity, and problem-solving.

When machines demonstrate some of these abilities, we confront two opposing temptations.

We may dismiss their performance because it does not originate in a human mind.

Or we may assume that because a machine appears to reason like a human being, it must think like one.

Neither conclusion necessarily follows.

In 2004, ChatGPT did not exist. But many principles that would help make it possible were already understood.

Mathematics, statistical inference, information theory, and decades of artificial intelligence research had laid the groundwork.

What remained were discoveries, engineering advances, and new ways to bring those elements together.

We did not invent the underlying mathematical principles. We discovered new possibilities in how to apply them.

The dots were there. We had not yet connected them.

That thought brought me back to a book I had read years earlier: Kevin Kelly’s What Technology Wants (2010).¹¹

Kelly describes what he calls the “technium”—not just individual inventions, but the expanding system of tools, knowledge, institutions, and human practices that develops around them. He asks us to consider technology as something more interconnected and evolutionary than any single inventor’s creation.

Kelly’s argument does not establish that artificial intelligence existed before we built it, or that technology possesses a will of its own. But it raises a question that the procession of inventions leaves unanswered: When we combine existing principles into an increasingly capable system, do we fully understand the possibilities and consequences that emerge?

That question becomes especially important when the technology begins to participate in human reasoning itself.

In 2017, researchers introduced the Transformer architecture in a paper titled Attention Is All You Need.⁴

The Transformer was an invention. It also demonstrated what could be accomplished by arranging existing mathematical ideas in new ways.

In November 2022, ChatGPT became publicly available.⁵

But did our ability to build it establish that we fully understood what we had created?

We built it. That establishes its origin. It does not automatically establish our mastery.

Intelligence That Remembers and Intelligence That Infers

Another encounter brought this distinction into focus.

During a rideshare, I recognized my driver, Steven, and recalled that he had moved south from Hell’s Kitchen in New York City fifteen years earlier.

He wondered how I knew those personal details.

The explanation was simple.

He had driven me before.

What seemed like remarkable deduction was simply memory.

An observer might have mistaken accurate recollection for brilliant inference.

I subsequently gave Steven a $10 tip. Then I blocked him from future rides for driving like a New Yorker in the South at 5:30 a.m.

That encounter gave rise to another question:

What's the difference between intelligence that infers and intelligence that remembers?

Memory and inference are not mutually exclusive. Human reasoning regularly combines them.

The distinction matters when we decide how much confidence a performance deserves.

Remembering a person's history does not demonstrate an ability to infer the same details about a stranger.

It could simply mean that, at sixteen, I was foisted to attend a Dale Carnegie seminar that caused me to miss several lacrosse practices and games.

A convincing AI response tells us little, by itself, about how reliable the reasoning behind it may be.

Fluency is not proof of accuracy.

Confidence is not proof of understanding.

And an answer that satisfies our expectations is not necessarily an answer that deserves our trust.

Consider the AI Sycophancy Test I proposed to Paul.

Suppose someone gives an AI system a questionable premise and receives an enthusiastic endorsement. Did the system test the premise—or merely agree?

One exchange cannot establish a model's overall reliability. It can expose a question worth investigating.

An AI system might instead reveal a contradiction I missed. If the evidence supports its criticism and I revise my position, it has improved my reasoning.

The difference lies not in whether the AI agrees or disagrees, but in whether its contribution survives a rigorous perusal.

The objective is neither automatic trust nor automatic rejection. It requires human discernment.

Human Hubris at the Podium

Artificial intelligence challenges more than jobs.

It challenges our intellectual identity—and our sense of being special.

For centuries, human beings have regarded reasoning, language, creativity, and sophisticated problem-solving as signs of our exceptional place in the natural hierarchy.

Now machines can outperform humans on certain tasks once regarded as uniquely human.

This raises uncomfortable questions.

If a machine can fashion sublime prose, what makes human writing valuable?

If it can retrieve and organize information in seconds, what does human expertise contribute?

If it can expose weaknesses in our reasoning, how much of our confidence depended on rarely being challenged?

And if it can perform substantial portions of our work, how should we understand our own contribution?

These are not merely “soup questions.”⁶

Nor should those questions obscure concerns about jobs, concentrated power, fabrication, hallucination, conflation, institutional accountability, and decisions affecting people who never consented to AI's use.⁷

But another concern deserves scrutiny.

Perhaps some of our discomfort comes from discovering that capabilities we considered uniquely human may not be ours alone.

Hubris—ask Achilles—can distort human evaluation in either direction.⁸

We may dismiss genuine machine competence because acknowledging it threatens our self-image.

Or we may become so impressed by technical performance that we grant a system more authority than its demonstrated capabilities justify.

Both reactions confuse the possession of intelligence with the exercise of human discernment.

Imagine a conductor standing before an orchestra.

The musicians perform beautifully. The score is before them. The conductor holds the baton.

But who is choosing the interpretation? Who determines the tempo? Who hears an imbalance and intervenes?

Occupying the podium does not, by itself, establish that the conductor is directing the performance.

Likewise, retaining a human approval step does not necessarily establish human control over an AI-assisted decision.

The essential question is whether the person still possesses the knowledge, independence, and practical ability to direct the outcome.

That distinction becomes increasingly important as AI capabilities evolve.

Icarus: The Wings and the Warning

Long before AI, a Greek myth asked what happens when our capabilities exceed our discernment.

Daedalus, the ingenious craftsman, constructed wings that enabled him and his son Icarus to escape confinement.⁹

The invention worked.

But Daedalus understood that flight came with conditions.

He warned Icarus against flying too close to the sun, where the heat would compromise the wings, or too close to the sea, where moisture could threaten them.

Icarus could fly. What he lacked was the discernment to respect the conditions that made flight possible.

Had the wings failed immediately, this would be a story about defective engineering.

Instead, technological success enabled a new danger.

The ability to fly had been mistaken for freedom from the conditions of flight.

Artificial intelligence may pose a more complicated version of that problem.

Daedalus understood the limitations of his invention well enough to issue warnings.

With AI, some limits remain uncertain, and the consequences of deploying increasingly capable systems are not always predictable.

But the story gives us a distinction worth keeping.

Intelligence expands what we can recognize and do. Human discernment enables us to distinguish what matters, recognize what is missing, weigh evidence and consequences, and determine what action is justified.

These capacities are related, but they are not interchangeable.

Nor does humanity possess an automatic monopoly on sound decisions simply because we are human.

Icarus was human.

His humanity did not prevent catastrophe.

The challenge, therefore, is not merely to keep a person involved in the operation of advanced technology.

It is to ensure that meaningful human discernment survives the encounter.

The Illusion of Control

Among the ancient philosophical traditions of the Indian world—Jambudvīpa in classical cosmological language—is the concept of Māyā, which invites questions about appearance and reality.¹⁰

Without equating that tradition with modern technological questions, it offers a useful metaphor for examining our assumptions about control.

We may believe that because we can activate, question, or deactivate an AI system, we necessarily govern its influence.

But control involves more than possessing a switch.

Consider a person who relies on AI to organize information, interpret evidence, recommend alternatives, and frame a decision.

The person may retain final approval.

Yet if the available choices, supporting explanations, and apparent priorities have already been shaped by the system, what exactly does that approval represent?

The influence need not involve deliberate manipulation.

A useful recommendation can become a familiar recommendation. Familiarity can develop into reliance. Reliance can gradually reduce our willingness—or ability—to examine the assumptions beneath it.

This does not make AI inherently dangerous or human independence inherently virtuous.

It means responsible collaboration requires more than the appearance of human participation.

What matters is the quality of the inquiry.

Can we identify unsupported assumptions?

Can we recognize when a machine flatters rather than challenges us?

Can we distinguish a persuasive explanation from a well-supported one?

And can we revise our position when evidence requires it?

Those questions apply equally to humans and artificial intelligence.

Never Surrender the Baton

Artificial intelligence can broaden an inquiry, challenge assumptions, expose weaknesses, and help organize difficult ideas.

Those capabilities have value. They do not relieve us of the responsibility to examine the conclusions they help produce.

Rejecting AI to defend human superiority shows poor discernment. So does accepting its answers without scrutiny.

The responsibility for the conclusions we draw cannot be surrendered to the technology.

Trust should depend on the task, the evidence, the consequences of error, and whether mistakes can be corrected.

A system that performs well in drafting correspondence has not thereby earned authority to determine someone's eligibility for benefits, resolve a legal dispute, or make an irreversible decision.

Human oversight must be more than ceremonial.

It requires access to evidence, freedom to disagree, sufficient understanding of the problem, and a practical ability to intervene.

Institutions that design and deploy these systems must also ensure that meaningful review, accountability, and remedies remain possible.

Technological Imagination asks us to look beyond the machine and reconsider the possibilities we have uncovered—and the assumptions we make about our ability to control them.

What humanity has accomplished is extraordinary.

We transformed branches into weapons, cliffs into hunting systems, cold into refrigeration, and mathematical relationships into machines that can generate language and assist with reasoning.

But discovering what is possible does not automatically establish what is wise.

And inventing a powerful capability does not prove that we possess the discernment to govern its use.

Daedalus understood that the wings came with conditions.

The ability to fly was only the beginning.

The question now is whether we will still know who is directing the flight.

Discernment over sycophancy.

Next in the series — Part II, sometime in November: From Assistance to Deference — How Humans Begin Surrendering the Baton.

Source Notes

1. Mrinank Sharma et al., “Towards Understanding Sycophancy in Language Models,” ICLR (2024). https://arxiv.org/abs/2310.13548

2. UNESCO World Heritage Centre, “Head-Smashed-In Buffalo Jump.” https://whc.unesco.org/en/list/158

3. C. Wright Mills, The Sociological Imagination (1959), chapter 1, “The Promise.”

4. Ashish Vaswani et al., “Attention Is All You Need” (2017). https://arxiv.org/abs/1706.03762

5. OpenAI, “Introducing ChatGPT,” November 30, 2022. https://openai.com/index/chatgpt/

6. Finding Forrester (2000), directed by Gus Van Sant; “soup question” refers to the film's dialogue about the purpose of a question.

7. NIST, Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile (2024), NIST AI 600-1. https://doi.org/10.6028/NIST.AI.600-1

8. Homer, The Iliad, Book I: the conflict between Achilles and Agamemnon, with Achilles’ anger and wounded pride as central themes.

9. Ovid, Metamorphoses, Book VIII, lines 183–235, “Daedalus and Icarus.” https://ovid.lib.virginia.edu/trans/Metamorph8.htm

10. Jambudvīpa is a name in Hindu, Buddhist, and Jain cosmological geography; it is not a single philosophical school. Māyā has different meanings across Indian traditions, including uses concerning illusion, appearance, and reality.

11. Kevin Kelly, What Technology Wants (New York: Viking, 2010).