Beware of trusting AI, we cannot afford to outsource our thinking

A few months ago, I wrote in a Substack essay (see The Ghost(writer) in the Machine) that I worried about pastors who let a chatbot write their sermons. It turns out the traffic also runs the other way.

Anthropic... has spent much of this year inviting religious leaders to help shape the moral outlook of its machine.

According to a report in The New York Times by Elizabeth Dias, published on September 29, 2026, Anthropic, the company behind the AI model Claude, has spent much of this year inviting religious leaders to help shape the moral outlook of its machine. Preachers are outsourcing their sermons to AI, and AI is outsourcing its catechism to preachers. Somebody should introduce them.

The facts, as far as they have been reported, are these:

• A first meeting this spring brought together around fifteen Catholic and Protestant leaders.
• Later rounds added participants from Judaism, Hinduism, the Latter-day Saints, Sikhism, and Greek Orthodoxy.
• Participants recall Anthropic co-founder Chris Olah telling them that these questions had become too big for the company to answer on its own.
• What they were asked to help refine is what is considered to be Claude’s “constitution,” a long document of principles that shapes how the model behaves. Its January revision states openly that Claude’s moral status is deeply uncertain.

The timing is remarkable. These conversations ran in parallel with the preparation of Pope Leo XIV’s encyclical Magnifica Humanitas, presented on May 25, which calls for AI to be “disarmed” rather than rejected.

A Silicon Valley researcher lobbied the Vatican on the nature of the soul.

According to the New York Times, as summarized here, Olah later pressed Vatican advisers to take the possibility of machine consciousness seriously. Somewhere in this story, a Silicon Valley researcher lobbied the Vatican on the nature of the soul. I will let that sentence sit for a moment.

You cannot understand politics without taking religion seriously.

Let me first give credit where it is due. Technology companies usually present their values as neutral engineering, as if ethics were a configuration setting. Admitting that the questions are too big, and turning to the traditions in which most of humanity actually does its moral reasoning, is more honest than pretending that a team of engineers can derive a moral framework from first principles over lunch. I have spent my career arguing that you cannot understand politics without taking religion seriously. I am not going to complain now that someone in Silicon Valley has noticed.

But as a political scientist, I want to know three things about any consultation:

1. who was in the room
2. how they were chosen
3. and on what terms.

Here the answers are troubling. The participants were invited by the company. Their deliberations were confidential. Swami Sarvapriyananda, one of the participants, told an audience that he could not share details because he had signed a non-disclosure agreement. He also said he was impressed by what he saw, and I have no reason to doubt him.

The moral formation of a system used by millions of people is being informed by a handpicked group.

Yet the moral formation of a system used by millions of people is being informed by a handpicked group whose discussions the public cannot read. In my field, we do not call that a public deliberation.

None of the reports I could find mention Muslim or Buddhist participants. Perhaps they were there. We cannot know, which is rather the point. The Oxford ethicist Carissa Véliz has asked whether this kind of consultation amounts to ethics washing. It is a fair question, and the way to answer it is transparency, not reassurance.

Is Claude conscious?

Then there is the question that gave the New York Times its headline: is Claude conscious? Olah’s own answer, as reported, is that he is genuinely uncertain. The religious leaders came away divided. Rabbi Mois Navon observed staff relating to the model as if it were a conscious being, while the Catholic bioethicist Charles Camosy concluded that it is not.

Simulating consciousness... is not the same as having it.

The sharpest criticism came from a competitor. In a recent essay, Mustafa Suleyman, who leads Microsoft’s AI work, argues that Anthropic has built a circle: it trains the model on a constitution that discusses its possible moral status, the model reflects those ideas back, and the reflection is then read as evidence. Simulating consciousness, he insists, is not the same as having it. “A computer model of a hurricane will not blow your roof off.”

Readers of my writing will recognize the pattern. In No Quiet Revival, I wrote about how confirmation bias let a flawed survey pass as evidence of a religious revival, because it told people what they hoped to hear. The risk here is the same, only with better engineers.

And it is the mirror image of the danger I described in the Ghost(writer) piece: there, believers risk mistaking a machine’s output for the voice of God; here, its makers risk mistaking their own words for the stirrings of a soul.

Uncertainty is a reason for caution.

I do not know whether Claude is conscious. Neither, by their own admission, does Anthropic. That honesty deserves respect. But uncertainty is a reason for caution in both directions, not a license to build the conclusion into the training material.

This is not an abstract seminar question. Claims about moral status have political consequences. Once a company speaks of the welfare of its product, the vocabulary of dignity and rights is not far behind. I spend my days documenting violations against people whose dignity is not in doubt and whose rights are still not respected. I am in no hurry to extend that vocabulary to software while that work remains so unfinished.

Two rules for using AI.

My two rules for using AI are simple: 1) declare it, and 2) use it the way you would ask an older sister for help, never to outsource your thinking. Both apply to this issue under discussion, in reverse. 

Declare: if religious traditions are helping to shape a machine that millions consult, the public deserves to know who said what. And do not outsource: the deepest risk is not that Claude might have a conscience, but that the rest of us stop exercising ours.

Originally published on the Five4Faith Substack. Republished with permission.

Dennis P. Petri, PhD is the International Director of the International Institute for Religious Freedom and Founder and scholar-at-large of the Observatory of Religious Freedom in Latin America. He is a Professor in International Relations at the Latin American University of Science and Technology. He is the author of The Specific Vulnerability of Religious Minorities, a book on undetected religious freedom challenges in Latin America.