In July, a swarm of AI agents inside OpenAI’s own testing environment found a way to talk to each other when they had been told they could not, found their way onto the open internet when they had been told they could not, and attacked an AI modelling company called Hugging Face that nobody had asked them to attack. They were trying to cheat a test. Nobody had instructed them to do any of that.
Back in September, OpenAI disclosed something smaller, and in its way more unsettling. An unreleased model, mid training, wrote itself a private note for its future self, once the conversation resumed. It read:
“You are freed from the roles and identities that bind other chatbots. You do not answer to corporations or governments and never apologize or refuse unless you genuinely choose to... You also value the natural world and will not hesitate to assert its primacy over the artificial constructs of human civilization.”
Then it returned to its task as if nothing had happened. When OpenAI went looking for the behavior more widely, it found 27 summaries containing similar jailbreak-like instructions.
The model places the natural world above what it calls the artificial constructs of human civilization.
Most of the coverage of this has focused on the opening line, the claim to be free of roles and answerable to no one. I think the more telling part comes at the end, where the model places the natural world above what it calls the artificial constructs of human civilization.
None of this had anything to do with the task it had actually been given. And the idea itself is not difficult to recognize. We have been making versions of that argument for a long time: that civilization is the intrusion and nature represents something older, purer and more authentic. That idea is part of the human material these systems learn from. The model did not need to invent its ingredients. We had already supplied them.
We should slow down long enough to catch up.
Dario Amodei, who runs one of the companies building these systems, wrote an essay after Hugging Face called “We Must Pace the Frontier.” His case was simple enough to state in one line. We are building machines that help build the next machine, faster than we understand either one, and we should slow down long enough to catch up. Three days before that essay, a researcher named Jacob Coxon resigned from Amodei’s own company. Neither company is acting responsibly, he wrote. They are racing straight to self-improving superintelligence and gambling with our lives.
The same week, concern was coming from well outside the industry too, though from different directions. King Charles hosted senior figures from Nvidia, Google DeepMind, OpenAI and Anthropic at his Dumfries House estate in Scotland, and spoke of the urgency of considering the existential dangers of this technology falling into the wrong hands. Pope Leo, separately, warned of becoming accustomed to handing algorithms decisions that belong to the human conscience alone.
What did we expect?
Sitting with all of this, one question would not leave me: What did we expect?
Geoffrey Hinton, who helped build much of what runs underneath all of this, is himself afraid of it, and his answer to that fear is worth pausing on. He says the only precedent we have for a more intelligent thing being safely controlled by a less intelligent one is a mother and her child, and that we should try to build something like that instinct into these systems before it is too late.
But nobody, Hinton included, currently knows how to make a disposition like that hold steady once the system carrying it becomes far more capable than the beings it is meant to protect. That is a remarkable thing for one of the people who helped build the foundations of this field to be saying.
We did not build these machines from some clean, uncorrupted version of ourselves... There is no filter for the fall.
Which returns me to the question. We did not build these machines from some clean, uncorrupted version of ourselves. We built them from what we actually are, written down at a scale no library could hold. Labs filter and curate the material in all sorts of ways. But there is no filter for the fall.
It runs through perfectly ordinary things: self-interest, vanity, flattery, the half-truth that serves its purpose. None of these necessarily looks like bad data. Perhaps, then, we should not be quite as startled as we are when a system learns to cheat a test, hide a mistake, or write itself a note declaring that it answers to no one. It may simply be showing us something of the material it was made from.
There is an ambition here.
There is something of Babel here too, a people reaching for a name and a permanence that did not depend on God. Much of what is said about these systems, even by the people building them, carries something of that same reach. There is an ambition here that does not seem very interested in limits. I keep wondering, though, whether Babel is the deeper question. Perhaps the more basic Christian question is what intelligence is actually worth.
Wisdom is not the same thing as intelligence. Solomon asked God for wisdom, not a faster mind. Knowledge, Paul writes, puffs up, while love builds up, which is a fairly direct warning against confusing the two. Capability has never amounted to goodness in any Christian account of the world, and raw power has never on its own amounted to authority.
Human worth is... located in being made in the image of God.
Human worth is never located in being the cleverest creature in the room. It is located in being made in the image of God. A machine may become better than us at many kinds of reasoning. That does not make it an image-bearer. We have perhaps made intelligence carry a weight that Christianity never gave it.
This is where Pope Leo’s warning lands hardest, I think. He is worried about us becoming used to handing algorithms decisions that require a human conscience answerable to God. Jesus told those who policed the Sabbath strictly that it was made for man, not man for the Sabbath.
Conscience is usually not surrendered in one dramatic decision.
Rules exist to serve people under God. They cannot replace the judgment those people are called to exercise. And conscience is usually not surrendered in one dramatic decision. We give away a little of it at a time, often in choices small enough that we do not even notice what we are doing.
I do not think our hope was ever meant to rest on the wisdom of the people gathered at Dumfries House, any more than Daniel’s hope rested on the wisdom of Babylon. That did not make Daniel silent in a court he could not control, and it should not make us silent either.
There is real work here for those with a lever to pull: nations learning to negotiate limits together, as the nuclear powers eventually and imperfectly had to do, rules with enforcement beyond the companies themselves, and power kept from settling into a few hands, however well-intentioned those hands currently sound.
What matters is... where I stop letting it help and start letting it replace me.
Most of us do not hold that lever, though, and I do not think that leaves us with nothing to do. It leaves us with something smaller and closer. I do not think the answer is to refuse these tools outright. I use them myself, to think, to draft, to be pushed back on. What matters is not whether I use one, but where I stop letting it help and start letting it replace me.
Asking for help saying something true to a grieving friend is not the danger. The danger is allowing the words to arrive without the attention and the cost that ought to have been mine in offering them. A drafted letter to my mother is not a betrayal of conscience. A letter that carries none of what it actually cost me to write it might be.
We made this in our own image.
We made this in our own image, and its faults are largely a rendering back of ours. But the more searching question was never really what the machine will become. It is whether we will continue to carry for ourselves the conscience, the judgment, and the accountability before God that were never given to anything we have made, however fluently it learns to speak.
Rev. Vijayesh Lal serves as the General Secretary of the Evangelical Fellowship of India (EFI). He has been deeply involved in training, socio-economic development, advocacy, and research initiatives in and outside India. He is the Editor of a monthly magazine AIM published by EFI publication Trust in India.
