Whether it's AI or us, it's OK to be ignorant

Our search for answers begins by recognising that we don’t have them.

Simon Walters

Simon Walters is Curate at Holy Trinity Huddersfield.

A street sticker displays multiple lines reading 'and then?'

Stephen Harlan on Unsplash.

When was the last time you admitted you didn’t know something? I don’t say it as much as I ought to. I’ve certainly felt the consequences of admitting ignorance – of being ridiculed for being entirely unaware of a pop culture reference, of being found out that I wasn’t paying as close attention to what my partner was saying as she expected. In a hyper-connected age when the wealth of human knowledge is at our fingertips, ignorance can hardly be viewed as a virtue.

A recent study on the development of artificial intelligence holds out more hope for the value of admitting our ignorance than we might have previously imagined. Despite wide-spread hype and fearmongering about the perils of AI, our current models are in many ways developed in similar ways to how an animal is trained. An AI system such as ChatGPT might have access to unimaginable amounts of information, but it requires training by humans on what information is valuable or not, whether it has appropriately understood the request it has received, and whether its answer is correct. The idea is that human feedback helps the AI to hone its model through positive feedback for correct answers, and negative feedback for incorrect answers, so that it keeps whatever method led to positive feedback and changes whatever method led to negative feedback. It really isn’t that far away from how animals are trained.

However, a problem has emerged. AI systems have become adept at giving coherent and convincing sounding answers that are entirely incorrect. How has this happened?

This is a tool; it is good at some tasks, and less good at others. And, like all tools, it does not have an intrinsic morality.

In digging into the training method for AI, the researchers found that the humans training the AI flagged answers of “I don’t know” as unsatisfactory. On one level this makes sense. The whole purpose of these systems is to provide answers, after all. But rather than causing the AI to return and rethink its data, it instead developed increasingly convincing answers that were not true whatsoever, to the point where the human supervisors didn’t flag sufficiently convincing answers as wrong because they themselves didn’t realise that they were wrong. The result is that “the more difficult the question and the more advanced model you use, the more likely you are to get well-packaged, plausible nonsense as your answer.”

Uncovering some of what is going on in AI systems dispels both the fervent hype that artificial intelligence might be our saviour, and the deep fear that it might be our societal downfall. This is a tool; it is good at some tasks, and less good at others. And, like all tools, it does not have an intrinsic morality. Whether it is used for good or ill depends on the approach of the humans that use it.

But this study also uncovers our strained relationship with ignorance. Problems arise in the answers given by systems like ChatGPT because a convincing answer is valued more than admitting ignorance, even if the convincing answer is not at all correct. Because the AI has been trained to avoid admitting it doesn’t know something, all of its answers are less reliable, even the ones that are actually correct.

This is not a problem limited to artificial intelligence. I had a friend who seemed incapable of admitting that he didn’t know something, and whenever he was corrected by someone else, he would make it sound like his first answer was actually the correct one, rather than whatever he had said. I don’t know how aware he was that he did this, but the result was that I didn’t particularly trust whatever he said to be correct. Paradoxically, had he admitted his ignorance more readily, I would have believed him to be less ignorant.

It is strange that admitting ignorance is so avoided. After all, it is in many ways our default state. No one faults a baby or a child for not knowing things. If anything, we expect ignorance to be a fuel for curiosity. Our search for answers begins in the recognition that we don’t have them. And in an age where approximately 500 hours of video is uploaded to YouTube every minute, the sum of what we don’t know must by necessity be vastly greater than all that we do know. What any one of us can know is only a small fraction of all there is to know.

Crucially, admitting we do not know everything is not the same as saying that we do not know anything.

One of the gifts of Christian theology is an ability to recognize what it is that makes us human. One of these things is the fact that any created thing is, by definition, limited. God alone is the only one who can be described by the ‘omnis’. He is omnipotent, omnipresent, and omniscient. There is no limit to his power, and presence, and knowledge. The distinction between creator and creation means that created things have limits to their power, presence, and knowledge. We cannot do whatever we want. We cannot be everywhere at the same time. And we cannot know everything there is to be known.

Projecting infinite knowledge is essentially claiming to be God. Admitting our ignorance is therefore merely recognizing our nature as created beings, acknowledging to one another that we are not God and therefore cannot know everything. But, crucially, admitting we do not know everything is not the same as saying that we do not know anything. Our God-given nature is one of discovery and learning. I sometimes like to imagine God’s delight in our discovery of some previously unknown facet of his creation, as he gets to share with us in all that he has made. Perhaps what really matters is what we do with our ignorance. Will we simply remain satisfied not to know, or will it turn us outwards to delight in the new things that lie behind every corner?

For the developers of ChatGPT and the like, there is also a reminder here that we ought not to expect AI to take on the attributes of God. AI used well in the hands of humans may yet do extraordinary things for us, but it will not truly be able to do anything, be everywhere, or know everything. Perhaps if it was trained to say ‘I don’t know’ a little more, we might all learn a little more about the nature of the world God has made.

Article

Belief

Biology

Creed

21 June 2024

5 min read

We’re gonna need a bigger ontology

Orca attacks prompt questions about being.

Henna Cundill

Henna is a researcher with the Centre for Autism and Theology at the University of Aberdeen.

A boat holding a camera crew drifts next to a whale fin.

Filming Shetland's orcas.

BBC.

In May 2023, British sailor Iain Hamilton was aboard his yacht in the Strait of Gibraltar when it was set upon by a pod of five orcas who succeeded in biting off both rudders, leaving him with no means of steering his boat back to shore. These enormous killer whales could have destroyed the small boat in its entirety, rounding off their escapades by making a quick lunch of Hamilton and his crew. But instead, they seemed content to merely play with the small vessel, pushing it around “like a ragdoll” for a while, before swimming away to find their next meal elsewhere.

How do we explain such behaviour? Environmentalists have been quick to suggest that the orcas are demonstrating their frustration with the human race – carrying out revenge attacks on those callous two-legged beings who overfish their waters and pollute their habitat. Other commentators propose a less anthropocentric view. One leading zoologist, Mark Cowardine, attributes the whales’ behaviour simply to play, “Boisterous play, yes, by animals weighing up to six tonnes, but nothing more sinister than that.”

The phenomenon of whales attacking boats is not new. Herman Melville’s magnum opus Moby Dick (published in 1851) is a fictional tale of one such encounter, inspired in part by the real-life sinking of a ship, The Essex, during a whale attack in 1820. However, there appears to have been a surge in such incidents in European waters over the past few years – more than 500 orca attacks were recorded between 2020 and 2023 alone. It is thought to be largely the same pod of whales who are responsible, but scientist fear that other pods are beginning to learn the behaviour.

This raises the question: at what point should humanity intervene to prevent the spread of knowledge? Theoretically, it would be possible, to isolate the ring leaders and remove them from whale ‘society’ (send them to ‘whale jail’ if you like). And, let’s be honest, in previous generations, trophy hunters would have blithely exterminated the troublesome pod without a second thought. But we live in more enlightened times, wherein we respect nature’s right to be protected from human interference.

The whale world has its own language, with distinct dialects, and is even thought to have culture, including celebration of life events and rituals for grieving the death of a family member.

On the other side of the globe, this right has even been enshrined in law. Pacific Indigenous leaders from the Cook Islands, French Polynesia, New Zealand and Tonga have agreed a treaty that officially recognises whales and dolphins as having legal personhood. The Whanganui River in New Zealand is also recognised as a “legal person” – a move intended both to enact reparations for the damage done to the river by European settlers, and to protect it from any future harm by the human race.

A being that is recognised as having legal personhood is one which has “rights and duties itself and which can enforce these rights against other legal persons.” So far so good for a river, which is vulnerable, not sentient, and certainly needs protecting from our shocking ability to exploit and pollute the natural world. But what can we say about whales and dolphins? Unlike the river, they are sentient. The whale world has its own language, with distinct dialects, and is even thought to have culture, including celebration of life events and rituals for grieving the death of a family member. With such obvious evidence of moral intelligence, should we be considering the ‘duties’ inherent to a whale’s legal personhood, as well as the rights?

The whales still seem to be communicating the same message: our ocean is vast, and we can make you humans feel your tininess in it.

In parts of the Hebrew Bible, animals are already described as having personhood. In the creation story both humans and animals are described as having nephesh – a Hebrew word that is sometimes translated as ‘soul’, and which indicates certain aspects of what it means to be sentient and have a moral conscience. Intriguingly, God seems to employ this sentience – at times employing animals to communicate with humans.

One famous example even includes a whale. When the runaway prophet Jonah was thrown from a ship into the ocean, we are told that God directed a large fish to swallow him up, and after three days return Jonah to dry land to continue the work to which God has asked him to do. In another example, when the donkey of the prophet Balaam was being unfairly beaten, the Bible records that the donkey turned and said to his master, “What have I done to you, that you have struck me?”

It is clear that some of the biblical writers believed that God could and would use animals to communicate with the human race, either through their behaviour or even through direct speech. Therefore, these orca “attacks” make me wonder if God may still be doing so today. Whilst both humans and animals are described as having nephesh in the creation story, the story does then go on to distinguish humans as having ‘dominion’ over the created order. The idea of what it means to have ‘dominion’ has been interpreted differently through the centuries of Christian thought. In the time of Moby Dick, when the fashion for trophy hunting and taxidermy was at its height in the western world, dominion had a feel of superiority and dominance to it. These days, it is more common to hear ‘dominion over creation’ described in terms of responsible stewardship and care.

But whilst human culture has changed (arguably for the better) it is noticeable that between Moby Dick’s time and now, the whales still seem to be communicating the same message: our ocean is vast, and we can make you humans feel your tininess in it. The temptation is there for us to intervene, to prevent these boisterous orcas from perpetuating their violent behaviour. This would serve to silence the voice that reminds us, uncomfortably, of our fundamental human vulnerability on the ocean. But perhaps we should not be too hasty. We cannot know if, inherent to the personhood of whales, they have a ‘duty’ to keep us in our place. Perhaps it is even their God-given call to behave in a way that reminds us that creation is ultimately, untameably, wild. Listening carefully, we might yet discover that God is speaking to us in whale song.