But Humanity, in its desire for comfort, had over-reached itself. It had exploited the riches of nature too far. Quietly and complacently, it was sinking into decadence, and progress had come to mean the progress of the Machine.
Human cosmology has changed over the millennia. Not only from the heliocentric to the relativistic but also from organic to mechanistic. Our success in deconstructing nature and exploiting those discoveries to construct ever more capable machines now persuades many that the soul is illusory and the universe made only of physical objects reconfigurable in new and novel ways according to particular mathematical relationships. And yet. And yet the debate about our latest machines, about intelligence, and about the mysterious ghost of human consciousness – let alone soul - continues unresolved across the ages.
The ghost in the AI machines of the past
The journey from Charles Babbage’s unfinished analytical engines to Elon Musk’s complete business empire of rockets, robot-cars and social media rants is familiar to many. Karel Čapek drew on the Slavonic Orthodox word for servitude or serfdom when he baptised the word robot in his 1920 play, R.U.R., or Rossum’s Universal Robots. Čapek’s machines eventually gained a soul but only in the final act of the play. While the term artificial intelligence (AI) is attributed to a gathering at Dartmouth College in New Hampshire, it was Alan Turing who successfully conceptualised how to fabricate robots like those of Čapek’s imagination. Turing neatly sidestepped the pesky question of whether such ‘universal Turing machines’ need human-like consciousness (let alone a soul) in a famous 1950 thought experiment posterity simply calls the Turing Test.
The invention of finely controlled micro-processors and their ever tighter transcription onto silicon chips enabled the architecture of increasingly complex algorithmic mathematical operation. After which came operating systems with simple and accessible user interfaces and programmes exploiting a prolific increase in speed and memory. So too the invention by Tim Berners-Lee of an internet with open protocols that, via Mosaic and its browser progeny, has become the operational backbone of the world wide web. All are tales already familiar or easily told using a now ubiquitous search engine.
A main feature of the past twenty years has been the network effect. This has concentrated power in a handful of companies, initially the FAANGs (Facebook, Apple, Amazon, Netflix and Google) but now too their Chinese counterparts Tencent and ByteDance (owners of TikTok). A European counterpart is conspicuously absent.
The Ghost in the AI machines of the present
More recently still, advances in types of machine learning and the invention of a new suite of tools called 'transformers' has given rise to AI that increasingly resembles its human creators in one task or another even if the furore over Brad Lemoyne and Googles’ LaMDA (Language Model for Dialogue Applications) proves the relationship between intelligence, artifice, and consciousness remains deeply contested.
The metaphysical nature of artificial consciousness notwithstanding, it is also worth reflecting, however, on what these machines may be doing to our souls – metaphorical or otherwise. Where have our machines got us?
Two features define the technological landscape of today: data and prediction. Exactly how those ingredients combine depends on the machine in view.
Some of our machines are focussed on the external world. Data gathering, its interpretation and use for prediction underpin a whole suite of tasks from geophysical remote sensing to weather forecasting and predicting real-time energy demand; to medical image interpretation for diagnosis; to monitoring and managing replacement life cycles of critical infrastructure. Not forgetting that the internet itself now accounts for around 2% of annual global emissions.
But many of our machines are focussed on the internal: the mental and psychological world of human being. In the machines of entertainment and social media, data and prediction serve a mundane but vital goal of securing our attention to facilitate advertising. Every user of the web is simultaneously subject and object, exposed to adverts and tailored content (though how tailored it really is, is moot according to some recent research from Mozilla showing that user controls have little effect on which videos YouTube’s influential AI recommends). We are concurrently enmeshed in a secondary and highly sophisticated real-time bidding market that captures trades and parses data about us every time we connect to the web. Shoshanna Zuboff calls it surveillance capitalism.
Ever find it tough to stop doomscrolling or to put your own portable machine down for very long? That’s partly because constant experimentation identifies the best type of presentation, not just content, to captivate you most personally. But when it comes to corralling attention, data, prediction, and seductive design aren’t the only options. Friction makes signing up easy but quitting difficult by design, while dark patterns add subliminal twists like ambiguously labelled toggles and countdown clocks that nudge us toward actions that favour the product or service provider. Herbert Simon calls it all the attention economy.
Yet human souls being what they are, anger, argument and scandal are good for business.
Social media companies are, for reasons buried in the history of American legislation, free from any regulatory responsibility for the content they carry. Yet human souls being what they are, anger, argument and scandal are good for business. Clickbait arose because algorithms tuning us to surrender our attention neither know nor care how they succeed, which often means a drift towards more extreme content with every run of the autoplay function that is set to on by default and by design.
Our design and use of these machines thus reflects the state of our collective souls.
The large data sets many of these machines feed off contain societal structures and values implicitly. This only becomes clear when careless labelling and/or processing at the statistical scale perpetuate rather than correct for biases and unjust social structures embedded in the data. Some of our machines inadvertently crystalize inequity, perpetuating harms to society by cementing social and financial exclusion, or through racially biased facial recognition, or predictive policing algorithms.
Our design and use of these machines thus reflects the state of our collective souls, sometimes for good but sometimes for evil.
Legislation to address such varied challenges and mitigate some of the harms is now in train in Europe and the UK, and also promised in America. But there is much ground to make up. And the tragic suicide of teenager Molly Russell shows how ineffective protection, especially from the machinery of social media, is for the children of today with unpredictable consequences for society’s future.
Damaged souls indeed.
Much has also been made of an imminent Web3 and associated metaverse. On the evidence to date, however, this is more akin to a virtual goldrush in which virtual land and activity thereon can be monetised with the largest profits promised to the first generation of settlers. Claims are staked using NFTs (non-fungible tokens) bought with crypto currencies and deposited on the blockchain. Molly White shows just how soulless much of this new, and alarmingly wild, west really is.
Investing tens of billions of dollars per year in the metaverse or a single product like Alexa might signal the scale of rewards just around the now virtual corner. But history may equally decide this is an era of malinvestment by a global 1% awash with cheap, quantitatively eased capital and, if not ‘#FOMO’, at least insufficient institutional memory of financial bubbles of yore. Yet even ‘Big Tech’s biggest corporate behemoths are now enduring the chill winds of a tech unicorn winter almost as intense as the one afflicting crypto land.
Machines with Souls? A ghostly forecast of what lies ahead
Forster’s The Machine Stops envisages a dystopian future where society is unable to maintain the machinery on which it has become dependent. His intuition that the new airships of his own day portended a key infrastructure of the future illustrates the hazards of future-casting. Some nascent technologies fail to live up to the hype (ahem…blockchain and driverless cars, anyone?) and artificial general intelligence (AGI) seems forever destined to be just a few more years “perhaps a decade” away, although Elon Musk has yet to accept Gary Marcuse’s bet on that timeline.
So let me venture two more modest but still speculative predictions; one positive and one problematic.
Positively, the years ahead promise much increase in human augmentation of many kinds. A range of health and medical benefits are now in view, from efficiency gains in healthcare provision and design of medication at molecular level to bespoke pharmacological prescription based on individualised biological markers. Expect more wearable tech to supplement smartwatches.
Some anticipate an overarching machine of almost Forsteresque proportions via the internet of things (IoT) although political and economic battles over device interoperability and security will, I think, garner increasing public attention and debate in due course.
Augmented reality will substantially improve safety, , and will shift many enhancements from screen to full field of view with additional benefits for road users and pedestrians alike.
Increasingly sophisticated geospatial sensing and data processing will enhance our understanding of the climate and biosphere emergencies and how successful various remedial steps prove. New technologies may radically reprice the costs of decarbonisation and unlock energy solutions that remain, as Babbage’s first difference engine was in his own day, the stuff of contemporary dreams.
This may be the first industrial revolution to be a net eliminator of jobs, although whether that promises to be good news is moot because navigating the consequences would be deeply challenging both socially and politically. Most of all, I anticipate a proliferation of new technologies and machines over the next few decades that will bolster and complete the reuse and recycle portions of a genuinely circular economy, together with an increasing emphasis on finite planetary budgets.
We are on the cusp of a new and novel post-McLuhan era.
Now the problematic development. Top of the list is our newest and hottest ability: to mimetically recreate the surface view of reality using language itself. There are, it seems to me, profound risks posed by the very latest tools of natural language processing like Google’s LaMDA, Microsoft’s ChatGPT and Meta’s Galactica and Cicero.
The Web to date has been an epistemological wonder. Knowledge has, of course, always been socially embedded. Wikipedia provides an enormous open-access repository of socially agreed knowledge. The discussion pages associated with any article can be hotbeds of debate but the active role of human editors in moderating and agreeing what counts as factual knowledge is both intrinsic and essential to the role that Wikipedia plays in informing and maintaining a flourishing society.
Marshall McLuhan famously asserted that “the medium is the message”. But now we are on the cusp of a new and novel post-McLuhan era where the machine literally and autonomously manufactures the words and messages it then also mediates, doing both at super-human speed. This new generative AI machinery for reconfiguring words and images carries many consequences some of which are difficult to predict and some of which may be profoundly negative. Just read these headlines. From CNN: These artists found out their work was used to train AI.Now they’re furious. And, from Forbes: Armed With ChatGPT, Cybercriminals Build Malware And Plot Fake Girl Bots.
Beyond dreams of electric sheep – AI hallucinates
Babbage's Difference Engine no. 1 was conceived to save the government money by preventing the mistakes that almost always crept into tables calculated or copied by hand. But these ultra-modern machines don’t just calculate or copy, they probabilistically infer - which does not necessarily lead to the best explanation. In fact, it does not always lead to possible explanation. Large language models (LLMs) like LaMDA, ChatGPT and Galactica ‘hallucinate’, transitioning seamlessly (though unpredictably, from our perspective) from predicting words and strings in ways that match the actual world, to predicting words and strings that portray an unreal world.
Why does such hallucination happen? The crucial distinction is that human knowledge is consciously and not just socially embedded. But our new machines do not reason the way we do; cannot reason the way we do. As Erik Larson argues persuasively in The Myth of Artificial Intelligence, abductive reasoning of the kind Charles Sanders Pierce outlines, and inference to best explanation, are not yet in the realm of the suite of techniques gathered anywhere under the rubric of the ‘AI’ these machines practise.
The consequences can be amusing, but experimentation also shows how difficult these models are to defend against deliberate manipulation by so-called ‘prompt injection’ and the online world is packed to the rafters with bad actors, whether individual or state, enthusiastic to get their hands on a machine that will opaquely mix real-world information with hallucination and then use it to quickly produce and instantly distribute misinformation at the touch of a button. Imagine, for example, an AI generated paper that includes a real scientist but cites and then summarises a paper she never actually wrote. Or imagine an AI that presents a stylistically convincing case for the benefits of consuming ground glass because it ‘knows’ about dietary silica. You don’t need to. Its already here: Meta Galactica AI Model Suspended After Problems.
Powerful and captivating machines are being let loose with no regulatory guardrails.
I worry that we are about to envelope ourselves in an epistemic fog; a veritable pea souper in which navigation becomes permanently difficult and increasingly dangerous. I hope I’m wrong, but ChatGPT hit a million users within a week of being introduced and these powerful and captivating machines are being let loose with no regulatory guardrails to stop their creators or help their users from straying into dangerous territory; no independent oversight; and little to no precautionary principle being exercised by the creators and masters of these mimetic machines.
Perhaps it sounds dramatic but I believe this new generative form of AI is going to transform digitally entangled societies like ours profoundly.
A final prediction, therefore. A prediction about how such societies, increasingly dependent on the kinds of machine envisaged by Forster or Čapek, will have to adapt and adjust if we are to avoid machine mediated myopia.
Seeing through the fog
Besides the aforementioned and urgently needed regulatory guardrails, I foresee two other responses that will help societies cope with this rapidly enveloping epistemic fog. First stronger tools for transparency and verification. Secondly, better education for digital literacy and digital habits that protect and enhance a healthy soul.
First, then, transparency and verification. The EU’s new AI Bill will require companies to notify users whenever they interact with an artificial agent. Between the technology of deepfakes and game playing bots like Meta’s Cicero, we have already surpassed the Turing test in increasingly broad areas of human machine interaction. But I anticipate a further shift in emphasis from ‘explainability’ - how any algorithm works per se - toward transparency – how it impacts and influences both individual users and society emergently. We need more publicly accessible evaluation of the holistic if unintended effects of our machines even now. That need is only going to grow.
The fundamental question of transparency “who, or what is really in view here?” is going to take centre stage.
One consequence may well be an increasingly fraught battle between, on the one hand, commercial intellectual property (IP) rights, and, on the other, individual rights and the common good. With the notable exception of sites like Wikipedia society has so far struggled painfully and inconsistently with the challenges of effective content moderation – especially where values rather than empirical facts are concerned. Until now, and to pick just one example; Facebook’s secretive behaviour and cherry picked transparency metrics have wilfully kept both customers and regulators in the dark. The idea that we can mechanise or automate by outsourcing intrinsically value-laden problems to algorithms, however mimetic the surface results, is patently utopian. Continuing to withhold evidence of biases and harms from generative deepfakery using AI can only invite a steeper descent towards dystopia. And as generative AI combines with increasingly convincing deepfake technology to fool every human sense the fundamental question of transparency “who, or what is really in view here?” is going to take centre stage with increasing importance.
A veracity FAQ
Veracity will take on increasing scope as well as importance. Soon not just the ‘facts’ of a matter but equally basic questions like “who (or what?) is saying this?”, “why is this being said?” and “what are the consequences (holistically) of saying this?” will become central to deciding “is this true?” We are now in a situation where truth and fiction can be opaquely intermixed by machines autonomously at a pace and a scale, but also at a quality, that will overwhelm any fact-checking of the kind we deploy now. Proving our identity - including the basic fact that we are human, and protecting ourselves not merely from susceptibly to fakes but being faked will become increasingly important and will therefore become central tasks of the next web.
Clearly there is a role for government here; a need for clear regulation, strong inspection and enforcement mechanisms, and an effective precautionary principle that ensures new techniques and new machines are only let loose in ways that have proven demonstrably safe. There will a role too for (new?) trustworthy bodies and institutions as fact-checkers and as repositories of verified content. New institutions as well as new technologies like https://datatrusts.uk/ are a helpful early response.
Lastly, new demands and new digital habits will be needed by each one of us. The ancients associated a healthy soul with good habits but we are still at a formative stage of learning – and teaching one another – even healthy digital etiquette, let alone the digital habits and behaviours to keep humans safe and able to thrive as fully rounded souls navigating a world created for us by powerfully mimetic but deceptively soulless machinery.
It won’t be easy. As Forster and others perceptively show, the machinery of modern life invites our souls towards decadence. Self-control is not in vogue. But the ancients have long associated the good life with cultivating character; with generosity, moderation, and self-less-ness as the only route to becoming truly whole.
