Are you sure?

Why I will always do my own writing and my own thinking

I was in the middle of a much-needed digital detox – a proper holiday – when I hopped back online briefly to check whether any venues were planning to screen the World Cup final the next day. I'd clocked that Spain and Argentina were through to the final and thought maybe, just maybe, somewhere on the Queensland island where we were staying would play it live, despite the brisk 5am kickoff time.

Years ago, Microsoft Word would treat its users to the presumptuous paperclip popping up with questions like: "It looks like you're trying to write a letter – would you like some help?" Now, Google forces its AI summary onto us, accuracy be damned. I wanted real-life listings from the websites of venues, but instead the unrequested AI summary told me confidently of three specific venues where I could "catch the World Cup final live". Due diligence required. I opened the links cited as sources and the pages did not actually say what the "summary" said they said.

Asked if it was sure (and it is an it – we must always remember this is not a person even if chatbots adopt personable language to foster the illusion they are friendly human-like helpers), it said directly, "Yes." This time it added a small caveat to check with the venues, which would seem a prudent step as none of them were normally open that early. Asked once more to explain specifically what source it was using to claim the screening was confirmed, it finally changed its tune: "There is no official, confirmed screening ... The previous information provided was incorrect."

I’m dropping a few screenshots of the subsequent exchange below, not because I’m suffering AI psychosis but because I genuinely believe a lot of people have absorbed a lot of corporate hype about AI supposedly changing everything but aren’t familiar with the nuts and bolts of how large language models actually work and their limitations.

That is crucial. Of course, having sucked up vast amounts of data for training, these large language models (LLMs) are reasonably good at predicting the right word combinations, but they are not truly thinking in the way we do. To oversimplify: an LLM divides text into chunks (think a single word or even smaller chunks like a few letters) and converts them to numbers while working out how likely it is that another chunk of letters will come next, and it keeps repeating this process until it’s finished constructing its answer. Sometimes, in trying to come up with a plausible-sounding answer, a chatbot will spit out wrong or misleading information or will "hallucinate" a source that doesn't exist. Near enough is good enough. That might be why, until recently, the terms of use for Microsoft Copilot cautioned that it should be used “for entertainment purposes only”. The best marketing trick, it seems, was to put "intelligence" in the title.

The next morning I was up at 4.55am ready to watch the game streamed live on my phone via SBS On Demand. Out of curiosity, I repeated on the same device the same basic Google search about whether there was a live screening on the island. Again, the AI panel turned up at the top of the results to say yes – even though a version of the same panel had admitted the previous night that info was fake.

**

It was a few years ago that the Canadian-British writer Cory Doctorow popularised the term "enshittification" to account for how digital platforms become worse over time. More recently, Doctorow has described AI as "the asbestos we are shoveling into the walls of our society and our descendants will be digging it out for generations". He maintains "it is imperative to see through that hype to the real story, to understand the technology not just for what it does, but for who it does it to and who it does it for".

No doubt the objection will be that some models are better than others, that they will get better over time, that they are particularly good at some types of usages (particularly around data processing or coding tasks), and that AI agents are being developed that can do complicated chains of tasks quickly. Maybe. But I'm suspicious of the god-like aura that is being placed around these tools. In a newly published special report for Reuters, the journalist Katie Paul wrote about how Meta employees' use of AI "had resulted in a vast increase in the code they generated, but with questionable impact on productivity". Paul explained:

"As early as March, infrastructure teams were also warning of 'reliability warning signs' caused by the AI coding surge, according to an internal post. Another post, in April, said that unchecked AI agents were performing 'large-scale, disruptive actions that humans are unlikely to execute.' The result: Major technical and security incidents, such as service disruptions and possible data leaks, spiked 40% from the previous year, with the time staffers had to spend 'firefighting' them up 70%, according to the internal posts."

A recent study on a phenomenon known as “cognitive surrender” found participants in the research were willing to accept faulty AI reasoning 73.2% of the time, compared with overruling it just 19.7% of the time, Ars Technica reported.

As a writer I do have specific concerns about generative AI. I worry that if we as a society increasingly rely on this technology, we will see the proliferation of formulaic slop, further muddying the waters on what is true and what is not. It has the potential to devalue the creative industries and degrade our ability to engage in deep, critical thinking.

The Turkish American author and critic Merve Emre sounded the alarm in interview with Kill Your Darlings published a few weeks ago: "We are living through an appalling de-skilling of writing." According to Emre, AI has given us the ability to detach the process of writing "from spontaneous, vigorous processes of thinking".

I also find it hard to detach the AI hype from the ideologies and visions of the big tech figures who are promoting it as The Next Big Thing and who have cozied up to power:

(Elon Musk wasn't at that dinner, but I stopped tweeting back in late 2022 shortly after Twitter's new owner reinstated a raft of Islamophobic, antisemitic, transphobic and misogynistic accounts. At the time, I found this to be a clear statement of intent about the hateful direction of travel we would see on the platform under his watch. Fast forward a few years and the ABC's Media Watch program has taken an in-depth look at what the world's first trillionaire has been up to on his platform, now rebranded as X, in connection with race-based attacks. Musk has also endorsed posts talking about a "Reconquista".)

Sam Altman, the CEO of OpenAI (of ChatGPT fame), is heavily boosting the AI boom. "We are close to creating a genie that can grant any wish," he claims. More interesting, though, are his pronouncements about how to monetise these tools. Earlier this year, Altman, described the economic opportunity of AI as "this new incredible wealth fountain" and said:

"Fundamentally our business and I think the business of every other model provider is going to look like selling tokens … We see a future where intelligence is a utility like electricity or water and people buy it from us on a meter and use it for whatever they want to use it for."

Altman predicted consumer demand for AI products would continue to grow, and then laid out what was meant to sound like a benevolent vision of benefit-sharing but in reality was an entirely self-interested argument to remove regulatory limits on his business:

"If we don’t have enough we either can’t sell it or the price gets really high and it kind of goes to rich people, or society makes a bunch of central planning decisions that I think almost always go badly about, you know, 'we’re going to use our limited compute supply for this and not that'. So the best thing to me, throughout all the history of capitalism, innovation, whatever you want, is to just flood the market."

Meta chief Mark Zuckerberg is another tech leader who is desperately trying to frame his business interests in benevolent terms. On 10 August, Zuckerberg published a 6500-word essay titled 'The Future is for Everyone'. He set out "a few specific ways Meta is building personal superintelligence to improve all of our lives", arguing:

"Everyone will have an exceptionally capable personal agent that understands you, your goals, and everything you care about. Your agent will work 24/7 on your behalf to improve your relationships, health, career, finances, home management, hobbies, and more."

Um, thanks but no thanks. The tech companies already know a lot about us but do we really trust Meta to have vastly more visibility of (and intrusion into) our personal lives? It's hardly a surprise that concerns are being raised about the increased scope for mass surveillance and repression.

Zuckerberg again:

"You’ll be able to interact with your agent through any device, including your glasses to keep you present in the moment with the people you care about."

Oh, yes, that sounds very "present". What a hellscape.

Zuckerberg seemed at pains to claim "everyone" would have "free or affordable access to these tools" (wonder how they can make it profitable, if that's true). He wrote:

"We will offer free versions that will be accessible to billions of people. For those who want to pay to use more compute, there will be a dynamic auction mechanism that will guarantee that everyone gets the lowest price possible for the intelligence and compute they’re using while also ensuring the capacity is used for whatever people collectively find most valuable. This will ensure the benefits of superintelligence are distributed widely."

Jason Koebler from 404 Media counters: "Like most Silicon Valley 'utopian' essays, to believe that any of this is going to go how Zuckerberg suggests it will requires one to have been recently concussed or to willfully ignore how this technology is being used today and believe that thousands of years of human nature will suddenly shift."

**

This all coincides with upheaval and fragmentation in the media industry, overlapping with financial pressures and the emergence of smaller independent outlets that make a deliberate decision to reflect the values of their audiences. Some news outlets are “embracing” AI, almost certainly driven by budget pressures to do more with less. But this is likely risky for trust. Multiple US newspapers publishing an AI-generated summer reading list that included fake titles by real authors should be a cautionary tale.

Just 43% of people say they trust the news as a whole within Australia, according to the latest Digital News Report for Australia. The figure has bounced around over the years but was as high as 50% in 2018. The latest survey also found just 19% of Australians had trust in news from AI chatbots. More generally, more than three quarters of Australians say they are concerned about “what is real and what is fake on the internet”.

I don't know exactly what the future of journalism holds, but I am sure it will depend on doing things that build trust we're giving audiences our best attempt at telling them the truth. It will include a more direct (and yes, accountable) relationship between journalist and reader. The unaccountable "voice of God" style of journalism ("this is how it is") is past its use-by date. So I'm launching this blog and newsletter as a way of creating that two-way channel with my reading audience, unmediated by big tech platforms.

A couple of years ago a hairdresser told me she had stopped paying attention to the news because there were so many competing claims around and she didn't know who or what to believe; why should she trust the media? The words have stuck with me. And it's true – everything is getting noisier and contested. I will use this newsletter to explain my journalistic processes and publish underlying documents (such as material obtained under freedom of information laws) so readers can see for themselves whether my interpretation stands up to scrutiny. This amounts to showing you my working. My guarantee to you is that I'll never use generative AI for writing. If I make errors, they will be entirely my own, human errors.

You might be wondering about the name, Dominoes Under the Mulberry. It is a nod to the old ritual of my grandfather's family in La Línea in southern Spain decades ago. Abuelo and his family used to gather under a mulberry tree and play dominoes. I plan to write about this in more detail when I have time, but the name is meant to signify stepping back from the noise and thinking about both the big and small things in life.

So sign up to my newsletter now to receive an email in your inbox every time I publish a new post. And importantly, write back!