But large language models and machine learning have the potential to improve public safety, boost productivity and transform some professions, both for better and worse.
The American and British internet entrepreneur revealed his thoughts on the emerging technology at Brisbane City Hall on Tuesday night in an event to promote his book, The Seven Rules of Trust.
His predictions come after research firm Gartner lifted its forecast for global spending on AI by 47 per cent to $US2.59 trillion, and after the federal government revealed plans to launch a $70 million AI Accelerator program in its budget.
Despite huge potential in multiple fields, Mr Wales said large language models remained deeply flawed and prone to making mistakes that were challenging for humans to identify.
"One of the dangers of the mistakes that they make is that they tend to make very plausible mistakes and so it's harder to catch," he said.
"If you're just a casual reader and you want to know about something, it's going to make something up to tell you but unfortunately, it will be hard to tell because it won't be completely crazy."
To test the accuracy of generative AI models, Mr Wales said he regularly quizzed them about his wife, Kate Garvey, who had achieved moderate fame after serving as UK Prime Minister Tony Blair's chief-of-staff.
In one case, an AI tool made up the name of her business partner and in others it gave her a different spouse, ranging from politicians and journalists to family friends.
Other elements of artificial intelligence and machine learning technology had potential to improve productivity and safety, Mr Wales said, such as unlocking autonomous driving, reducing administrative tasks and developing web resources.
"The technology is amazing if we think beyond just large language models into all kinds of things," he said.
"At the same time, we're going to experience quite a lot of disruption in the labour market and in ways that are hard to predict right now."
Among the first people to be affected by AI advances would be long-haul truck drivers who would need to re-train, Mr Wales said, and junior lawyers who were often asked to analyse and search through large contracts - a task that could be performed by software.
Maintaining transparency about how the technology was used would be vital to establishing trust in it, he said, just as it was for other types of technology and media, including Wikipedia.
15°C