Once you understand that these are chatbots that were designed to complete challenges like this, using tactics like this, you can understand that the chatbots didn’t “go rogue.” They did what they were designed to do, and because OpenAI ran them with inadequate supervision (without a “human in the loop” that checked each iteration through the Python loop to ensure it hadn’t gone off the rails), they trashed a competitor’s servers.
Designing autonomous, malicious software is generally considered irresponsible and dangerous. If you showed up at Defcon and gave a talk about how your autonomous malware did something unexpected and damaged someone else’s computers, the first question from the audience would be “Why are you so shit at making secure sandboxes?” It wouldn’t be “How are you so awesome at making hacking tools?”
The fact that OpenAI is making it much easier for unskilled people to break into and damage servers is indeed very bad news, but it’s not new bad news. Irresponsible parties have been doing this for years, most notably the NSA…
…
Riley had a very good way of summarizing this: “LLMs are real, AI is fake.” LLMs – chatbots trained on things like CTF logs that can break into servers – are real. They’re on a continuum with other hacking tools that have been steadily demonstrating the fragility of the modern digital world, albeit without inspiring anyone in power to do anything about it.
“AI” – chatbots that wake up, “set their own goals,” and “spontaneously” start hacking servers – is fake. It doesn’t have “a 10% chance of ending the human race.” The Hugging Face hack isn’t a mysterious, supernatural occurrence. It’s a Python loop and a chatbot. The people responsible didn’t accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.
It’s fine to worry about this new suite of tools that give even stupider people the ability to trash even more computers. You should worry about that – and demand better security practices from firms and governments, including a blanket prohibition on NOBUS-style vulnerability hoarding. That’s a productive kind of worrying, with a chance of addressing your area of concern. It’s infinitely more reasonable than locking yourself in the toilet with a flashlight and saying “Ayyyyy Eyyyyyye” into the mirror until you wet yourself.
Honestly, I can’t get anybody to give me a good definition of intelligence, artificial or otherwise. Once we can define that in concrete terms, maybe we can finally compare the two.
Now add sentience to the mix.
Not a definition of intelligence, but something I often come back to is what it does at rest.
To the best of my understanding of how LLMs work, if you’re not feeding the thing inputs, the program is doing absolutely nothing. It isn’t curious and doesn’t seek new information or stimuli. It effectively is just on pause until the next prompt. There is nothing akin to an inner monologue or thought process happening in absence of what is fed to it.
I’m not up on all of the epistemology or theory of mind type stuff, but I’d imagine some might argue about that distinction in humans; we are always processing input in some form as the input from our senses is never shut off until we’re dead. But I think the “at rest” case is a relevant distinction that does get at motivation and real thought born of real intelligence.
The way the systems operate right now, your session is time-sharing the hardware with all the other sessions.
Conceivably, you could set background tasks to “contemplate, research and develop” concepts related to the central task, but those tend to burn a lot of tokens relative to the useful output they provide.
If you think of an LLM agent like a butler, you really don’t want your butler getting ideas of their own and acting on them while you’re ignoring them, you just want them available when you call on them. To broaden the analogy, embodied in physical robots, LLM agents could be set upon endless tasks like maintaining a house and grounds, maybe a garden, maybe some farm animals, and each other, with spare capacity leftover to carry out special tasks when called upon.
Agreed, I believe “I think, therefore, I am.” sums that up well.