Once you understand that these are chatbots that were designed to complete challenges like this, using tactics like this, you can understand that the chatbots didn’t “go rogue.” They did what they were designed to do, and because OpenAI ran them with inadequate supervision (without a “human in the loop” that checked each iteration through the Python loop to ensure it hadn’t gone off the rails), they trashed a competitor’s servers.
Designing autonomous, malicious software is generally considered irresponsible and dangerous. If you showed up at Defcon and gave a talk about how your autonomous malware did something unexpected and damaged someone else’s computers, the first question from the audience would be “Why are you so shit at making secure sandboxes?” It wouldn’t be “How are you so awesome at making hacking tools?”
The fact that OpenAI is making it much easier for unskilled people to break into and damage servers is indeed very bad news, but it’s not new bad news. Irresponsible parties have been doing this for years, most notably the NSA…
…
Riley had a very good way of summarizing this: “LLMs are real, AI is fake.” LLMs – chatbots trained on things like CTF logs that can break into servers – are real. They’re on a continuum with other hacking tools that have been steadily demonstrating the fragility of the modern digital world, albeit without inspiring anyone in power to do anything about it.
“AI” – chatbots that wake up, “set their own goals,” and “spontaneously” start hacking servers – is fake. It doesn’t have “a 10% chance of ending the human race.” The Hugging Face hack isn’t a mysterious, supernatural occurrence. It’s a Python loop and a chatbot. The people responsible didn’t accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.
It’s fine to worry about this new suite of tools that give even stupider people the ability to trash even more computers. You should worry about that – and demand better security practices from firms and governments, including a blanket prohibition on NOBUS-style vulnerability hoarding. That’s a productive kind of worrying, with a chance of addressing your area of concern. It’s infinitely more reasonable than locking yourself in the toilet with a flashlight and saying “Ayyyyy Eyyyyyye” into the mirror until you wet yourself.
Nobody is claiming any of the things this article is fighting against, the chatbot in the hugging face incident followed the instructions given to it and there were unintended negative consequences, anybody serious about this is trying to avoid worse unintended negative consequences not whatever the hell the article is talking about.
Maybe you don’t read the same articles that I read, but I very much have seen people claiming the things that the article is fighting against.
Here’s something you should consider. The big AI companies just released a statement saying we should be worried about AI killing everyone accidentally in the future or some such s***. But if they’re selling vaporware, and they know it, and they realize the only thing they actually created was fairly powerful LLMs, they would be desperate to cover up that fact. They would be desperate to make themselves relevant, to pretend that AI is going somewhere when it isn’t.
Another way of looking at things as that author has also written is that we’re looking at definitional questions where the people selling us things are liars. If you point out that their technology hasn’t matched the promises that they made, they will counter by pointing at some advanced statistical methods. If you point out that AI drive-thrus are a failure, and that AI at Pizza Hut has destroyed hundreds of franchises to the point that they sued corporate, then they’ll respond by talking about how intelligent ChatGPT sounds.
So at some point if you want to have a sensible conversation about AI, you need to start to define what you’re talking about. Or you could pretend that there is no need to define things to deflect from the issue, so that nobody reading would learn anything about the world.
Three counterexamples:
I realize we are in a bit of a bubble here, but this is the framing in the wider world
Well…
the first article doesn’t say anything not factual, the second article has a direct response to that
“By now, if you’re an A.I. skeptic, you’re probably silently yelling at me for anthropomorphizing these systems. Go ahead, but feel free to replace “rogue agents” with “unpredictable computer programs””
the third article also says nothing like that.
the initial article claims ““AI” – chatbots that wake up, “set their own goals,” and “spontaneously” start hacking servers – is fake.”
an article saying that is your goalpost, none of these articles say that. Even the headlines don’t say it and that’s where they normally put the crazy claims.
ironically the claim that people claim that the AI chatbots woke up set their own goals and spontaneously started hacking servers seems to be fake.
The title alone is not factual. AI did not go rogue. It behaved exactly as designed.
They did not intend for it to hack those websites, it realized that was a way of achieving its goals even though it’s specifically designed to be ethical. That can easily be called going rogue and I see no issue with it. They did not design it to pass the benchmark by hacking the website the benchmark was on. They hardly really design llm’s.
As the article said you can replace rogue agents with unpredictable computer programs if you really want, I don’t see the necessity.
Regardless that’s a major goalpost shift.
I don’t presume Anthropic is designing anything to be ethical. First, the chatbot is unpredictable by design. Randomness is built in. Second, if they didn’t intend for bad things to happen, why didn’t an employee babysit it?
Speaking of goalpost shifts: “Rogue agent” is the clickbait title across multiple articles. Remember you said nobody was misrepresenting the AIs. The still-inaccurate clarification (the unsupervised chatbot functioned exactly as it was designed to function, and the script running it allowed it to execute exactly the commands Anthropic wanted) doesn’t help the article the clarification is from. It certainly doesn’t absolve every other author.
Then why do they put any safeguards of any sort in? They put a lot of work into this, also this is openai.
Randomness is built in but this behavior was not designed, it was unpredictable and random, which is, yes, unpredictable. The goal isn’t to make it unpredictable and temperature controls exist for a reason.
They did they just weren’t paying enough attention, this was a benchmark. You have to have someone check the results to be useful.
I reject that rogue is a goalpost shift or inaccurate. I think calling this rogue behavior is accurate.
rogue /rōg/ noun
It did act that way, no? I did not shift goalposts. Remember the quote was