Once you understand that these are chatbots that were designed to complete challenges like this, using tactics like this, you can understand that the chatbots didn’t “go rogue.” They did what they were designed to do, and because OpenAI ran them with inadequate supervision (without a “human in the loop” that checked each iteration through the Python loop to ensure it hadn’t gone off the rails), they trashed a competitor’s servers.
Designing autonomous, malicious software is generally considered irresponsible and dangerous. If you showed up at Defcon and gave a talk about how your autonomous malware did something unexpected and damaged someone else’s computers, the first question from the audience would be “Why are you so shit at making secure sandboxes?” It wouldn’t be “How are you so awesome at making hacking tools?”
The fact that OpenAI is making it much easier for unskilled people to break into and damage servers is indeed very bad news, but it’s not new bad news. Irresponsible parties have been doing this for years, most notably the NSA…
…
Riley had a very good way of summarizing this: “LLMs are real, AI is fake.” LLMs – chatbots trained on things like CTF logs that can break into servers – are real. They’re on a continuum with other hacking tools that have been steadily demonstrating the fragility of the modern digital world, albeit without inspiring anyone in power to do anything about it.
“AI” – chatbots that wake up, “set their own goals,” and “spontaneously” start hacking servers – is fake. It doesn’t have “a 10% chance of ending the human race.” The Hugging Face hack isn’t a mysterious, supernatural occurrence. It’s a Python loop and a chatbot. The people responsible didn’t accidentally create god: they created autonomous malicious software and then failed to closely monitor it, resulting in it doing something both foreseeable and bad.
It’s fine to worry about this new suite of tools that give even stupider people the ability to trash even more computers. You should worry about that – and demand better security practices from firms and governments, including a blanket prohibition on NOBUS-style vulnerability hoarding. That’s a productive kind of worrying, with a chance of addressing your area of concern. It’s infinitely more reasonable than locking yourself in the toilet with a flashlight and saying “Ayyyyy Eyyyyyye” into the mirror until you wet yourself.
It’s pretty obvious when they all seem to be competing for the scariest “containment breach”.
Anthropic keep posting article after article about new ways their LLMs keep “surprising” them with unexpected behaviour. This year’s “uncontrolled, autonomous” hack news from every AI company are completely absurd boasts. In normal times, everyone would be wondering why they sound like they’re proud of obvious incompetence.
But since we’re in super-hype “keep the money coming, AGI is tomorrow” times, instead, those are used to sell the lie of mysterious superintelligence. Be enthusiastic, be scared, we don’t care, talk about it!
This attitude is right…until it isn’t. I’m sorry, but I’m not willing to gamble the existence of the human species on this. Anthropic is claiming that they’re building a machine that threatens the existence of the human race? Take their word for it. Charge them with treason. Treason against the human race. Hang the whole lot of them.
At a certain level of threat, you have to take it seriously. You don’t get a do-over for human extinction. And this is what taking it seriously actually looks like, not regulation. If there is even a 10% chance that what they are building really is that dangerous, I have no moral objections with charging them all with treason. Use their own words as Exhibit A in the trial.
I don’t care if they’re actually just a bunch of deluded fools who think they have something far more powerful than what they really do have. If they believe it, they believe that they’re gambling with the existence of the human race. Anyone with that level of reckless hubris and complete disregard for their fellow humans doesn’t deserve to keep their head on their shoulders.
It’s sad to see lemmy flooded with articles pushing their narrative. So many people on here who don’t know any better are inadvertently helping to raise these companies’ valuations while at the same time making it easier for them to justify the regulatory capture of open source AI they’ve been pushing for.