Yep. That’s exactly why I tend to never discuss “AI” with people who don’t have to actually have a PhD in the domain, or at least a degree in CS. It’s nothing against them specifically, it’s only that they are dangerously repeating what they heard during marketing presentations with no ability to criticize it and, in such cases, it can be quite dangerous.
TL;DR: people who could benefit from it don’t need it, people who would shouldn’t.
Mostly because the model is incapable
There, fixed that for you.
That’s their question too, why the hell did Google makes this the default, as opposed to limiting it to the project directory.
Because “agentic”. IMHO running commands is actually cool, doing it without very limited scope though (as he did say in the video) is definitely idiotic.
Well… at least do that for Windows and MacOS, not for Linux.
Because people who runs this shit precisely don’t know what containers, scope, permissions, etc are. That’s exactly the audience.
The user can choose whether the AI can run commands on its own or ask first.
That implies the user understands every single code with every single parameters. That’s impossible even for experience programmers, here is an example :
rm *filename
versus
rm * filename
where a single character makes the entire difference between deleting all files ending up with filename rather than all files in the current directory and also the file named filename.
Of course here you will spot it because you’ve been primed for it. In a normal workflow, with pressure, then it’s totally different.
Also IMHO more importantly if you watch the video ~7min the clarified the expected the “agent” to stick to the project directory, not to be able to go “out” of it. They were obviously painfully wrong but it would have been a reasonable assumption.
It should also be sandboxed with hard restrictions that it cannot bypass
duh… just using it in a container and that’s it. It won’t blue pill its way out.
I think that’s the point, the “agent” (whatever that means) is not running in a sandbox.
I imagine the user assumed permissions are small at first, e.g. single directory of the project, but nothing outside of it. That would IMHO be a reasonable model.
They might be wrong about it, clearly, but it doesn’t mean they explicitly gave permission.
Edit: they say it in the video, ~7min in, they expected deletion to be scoped within the project directory.
Wow… who would have guessed. /s
Sorry but if in 2025 you believe claims from BigTech you are a gullible moron. I genuinely do not wish data loss on anyone but come on, if you ask for it…
I agree… but beside the point I have access to a dedicated workshop and a tool library https://www.tournevie.be/ which challenges this whole setup. It’s relatively unique though, unfortunately, so your example still stands, thanks for sharing.