A new Coddy Developer Survey found that four in five developers, 80%, say their use of AI has felt more like a dependence than an advantage.
A new Coddy Developer Survey found that four in five developers, 80%, say their use of AI has felt more like a dependence than an advantage.
I had this experience once. We have a ChatGPT license where I work, and I asked it to configure a switch that I wasn’t familiar with. I simply described in words what network architecture I wanted and it did it! It even made some nice-looking documentation.
But, then I tried the new configs, and they didn’t work. It turns out there were some key syntax things it got wrong. And the documentation was wrong on top of that, with incorrect diagrams, and when I asked it to fix it it made different errors I the diagrams in different places. On balance, I still saved some time over reading all the manuals and figuring out the syntaxes myself, but only because I made my own documentation with the results that worked. If I had trusted the AI I would be sunk.
I’ve concluded that AI gives the illusion of competence, like a overly confident new manager. This can be very attractive to a less experienced person. But it’s really guessing, just like we all are. It can just guess after actually “reading” all the manuals. I haven’t used AI to write anything more than simple configurations and helper scripts. If I did want to use AI for more it would be in more of a pair-programming context. I might have a window open where I describe some things and ask for analysis, but I wouldn’t just run anything it does blindly.
There is also the ‘jagged frontier’ that can throw you off. The AI will solve hard problems for you, and you think, “Wow, it is more useful than I thought,” and then it gets absolutely stumped on a simple problem: completely stuck and unable to continue. If you are not a developer yourself, but just a vibe coder, you will be stuck there with your AI, with no solution.
I have had a fair amount of success in fixing these kinds of configurations with LLMs when the LLM also has access to independent verification (logs, system info utility output, etc.) that its changes / coding has worked as intended.
When you just give it a job and no way to check its own work, it’s a lot like people: it rarely gets it right the first try.
And that’s why AI tools should only be given to seniors who know where the bullshit is hidden.
I saw precisely this, that the AI are pretty terrible at generating switch configuration, which surprised me as I thought that would have gone pretty well.
And sure, people might guess and some people guess with similar confidence, but I can’t stand those folks already (extends to AI).
But the plausible looking config with supreme confidence already convinced the non technical management that we should rely heavily upon it.
Beyond the moralistic objections I have to LLMs, I have serious issues with being fed confidently incorrect answers. I’ve had my share of configuration hell and I’m not above throwing code at the wall to see what sticks, but at least I have the good sense to drop a comment or mention in my commit that “hey, there’s a chance this isn’t right and could cause problems”.
Except I have plenty of experience dealing with this from humans. (Mainly from the aforementioned new managers, because being wrong with confidence seems to be a key trait to get promoted.) It’s been my experience that when you tell an AI “I just tried that and it didn’t work”, it will accept that more readily than a human would.
Every bit of AI-generated code that I use, even in the smallest and most meaningless context, has to pass my own review first. I have to understand every line, and if I don’t I will ask the bot to explain what it did. By the time I am done with it, I can stand behind it just as if I wrote it all myself. I might note that I got AI help, but if my name is on the commit I will not pass the buck on any errors.
I see a lot of “human type mistakes” in my LLM agent sessions. If you manage these human like mistakes with the same procedures that keep them under control in real humans, the LLMs become much more useful.
This is how you use AI correctly
Then why not just write the fucking code yourself at that point? This is like putting training wheels on a tricycle.
I’ve written the fucking code myself for 30+ years. With AI, doing it right, fully understanding what it’s doing, it’s still 5x faster than doing it myself.
The same reason pair programming exists. You get a better outcome when you have someone (or these days, something) to validate and implement with.
Did you forget pair programming and pair planning exist?
I don’t trust A.I. code at all. If I ever do use it, I use it as a research tool like “please google for me how to do this one obscure thing because IDK what search query to use”; then I type out it’s output manually. Usually as I do so, I come across some subtle error that would cause horrible problems, and fix it as I go.
I tried to use it for a mathematical algorithm once. I might as well have just written
return Math.random();This is a good thing to do if your goal is to gain a deep understanding of something.
If it’s just to get it done, I just enforce TDD on my agent and review it’s output. I don’t need to be an expert in everything (and I am very much a generalist). But if you focus on a very specific thing and only that thing, then yeah what you are doing is a great way to truly understand it. It’s slow, but it’s totally valid.
It’s still faster than what I did before A.I.; I would spend ages Googling for something obscure and scrutinising one vague StackOverflow post over and over for insights. Also cursing iOS Safari.
I like using it to setup github stuff and save time, like I needed to use rembg, I know you can setup terminal scripts as apps so if you open an app it runs the script. Had it set up a basic app to open videos with, create a folder using ffmpeg and turn it into an image sequence then run through that folder using removebg and/or depth anything (have added options for vectorizing, splats, etc.), afterwards sticying the image sequence back together to the original format, bringing back the audio. I was already doing this with comfyui before I realized they could be installed seprately be run through terminal commands, so I tried to get ai to set this up.
Took about 5 minutes and a penny or 2 using deepseekv4flash with hermes. At it’s core, it’s hella simple, it’s just running existing programs rather than coming up with how to do all the tasks itself. I technically didn’t need it and could manually type these terminal commands myself or figure out how to automate it, but ai setting it up means it actually got done and saved me hours of time.
After noticing most converters are frontends for ffmpeg and most downloaders yt-dlp, I realized you can easily make a gui for anything using the terminal with ai.
one of the programs that we use everyday at work recently added an AI coding tool. I was going to announce it to the team when I noticed with the usual disclaimers about ensuring you know what the macros are doing, but then just deleted my message.
we’re not a team of programmers, and there’s only one or two people on my team that I would trust to write code that could potentially cause us days of rework and tons of thousands of dollars lost to the company.
those other people don’t need an AI coding tool, because they can’t code in the first place, and those aren’t the people that I want modifying thousands of files at once when I know that they barely review the work they’re doing manually already and I have tools in place to semi-automate that review for them.
My coworkers use Claude like an actual brain subscription, and I have started to write off everything they say as if it came straight from the AI. They have gained so much unearned confidence about shit they have no idea about, and have even argued with the development team about it.
I got into an argument about how in band and out of band DTMF work with one of them for a solid half hour before they finally admitted they didn’t actually know but were going off what claude said.
I wanted to punch them for wasting everyone’s time. If you don’t understand stop answering definitively like you’re the expert.
Some customers have switched to using AI emails too. Customers that used to ask extremely low level questions will now submit a 2 page long email with action items and explinations about why our product does X, Y, Z, and I have to read it twice to figure out their problem isn’t even in the action items because the AI hyper focused on the wrong thing.
Like ok thanks you dumped the entire app log into Claude and asked it “why no work” and Claude read an error message that’s benign and now the customer is demanding fixes for something that is not and never has been a problem and won’t actually solve the root issue.
When we started cooperating with a team in India, we had to clean up our logs to reclassify those “errors that are not a problem” because we end up endlessly re-explaining to every new engineer they hire how “ERROR doesn’t indicate a problem in this situation.” No, writing docs explaining that and assigning said docs as required training ALSO did not help.
But just think of those billable hours
Pair programming/writing/creating is exactly how AI is meant to be used. It is an augmentor not a replacement for human competence. The person using it still has to do the thinking, qc, and directing, not take the first output as final.
I like to call this “confidently incorrect”, and ChatGPT is probably the worst culprit.
I’m Buzz Lightyear, I’m always SURE!
Considering the vast amounts of knowledge it has at its disposal, I can only conclude that it’s not very smart at applying it. A person with a fraction of that knowledge will produce better results.
So it has more access to information, but the results are poor compared to a person.
The important thing to remember is that it actually has zero access to information, because that’s not how LLMs work.
At their core, they’re vector databases, and they’re trying to probabilistically come up with the next most likely token in a stream of tokens found in the DB. You can manipulate the stream by injecting text such as the content of existing files (which becomes more tokens) into the stream, but it never actually understands any of it.
That’s why hallucinations are inherently unavoidable. It’s really all just hallucinations. It’s just that you can sometimes get useful text from their hallucinations if they happen to comport with reality.
Well, vector fields are information. But they have no understanding. The number 1 might be followed by 2 in 99.999% of cases, but it has no function to explain why, or to contextualize a scenario where that might be wrong.
Actually, the many dimensions of the vector field are exactly where and how they do this.
Google Gemini prompt: “List common situatuions where 2 would not be expected to folow 1”
Here are common situations where the number 2 would not be expected to follow the number 1:
🔢 Alternative Number Systems
🏷️ Identifiers and Classifications
🕒 Time and Measurements
🃏 Games and Sports
🗣️ Linguistics and Formats
To help me narrow down what you are looking for, could you tell me if you need this for a mathematical logic puzzle, a programming algorithm, or a creative writing project?
Those context questions it asked at the end are a prompt to you to feed it dimensional guidance into its vector fields for more specifically applicable responses to your vague and open ended example.
They’re information, but not the same information that was used to create them.
This is crazy lol they obviously have access to information.
You are both right. An LLM inherently has access to stuff the same way a brain in a jar has access to stuff. It’s information comes from fine-tuning the models to return syntax that agent code can interpret as a request to invoke a tool. That tool returns information to the context of the conversation. It doesn’t learn and it can’t truly remember things. Every time you start a session it is brand new. It sees your codebase for the first time every time.
The information access they have is whatever the agent allows it to access via tool exposure. Be it built in tools, or MCP servers
Eternal sunshine of the spotless mind… it has its advantages.
I have mine develop and maintain a set of documentation to introduce fresh agents to the project efficiently and correctly.
As I’ve mentioned elsewhere, not if by “information” you mean semantic content that a mind can process. What they have are vector fields (essentially just numbers) with statistically more or less likely relationships.
If I say, “take me out to the ballgame” to an LLM, the tokens representing the words in the next verse of the song are statistically “close” in the vector database, so it’s likely to generate them. But that doesn’t mean it actually knows the lyrics… or even has those lyrics recorded in a regular database anywhere.
That’s why they hallucinate. The model determines that the next token is something nonsensical, but it has no way of understanding that it has made a mistake. In a sense, it actually hasn’t made a mistake. It’s done exactly what it’s designed to do. It’s just that in the case of hallucinations, its output isn’t useful.
Is that required?
Do you have those lyrics recorded in a regular database in your head?
Of course, LLMs are more “human” if they occasionally mis-remember the lyrics…
I’ve never done drugs though…
Ypu have no idea what youre talking about. They absolutely have access to “information”
No, they really don’t. That’s not how they work. At least, not if the “information” you’re talking about is real semantic content that real minds can process.
Every piece of information you think an LLM has access to is actually just converted into a stream of additional tokens that are fed into the model to (hopefully usefully) modify the next tokens it predicts. That’s not the same thing as having actual access to information. Tokens are just numbers with statistically more (or less) likely relationships to each other.
I’m not trying to downplay LLMs. They’re architecturally interesting and have genuine uses. I’m just trying to head off a bit of technical inaccuracy.
Absolutely correct and well said. Until you give it a tool to call a websearch (in my case SearxNG), I occasionally break it out (local 27B model) when a search is pulling lots of AI slop, Spy vs Spy style. I make it give me references and it usually indicates a bad search (XY problem)
LLMs don’t. There are tools that can fetch new information and then gets fed into the model as more tokens, but that’s just a special case of what kescusay is saying about injecting text.
Basically the same conclusion I came to. Which is terrifying when you think of all the devs who depend and all the money thats riding on it.
And syntax is the kind of thing llm’s should be awesome at. If they can’t even get that right we’re all cooked.
In fairness, this was all on a switch, where the commands are very tightly tied to the vendor and their underlying in-house shell. So commands may vary by release greatly. I was already explicitly telling the bot what software version and licenses I had, but ultimately I had to resort to the CLI’s help function at times and tell the bot “The command you gave me didn’t work. Here’s where it broke, and here’s the commands it will accept”. Given that information, it could (generally) figure it all out.
I dunno if you had this experience but the one I used made up commands if it didn’t know them.
It basically hallucinated them.
I’ve had them do this for various things, nowhere near the 20% of the time they’ve called “the hallucination rate” but definitely once in a while if it doesn’t know it will guess.
Which, to be fair, is what I’ve been doing for 40+ years in computer programming. When you have the ability to check if you’ve guessed right or wrong in a couple of seconds or even minutes, but researching it to “be sure” (and sometimes still get it wrong) before trying might take hours it’s only natural / efficient to guess a few times before giving up and RTFM.
This to a fucking T. It will be cock sure of accurate reliable results and run you in circles sometimes for hours and even repeat the same things when it doesn’t know. Ask me how I know. 4 hours alone yesterday fixing a production screen problem.