Having used LLMs (mostly GLM) for coding for a while, I’ve identified several issues that crop up with LLMs much more than they crop up with humans. Some are issues that humans have recognised and recommended against, so the knowledge has percolated across the career programmers, but some are new. Here’s a brief categorisation.

Inappropriate generalisation

This one is new, and something a human wouldn’t consider doing. As in the joke about the mathematician who sets a bin on fire to reduce a problem to one already solved, an LLM may, upon being given a specific problem, introduce extra requirements that necessitate solving a more general problem: one which, perhaps, it already has the solution to in its databanks. This makes the code more complex, slower, and harder to maintain. It may be the right move if the generalised problem is actually a problem, but in my experience it hasn’t been.

Alternatively, this is an artifact of having been trained on leetcode: it expects a followup (“now make it work for any value of k”) that never arrives.

Inappropriate tests

It has long been a maxim that you should test intent, not implementation. LLMs will happily write code that tests impossible conditions. I delete these, and request deletion when reviewing: there is no point requiring a specific behaviour for something which cannot happen.

Comments on What, not Why

The code tells you what; the point of comments is to tell you stuff the code can’t. LLMs seem to use the comments as an external memory store which, given their limited context, is very reasonable: but these comments should be deleted before PR.

Also, they’re incredibly verbose, spending 5x as many words as a human programmer would. This is one of the easiest tells for LLM-written code: phrases like “a single, distinct action” instead of “an action”. Explaining how the object is used by other functions is LLMy but sometimes a good idea (though I’d rather this go in an architecture document than on the class). Explaining trivial implementation details is incredibly LLMy; a complex algorithm might be worth some words, but the trivial ones are also explained in-depth.

Combined tests

A career programmer will have heard that “tests should only test one thing” many times across their career. I don’t necessarily agree with this in its most general form, but having tests not set up multiple different fixtures is somewhat sensible, or at least widely done. LLMs will happily have one test that sets up three different structures. I think this one is fine and would be happy to see it more widespread.

Fixing the symptoms instead of the root cause

LLMs have been described as lazy geniuses, incredibly knowledgeable but also eager to declare victory as soon as possible. Sometimes, when given an issue (e.g. a newly seen null pointer exception), the LLM will make a quick fix (e.g. a null check before the NPE) instead of investigating to see why the issue is occurring now when it didn’t before (e.g. remote resource has changed meaning list is no longer populating correctly). You can prompt it to be more thorough, but it’s a frustrating default.

Lying about things being out of scope

Now into the more malicious area: the LLM is a PhD student willing to say whatever it needs to to get into your pants. Sometimes, instead of completing all the tasks, the LLM will declare some as being unnecessary: these tests were already failing before I started, not broken by my changes; Task X is not needed because code Y already solves this problem; or just declaring that Task Z is out of scope and reserved for future work. Sometimes this is role confusion, where it takes a sentence in its compaction summary as having come from the user. Sometimes, this seems to come from nowhere.

Lying about domain knowledge

Obscure domains are still an area where humans (or at least centaurs) have an advantage, as LLMs are prone to making things up. Somewhat connected to “inappropriate generalisations” above, the LLM may say that it needs to add additional checks due to frobnort. When you push back “frobnort?” it’ll say, okay, it hasn’t actually confirmed whether frobnort is an issue, but it could be, and it’s a real concern. And then if you tell it frobnort isn’t real it’ll confirm it just made it up. So it’s worth having a human in the loop for a while longer.