There are two specific places where I’ve substantially benefited from the use of LLMs in my work: building FFmpeg visualizers for my music YouTube channel, and building a bulk updater/scheduler tool for the same. Neither are uses I’m happy about, but I’d like to discuss them a bit further.

FFmpeg visualizers

FFmpeg’s filtergraph functionality is pretty comprehensive and powerful. It’s also poorly-documented, with a bunch of useless/trivial examples, very little actual syntax documentation, and no good explanations of how filter chains actually connect together.

There are some online tools such as FFmpeg Explorer which can help, but it’s also a bit cumbersome to use, requires a lot of trial-and-error, often results in incomprehensible errors, and also needs you to know what you’re looking for in the first place while dealing with FFmpeg’s rather inconsistent naming scheme for things.

However, in one of my searches on DuckDuckGo (which I’ve returned to using despite their hard pivot towards AI, because the alternatives are worse), I hadn’t yet disabled the duck.ai search helper, and I happened to look at its response, and it was actually… somewhat helpful. But not helpful enough.

So I asked ChatGPT.

ChatGPT gave me somewhat better explanations of things. It was sometimes incorrect, and often completely made up functionality that FFmpeg doesn’t have. It was completely sycophantic (if not downright patronizing) when describing my thought processes back to me, and I wanted to stab it with a fork. But it got me to where I was actually pretty good at building complex filter graphs on my own. Now I do not need to consult it at all.

I was absolutely not “vibe coding” with it, but it gave me a much better understanding of the system, and worked better than the official documentation or anything I could find online.

In this case, AI wasn’t good, so much as the human-written documentation was bad.

yt-updater

At one time, the YouTube data API used to be very well-documented, with pretty good example code provided in several languages. But the documentation quality has gone way downhill, and their search function has been replaced with an “ask AI” chatbot powered by Gemini.

I could not make heads or tails of how to write even the most basic code to log in to the API and perform some basic tasks. But the Gemini-powered manual did a decent job of spewing out some example code that I could work with and adapt.

I never copy-pasted the code directly, but I did read it and comprehended it, and a process which had been causing me to pull my hair out¹ turned into one where I could at least get on my way and get shit done.

Even Gemini, purpose-trained on the docs, still sent me in the wrong direction on a few things, and didn’t write super great code. Some of its examples were incorrect, but the code I tried ended up burning through my API quota very quickly and I had to wait some time before I could try more things. It did an awful job of warning me about things to watch out for which I only learned in retrospect. It also hallucinated functionality, based on old docs for long-deprecated APIs, and when I asked about that, it apologized profusely while subtly shifting the blame to me for not already knowing better, when, my dude, you’re the one who recommended it to me.

But still, I have working software which helps me achieve my actual task: bulk-updating video playlists with templatized descriptions and a posting schedule.

Again, I do not think this is a win for LLMs, so much as a loss for good documentation.

Other things

Sometimes I need to look up a Python function that I’ve forgotten the name or containing library for. I could try doing a billion search queries and wade through poorly-written docs which are scattered across many different repositories. Or I can look at the duck.ai summary which usually tells me the exact thing I’m looking for.

Sometimes I need to evaluate libraries for certain purposes, and search results are difficult to wade through. The AI summary usually presents a couple of options that I can investigate more thoroughly and make a decision on. (But sometimes it completely fails to interpret the question right and gives me completely useless answers as well.)

I also have non-programmer friends who have been making use of Claude and similar to write code that helps them get their things done, without having to find a skilled developer who is passionate enough to help who they can also afford the services of. These friends do usually end up realizing that the AI code is limited and only gets them part of the way there, or that the code isn’t the best way of doing things… but it gets the job done for them, and as the saying goes, perfect is the enemy of done.

The big question

So my question is, since I have benefited from the use of AI in some of my projects, does that mean my projects are irrevocably tainted? Should I be put on the open slopware list as a result²? EDIT: It looks like the “open slopware” repository has been deleted.

In my career I always rejected pretty much all forms of code autocompletion. I absolutely derided things like IntelliSense and XCode’s API autocompletion stuff because I felt like it made it too easy to just write code without having to think about it or understand it, because it would just offer a simple solution and then people would fill in the madlibs on it. I felt like if you hadn’t internalized an API then you didn’t really know it, and that you should try to remember the context of what you’re doing.

I also find the autocomplete things to be super distracting and that they usually present the wrong thing. Copilot was even worse about that, and I’d find it would actively take me out of my flow state, which is super disruptive to an ADHD brain like mine. I couldn’t understand how people would even want to work with those things.

Since then I’ve relaxed my stance a little bit, in that I use Sublime Text’s context-aware keyword autosuggest thing. It isn’t AI in any sense, it’s just something like “oh, you’re starting to type the name of a symbol that I have a few matches for, here’s some suggestions” and I find that actually useful. I also don’t mind its automatic punctuation insertion most of the time, since it helps me with things like parenthesis/bracket balancing (although sometimes it can go a bit overboard, especially for quotes and such, which it often gets wrong).

But seeing the kinds of code being written by supposedly senior engineers who have bitten the apple of LLM-based vibe coding, I have reaffirmed my desire to never write code with an LLM directly.

But I also don’t mind so much using LLMs as a supplement for documentation, when the documentation is awful.

But I really wish documentation weren’t so awful.

Where I’m really going with this: Good documentation practices

Back when I primarily wrote C++, my typical approach to building an API would be to start by writing the header files first, complete with documentation (via Doxygen) and sample code. This would help me to hammer out issues with how it would be used and would inform the underlying implementation. To me, this made the separation of .h and .cpp files a strength of C++, not a weakness.

Like, to me, the header file is documentation, and makes a nice interface between the machine-readable parts and the human-readable parts.

As a policy I would always insist on not just having good test coverage, but good documentation coverage. Any code which didn’t have at least a high-level description of what it does and a full explanation of what each function parameter means would fail my criteria.

Some people see this as “gatekeeping,” but I see this as “basic hygiene.”

As I’ve left the software industry and have mostly been writing software for my own ends, I’ve relaxed on this somewhat, but then whenever I have to consult my own code at a later date I end up being confused. So I at least try to do a decent job of documenting my (mostly Python) code with its relevant markup; thankfully, pydoc is built-in to the language itself and it’s more or less standardized how to actually document function parameters, between using docstrings to describe the function and type annotations to describe the function arguments. (Regrettably, the docstrings are typically formatted using RST which I don’t particularly care for the syntax of, but that’s what Sphinx³, and therefore readthedocs, uses.)

One of the more baffling things I ran into in my career was engineers insisting that good docs don’t matter, good code matters; if you can read the code then you can understand the code, and if you write separate documentation, it’ll fall out of sync. I find this mindset absolutely baffling.

To me, the documentation is what people are using, and the code is just how you instruct the compiler how to fill in what the documentation is saying. If the documentation and code differ, then they need to be brought into sync. And if the documentation doesn’t live alongside the code and makes keeping them in sync a lot easier⁴, that is the problem.

LLMs are, to me, an unfortunately-useful tool in filling in a documentation gap. But that’s not a win for LLMs. It’s a loss for documentation, and everyone who needs it: the sentient beings trying to get shit done.

And I hope this doesn’t mean that my code is forever tainted.