Software projects have very short shelf lives. The moment a commit lands on main the software is spoiling; new versions of runtimes and compilers are shipped, dependencies cut new releases with bug fixes and exciting new features, and connected APIs are deprecated.
Great engineering organizations always keep their tooling up to date on the latest stable versions. As language usage evolves and tools get better, running the deprecated versions means leaving a lot of automated value on the table.
Maintaining a modern project is a Sisyphean task that if not managed will lead to rot. Every poorly maintained project will eventually suffer from a severe case of tool rot. The shiny new version of a tool that was enabled when the repo was first getting started slowly ages and if not updated — gets rusty, and breaks.
When repositories are adopting their complementary dev tools, the tools are always shiny and recently released. But set-it-and-forget-it-ism sets in. DevEx tools, usually lacking a clear owner or champion inside a repo, aren’t refreshed, and tool rot sets in.
To see how pervasive tool rot is in the open-source world I did a quick survey of legacy ESLint usage throughout the entirety of the GitHub Open-Source codeset. I chose ESLint because it is pervasive and also one the easiest tools to upgrade; npm makes it stupid-easy to keep up-to-date. So let’s take a look at how even a near-zero effort tool is in practice subject to tool rot.
To perform this analysis I used GitHub’s still-in-technology-review Code Search tool (jump here to join the waitlist).
ESLint was an easy target for analysis because it is almost always version checked with a package.json file. The first thing I wanted to understand was how many repositories were an entire major version behind — or still pinned to eslint:7.x.x . So I searched:
path:**/package.json "\"eslint\": \"7"
And as you can see — there are over 31,000+ instances of repositories still pinned to ESLint 7. It has been over a year since ESLint major version 8 was released, but still, a boatload of projects are pinned to the legacy version. That means they are not taking advantage of all the massive improvements to the tools and ruleset over the last year.
But wait, there’s more. We found a large number of projects using outdated versions of ESLint dating all the way back to version zero. Here is the breakdown of version distribution of open source projects using ESLint:
It’s important to call out that up until v4 ESLint was vulnerable to a Regular Expression Denial of Service attack (ReDoS).
As you can see, we found 9500 open-source projects with instance of ESLint potentially vulnerable to this attack vector. In all fairness, there could be a lot of old repos that people set up a long time ago and abandoned, but there are also some big tech brand names (which I won’t list here) in the list still using unsafe versions of tools.
Tool Rot is Everywhere
I also dug a little into the prevalence of tool rot in other major ecosystems, specificially black the awesome python formatter and prettier the awesome lots-of-stuff formatter.
For those following along at home — prettier is currently on release v2.7.1 and black is up to release v22.10.0. This dataset is also smaller than what is really out there because prettier can be referenced in a myriad of ways and this was just projects directly running prettier through a package.json setup.
How to keep everything always-up-to-date
We, technologists, love creating and solving our own problems. There are bots that can attempt to keep your tools up to date but they have to be manually configured and can lead to a tremendous amount of PullRequest spam.
The bot solutions are pieces of the larger jigsaw puzzle of self-managed developer tooling. But my, slightly biased, recommendation, would be to use trunk check.
Any self-rolled solution requires the discovery of the best tools to run, the scripting of those tools, and the maintenance of them as well. You can of course cobble together all the parts of a reasonable devex solution, but the alternative is a ready-to-eat, shrink-wrapped solution trunk check.
With trunk check all you have to do is init and leave the rest to us. Keeping the myriad tools you run or should be running across your multiple repositories is a chore. Trunk takes care of that chore for you, so you can focus on what matters in your repo.
Trunk will by default scan for new releases of tools in the background and alert you to upgrades. After running trunk upgrade you can quickly confirm the behavior of the newly updated tools by running
1# the sample command will execute trunk check against a sampling of 5 files2trunk check --sample=5
Once you’ve confirmed the tools are running as expected, upgrading all your fellow engineers is as simple as committing the updated trunk.yaml file into the repository.
To find out more check us out at docs.trunk.io
