A few thoughts on xz-utils

There will be a lot written about xz-utils this week. I am not going to add to the hair-on-fire fray.  I do have a thought about the larger implications. That's what they pay me for, the forest thinking, not the tree thinking. If you aren't familiar with the xz-utils debacle, start here, where it was discovered, and go as far as the rabbit hole leads you. I'll wait.

You're back?  Ok, good. Clearly this is a real threat for some discreet population of systems. Being able to log into specific ssh endpoints, especially if there are a few in particular that you are after, is obviously A Good Thing (tm) for attackers and A Bad Thing (tm) for defenders. What is less clear, is how the hell is this possible?

It looks like the attacker was an Eastern European gang of very talented developers who planned this for years. (See Andy's excellent article in Wired.) This is clearly not good. Given the details of the payload itself, one can guess that they were after some specific systems that they knew ran the affected operating systems. (See Dan's excellent article in Ars.) This is also clearly a net negative. This potential attack was a lot of work, all for naught thanks to a performance focused Postgres developer.  While this is a good thing it still begs the question - what now?

There is a lot that needs to be done, but first there should be some bandages applied.  We aren't going to fix the open-source problems overnight, or soon, or maybe ever. We can, however, remediate the vulnerability.

I am recommending to my clients that third party libraries found hosted on and referenced at the original location should be considered a Low severity vulnerability, or Medium if the functions in which they are found are sensitive in nature. The remediation is to store known-good scripts in a central organizationally controlled location and depend on Snyk or retire.js to let the developers know if there are vulnerabilities found at a later time in the installed version.  When that happens, the new script can be retrieved, checked for vulnerabilities, and then rolled into the application.

Note that this would not have found xz.  This is a fact, but app libraries are different from internal operating system vulnerabilities.  Nevertheless, we have been battling this in the app space since leftpad (remember leftpad?) and it is common practice for developers to find a package that solves their problem and just use it, even if it has been abandoned. Hell, I did it just this week, writing some Ping to Spring MVC integration. The FOSS movement has changed development for the better, but it is time to put some formality around its use.

This isn't a be-all-and-end-all fix, but it should have been set as the standard for script includes a long time ago.  CI/CD processes make it very difficult to keep up with library vulnerabilities, as every build often goes and gets the latest libraries from jQuery, npm, GitHub, and wherever else. Local hosting of scripts will solve the problem, and not add a significant amount of work.  There will be some gnashing of teeth, but isn't there always? Local hosting of integrated libraries is a good first step.