We live in a world where applications run the technology that we all use. There was, once, a time where hardware was custom developed to solve certain problems, but these days we have general use hardware and applications designed to solve our problems. Everything from apps on our phones to websites to alarm systems to the management screens for our internet gateways are applications, coded in common languages, using common protocols.
For every 100 lines of code, there are five security vulnerabilities.
The average application is 15,000 lines of code.
Let’s take a minute to talk about pentesting. Pentesting, or “penetration testing” to expand the vernacular, is the art and science of finding a path through the security of a system in order to achieve a goal. It is different from red teaming, because rather than making for a continuous series of tests like contemporary attackers would, penetration testing is a scheduled event that is designed to move from point A to point B.
Vulnerability analysis is different. The goal is to take an application and find every single thing that could be used to circumvent the security of that application, then report on those items and provide a solution. It is similar to penetration testing because it is a scheduled event. It is is easier than penetration testing because you are far less likely to go to jail for the night. It is harder than pentesting because you don’t have to find one flaw, you have to find all of the flaws.
Quality assurance and vulnerability assessment have a lot in common. Both practices have the goal of making sure the application in question is as good as it can be. Quality assurance is focused on the end user experience. To that end, the both use a test plan. The plan has a starting state, an end state, and steps to get from one to the other. With QA it is best if those tests succeed. With vulnerability analysis, it is best if those tests fail.
That test plan is key. In quality assurance, the business owners give a detailed description of how the application should respond under every circumstance. If the user submits a valid application, it should be sent to the processing center. If it has an invalid date, then this error will be presented. If it fails in processing, then this message will be sent.
In vulnerability analysis, the test plan is determined by the attackers. Whatever the flavor of the week is, it’s added to twenty years of attacks on the HTTP Protocol, language specifics, side channel attacks, and other weirdness. The analyst will, rather than checking how the application responds to valid business requests, check how the application responds to this huge collection of known threats. Not just the ones that work. All of them. The goal is the find all of the vulnerabilities.
It is true, not all of the vulnerabilities can be found. There are a lot of tests, and a lot of fields, and a lot of POSTs, and a lot of URLs. They can’t all be checked and fixed with machines (at least not yet). There isn’t the time or money to check them all by hand. Things will be missed, and that’s how we end up with 81% of breaches in the last 10 years having an aspect of application security involved. This is why we still have SQL injection, even in the age of ORMs. This is why we still see CSRF even though it is a well understood vulnerability. There are far too many legacy applications and far too few talented developers.
So why am I writing about this? I performed my first vulnerability analysis in 2002. That seems like a recent date to me - I wrote my first paid application in 1986. But in the arena of application security this is a recent date. The folks that wrote the Internet didn’t design it for security. That wasn’t the goal - sharing was the goal. I am writing this because I participated in the process of the web becoming the hub for commerce and communication - where security mattered.
And I, along with a boatload of others, failed miserably.
I got paged at 11PM on a Saturday in 1997 by a trigger I’d set up on a web server because the hard drive was full. Long story short: the drive was full of German porn because of a SQL injection flaw I’d written into an application running on that server. Someone had broken in and set up a convenient FTP server.
At the time I didn’t know SQL Injection was even possible.
I’ll leave the path from then to now to the reader, but suffice it to say, I was the security guy on every project from then on. I still strive to teach developers how to write more secure code - I’m the Security Track advisor to three conferences, and I speak to developers monthly about security awareness. I train a thousand folks a year on secure coding standards.
That’s not why I am writing today, though. There are way too few people checking applications for vulnerabilities, and OWASP isn’t making things obvious enough. They have a greater reach than I, and that’s awesome, but I wanted to put together this book to lay out how I see vulnerability analysis in plain language, in hopes that it would help a few other folks get into the field.
What’s in this guidance certainly isn’t the only way. It probably isn’t the best way. It might be a bad way. I’m not sure, but it has worked for me, and I’m including things that you won’t hear in some breakdowns, like client management and report writing. Feel free to ignore everything, or take just the pieces that you like. And send me feedback! I’m more public than I should be on Twitter (@sempf) and Linkedin. My Skype is pointweb.net if you want to tell me how awful it was privately. I’ll take your perspective in any form.