Getting Secure Beyond CVE Scanning
This is not a post meant to bash CVEs, especially with the recent drama around the issues at the NVD. Instead, I’m arguing that CVEs are the baseline for supply chain security, not the end goal.
Table of Contents
This week I planned on further exploring API security or CNAPP, but I find myself returning to SCA after meeting with two vendors, Phylum and Tidelift. Phylum raised a $15 million series A in 2022, and Tidelift a $27 million series C (later expanded to 33.5) at around the same time. After meeting with these companies, I was wondering two things:
Why have I come across neither company before in researching application security tools?
Should supply chain security mean more than CVE scanning and remediation workflows?
Phylum shared some data with me that led me to question my sanity, having spent so much of my career on CVE mitigation:
Only 18% of malware detections in open source libraries received a CVE or known vulnerability in a third party database.
82% of packages never enter a known vulnerability database (e.g., NVD, Snyk, GHSA, etc.).
For packages that did get a known vulnerability/CVE, it took nearly two days to do so.
For packages that were removed, it took on average 31 hours to be taken down; max took 369 hours.
Roughly 18% of malware packages are still available in open source registries.
These stats led me to ask:
Are CVE's really the best we can do?
At this point, we’re all familiar with some of the problems of CVEs. Here’s some great stats from Qualys if you’re curious. These problems are:
They are reactive instead of proactive detection mechanisms
There is a substantial increase year over year
Only a small percentage are exploitable
Scoring fails to reflect the reality of user’s environments
And yet, my day to day is filled with tools who fundamentally exist to prioritize, evaluate, and detect the presence of these CVEs in your environment. Surely we can do better? Are there any proactive solutions to securing open source software?
Why SCA Scanners Built Upon CVEs
Even looking at the Wikipedia for SCA, security is tied to CVEs, “Security: risks of vulnerabilities in components - Common Vulnerabilities & Exposures (or CVEs).” Snyk Advisor, one of the better open databases for the health of open source packages, seems to determine the health of a repo by looking at the general interaction with the repo against the number of CVEs that get discovered in it.
In order to answer why SCA scanners build on CVEs, we could ask a simpler question, why does Snyk’s scanner not support failing builds if their own advisor health check is below a certain threshold? Why do security teams not value the health of a package, but are willing to stop pipelines if a CVE exists against it?
This bias towards CVEs over health reveals that security teams are narrowly focused on one thing: the probability of getting exploited. Security teams assume that CVE’s provide a database of exploits, letting them know exactly when there’s a threat to their app. As anyone who works in security will tell you, most CVE exploits only occur very specific circumstances. Step one to exploit CVEs is too often, “be administrator on the container.”
I’ve come to believe that the general health of a public repository offers a more holistic understanding of the security threat of using a package than CVEs. CVEs still definitely matter, and are of course an indicator of issues; however, what really matters is how likely a CVE is to be fixed, or even identified in the first place! The 3 star 1 maintainer open source repository powering your frontend is unlikely to get a CVE published against it, but very likely to be vulnerable in some way.
Here’s an example: one of the war-room exercises from my career was discovering that a library we used for authentication had 5 stars and 1 maintainer out of Poland. In the event that that maintainer was phished, malware could easily be added upstream, pulled in automatically, and from top to bottom everything from customers data to intellectual property would be breached. The problem? None of our SCA scanners would detect this. This forces us to ask:
What Outcome are we looking for?
Theoretically, we’re buying SCA scanners for a few reasons:
Compliance dictates we look for CVEs in our packages
Compliance wants SBOMs so no one can read them
We want to fix exploitable vulnerabilities in third party code that could negatively impact our business
It seems to me that number 3 has led us to miss the mark on evaluating SCA scanners, because we’re including the idea of a “vulnerability” in our assessment criteria. Instead, if we tried “we want to prevent our being exploited by third party code,” we end up prioritizing real security instead of the CVE shakedown.
If you want to prevent being exploited by third party code, then your approach to SCA scanning broadens to being less focused on CVEs, and more towards remediating actual risk. Instead of asking, “how can we stop vulnerabilities?” we need to ask, “how do we stop supply chain attacks?”
What are some different approaches?
Phylum sits neatly in what I’m calling malware focused SCA scanners. Alongside them are Socket and Xygeni. These are tools that primarily focus on looking for actual attacks in your supply chain ecosystem, as opposed to just scanning for CVEs. I hope tools like this can gain adoption, but they face two challenges:
First, many people aren’t aware that there are more valuable results you can get from this approach than by hunting CVEs. Second, the current industry trend is towards ASPM platforms, not best in class point solutions. Xygeni is the only tool I see trying to compete on both fronts - all in one scanning alongside the upstream malware scanning.
Tidelift fits into the “let’s actually fix this crap” category, but with the uniquely ethical approach of helping open source maintainers get paid to deal with it. I would put them alongside Moderne, Grit, and Seal - different technological approaches to the same outcome: actually fixing upstream vulnerabilities.
Oligo and Deepfactor offer a third solution - watching packages at runtime to see if something fishy is going on.
Companies that don’t rely fundamentally on CVE scanning
As I see it, here are the pros and cons for each approach at the moment:
When I meet with malware scanner vendors, they’ve made an obvious investment into real security and detection engineering. However, they tend to not have a great user experience (UX) for the CVE based workflows, and generally lack the bells and whistles of the traditional SCA leaders to deal with CVEs. They also have the unfortunate sales disadvantage of not being very shiny - you’re probably not using any active malware packages, but you definitely want their tool if you are!
The fixers don’t already have allocated budget to their category, and really require a healthy Dev/Sec relationship to operationalize. You’d think you’re buying a scanner in order to fix the results - but orgs tend never to get to that part, so these tools are unfortunately swimming upstream in getting adoption…for now.
Runtime scanners require instrumentation that can be tough to get organizational buy in to implement. It’s a unique solution in that it’s the only thing that could actually stop a zero day with no intervention. The question comes down to if orgs care enough about that to implement an agent.
Predictions on the future
First, CVE scanning can’t, won’t, and shouldn’t go away. CVEs form a bedrock of security by providing common disclosures for exploitable issues. Remediating CVEs absolutely has general value for making you more secure; even if that’s a 1 to 1,000 real fix to false positive ratio. Any product in this category needs CVE scanning as its base.
Second, the future of this space is ASPM and all in one scanning. As valuable as a real security approach to SCA scanning is, the reality is that it’s not enough to sell. I can’t convince CISOs that the pay off to this issue is comparable to the overall value of combining SCA, SAST, IaC, CSPM, SDLC, and Container tools in a single place.
Third, each of these tools is pushing against the tendency to look for biggest impact lowest cost to implement solution. They require the user to understand the up front problem with traditional scanners, and to be committed to actually improving security instead of mindlessly checking the compliance box year over year. Unfortunately, this is a tough sell.
There are a lot of market challenges to selling these platforms; however, if I was an existing ASPM platform, I’d rush to buy or invest in one of these differentiators. In a world where the scanning has become commoditized, any of these three approaches offers real differentiation from the competition. If I was choosing between ASPM providers that all check the same boxes, upstream malware detection would 100% move the needle on going with that vendor. Similarly, if I was in an environment where a zero day supply chain threat would cripple the business, I would absolutely invest in one of these tools.
Short of that, there are opportunities to wrap all of these tools in a “Code Fixer” type of solutions. All of these tools recognize the problem: fixing open source problems is harder than scanning CVE databases. There are two options: either the company is going to offer the patch for you, or they’re going to make it easier to do it yourself. All of these companies have a genuine desire to make open source better, and if enterprises aren’t going to actually pay for any of the code they’re stealing, they could at least use something like Tidelift to make their own lives easier.
Back to the original question: Should supply chain security mean more than CVE scanning and workflows? Absolutely, I’d call it Beyond CVE™ SCA scanning. CVE’s are the floor, not the ceiling.
Latio Updates:
Added Akto to API security and DAST - Open Source next gen DAST
Added Tidelift to SCA and Boundary Breakers - Working with open source maintainers to uplift repo health and security
Added Devici to Boundary Breakers - Threat Modelling collaboration platform
Added InstaSecure to Cloud Identity - Very unique approach to securing identities by focusing on permissions boundaries rather than endless least privilege tweaking
Added Phylum to SCA - Great upstream malware detection