Re: C++ static analysis tools for CII badge

David A. Wheeler
 

Daniel Heckhenberg:

> Daniel and David, you've both clarified the essential point: analysis tools detect errors or potentially error-prone code which only become identified vulnerabilities in larger contexts.  Most of the projects in the ASWF domain are used for making images, typically in environments that are not open to general network access or arbitrary inputs from untrusted users.  As far as I'm aware, there are no existing published vulnerabilities (e.g. CVSS scored examples) from these projects.  This is partly why our community seems to be struggling a little to know how to adhere to the spirit of the CII badging requirements.

Yes.  I hope it’s clear that if you don’t have any published vulnerabilities, fixing them takes zero time J.

> So... we can't map CVSS medium and high to any specific set of analysis checks or even particular coding errors.

Right.  Whether or not a particular coding error is a medium or high vulnerability is very dependent on the intended use of the component, not just the kind of error it is.

> But we'd still like to be able to provide some good-practice examples of specific analysis configurations for our projects to follow. 
> Looking at the curl example, and specifically the clang-tidy checks:
https://github.com/curl/curl/blob/52e27fe9c6421d36337c0b69df6ca2b3b2d72613/src/Makefile.am#L145
> This appears to be just running the default set of clang-tidy checks with a few globally disabled to avoid false positives.  Similarly, the lgtm config seems to be just the default.  These would be very easy to add for our projects.  Is this a reasonable setup to guide our community?

Yes, that’d be just fine.  In the criterion “static_analysis” we even list some similar tools as examples (e.g., SpotBugs, FindBugs, lintr, and goodpractice).

It’s really hard to give specific guidance for checks.  Different languages are often best handled by different tools.  There’s also always a trade-off of how far to configure checkers: if you turn them up too much & too quickly you get flooded by reports.  What you should do is set up at least one checker, and then slowly increase the rigor it enforces.  Adding more checkers over time, and gradually increasing their pickiness, is far more practical than trying to turn on everything at once (unless you’re a brand new project).  The static analysis criterion is focused on making sure you’ve at least started down that path; once you have *some* tools in place, it’s a lot easier to gradually increase what they check.

Criterion “static_analysis_common_vulnerabilities” SUGGESTs that at least one be used to look for common vulnerabilities.  But this is SUGGESTed, not a MUST or SHOULD; there are a long list of reasons that it might not worth be it for your project.

Let me know if you have other questions!

--- David A. Wheeler

Join CII-badges@lists.coreinfrastructure.org to automatically receive all group messages.