static_analysis: plan to add met_justification_required

David A. Wheeler

We intend to add a tweak to the criteria badging form, one which technically doesn’t change the criteria but will require a little more information from projects about static_analysis.  Comments welcome.


First, background.  One of the badge “passing” criteria is static_analysis:

* At least one static code analysis tool MUST be applied to any proposed major production release of the software before its release, if there is at least one FLOSS tool that implements this criterion in the selected language. (N/A allowed.) (Justification required for "N/A".)


However, we now have several projects that are claiming that this criterion is “met” but it’s not obvious *how* this criterion is met.  I’m concerned that some projects might not really be meeting this criterion even though they’re claiming it, and of course that would be a problem.  In addition, we think it’d be useful for passing projects to provide examples for other projects, so that the other projects can consider those approaches.


So we intend to make a tweak to require that projects provide a *justification* for how they meet static_analysis.  A vast number do provide that justification, but a few don’t.  We want to do this gracefully, so here’s what we’re thinking:

* We’ll modify the form so that projects that are editing from here on must provide the justification.

* If a project already has a badge, we will *display* (for now) that they have a badge, even without this information, to give them time to update their entry.  We may send an email to just passing projects without justification in this field, to make sure they’re aware of it.


Technically this isn’t a change in the criteria, because it’s always been a MUST.  All that’s changing is that we want some information *justifying* that the criterion is met.  If you meet a criterion within a project you should be able to easily justify it.  For the vast majority of criteria this hasn’t been an issue, but this one seems to be more problematic.  This seems like the simplest way to inhibit problems.


Comments welcome!


--- David A. Wheeler