David A. Wheeler
Christopher Sean Morrison:
I finally pushed enough off my plate and found time to finish filling out BRL-Congrats!! That's excellent.
Here’s a retrospective with feedback.I really appreciate the write-ups. The 1 hour is consistent with what we've been seeing & estimating.
Getting to 100% passing was relatively easy for BRL-CAD withGood to hear. I would expect most well-run projects to "mostly" do well. Fixing domain certificates, and doing a little documentation & elaboration, Is (to me) a *good* thing.
Here’s my top-7 critical feedback:We certainly *could* add 3rd-party review gating. For example, we don’t advertise this much, but I *do* review each passing badge to look for nonsense. But it's merely a brief look for nonsense, not a rigorous re-check.
This is a fair concern, and one that's been discussed from the beginning. The reason we didn't *require* 3rd-party review gating is because we're concerned that we would become the bottleneck as the number of projects increases. There are a lot of OSS projects out there.
So the question is... how could we scale that 3rd-party review? My current position is to focus on improving automation... which would make everyone happy (faster to get a badge, more rigorous checking). Are there good scaleable alternatives?
2) Taking a position on distributed vs centralized version control is(This is in reference to: "It is SUGGESTED that common distributed version control software be used (e.g., git). [repo_distributed]").
I certainly agree that centralized version control is viable (having used sccs, rcs, cvs, and subversion at one time or other). The argument for this criterion is that distributed systems tend to make it easier to collaborate (because you can easily re-heal changes initiated at different times)... and since it's only SUGGESTED, it does not *mandate* decentralized version control.
That said, we could certainly drop this criterion. It's only SUGGESTED (which hints at your next point), so it doesn't have much "oomph" - and shortening the criteria a little bit is a good thing. I think we should *expect* that as we get more projects & experience there will improvements and tightening of the criteria. The *much* more important issue is to *have* version control, and I think it's generally agreed that version control should stay as a MUST.
3) Most of the SUGGESTED items devalue the badge through dilution. SomeReasonable enough. I think the SUGGESTED items have some value, because psychologically people don't like to admit they don't do something (if they think they should be doing it). But I could be mistaken.
I'd like to hear others' opinions on this. I note that Daniel Stenberg agrees with this comment.
4) Private reports MUST … be privately reportable. N/A notwithstanding, IThat's certainly true in a sense. The point, though, is that the project has to *tell* reporters how to do the private reporting. A lot of projects just don't tell people how to report vulnerabilities (they've never considered the possibility), so the real goal is to get them to write it down ahead-of-time.
5) "Working build system” is not strictly defined (perhaps intentionally) but:-).
Well, I've seen some non-working build systems, but you're right, people don't want to *admit* they're not working.
Would changing "Working" to "Automated" be an improvement?
6) Treating warnings as errors shouldn’t be a suggestion. Projects SHOULD beThe problem is that it depends on the overall platform (including language and underlying OS) & the tools available. Making it a MUST would be too harsh for many cases. For example, you can find more problems by enabling more warnings, but those higher levels tend to be much noisier. We don't want to *discourage* people from using more sensitive (though noisier) tools.
7) The “BadgeApp” title on individual badging pages make for a terrible titleI presume you mean the <title>...</title> value in the HTML page, which shows up on browser tabs.
You're absolutely right, and we can quickly fix this too.
--- David A. Wheeler