Topics

Feedback on current criteria


 

Hi All,

Been lurking for a few weeks, but enjoying the conversation. It’s REALLY nice to see some organized effort around the topic of best practices and code quality. As someone that has focused heavily on code quality, security, and community for a number of high-profile OSS projects, this is all very interesting and relevant to me. Lots of potential.

That said — putting on a critical project management hat and looking over the current criteria, I’m a bit disillusioned. The list feels very much like a brain dump (or several) than a cohesive set of testable criteria. There are so many subjective points, unnecessary elaborations, and sub-bullet distractions that it feels more like a disorganized academic paper draft than badging criteria. That’s certainly not to put ANYONE’s efforts down — it’s clearly a work in progress, respectable effort, and will improve. Cannot stress that enough.

So critique aside, here are my top 5 suggestions in priority order:

1) Less is more. This problem is probably author symptomatic. I suggest having a non-technical editor do an elimination pass on criteria.md to tighten up language, eliminate distracting prose, duplication points, etc. I suggest having an experienced dev identify criteria for elimination (is it really important?).

Example: Instead of "The project website MUST succinctly describe what the software does (what problem does it solve?), in language that potential users can understand (e.g., it uses minimal jargon)”, condense to “The project website MUST describe the software’s purpose in 50 words or less.” The intentional elimination of “succinctly” pertains to the next suggestion...

2) No subjective MUST clauses. If something is an absolute requirement, it should not be subjective. Make it testable or make it SHOULD.

Example 1, instead of “The project MUST have a public website with a stable URL”, rephrase as “The project MUST have a public website hosted on a static IP address” or add a secondary criterion “The project SHOULD be hosted on a stable URL” and define stable somewhere. In this case, the word “public” and/or “stable” would ideally be defined elsewhere to avoid misinterpretation.

3) No badge levels. This is riddled with issues like manipulation, subjectiveness, inflation, or unintentional devaluation / discrimination against lower levels (e.g., like restaurant ABC grades or CMMI levels). Nobody want’s to showcase their “B” rating. If you must, the labels should not diminish lower levels. E.g., Bad: Level I vs Level II or Silver vs. Gold; Good: Advocate vs Leader vs Visionary, etc. Instead of levels, though, I suggest badge compartmentalization common with games, described next.

4) Balanced compartmentalization. I suggest embracing “gamification” by creating independent badges that cover an entire aspect like “Public Relations”, “Source Code Management”, “Contributor Relationship Management”, etc. By design, the categories should roughly have the same number of criteria, e.g., 10-20 each. This is not the same as the current top-level topic categories that heavily overlap and vary in scope (e.g., user interaction vs development practice). Acquiring a set of badges could result in badge “certification” (e.g., Advocate vs Leader vs Visionary certification) equivalent to the proposed “baseline” badge.

5) Ditch self-assertions. That’s worthless, imho. I suggest incorporating peer-review. Similar to Professional Engineer (PE) certification or academic paper submission, a project/org would submit their application for a badge and, e.g., 3 others that have attained that badge would review. As a technical aside, I suggest publicly databasing the submissions and reviews in open source spirit (in a repo).

Please let me know if / how I can help beyond feedback and discussion (talk and ideas are cheap). Maybe a pull request that contains some of the changes I suggested? Regardless, this effort will benefit greatly with some diversity, especially inputs from non-technical folks (which I can’t help with).

Cheers!
Sean


David A. Wheeler
 

Christopher Sean Morrison:
Please let me know if / how I can help beyond feedback and discussion ... this effort will benefit greatly with some diversity,
I agree. That's why we have a mailing list, public issue tracker, and pull request mechanism - they're all there to encourage participation.

Welcome aboard!

--- David A. Wheeler


Enos <temp4282138782@...>
 

Sean Morrison wrote:

[..] The list feels [...] like a brain dump [...]
There are so many subjective points, unnecessary elaborations,
and sub-bullet distractions [...]
I would not label the criteria itself "a brain dump" (contrary to some of my posts here which are - and are meant to be - exactly that), yet I too would prefer the criteria without the "decorations" listed above.


[...] my top 5 suggestions in priority order:

1) Less is more [...] do an elimination pass [...]
+1 (although, quote for quote, "Make everything as simple as possible, but not simpler"... sometimes it may be just a matter of simplifying the writing style)


2) No subjective MUST clauses [...]
+1


3) No badge levels. [...]
I disagree. My favorite text editor and the Linux kernel play entirely different games: I don't measure a table and a planet with the same unit, regardless of whether I can attribute specific qualities to each.
Also, many small projects would be honored of being on the same ladder as the Linux kernel, even if a few steps below.


4) Balanced compartmentalization.I suggest embracing “gamification”
by creatingindependent badges that cover an entire aspect [...]
+1


5) Ditch self-assertions [...]
I disagree (at least for the first level) because self-assertions would allow small projects to dip their toes into code security without the pressure of being publicly judged.


I suggest incorporating peer-review. [...]
+1


Please let me know if / how I can help [...] Maybe a pull request
that contains some of the changes I suggested? [...]
+1 (maybe start with a single a chapter)

Or you could formally enumerate the list of changes most agree with.

Personally I'm looking forward to David's reply to your post (the one before does not count as one ;-).


Kind Regards
--
Enos