Date   

Stuff that's happening in the CII best practices badge project

David A. Wheeler
 

FYI, I thought I’d share some of the things that are going on in the CII best practices badge project.  I think it’s especially important because much of this may be otherwise invisible to you.  Details below.

 

The number of participating projects, and the number of projects with passing badges, continue to grow.  You can see the statistics & graphs here:  https://bestpractices.coreinfrastructure.org/project_stats

 

Of course, the *real* goal is to get projects to improve themselves, & help them show users that they’re well-run.  I think the badging project continues to do that, and I hope you do too.

 

Thanks!

 

--- David A. Wheeler

 

============= DETAILS =======================

 

We’ve improved our ability to provide data to others.  We’ve supported a REST API and JSON from essentially the beginning, but we’ve significantly improved the API documentation and created a separate page for it <https://github.com/coreinfrastructure/best-practices-badge/blob/master/doc/api.md>.  In addition, we’ve added support for CORS, so that JavaScript clients on browsers can access some of our data.  This should make it easier to create dashboards, analysis tools, and other such things that base themselves on tool data.  We *want* people to build on data as long as it doesn’t interfere with personal privacy.   We still don’t have the API defined using OpenAPI/Swagger, but we’d love the help: <https://github.com/coreinfrastructure/best-practices-badge/issues/129>

 

The master branch has a new form for those who want to delete projects.  Project entries aren’t deleted often, but when they are, we currently don’t know why.  The new form requires that there be some text that explains *why* they are deleting a project entry.  I’m sure we can’t please everyone, but if there’s a common issue we can resolve in the future, this will at least help us find out.  This also makes it harder to accidentally delete a project, which has happened.  Some people habitually say “yes” to any “are you sure” message, so having a special form should reduce the risk of accidental deletion.  If someone really does want to delete their project entry, we will of course honor that, but I think this change will make things better for all.

 

A number of projects are making slow & steady progress towards getting silver & gold badges.  These are (intentionally) much harder levels to achieve; even the “passing” level is challenging for many projects. This is very encouraging for the long term.

 


Projects that received badges (monthly summary)

badgeapp@...
 

This is an automated monthly status report of the best practices badge application covering the month 2017-12.

Here are some selected statistics for most recent completed month, preceded by the same statistics for the end of the month before that.

Ending dates2017-11-292017-12-30
Total Projects11671248
Projects 25%+419442
Projects 50%+345364
Projects 75%+273287
Projects passing119130

Here are the projects that first achieved a passing badge in 2017-12:

  1. distro-tracker at 2017-12-04 20:53:54 UTC
  2. linkerd at 2017-12-05 20:13:47 UTC
  3. Vita Genetic Programming Framework at 2017-12-12 16:24:14 UTC
  4. NSD: Name Server Daemon at 2017-12-14 21:07:01 UTC
  5. perl-test-timer at 2017-12-16 21:36:23 UTC
  6. Jobber at 2017-12-17 23:20:12 UTC
  7. EPICS Base at 2017-12-20 03:45:23 UTC
  8. Node-Data at 2017-12-20 13:04:47 UTC
  9. crypto-tk at 2017-12-21 12:29:21 UTC
  10. Fruit at 2017-12-27 10:41:24 UTC
  11. edumips64 at 2017-12-29 20:53:32 UTC

We congratulate them all!

Do you know a project that doesn't have a badge yet? Please suggest to them that they get a badge now!


CommonMark has received a silver badge!

David A. Wheeler
 

The league/commonmark project (which implements a Markdown processor) has received a silver badge!!

 

More details here:

  https://bestpractices.coreinfrastructure.org/projects/126

 

As always, if there’s a problem please let us know.  However, on a brief spotcheck it looks fine.  For example, they expressly note that they exceed the 80% statement coverage requirement.

 

Because of the *kind* of project it is, it’s a little easier for them to meet the criteria than some others.  They don’t do anything with cryptography, and they don’t produce compiled executables, so some criteria are no-ops.  Even so, getting silver is not an easy thing!!

 

I know a number of projects are pursuing the silver badge, but relatively few have achieved it, so congrats!!

 

--- David A. Wheeler

 


Projects that received badges (monthly summary)

badgeapp@...
 

This is an automated monthly status report of the best practices badge application covering the month 2017-11.

Here are some selected statistics for most recent completed month, preceded by the same statistics for the end of the month before that.

Ending dates2017-10-302017-11-29
Total Projects11101167
Projects 25%+401419
Projects 50%+333345
Projects 75%+264273
Projects passing114119

Here are the projects that first achieved a passing badge in 2017-11:

  1. git-buildpackage at 2017-11-01 18:12:26 UTC
  2. systemd at 2017-11-04 19:48:26 UTC
  3. CAIRIS at 2017-11-24 08:10:25 UTC

We congratulate them all!

Do you know a project that doesn't have a badge yet? Please suggest to them that they get a badge now!


Current plan: No criteria changes, focus on getting projects involved & getting badges

David A. Wheeler
 

All:

We want to slowly & carefully update the criteria as community norms improve. However, I don't see any need to update the criteria at this time. The passing criteria in particular appear to be "just hard enough" that a badge is worth having, without being so hard that it's impossible to earn one. Silver & gold are harder, but they are *supposed* to be harder, and I know several projects are slowly pursuing higher-level badges.

Instead, I think we need to focus on getting more projects *involved* in the badging process and eventually *getting* badges. We'll also continue to review badge claims to make sure that the badges have really been earned (nonsense badges drag down the badge's value for everyone). The good news is that projects really are getting involved & earning badges, as evidenced by the project statistics here: https://bestpractices.coreinfrastructure.org/project_stats

SO: Please try to convince other projects to work on a badge. As you can see on the front page, a lot of well-known projects have received badges! Of course, the real goal isn't badges, the goal is to improve practices in the projects that we all rely on.... but badges are the visible manifestation of that. If you know that a project made an improvement because of the badging project, please let us know; I'd like to record that!

Of course, if a criterion is misunderstood, or doesn't make sense in specific narrow cases, we can review clarifications & narrow reasonable exemptions. But for now, I think we should stay the course.

We should revisit the criteria around August 2018. I expect any future changes to be very gradual, anyway. People work hard to earn badges, and we don't want to make their hard work irrelevant.

As far as the site itself goes, it seems to be working fine, so the current plan is to make at most modest improvements/updates. Everything can be improved, but again, I think we should focus on getting more projects involved & eventually earning a badge.

If you think this is a terrible idea, please let us know!

--- David A. Wheeler


Projects that received badges (monthly summary)

badgeapp@...
 

This is an automated monthly status report of the best practices badge application covering the month 2017-10.

Here are some selected statistics for most recent completed month, preceded by the same statistics for the end of the month before that.

Ending dates2017-09-292017-10-30
Total Projects10491110
Projects 25%+388401
Projects 50%+321333
Projects 75%+251264
Projects passing110114

Here are the projects that first achieved a passing badge in 2017-10:

  1. AuthZForce Core PDP engine (Community Edition) at 2017-10-10 21:40:20 UTC
  2. libvirt at 2017-10-13 11:50:57 UTC
  3. Material-UI at 2017-10-17 18:47:47 UTC
  4. A Python library for automating interaction with websites. at 2017-10-22 11:37:46 UTC

We congratulate them all!

Do you know a project that doesn't have a badge yet? Please suggest to them that they get a badge now!


Someone created a long-term badge entry on a test tier.. please don't do that, & here are our steps to prevent future problems

David A. Wheeler
 

All:

 

We’ve just learned that someone had created a project badge entry on a *test* tier (master), with the intent of using that as the main information source.  Since that’s not what the test tiers are for, that did not work out well.  We occasionally copy data from the production site to the test tiers (master and staging), and we only learned about this situation after the data had already been overwritten (eek!).  We have many backups to prevent the loss of *production* data, but no one had considered the *test* tier data as something that needed backups, so some data was lost from that project’s badge entry.  The data can be recreated, but that’s not something I want to do often (!).

 

So: If you know of a badge entry that’s starting on master.bestpractices.coreinfrastructure.org or staging.bestpractices.coreinfrastructure.org, please save the work (say as a JSON file, just add “.json” to the project URL), and let us help you move that data.  Please do NOT put “real” data on a test tier!!

 

We’re also taking these steps to prevent a recurrence:

1. We’ve turned on a warning message on non-production tiers (including master and staging) for when you try to create a new project, view a project, or edit a project (thanks to Jason Dossett, who implemented this).  The warning is at the top and will clearly state that this is a test tier.  That should make a recurrence unlikely.

2. We’ve turned on daily backups for master and staging.  Normally that’s pointless, but if someone does this again & lets us know within a few days, we should be able to recover the data.  Thus, even if this problem recurs, the backups should reduce the probability or impact of data loss.

 

Thanks!

 

--- David A. Wheeler

 


CII Best Practice badge: Sharing, logos of sample projects, cookie law - PLEASE TELL OTHERS about the badge!

David A. Wheeler
 

FYI, there have been a few changes to the CII Best Practices badge site:

 

1. We now have new “sharing” links at the bottom of the front page <https://bestpractices.coreinfrastructure.org/> for Twitter, Reddit, Facebook, LinkedIn, Google+, and Email.  Unlike “like” buttons, these are NOT tracking devices – they do nothing unless you click on them.  Credits to “Responsible Social Share Links” by Jonathan Suh for explaining how to do this in a privacy-respecting way (<https://jonsuh.com/blog/social-share-links/#use-share-urls>).

2. Also on the front page, we have a list of logos for some of the projects who have earned a badge.  Hopefully that will convince some people that “projects I know got a badge, I want to join the party!”.

3. On the /login page we link to a “/cookies” page, which explains how we use cookies.  This is an honest effort to comply with certain EU/UK laws.  The laws exempt session cookies, and we only use permanent cookies when local users enable “Keep me logged in” (which is not the default), so linking *just* there seems appropriate. I think these cookie laws are misguided, fail to provide *real* privacy, and it’s not clear that we have to comply anyway.  But it doesn’t seem hard to comply, and we want to make everything a *positive* experience for our friends in the EU/UK, so we’ve done our best.

 

PLEASE tell others about the CII Best Practices badge – and by all means, use the new “sharing” links to help you do that.  We are generally growing, both in terms of the number of participating projects and the number of passing projects, but I’d love to see faster growth.  The more FLOSS projects that get a badge, the better the results.

 

Thanks!

 

--- David A. Wheeler

 


Re: CII Best Practices co-developers - need to update Ruby & gems

David A. Wheeler
 

Dan: Thanks!

 

I tried to merge this logic into “update-ruby” so that the script just “works everywhere”.  I don’t have a Mac, so I don’t know if it actually works.  Testing welcome J.

 

--- David A. Wheeler

 

 

From: Dan Kohn [mailto:dan@...]
Sent: Monday, October 23, 2017 1:54 AM
To: Wheeler, David A
Cc: cii-badges@...
Subject: Re: [CII-badges] CII Best Practices co-developers - need to update Ruby & gems

 

On a Mac, it's:

 

brew update && brew upgrade ruby-build

rbenv install 2.4.2

gem install bundler

bundle


--

Dan Kohn <dan@...>

Executive Director, Cloud Native Computing Foundation https://www.cncf.io

+1-415-233-1000 https://www.dankohn.com

 

On Sun, Oct 22, 2017 at 5:27 PM, Wheeler, David A <dwheeler@...> wrote:

FYI:

 

If you are creating proposed changes to the CII Best Practices Badge code, I’ve recently updated the Ruby implementation (as well as all the gems I can).  So if you’ve had the badge application running, the next time after doing a “git pull” you’ll need to run this (in its directory):

 

./ruby-update   # Update Ruby

bundle install   # Install current gems

 

This is documented in the CONTRIBUTING.md file, but I thought I’d point this out here, because I want to make it easy for people to propose changes.

 

Thanks!

 

--- David A. Wheeler

 


_______________________________________________
CII-badges mailing list
CII-badges@...
https://lists.coreinfrastructure.org/mailman/listinfo/cii-badges

 


Re: CII Best Practices co-developers - need to update Ruby & gems

Dan Kohn
 

On a Mac, it's:

brew update && brew upgrade ruby-build
rbenv install 2.4.2
gem install bundler
bundle

--
Dan Kohn <dan@...>
Executive Director, Cloud Native Computing Foundation https://www.cncf.io
+1-415-233-1000 https://www.dankohn.com

On Sun, Oct 22, 2017 at 5:27 PM, Wheeler, David A <dwheeler@...> wrote:

FYI:

 

If you are creating proposed changes to the CII Best Practices Badge code, I’ve recently updated the Ruby implementation (as well as all the gems I can).  So if you’ve had the badge application running, the next time after doing a “git pull” you’ll need to run this (in its directory):

 

./ruby-update   # Update Ruby

bundle install   # Install current gems

 

This is documented in the CONTRIBUTING.md file, but I thought I’d point this out here, because I want to make it easy for people to propose changes.

 

Thanks!

 

--- David A. Wheeler

 


_______________________________________________
CII-badges mailing list
CII-badges@lists.coreinfrastructure.org
https://lists.coreinfrastructure.org/mailman/listinfo/cii-badges



Re: Daniel Stenberg won the Polhem Prize for his work on curl!

Daniel Stenberg
 

On Fri, 20 Oct 2017, Wheeler, David A wrote:

I'd just like to publicly congratulate Daniel Stenberg for winning the Polhem Prize for his work on curl! The Polhem Prize is awarded "for a high-level technological innovation or an ingenious solution to a technical problem."
Thank you!

--

/ daniel.haxx.se


CII Best Practices co-developers - need to update Ruby & gems

David A. Wheeler
 

FYI:

 

If you are creating proposed changes to the CII Best Practices Badge code, I’ve recently updated the Ruby implementation (as well as all the gems I can).  So if you’ve had the badge application running, the next time after doing a “git pull” you’ll need to run this (in its directory):

 

./ruby-update   # Update Ruby

bundle install   # Install current gems

 

This is documented in the CONTRIBUTING.md file, but I thought I’d point this out here, because I want to make it easy for people to propose changes.

 

Thanks!

 

--- David A. Wheeler

 


Daniel Stenberg won the Polhem Prize for his work on curl!

David A. Wheeler
 

All:

 

I’d just like to publicly congratulate Daniel Stenberg for winning the Polhem Prize for his work on curl!  The Polhem Prize is awarded “for a high-level technological innovation or an ingenious solution to a technical problem.”

 

As you know, curl was one of the early projects in the badging project.  Daniel’s commentary & insights have been very helpful to the badging project, and it’s nice to see him honored elsewhere as well.

 

More info here:

https://daniel.haxx.se/blog/2017/10/16/polhemspriset-2017/

https://daniel.haxx.se/blog/2017/10/20/my-night-at-the-museum/

 

--- David A. Wheeler

 


CII Best Practices Badge, 1.5 years later

David A. Wheeler
 

At the Linux Security Summit 2017 I gave a presentation titled “CII Best Practices Badge, 1.5 years later”.  It’s basically a status report about the CII Best Practices badging project.

 

A few highlights:

* We continue to grow (e.g., in the number of participating projects and the number passing badges).

* In general, the number of passing projects seems to be consistently about 10% of the participating projects.  I’m not sure why!

* The most commonly-missed criteria (at first), among projects close to passing, are now vulnerability_report_process (make sure you tell people how to report vulnerabilities) and sites_https_status (HTTPS).

* The criterion tests_are_added (tests are added for new major functionality) is the third most likely problem (it *was* #1).  Hopefully that’s because more projects are working to create & maintain test suites, but trends are always suspicious when there are only 2 data points J.

* Most importantly: We’re making a difference.  OSS projects are reporting that they’re changing things in their projects to meet the criteria, and as a result improving their project.  In many cases, they kept putting it off (e.g., test suites and HTTPS).  In other cases, they just didn’t think of it (e.g., telling people how to report vulnerabilities).  None of this guarantees that there will never be vulnerabilities, of course, but taking care of these things makes it easier to produce software more resistant to attack & more responsive when vulnerabilities are found – and that helps all of us.

 

You can see much more here:

  http://events.linuxfoundation.org/sites/events/files/slides/cii-bp-badge-2017-09_0.pdf

 

Once again, I want to say THANK YOU VERY MUCH for everyone who has participated in this badging project.  No doubt there are improvements that can be made and need to be made.  That’s true for any project J.  But in my mind, the right test for a project is, “is the world a better place because this project exists?”  I believe the badging project is passing that test with flying colors, and that could not have happened without you.  Thank you very much for ALL the effort that ALL of you have put into the badging project.

 

--- David A. Wheeler

 


Dan Kohn on FLOSS Weekly, & CIL providing free computing power

David A. Wheeler
 

FYI:

 

Dan Kohn was instrumental in getting the CII Best Practices Badge project up-and-running.  Dan is now the Executive Director of the Cloud Native Computing Foundation, which sustains and integrates open source technologies like Kubernetes and Prometheus.  He was recently interviewed on “FLOSS Weekly”, and I suspect many of you will be interested in that interview:

  https://twit.tv/shows/floss-weekly/episodes/452

 

In particular, the CNCF Community Infrastructure Lab (CIL) has free access to state-of-the-art computing resources for open source developers working to advance cloud native computing. They offer access to both x86 and ARMv8 bare metal servers.  If you happen to need some real computing power, and it relates to open source software, this might be the free capability you’ve been looking for.  For example, it could be used for software builds, continuous integration, scale testing, or demonstrations.  Dan’s of the opinion that it’d be sad to see this free capability go unused, and I agree, so please let them know if you need something like this.  More info here:

  https://www.cncf.io/community/infrastructure-lab/

 

--- David A. Wheeler

 


Projects that received badges (monthly summary)

badgeapp@...
 

This is an automated monthly status report of the best practices badge application covering the month 2017-09.

Here are some selected statistics for most recent completed month, preceded by the same statistics for the end of the month before that.

Ending dates2017-08-302017-09-29
Total Projects10031049
Projects 25%+371388
Projects 50%+303321
Projects 75%+239251
Projects passing103110

Here are the projects that first achieved a passing badge in 2017-09:

  1. bolt at 2017-09-01 20:15:38 UTC
  2. User mode file system library for windows with FUSE Wrapper at 2017-09-01 20:27:25 UTC
  3. CoreDNS at 2017-09-12 13:33:53 UTC
  4. C++ front/service proxy at 2017-09-19 22:51:05 UTC
  5. chrony at 2017-09-26 15:00:02 UTC
  6. ibmkr at 2017-09-27 04:57:44 UTC

We congratulate them all!

Do you know a project that doesn't have a badge yet? Please suggest to them that they get a badge now!


Badging project continuing to grow!

David A. Wheeler
 

All – there is continuous steady growth in badging project participation.  We have over 1000 participating projects, and over 100 badges.  I’m not sure why 10% of the participating projects at any one time have a badge, but it’s been approximately true for some time.

 

I gave a presentation about the badging project at the 2017 Linux Security Summit; slides here:

  http://events.linuxfoundation.org/events/linux-security-summit

 

Steady growth isn’t something that “jumps out” at you, but I thought you’d like to know!

 

I thank *everyone* here for your help and insights.

 

--- David A. Wheeler

 


Projects that received badges (monthly summary)

badgeapp@...
 

This is an automated monthly status report of the best practices badge application covering the month 2017-08.

Here are some selected statistics for most recent completed month, preceded by the same statistics for the end of the month before that.

Ending dates2017-07-302017-08-30
Total Projects9341003
Projects 25%+348371
Projects 50%+285303
Projects 75%+224239
Projects passing95103

Here are the projects that first achieved a passing badge in 2017-08:

  1. Kubernetes at 2017-08-16 14:52:28 UTC
  2. dash at 2017-08-21 15:29:31 UTC
  3. libpki at 2017-08-24 17:43:06 UTC
  4. LDAP Tool Box Self Service Password at 2017-08-30 12:44:34 UTC

We congratulate them all!

Do you know a project that doesn't have a badge yet? Please suggest to them that they get a badge now!


Projects that received badges (monthly summary)

badgeapp@...
 

This is an automated monthly status report of the best practices badge application covering the month 2017-07.

Here are some selected statistics for most recent completed month, preceded by the same statistics for the end of the month before that.

Ending dates2017-06-292017-07-30
Total Projects877934
Projects 25%+328348
Projects 50%+269285
Projects 75%+213224
Projects passing8895

Here are the projects that first achieved a passing badge in 2017-07:

  1. umoci at 2017-07-02 13:10:18 UTC
  2. lxd at 2017-07-03 16:34:14 UTC
  3. LXC - Linux Containers at 2017-07-03 16:56:04 UTC
  4. A library for exploring persistent homology at 2017-07-05 10:33:27 UTC
  5. Viua Virtual Machine at 2017-07-08 19:09:29 UTC
  6. ruamel.yaml at 2017-07-18 17:01:35 UTC
  7. QtOSG at 2017-07-25 13:35:57 UTC

We congratulate them all!

Do you know a project that doesn't have a badge yet? Please suggest to them that they get a badge now!


Projects that received badges (monthly summary)

badgeapp@...
 

This is an automated monthly status report of the best practices badge application covering the month 2017-06.

Here are some selected statistics for most recent completed month, preceded by the same statistics for the end of the month before that.

Ending dates2017-05-302017-06-29
Total Projects820877
Projects 25%+306328
Projects 50%+250269
Projects 75%+195213
Projects passing8288

Here are the projects that first achieved a passing badge in 2017-06:

  1. Cypht at 2017-06-03 19:01:49 UTC
  2. Iroha - A simple, decentralized ledger at 2017-06-11 02:59:07 UTC
  3. OWASP dependency-check at 2017-06-19 11:00:16 UTC
  4. The Prometheus monitoring system and time series database. at 2017-06-23 08:01:41 UTC
  5. naxsi at 2017-06-23 13:38:29 UTC
  6. LDAPImporter at 2017-06-29 18:38:08 UTC

We congratulate them all!

Do you know a project that doesn't have a badge yet? Please suggest to them that they get a badge now!