Here's some data to work from. Below is a short Ruby snippet to sum up the results for each criterion, followed by a CSV formatted result from when it was executed.

Anyone who's interested can load the CSV file into something... and report what they learn!!

--- David A. Wheeler

===================================

results = {}
Project::ALL_CRITERIA_STATUS.each do |criterion|
data=Project.select(criterion.to_s).group(criterion.to_s).unscope(:order).count
results[criterion] = data
end
puts 'criterion,Met,Unmet,?,N/A'
results.each do |k,v|
puts "#{k.to_s}, #{v.key?('Met') ? v['Met'] : 0}, #{v.key?('Unmet') ? v['Unmet'] : 0}, #{v.key?('?') ? v['?'] : 0}, #{v.key?('N/A') ? v['N/A'] : 0}"
end
true

===========================================


criterion,Met,Unmet,?,N/A
description_good_status, 58, 2, 63, 0
interact_status, 53, 4, 66, 0
contribution_status, 113, 4, 6, 0
contribution_requirements_status, 41, 8, 74, 0
floss_license_status, 88, 0, 35, 0
floss_license_osi_status, 82, 14, 27, 0
license_location_status, 95, 23, 5, 0
documentation_basics_status, 59, 1, 63, 0
documentation_interface_status, 55, 3, 65, 0
sites_https_status, 100, 21, 2, 0
discussion_status, 115, 2, 6, 0
english_status, 62, 1, 60, 0
repo_public_status, 112, 0, 11, 0
repo_track_status, 113, 1, 9, 0
repo_interim_status, 43, 1, 79, 0
repo_distributed_status, 114, 0, 9, 0
version_unique_status, 46, 0, 77, 0
version_semver_status, 34, 12, 77, 0
version_tags_status, 46, 0, 77, 0
release_notes_status, 50, 63, 10, 0
release_notes_vulns_status, 37, 4, 82, 0
report_process_status, 44, 0, 79, 0
report_tracker_status, 45, 1, 77, 0
report_responses_status, 42, 1, 80, 0
enhancement_responses_status, 38, 3, 82, 0
report_archive_status, 44, 0, 79, 0
vulnerability_report_process_status, 34, 6, 83, 0
vulnerability_report_private_status, 30, 4, 82, 7
vulnerability_report_response_status, 38, 2, 83, 0
build_status, 56, 49, 9, 9
build_common_tools_status, 55, 51, 10, 7
build_floss_tools_status, 38, 0, 78, 7
test_status, 45, 0, 78, 0
test_invocation_status, 42, 0, 81, 0
test_most_status, 29, 12, 82, 0
test_continuous_integration_status, 38, 5, 80, 0
test_policy_status, 37, 3, 83, 0
tests_are_added_status, 35, 3, 85, 0
tests_documented_added_status, 24, 14, 85, 0
warnings_status, 34, 2, 82, 5
warnings_fixed_status, 33, 2, 83, 5
warnings_strict_status, 27, 6, 85, 5
know_secure_design_status, 38, 2, 83, 0
know_common_errors_status, 39, 1, 83, 0
crypto_published_status, 25, 0, 84, 14
crypto_call_status, 22, 1, 84, 16
crypto_floss_status, 26, 0, 83, 14
crypto_keylength_status, 18, 1, 86, 18
crypto_working_status, 25, 2, 82, 14
crypto_weaknesses_status, 24, 4, 83, 12
crypto_pfs_status, 13, 0, 86, 24
crypto_password_storage_status, 13, 1, 85, 24
crypto_random_status, 19, 0, 85, 19
delivery_mitm_status, 39, 1, 83, 0
delivery_unsigned_status, 35, 2, 86, 0
vulnerabilities_fixed_60_days_status, 37, 2, 84, 0
vulnerabilities_critical_fixed_status, 37, 1, 85, 0
no_leaked_credentials_status, 41, 0, 82, 0
static_analysis_status, 30, 2, 88, 3
static_analysis_common_vulnerabilities_status, 23, 4, 88, 8
static_analysis_fixed_status, 27, 2, 87, 7
static_analysis_often_status, 23, 8, 88, 4
dynamic_analysis_status, 23, 12, 88, 0
dynamic_analysis_unsafe_status, 17, 3, 88, 15
dynamic_analysis_enable_assertions_status, 23, 10, 90, 0
dynamic_analysis_fixed_status, 29, 4, 90, 0
installation_common_status, 25, 1, 97, 0
build_reproducible_status, 11, 6, 101, 5
crypto_used_network_status, 15, 0, 98, 10
crypto_tls12_status, 12, 0, 98, 13
crypto_certificate_verification_status, 10, 1, 99, 13
crypto_verification_private_status, 7, 1, 99, 16
hardened_site_status, 18, 11, 94, 0
hardening_status, 17, 2, 104, 0

I some improvements to my previous script, in particular, I sorted by how many projects "met" each criterion. You can see the results below. I'd love to hear what people's insights are...

--- David A. Wheeler


=== Ruby ===

results = []
Project::ALL_CRITERIA_STATUS.each do |criterion|
data=Project.select(criterion.to_s).group(criterion.to_s).unscope(:order).count
results.append([criterion.to_s, data.fetch('Met',0),
data.fetch('Unmet',0), data.fetch('?',0), data.fetch('N/A',0)])
end
results.sort! { |x,y| x[1] <=> y[1] }
puts 'criterion,Met,Unmet,?,N/A'
results.each do |row|
puts row.join(',')
end

=== Data ===

criterion,Met,Unmet,?,N/A
crypto_verification_private_status,7,1,99,16
crypto_certificate_verification_status,10,1,99,13
build_reproducible_status,11,6,101,5
crypto_tls12_status,12,0,98,13
crypto_pfs_status,13,0,86,24
crypto_password_storage_status,13,1,85,24
crypto_used_network_status,15,0,98,10
dynamic_analysis_unsafe_status,17,3,88,15
hardening_status,17,2,104,0
crypto_keylength_status,18,1,86,18
hardened_site_status,18,11,94,0
crypto_random_status,19,0,85,19
crypto_call_status,22,1,84,16
static_analysis_common_vulnerabilities_status,23,4,88,8
static_analysis_often_status,23,8,88,4
dynamic_analysis_status,23,12,88,0
dynamic_analysis_enable_assertions_status,23,10,90,0
tests_documented_added_status,24,14,85,0
crypto_weaknesses_status,24,4,83,12
crypto_published_status,25,0,84,14
crypto_working_status,25,2,82,14
installation_common_status,25,1,97,0
crypto_floss_status,26,0,83,14
warnings_strict_status,27,6,85,5
static_analysis_fixed_status,27,2,87,7
test_most_status,29,12,82,0
dynamic_analysis_fixed_status,29,4,90,0
vulnerability_report_private_status,30,4,82,7
static_analysis_status,30,2,88,3
warnings_fixed_status,33,2,83,5
version_semver_status,34,12,77,0
vulnerability_report_process_status,34,6,83,0
warnings_status,34,2,82,5
tests_are_added_status,35,3,85,0
delivery_unsigned_status,35,2,86,0
release_notes_vulns_status,37,4,82,0
test_policy_status,37,3,83,0
vulnerabilities_fixed_60_days_status,37,2,84,0
vulnerabilities_critical_fixed_status,37,1,85,0
enhancement_responses_status,38,3,82,0
vulnerability_report_response_status,38,2,83,0
build_floss_tools_status,38,0,78,7
test_continuous_integration_status,38,5,80,0
know_secure_design_status,38,2,83,0
know_common_errors_status,39,1,83,0
delivery_mitm_status,39,1,83,0
contribution_requirements_status,41,8,74,0
no_leaked_credentials_status,41,0,82,0
report_responses_status,42,1,80,0
test_invocation_status,42,0,81,0
repo_interim_status,43,1,79,0
report_process_status,44,0,79,0
report_archive_status,44,0,79,0
report_tracker_status,45,1,77,0
test_status,45,0,78,0
version_unique_status,46,0,77,0
version_tags_status,46,0,77,0
release_notes_status,50,63,10,0
interact_status,53,4,66,0
documentation_interface_status,55,3,65,0
build_common_tools_status,55,51,10,7
build_status,56,49,9,9
description_good_status,58,2,63,0
documentation_basics_status,59,1,63,0
english_status,62,1,60,0
floss_license_osi_status,82,14,27,0
floss_license_status,88,0,35,0
license_location_status,95,23,5,0
sites_https_status,100,21,2,0
repo_public_status,112,0,11,0
contribution_status,113,4,6,0
repo_track_status,113,1,9,0
repo_distributed_status,114,0,9,0
discussion_status,115,2,6,0

David,

Thanks for taking the time to do this. I plan on taking a look
at it this weekend and see if anything interesting jumps out at me.

Just one question though...I'm trying to recall if '?' is the default
answer for everything or if by default there is no answer and you are
required to pick something. I seem to recall it is the former, but
I don't want to start a dummy project just to check.

Thanks again,
-kevin

--
Blog: http://off-the-wall-security.blogspot.com/ | Twitter: @KevinWWall
NSA: All your crypto bit are belong to us.

Kevin W. Wall:

Thanks for taking the time to do this. I plan on taking a look at it this weekend and see if anything interesting jumps out at me.

Thanks, looking forward to it. I've been intended to do that anyway, so you spurred me on.

Just one question though...I'm trying to recall if '?' is the default answer for everything or if by default there is no answer and you are required to pick something. I seem to recall it is the former, but I don't want to start a dummy project just to check.

Yes, '?' is the default answer that just means unknown. A new project will have mostly '?'. However, when you start a project, we *do* try to fill in some values, so even in a whole new project some criteria may have a status other than '?'.

--- David A. Wheeler

On Mon, Jun 13, 2016 at 8:42 AM, Wheeler, David A <dwheeler@ida.org> wrote:

Kevin W. Wall:

Thanks for taking the time to do this. I plan on taking a look at it this weekend and see if anything interesting jumps out at me.

Thanks, looking forward to it. I've been intended to do that anyway, so you spurred me on.

Okay, well here it is in
<https://drive.google.com/file/d/0B3Yc2oc1Z9n5UjBkb2dWWWJpRGM/view?usp=sharing>;
as an Open Document Spreadsheet (.ods, like what LibreCalc uses).

The first worksheet is just the raw data gathered from the data that
David published.
The 2nd is some very rudimentary analysis...looking at things like top
5 and bottom 5
for 'met', 'unmet', '?', and 'N/A' and some of my rambling
observations. I guess
the only think that partially surprised me was there that I was
somewhat expecting
there to be an overlap between the criteria that had the "top 5 'met'" and the
"bottom 5 'unmet'", but there was none. Perhaps this was because most of the
'top 5 met' criteria were those that were automatically filled-in once
the source
repository link had been provided.

Anyhow, maybe those of you who understand the relationships between
the criteria can glean some deeper meaning from this.

It's at least a start,
-kevin
--
Blog: http://off-the-wall-security.blogspot.com/ | Twitter: @KevinWWall
NSA: All your crypto bit are belong to us.

I guess the only think that partially surprised me was there that I was somewhat expecting there to be an overlap between the criteria that had the "top 5 'met'" and the "bottom 5 'unmet'", but there was none. Perhaps this was because most of the 'top 5 met' criteria were those that were automatically filled-in once the source repository link had been provided.

I'd expect the most common response in many cases to be '?'. Unless the system can automatically determine the answer, '?' is the default. Having "Unmet" means that either the system determined it wasn't met, or that a human was willing to admit that it wasn't met. Humans don’t like admitting they don't do something they don't do, so I suspect in many cases the "unmet" is due to automation.

It's at least a start,

Certainly is!

--- David A. Wheeler