Topics

Draft criteria for passing+1 and pasing+2 - comments?


David A. Wheeler
 

We have an updated draft set of criteria for “passing+1” and “passing+2”:

  https://github.com/linuxfoundation/cii-best-practices-badge/blob/master/doc/other.md

 

These are *not* the final product – but I’d really like to hear comments about them.

 

I really want to make it *possible* for small projects to get higher-level badges, but clearly *some* activities that reduce risk for users *do* require more people (e.g., for multi-person review, bus factor, etc.).  The current draft attempts to make passing+1 possible for small projects, while passing+2 includes criteria that help a project but are not practical for small projects.  Well, that was the idea anyway.

 

--- David A. Wheeler

 


Mark Rader
 

David

While browsing through the list this just caught my eye.

Upgrade: Reporting
  • Upgrade
    • enhancement_responses: SHOULD to MUST. "The project MUST respond to a majority of enhancement requests in the last 2-12 months (inclusive)."
Ok, what do we mean by most?  50.1%, 99.9% there is a lot of wiggle room with "majority".

On Mon, Feb 20, 2017 at 8:44 PM, Wheeler, David A <dwheeler@...> wrote:

We have an updated draft set of criteria for “passing+1” and “passing+2”:

  https://github.com/linuxfoundation/cii-best-practices-badge/blob/master/doc/other.md

 

These are *not* the final product – but I’d really like to hear comments about them.

 

I really want to make it *possible* for small projects to get higher-level badges, but clearly *some* activities that reduce risk for users *do* require more people (e.g., for multi-person review, bus factor, etc.).  The current draft attempts to make passing+1 possible for small projects, while passing+2 includes criteria that help a project but are not practical for small projects.  Well, that was the idea anyway.

 

--- David A. Wheeler

 


_______________________________________________
CII-badges mailing list
CII-badges@lists.coreinfrastructure.org
https://lists.coreinfrastructure.org/mailman/listinfo/cii-badges



David A. Wheeler
 

I think the normal meaning for “majority” is “more than 50%”, but we can say that directly.

 

Of course, the other question is, “is that the ‘right’ criterion in the first place?”

 

--- David A. Wheeler

 

 

From: Mark Rader [mailto:msrader@...]
Sent: Monday, February 20, 2017 11:38 PM
To: Wheeler, David A
Cc: cii-badges@...
Subject: Re: [CII-badges] Draft criteria for passing+1 and pasing+2 - comments?

 

David

 

While browsing through the list this just caught my eye.

 

Upgrade: Reporting

  • Upgrade
    • enhancement_responses: SHOULD to MUST. "The project MUST respond to a majority of enhancement requests in the last 2-12 months (inclusive)."

Ok, what do we mean by most?  50.1%, 99.9% there is a lot of wiggle room with "majority".

 

On Mon, Feb 20, 2017 at 8:44 PM, Wheeler, David A <dwheeler@...> wrote:

We have an updated draft set of criteria for “passing+1” and “passing+2”:

  https://github.com/linuxfoundation/cii-best-practices-badge/blob/master/doc/other.md

 

These are *not* the final product – but I’d really like to hear comments about them.

 

I really want to make it *possible* for small projects to get higher-level badges, but clearly *some* activities that reduce risk for users *do* require more people (e.g., for multi-person review, bus factor, etc.).  The current draft attempts to make passing+1 possible for small projects, while passing+2 includes criteria that help a project but are not practical for small projects.  Well, that was the idea anyway.

 

--- David A. Wheeler

 


_______________________________________________
CII-badges mailing list
CII-badges@...
https://lists.coreinfrastructure.org/mailman/listinfo/cii-badges

 


liujin
 

David,

 

It is noticed code review suggestion and rules had set to “Probably not”,  and it may not be a right choice.

 

The OSASP Top 10 secure issues are too rough to operate, and there are hundreds rules in  CERT suggestion. Both are hard to be used directly.

 

Even today, buffer overflow, integer overflow, and improper validation of array index is still the most common C language code security issues.For small software, to do code review is very easy to operate, and it is the best way to find out most of the ootential code secure fault in small software, e.g., the software less than 100 thousand lines of code. Code view convenient and cheap.For large software, code review has almost become the industry standard. For example, Google and Microsoft have established a clear code view system. This is because the code view is always more accurate and can make up for the lack of static code checking tools.

 

The code review suggestions as following:

1.The direct data or the indirect data from the untrusted sources which is used as the array index MUST be ensured within a legal range.

2.The direct data or the indirect data from the untrusted sources which is used as the buffer length for read/write MUST be ensured within a legal range.

3.The direct data or the indirect data from the untrusted sources which is used as the loop ending condiction MUST be avoided infinite loop or other logic mistake.

4.When copying from a string that is not a trusted source, it MUST ensure that there is enough space to hold the data and the end.

5.The integer values from untrusted sources MUST be avoided the integer overflow or wraparound.

6.Appropriate size limits SHOULD be used to allocate memory from an unreliable source, and MUST check the return value of the allocate function. 

 

The above 6 rules are very simple and very detail, but they can help us to avoid the most dangerous codes’ mistakes. It may be the beginning that helps us improve the efficiency of the code review by specifying rules. We hope that through the contribution of the people, the extraction of limited code view rules can reduce most of our security coding problems.

 

Best Regards

 

--- Jin Liu

 

发件人: cii-badges-bounces@... [mailto:cii-badges-bounces@...] 代表 Wheeler, David A
发送时间: 2017221 10:44
收件人: cii-badges@...
主题: [CII-badges] Draft criteria for passing+1 and pasing+2 - comments?

 

We have an updated draft set of criteria for “passing+1” and “passing+2”:

  https://github.com/linuxfoundation/cii-best-practices-badge/blob/master/doc/other.md

 

These are *not* the final product – but I’d really like to hear comments about them.

 

I really want to make it *possible* for small projects to get higher-level badges, but clearly *some* activities that reduce risk for users *do* require more people (e.g., for multi-person review, bus factor, etc.).  The current draft attempts to make passing+1 possible for small projects, while passing+2 includes criteria that help a project but are not practical for small projects.  Well, that was the idea anyway.

 

--- David A. Wheeler

 


Daniel Stenberg
 

On Tue, 21 Feb 2017, liujin (K) wrote:

Even today, buffer overflow, integer overflow, and improper validation of array index is still the most common C language code security issues.For small software, to do code review is very easy to operate
Code reviews are technically easy to operate, sure, but the problem is rarely the actual review process. The problem for all my projects that I don't do as part of my paid job, is to actually get volunteers to do the job. To spend their spare time and energy on reading someone else's work just to make sure it is good enough.

So then we get in the situation where we can either block process and wait for someone to magically appear at some point and review, or we merge the code anyway to get progress and rely on users in the wild to test it and report back when they find problems. It's a balance.

--

/ daniel.haxx.se


David A. Wheeler
 

From: liujin (K) [mailto:liujin1@huawei.com]
It is noticed code review suggestion and rules had set to "Probably not",  and it may not be a right choice.
Even today, buffer overflow, integer overflow, and improper validation of array index is still the most common C language code security issues.
These are certainly common security issues in C!

For small software, to do code review is very easy to operate, and it is the best way to find out most of the potential code secure fault in small software, e.g., the software less than 100 thousand lines of code. Code view convenient and cheap.
In passing+2 we have two proposed requirements: two_person_review (which I think is somewhat similar to your proposal) and security_review. Currently two_person_review requires that at least 50% of modified code be reviewed; we could make larger (even 100%).

Certainly code review can be very effective. I co-edited the IEEE book, "Software Inspectino: An Industry Best Practice", where I identified many good arguments for code review (especially a specific code review technique called "software inspections"). So there's plenty of evidence for its use.

However: It is *not* cheap. Yes, if you compute the number of defects found per hour of review, it can be very efficient. But not all defects are of equal importance, and *many* projects simply do not have the resources available to require code review no matter how low the cost/defect-found is.

Putting this at passing+2 enables projects that are not as resource-flush to still make forward progress.

For large software, code review has almost become the industry standard. For example, Google and Microsoft have established a clear code view system. This is because the code view is always more accurate and can make up for the lack of static code checking tools.
Both Google and Microsoft have literally billions of dollars to spend. Not everyone has their resources. Also, code review techniques are best *combined* with static & dynamic tools, since each have their strengths & weaknesses.

The code review suggestions as following:
1.The direct data or the indirect data from the untrusted sources which is used as the array index MUST be ensured within a legal range.
2.The direct data or the indirect data from the untrusted sources which is used as the buffer length for read/write MUST be ensured within a legal range.
3.The direct data or the indirect data from the untrusted sources which is used as the loop ending condiction MUST be avoided infinite loop or other logic mistake.
4.When copying from a string that is not a trusted source, it MUST ensure that there is enough space to hold the data and the end.
5.The integer values from untrusted sources MUST be avoided the integer overflow or wraparound.
6.Appropriate size limits SHOULD be used to allocate memory from an unreliable source, and MUST check the return value of the allocate function. 
First: We should avoid language-specific requirements; tech changes! If we use a list like this, it needs to be recast as additional requirements for memory-unsafe code (C, C++, and some other languages when memory safety is disabled). Almost all of these seem to be specific to memory-unsafe code.

#3 won't work. Many programs are specifically designed as infinite loops, that's the whole point. For example, many real-time systems, after boot, look like this:
while true {
read_input
do_stuff
}

One possibility is turning this into a set of minimum criteria for code reviews (in passing+2) when using a memory-unsafe language. Or maybe it's a new "code_review" or "code_review_guidelines" criterion.

The above 6 rules are very simple and very detail, but they can help us to avoid the most dangerous codes' mistakes. It may be the beginning that helps us improve the efficiency of the code review by specifying rules. We hope that through the contribution of the people, the extraction of limited code view rules can reduce most of our security coding problems.
--- David A. Wheeler