Topics

Plan to modify assurance case format (more claims, use SACM notation) - any thoughts?


David Wheeler
 

For the BadgeApp we include an “assurance case”, that is, a set of claims/arguments/evidence explaining why we think it’s secure. You can see the assurance case here:

Some folks at MITRE have been reviewing our assurance case. My thanks - reviews make things better! Two overall suggestions have been made:
1. Use nested claims instead of nested arguments. Claims are simple true/false statements, so this change should make the material easier to follow.
2. Switch to the new SACM graphical notation instead of the older CAE graphical notation. CAE is more common, but SACM notation has many advantages.

I think these are good ideas & I currently plan to implement them over time. We want the assurance case to be maximally clear. I also want its assurance case to be potentially easy to use by others as a starting point (many assurance cases aren’t public, making them hard to learn from).

However, before committing to them, please let me/us know if there are any objections / concerns. If this is a bad idea, I don’t want to do it :-).

Details below.

--- David A. Wheeler


=== DETAILS ===

Up to this point we’ve used claims/argument/evidence (CAE) notation, which is wonderfully simple. Claims (including subclaims) are ovals, arguments are rounded rectangles, evidence (references) are rectangles. You can see its definition here:

Object Management Group (OMG) Structured Assurance Case Metamodel (SACM) specification here:
Historically this specification has worried about defining a standard interchange format for assurance case data. We aren’t trying to exchange with others, and I don’t know of any mature OSS tools that directly support the SACM data format, so this specification hasn’t been focused on a problem we’re trying to solve. However, the newest version of SACM has a new graphical notation. Claims (including subclaims) are rectangles, ArgumentReasoning (aka arguments) are half-rectangles, and evidence are shadowed rectangles. In addition, it uses “big dots” on connections.

Here’s a fragment of our assurance case in CAE graphical notation, including evidence symbols (we often suppress them due to space limitations):

Here’s the same fragment using SACM graphical notation (really a simplified subset of SACM):


Here are advantages of the SACM graphical notation over CAE’s graphical notation:

1. CAE Claim vs. SACM Claim. CAE uses ovals, while SACM uses rectangles. SACM has a *BIG* win here: Rectangles use MUCH less space, so complex diagrams are much easier to create & easier to understand.
2. CAE Argument vs. SACM ArgumentReasoning. CAE uses rounded rectangles, while SACM uses a shape I’ll call a "half-rectangle”. CAE’s rounded rectangles are not very distinct from its evidence rectangles, which is a minor negative for the CAE notation. SACM initially presented some challenges when using our drawing tool (LIbreOffice Draw), but I overcame them:
  - SACM’s half-rectangle initially presented me with a problem: that is *NOT* a built-in shape for the drawing tool I’m using (LIbreOffice Draw). I suspect it’s not a built-in symbol in many tools. I was able to work around this by creating a polygon (many drawing tools support this, and this is a very easy polygon to make). It took a little tweaking, but I managed to create a simple polygon with embedded text. In the longer term, the SACM community should work to get this easy icon into other drawing tools, to simplify its use.
  - SACM’s half-rectangle is VERY hard to visually distinguish if both it & claims are filled with color. I use color fills to help the eye notice type differences. My solution was simple: color fill everything *except* the half-rectangle; this makes them all visually distinct.
3. CAE Evidence vs. SACM ArtifactReference. In CAE this is a simple rectangle. In SACM this is a shadowed rectangle with an arrow; the arrow is hard to add with simple drawing tools, but the shadow is trivial to add with a “shadow” property in LibreOffice (and many other drawing tools), and I think just the shadow is adequate. The shadow adds slightly more space (but MUCH less than ovals), and it takes a moment to draw by hand, but I think that’s a reasonable trade-off to ensure that they are visually distinct. In addition: I tend to record evidence / ArtifactReferences in *only* text, not in the diagrams, because diagrams are time-consuming to maintain. So making *claims* simple to draw, and making evidence/ArtifactReferences slightly more complex to draw, is exactly the right tradeoff.
4. Visual distinctiveness. In *general* the CAE icons for Claim/Argument/Evidence are not as visually distinct as SACM’s Claim/ArgumentReasoning/ArtifactReference, especially when they get shaped to the text contents. That’s an overall advantage for the SACM graphical notation.
5. SACM’s “bigdot”. The bigdot, e.g., in AssertedInference, make the diagrams simpler by making it easy to move an argument / ArgumentReasoning icon away from the flow from supporting claims/evidence to a higher claim. You could also informally do that with CAE, but it’s clearly a part of SACM. In the SACM (like?) diagrams I’ve drawn I’ve omitted the bigdot in some cases, which may not be strictly compliant. I’m not sure how important that is, although I guess one advantage of “bigdot” is that it makes it much easier to add an ArgumentReasoning later.

Nothing is perfect. One problem with SACM’s ArgumentReasoning symbol - a half-rectangle - is that while it’s easy to connect on the left/top/bottom, it’s someone unclear when trying to connect from its bare right-hand-side. A simple solution is to prefer to put them on the right-hand-side. I wish they’d chosen another symbol that was still clearly distinct from the others, easy to hand-draw, already available in simple drawing tools, and did not take a lot of extra space. For example, they could have chosen an uneven pentagon (“pointer”) or callout symbol (with the little tail). But that’s a nit, it’s still an improvement & I try to use standard symbols when it’s reasonable to do so.


Kevin W. Wall
 

Other than describing the SACM's ArgumentReasoning symbol as a "half-rectangle", I have no objections. (A "half-rectangle" is also itself a rectangle, so I think some alternate description would be better. Indeed, in certain cases people might even think "square".)

For those playing along at home, this is the symbol David is referring to:

image.png
I think "open rectangle" would be a little better, but there probably is some formal name for this. (Anyone know?)

Overall, I think the notation is similar in flavor to (i.e., has the look and feel) of UML diagrams, so perhaps a UML drawing tool would work better than Libre Draw. Long ago (at a different job) I used both Dia and Umbrello and they were acceptable, but according to this (https://www.linuxlinks.com/best-free-unified-modeling-software/) there are some better choices. Eclipse also has some UML plugins that might work.

So, FWIW, no objections from me.
-kevin

On Wed, Oct 21, 2020 at 7:59 PM David Wheeler <dwheeler@...> wrote:
For the BadgeApp we include an “assurance case”, that is, a set of claims/arguments/evidence explaining why we think it’s secure. You can see the assurance case here:

Some folks at MITRE have been reviewing our assurance case. My thanks - reviews make things better! Two overall suggestions have been made:
1. Use nested claims instead of nested arguments. Claims are simple true/false statements, so this change should make the material easier to follow.
2. Switch to the new SACM graphical notation instead of the older CAE graphical notation. CAE is more common, but SACM notation has many advantages.

I think these are good ideas & I currently plan to implement them over time. We want the assurance case to be maximally clear. I also want its assurance case to be potentially easy to use by others as a starting point (many assurance cases aren’t public, making them hard to learn from).

However, before committing to them, please let me/us know if there are any objections / concerns. If this is a bad idea, I don’t want to do it :-).

Details below.

--- David A. Wheeler


=== DETAILS ===

Up to this point we’ve used claims/argument/evidence (CAE) notation, which is wonderfully simple. Claims (including subclaims) are ovals, arguments are rounded rectangles, evidence (references) are rectangles. You can see its definition here:

Object Management Group (OMG) Structured Assurance Case Metamodel (SACM) specification here:
Historically this specification has worried about defining a standard interchange format for assurance case data. We aren’t trying to exchange with others, and I don’t know of any mature OSS tools that directly support the SACM data format, so this specification hasn’t been focused on a problem we’re trying to solve. However, the newest version of SACM has a new graphical notation. Claims (including subclaims) are rectangles, ArgumentReasoning (aka arguments) are half-rectangles, and evidence are shadowed rectangles. In addition, it uses “big dots” on connections.

Here’s a fragment of our assurance case in CAE graphical notation, including evidence symbols (we often suppress them due to space limitations):

Here’s the same fragment using SACM graphical notation (really a simplified subset of SACM):


Here are advantages of the SACM graphical notation over CAE’s graphical notation:

1. CAE Claim vs. SACM Claim. CAE uses ovals, while SACM uses rectangles. SACM has a *BIG* win here: Rectangles use MUCH less space, so complex diagrams are much easier to create & easier to understand.
2. CAE Argument vs. SACM ArgumentReasoning. CAE uses rounded rectangles, while SACM uses a shape I’ll call a "half-rectangle”. CAE’s rounded rectangles are not very distinct from its evidence rectangles, which is a minor negative for the CAE notation. SACM initially presented some challenges when using our drawing tool (LIbreOffice Draw), but I overcame them:
  - SACM’s half-rectangle initially presented me with a problem: that is *NOT* a built-in shape for the drawing tool I’m using (LIbreOffice Draw). I suspect it’s not a built-in symbol in many tools. I was able to work around this by creating a polygon (many drawing tools support this, and this is a very easy polygon to make). It took a little tweaking, but I managed to create a simple polygon with embedded text. In the longer term, the SACM community should work to get this easy icon into other drawing tools, to simplify its use.
  - SACM’s half-rectangle is VERY hard to visually distinguish if both it & claims are filled with color. I use color fills to help the eye notice type differences. My solution was simple: color fill everything *except* the half-rectangle; this makes them all visually distinct.
3. CAE Evidence vs. SACM ArtifactReference. In CAE this is a simple rectangle. In SACM this is a shadowed rectangle with an arrow; the arrow is hard to add with simple drawing tools, but the shadow is trivial to add with a “shadow” property in LibreOffice (and many other drawing tools), and I think just the shadow is adequate. The shadow adds slightly more space (but MUCH less than ovals), and it takes a moment to draw by hand, but I think that’s a reasonable trade-off to ensure that they are visually distinct. In addition: I tend to record evidence / ArtifactReferences in *only* text, not in the diagrams, because diagrams are time-consuming to maintain. So making *claims* simple to draw, and making evidence/ArtifactReferences slightly more complex to draw, is exactly the right tradeoff.
4. Visual distinctiveness. In *general* the CAE icons for Claim/Argument/Evidence are not as visually distinct as SACM’s Claim/ArgumentReasoning/ArtifactReference, especially when they get shaped to the text contents. That’s an overall advantage for the SACM graphical notation.
5. SACM’s “bigdot”. The bigdot, e.g., in AssertedInference, make the diagrams simpler by making it easy to move an argument / ArgumentReasoning icon away from the flow from supporting claims/evidence to a higher claim. You could also informally do that with CAE, but it’s clearly a part of SACM. In the SACM (like?) diagrams I’ve drawn I’ve omitted the bigdot in some cases, which may not be strictly compliant. I’m not sure how important that is, although I guess one advantage of “bigdot” is that it makes it much easier to add an ArgumentReasoning later.

Nothing is perfect. One problem with SACM’s ArgumentReasoning symbol - a half-rectangle - is that while it’s easy to connect on the left/top/bottom, it’s someone unclear when trying to connect from its bare right-hand-side. A simple solution is to prefer to put them on the right-hand-side. I wish they’d chosen another symbol that was still clearly distinct from the others, easy to hand-draw, already available in simple drawing tools, and did not take a lot of extra space. For example, they could have chosen an uneven pentagon (“pointer”) or callout symbol (with the little tail). But that’s a nit, it’s still an improvement & I try to use standard symbols when it’s reasonable to do so.



--
Blog: http://off-the-wall-security.blogspot.com/    | Twitter: @KevinWWall
NSA: All your crypto bit are belong to us.