A framework for counting problems and defects

Last week I came across this technical report from SEI:

Software Quality Measurement: A Framework for Counting Problems and Defects

Abstract. This report presents mechanisms for describing and specifying two software measures–software problems and defects–used to understand and predict software product quality and software process efficacy. We propose a framework that integrates and gives structure to the discovery, reporting, and measurement of software problems and defects found by the primary problem and defect finding activities....

I haven't yet read through the report thoroughly, though the bits I have read seem immensely sensible. This report doesn't attempt anything too grand. It simply lays out clear definitions, and provides a set of questions to ask yourself when going about trying to understand and count problems and defects. Cool.

In fact, I think these questions will also be great to use when following up with climate modellers about specific bugs. Here they are:

• Identification: What software product or software work product is involved?
• Finding Activity: What activity discovered the problem or defect?
• Finding Mode: How was the problem or defect found?
• Criticality: How critical or severe is the problem or defect?
• Problem Status: What work needs to be done to dispose of the problem?
• Problem Type: What is the nature of the problem? If a defect, what kind?
• Uniqueness: What is the similarity to previous problems or defects?
• Urgency: What urgency or priority has been assigned?
• Environment: Where was the problem discovered?
• Timing: When was the problem reported? When was it discovered? When was it corrected?
• Originator: Who reported the problem?
• Defects Found In: What software artifacts caused or contain the defect?
• Changes Made To: What software artifacts were changed to correct the defect?
• Related Changes: What are the prerequisite changes?
• Projected Availability: When are changes expected?
• Released/Shipped: What configuration level contains the changes?
• Applied: When was the change made to the baseline configuration?
• Approved By: Who approved the resolution of the problem?
• Accepted By: Who accepted the problem resolution?

I might add more why and how questions to this list: why did the bug go unnoticed? why is it important to have fixed this bug, at that time? how was the bug fixed? why is the fix appropriate?