By summarising several studies, Hatton concludes that the number of faults has a logarithmic relationship to code size (or, more generally, code complexity). One implication of this is that smaller components have a higher density of faults (since the logarithmic curve rises sharply at first, and then grows more slowly). The interesting bit of the paper comes when Hatton models these observations from a psychological basis beginning with G. Miller's observation that a person can only cope with 7±2 independant pieces of information in short-term memory. Hatton's model specifies one behaviour (logarithmic) for components as they increase in size up to the "Miller threshold" (Hatton uses a capital Omega prime symbol to denote this), and second behaviour (quadratic) for components as they increase past this threshold.
This paper is simultaneously cool and confusing. Cool because Hatton is trying to rigorously use our knowledge from psychology to explain the underlying causes for defect density, rather than just fit an equation to the data. And he tries to back up his models with lots of data, or clearly state when he can't back it up without doing more empirical work.
But the paper is also confusing for a number of reasons. Some are technical: many of the figures are hard to read: the axis titles are vague or ambiguous --one plot is actually missing an x-axis!; another plot's description conflicts with the description given in the paper text. Others are less technical: Hatton switches from talking about different models throughout the paper and isn't very clear as to the situations he's discussing. I had to re-read the paper a few times because of this (as always, YMMV, maybe it's just me being slow). He's also very vague at points, especially when it comes to explaining why fault density ought to be proportionately higher for components below the Miller threshold,. He says,
"... if a system is decomposed into pieces much smaller than the short-term memory cache, the cache is used inefficiently because the interface of such a component with its neighbours is not 'rehearsed' explicitly into the cache in the same way, and the resulting components tend to exhibit higher defect densities."Erm.. that's all we get on the matter. In short, I like what he's trying to do here by backing up his models, he just doesn't convince me in this paper that he's got the right explanation.
That said, he concludes that there is an optimum component size range to achieve low fault density. The range differs across languages (because some languages pack less information in per line so the Miller threshold is reached sooner if you're just counting lines) and across programmers (because Miller's magic number is actually a range from 5-9 pieces of information). He mentions several implications of this that make for testable hypotheses. For instance, "manual [code] inspections would be most effective on components that fit into cache", and "only substantial reuse within the same system will likely improve reliability. Modest reuse ... is likely to make it worse."
1 comment:
In "A Critique of Software Defect Prediction Models" Fenton and Neil contradict this study. Their argument that made the most sense to me is that larger modules often hide defects more effectively than smaller modules. The Fenton/Neil paper is a worthwhile read if you are interested in fault prediction. They take a very process-oriented view, where I am interested in a more product-oriented one, but each has its proper applications.
Post a Comment