Software measurement

Defining Software Measures

How to accurately define software measures (according to the ISO/IEC/IEEE 15939:2017)

Definitions

Photo by Sandy Millar on Unsplash
  • Derived measure: measure that is defined as a function of two or more values of base measures.
  • Base measure: measure defined in terms of an attribute and the method for quantifying it.
  • Indicator: measure that provides an estimate or evaluation of specified attributes derived from a model with respect to defined information needs.
  • Measurement method: logical sequence of operations, described generically, used in quantifying an attribute with respect to a specified scale. The type of measurement method depends on the nature of the operations used to quantify an attribute. Two types can be distinguished:
  • subjective: quantification involving human judgment; and
  • objective: quantification based on numerical rules.
  • Scale: ordered set of values, continuous or discrete, or a set of categories to which the attribute is mapped.

Scale

Photo by Ag PIC on Unsplash
  • Nominal: the measurement values are categorical. For example, the classification of defects by their type does not imply order among the categories. Operations supported: Counting values.
  • Ordinal/Likert: the measurement values are rankings. For example, the assignment of defects to a severity level is a ranking. Many people try to do math with this scale, attributing numbers to the values. You have probably seen this with “ratings”. Imagine you are researching how much people like a particular wine. They taste it and rank it among “Strongly dislike”, “Dislike”, “Indifferent”, “Like”, and “Strongly like”. Then some magician turns these into numbers from 1 to 5 and calculates an average, reaching the conclusion that everyone liked it 3.5 in 5. Well, first of all, there’s nothing to say that the distance between any 2 of these values is the same as any other 2. That’s for another scale type (Interval). Second, having most people liking your product around the average is very different from having a group of lovers and a group of haters. Operations supported: Counting values and sorting values.
  • Interval: the measurement values have equal distances corresponding to equal quantities of the attribute. For example, cyclomatic complexity has a minimum value of one, but each increment represents an additional path. The value of zero is not possible. Operations supported: Counting values, sorting values, and since it has equidistant intervals, you can also perform addition and subtraction.
  • Ratio: the measurement values have equal distances corresponding to equal quantities of the attribute where the value of zero corresponds to none of the attribute. For example, the size in terms of the number of requirements is a ratio scale because the value of zero corresponds to no requirements and each additional requirement defined represents an equal incremental quantity. Operations supported: Counting values, sorting values, addition, subtraction, division, and multiplication.

Criteria for selecting measures

  • relevance to the prioritized information needs;
  • feasibility of collecting the data in the organizational unit;
  • ease of data collection;
  • extent of intrusion and disruption of staff activities;
  • availability of appropriate tools;
  • protection of privacy;
  • potential resistance from data provider(s);
  • number of potentially relevant indicators supported by the base measure;
  • evidence (internal or external to the organizational unit) as to the measure’s fitness for purpose or information need, and its utility; and
  • The costs of collecting, managing, and analyzing the data at all levels should also be considered. Costs include the following:
  • Measures utilization costs: associated with each measure are the costs of collecting data, automating the calculation of the measure values (when possible), analyzing the data, interpreting the analysis results, and communicating the information products;
  • Process Change Costs: the set of measures may imply a change in the development process, for example, through the need for new data acquisition;
  • Special Equipment: system, hardware, or software tools may have to be located, evaluated, purchased, adapted or developed to implement the measures; and
  • Training: the quality management/control organization or the entire development team may need training in the use of the measures and data collection procedures. If the implementation of measures causes changes in the development process, the changes needs to be communicated to the staff.

Measurement method

Accuracy of a measurement procedure

  • the extent of missing data is within specified thresholds;
  • the number of flagged inconsistencies in data entry are within specified thresholds;
  • the number of missed measurement opportunities are within specified thresholds (e.g., the number of inspections for which no data were collected);
  • all base measures are well‐defined and those definitions are communicated to data providers. Poorly defined measures tend to yield inaccurate data. The repeatability and reproducibility of the underlying measurement method (see below) may also limit the accuracy achievable by a measurement procedure.

Repeatability of a measurement method

Reproducibility of a measurement method

Establishing a measure

  1. Requirements implemented by past projects
  2. Effort expended by past projects
  1. User Stories (requirements)
  2. Worked days (effort)
  1. Project X Requirements
  2. Project X Days of Effort
  1. Acceptance criteria items in all user stories
  2. Add the worked days of all people for Project X
  1. Objective
  2. Objective
  1. Integers from zero to infinity
  2. Real numbers from zero to infinity
  1. Ratio
  2. Ratio
  1. Criteria
  2. Day

--

--

Many management blogs focus on soft skills. This blog is about hard skills! Measurement, indicators, approaches, etc., for Software Engineering Management.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Peter P. Lupo

Many management blogs focus on soft skills. This blog is about hard skills! Measurement, indicators, approaches, etc., for Software Engineering Management.