Figure 4: Risk and Quality
Interactions
4.1. The Requirements Phase
4.1.1. Requirement Attributes and
Metrics
The goal for this first phase of software
development is a complete, unambiguous, and understandable requirements
document, the stabilization of requirements as quickly as possible, and
the traceability of all requirements from their source to the software
requirements document and then through design and implementation and test.
The primary risks are that the wrong software will be developed, that the
software will not be completed on schedule, and that the requirements will
not be testable. The associated attributes are:
-
Ambiguity - Potential multiple meanings.
-
Completeness - Items left to be specified.
-
Understandability - Readability of the
document.
-
Volatility - Rate and time within the
life cycle changes are made to the requirements.
-
Traceability - Traceability of the requirements
upward to higher level documents and downward to code and tests.
Table 1 shows the relationships
for the requirement attributes and applicable metrics for this phase.
Table 1: Requirement
Attributes & Metrics
Overall assessment of requirements
risk has to blend the impact of all of the requirements metrics. To reduce
risk, specific requirements that are ambiguous can be identified and revised,
as illustrated in Table 2:
Table 2: Weak Phrases
Completeness is measured by the number
of items not yet specified - that contain TBAs (to be added), TBDs (to
be determined), or TBSs (to be supplied). Measures of Understandability
rely on readability indices, the number of structure levels within the
document and at what level the requirements are specified [WRH96].
Volatility is an important factor in
risk, and extensive measures are often taken to reduce its impact. But
in order to measure volatility, a count of the requirements is needed.
Identifying and counting the imperatives and continuances is one method
to quantify the number of actual requirements in a document. Table 3
is a list of imperatives and continuances commonly found in requirement
documents. The count of the requirements is the sum of the number of imperatives
and the number of items that follow continuances [WRH96].
Table 3: Imperatives
& Continuances
Once the requirements have been counted,
the volatility can be measured by a count of the number of changes divided
by the number of total requirements. The count for the number of changes
is not by number of requirements changed, but by applying the same imperative/continuance
count used to determine the base number of requirements.
4.1.2. Requirement Risks
It is generally accepted that poorly
written, rapidly changing requirements are a principal source of project
risk (and indeed, of project failure). The metrics and attributes defined
above are related to project risk. For example, the metrics for ambiguity
- the count of weak phrases and optional phrases in the requirements document
- indicate problems in the requirements document that can result in confusion
and the need to take unplanned actions to resolve the questions raised.
Thus, the higher the count of ambiguous terms, the higher the project risk.
The requirements counting method discussed
above only provides an indicator of the number of requirements. Even though
it may not yield an exact count, it does provide the developer a working
estimate of tasks to be done. Below is an example of a requirement that
would be flagged by the SATC requirements metrics tool as ambiguous (because
of the use of "normal speed"). It is also difficult to determine
the number of real and implied requirements in the statement, although
the tool would only count the one "shall". (In order to preserve
the anonymity of the project, ITEM1 or ITEM2 are substituted
for project specific terms.)
Requirement: Each channel shall
be capable of processing and forwarding telemetry data to the ITEM1 and
the ITEM2 at normal speed.
If the imperatives are counted, it
has only one task, but in reading the requirement, it becomes obvious that
it contains two or more tasks. Requirements volatility is always associated
with project risk. In addition, the later in the life cycle changes are
made to requirements, the more resources needed to implement them. Late
requirement changes may also cause a ripple effect, causing additional
changes in associated areas. The earlier in the life cycle the requirements
stabilize, the lower the risk. The traditional cost increase for the impact
of errors shown in Figure 5 can be used for the impact of requirements
changes [Fen91].
Figure 5: Additional
Cost of Errors
Because of the high cost and associated
risk of changes made late in the life cycle, requirements attributes and
risk must be measured throughout the life cycle.
4.2. The Design Phase
4.2.1. Design Attributes and Metrics
The design process translates requirements
into a representation of the software that could be assessed for quality
before coding begins. Like requirements, the design is documented and becomes
a part of the software configuration baseline. However, the products of
the design phase are even more difficult to measure. Design products lack
the application of standard depictions and formats and a standard method
for developing the design. This makes metrics for the design phase difficult
to identify and to collect.
Design can be depicted by showing control
flow, data flow, or object-oriented/functional structures. An ideal set
of metrics covers all three, but most projects are designed by developing
only one of these structures. Since the design phase is the transition
between requirements and code, the attributes and metrics proposed for
design overlap these two phases. The attributes for the design phase are:
-
Ambiguity - Multiple meanings.
-
Completeness - Items left to
be specified.
-
Structure/Architecture - Amount
of work a module must perform.
-
Coupling - Strength of the interconnection
from one module to another.
The SATC has identified some design
metrics proposed in the industry as shown in Table 4. Due to insufficient
data, the SATC has been unable at this time to validate this set of metrics.
Since high complexity increases the propensity for errors and hence increases
risks to the schedule and maintainability, the goal of design quality minimizes
the complexity of the design. The applicable metrics are level and completeness
of the design and two types of complexity, similar to those used for code.
Table 4: Design Attributes
& Metrics
Ambiguity could be measured by the
amount of detail, and Completeness could be measured by the number of modules,
not at the lowest level, whose structure is not specified. Structural complexity
is measured using McCabe's cyclomatic complexity. It is a calculation of
the number of test paths within a module. Coupling is measured by Fan-in
and Fan-out. Fan-in is a count of the calls to a given module. This is
the number of local flows into the module plus the number of data structures
from which the procedure retrieves information. Fan-out is a count of calls
from a given module. This is the number of local flows from a module plus
the number of data structures which the module updates.
4.2.2. Design Risks
The design of the software is critical
to all future stages of development. If the design is very complex, the
coding requires more experienced programmers, more meticulous attention
to detail, and more time to code and test. Unfortunately, due to the lack
of standard design formats, metrics for this phase are often ignored or
omitted from risk evaluation. This increases not only the risk to correctness
but can lead to a snowball effect of schedule risk.
4.3. The Coding Phase
4.3.1. Code Attributes and Metrics
The primary goal of a software development
project is to develop code and documentation that will meet the project's
requirements. The primary risks in this phase are that the software be
maintainable and reusable. The risks to schedule and reliability must be
taken into account at this phase, but they are measured in the testing
phase. The specific attributes measured during software development are
as follows:
-
Structure/Architecture - Evaluation of
the constructs within a module to identify possible error-prone modules.
-
Maintainability - Ease of finding and
fixing errors.
-
Reusability - Feasibility of reuse of
the software.
-
Documentation - Adequacy and usability
of internal and external documentation.
The metrics used to evaluate these
attributes are specified in Table 5.
Table 5: Development
Attributes & Metrics
The attributes of Structure/Architecture,
Maintainability and Reusability use the same metrics for evaluation, but
with different emphasis and interpretation guidelines. The three areas
of metrics applied here are complexity, size, and the correlation of module
complexity with size.
There are many different types of complexity
measurements. These include Logical (Cyclomatic) Complexity, which is the
number of linearly independent test paths; Data Complexity, which measures
the data types and parameter passing; and Calling Complexity, which counts
the number of calls to and from modules [IEEE92]. The extent the GOTO command
is used is an indication of disruption of the planned logic flow.
One of the oldest and most common forms
of software measurement is size. There are many possibilities for representing
size: Lines of Code counts all lines containing program headers, declarations,
and executable and non-executable statements; Non-comment Non-blank (Source
lines of code) counts lines that are not a comment line or a blank line;
and Executable Statements counts the number of executable statements regardless
of the number of physical lines.
4.3.2. Code Risks
It is generally accepted that modules
with higher complexities are more difficult to understand and have a higher
probability of defects than modules with smaller values. Thus, complexity
has a direct impact on overall quality and specifically on maintainability
and reusability. Projects developing code designed for reuse should be
especially concerned with complexity since it may later be found to be
more expedient to rewrite highly complex modules than to reuse them.
No single metric is a precise risk
determinant in the analysis of a product. While poor quality of any product
is a risk to the specific goals of the project, some of the best measures
of risk come from correlations of the base metrics. Complexity and size
of modules are two common measurements of modules, but alone they provide
a very one dimensional picture. A more comprehensive view of the data can
be obtained by correlating the complexity to the size. A scatter plot of
the modules (procedure, function, method, class) is useful to evaluate
this data. The SATC uses the Extended Cyclomatic Complexity and the number
of Executable Statements. These metrics were chosen because they allow
maximum comparison across languages and are the least influenced by programmer
style. Using a historical database of code metrics, error data and discussions
with numerous programmers and software managers, the SATC developed the
diagram applied in Figure 6 (counts in Table 6). Each symbol
represents one module.
The purpose of this diagram
is to help prioritize the modules' risk and identify which modules should
be further investigated. The seven risk areas are comprehensive, including
probability of numerous errors, difficult testing, maintaining and reusing.
Figure 6: Modules at
Risk
Modules with an extended cyclomatic
complexity of less than 20 and the number of executable statements less
than 50 have a minimal risk (Region 0). These modules are within most industry
acceptable ranges. Modules with a complexity less than 20 but more than
50 executable statements are usually lists of assignment statements and
are considered low risk (Region 1). Modules with a complexity between
20 and 40 are more difficult to understand. Those with less than 50 executable
statements (Region 3) are terse and tend to be more complex and
hence a higher risk than those modules with 50 to 100 executable statements
(Region 2). The SATC has found that modules with proportional size
to complexity, approximately 2 statements for each unit of complexity (Region
4), are easier to re-engineer and tend to be at a lower risk than modules
in the two other "HIGH" risk areas. Modules that have a high complexity
for a few number of executable statements (Region 6) are the highest
risk. These modules are very terse and compact. They tend to be very difficult
to understand, increasing the difficulty of maintenance and decreasing
the possibility of reuse.
The percentage of modules in each region,
shown in Table 6, and a list of the names of the modules in Regions
4, 5, and 6 are supplied to the project management. It is recommended that
developers further investigate the modules in these regions using additional
metrics such as fan in/ fan out, comment percentage and number of errors,
especially the modules probably at risk in Region 6 with very high complexity
and size.
Table 6: Counts &
Percentages Figure 6
The diagram in Figure 6, when
completed, identifies the potential risks to reliability and, especially,
maintainability and reusability based on size and complexity. The diagram's
values are used to show risk for maintainability by the number of modules
in the higher risk areas. Lower values for complexity and size would be
used to redefine the risk areas for reusability since the candidates for
reusability have both upper and lower guidelines.
Since resources are usually limited
in any project development, another application of the diagram in Figure
6 is to assist developers and project managers in identifying where
to focus resources. Modules in the highest risk areas should be re-engineered
if possible, breaking them into smaller units or applying a different design
algorithm that decreases complexity. If code redesign and modification
is not feasible, high-risk modules should be rigorously tested and explicit
commenting verified.
4.4. Testing Phase
4.4.1. Testing Attributes and Metrics
Once code has been generated and completed,
unit testing, formal testing - System, Integration, and Acceptance Testing
- begins. During formal testing, all software modules are integrated into
a cohesive whole and a series of system integration and validation tests
are conducted. The purpose of this testing is to find both errors remaining
for unit testing and errors which result from unanticipated interactions
between subsystems and components. It is also concerned with validating
that the overall system provides functions specified in the requirements
and that the dynamic characteristics of the system match those required.
The goals for effective testing are
to locate and repair faults in the software, to identify error-prone software,
and to complete testing on schedule with sufficient number of faults found
and repaired that the software will operate as well as needed when it is
put into operation. The risks are to schedule and reliability. The attributes
measured are:
-
Correctness - The likely number of errors
remaining in the software.
-
Reliability - The likely number of highly
critical errors remaining in the software.
The associated metrics are shown
in
Table 7.
Table 7: Testing Attributes
& Metrics
Correctness and Reliability of a computer
program are the most important elements of its overall quality. If a program
repeatedly and frequently fails to perform, the users will not be satisfied,
regardless of the acceptability of other quality factors. Software reliability
is defined in statistical terms as the probability of failure free operation
of a computer program in a specified environment for a specified time.
Software reliability is often measured by the "mean time between failure",
but it can also be measured by estimating the number of errors left in
the software that are of high criticality - that is, that prevent the software
or some major function from operating [KP96].
Correctness can be measured by stating
the percentage of errors within a program that must be removed. This implies
the ability to estimate the number of errors in the software. One industry
guideline is to expect approximately 7 errors per 1000 Source Lines of
Code. This estimate is helpful in an overall estimate of the number of
errors, but does not take into account the rate at which errors are found.
If the rate of finding errors has not started to flatten out, the projection
is that if the software is released, the risk to reliability is high that
it will not meet the qualifications.
The SATC is working to release the
Waterman Error Trending Model for determining the status of testing by
projecting of the number of errors remaining in the software and the expected
time to find a specified percentage of errors. This model used the Musa
Reliability model and the Raleigh curve for effort estimations. The SATC
is developing this model rather than using the standard Musa model because
it is less sensitive to data inaccuracy and provides for non-constant testing
resource levels. Figure 7 is an example of the model's application.
Figure 7: Waterman Error
Trending Model
During the testing phase, as the errors
are identified, they may result in changes to the code. Code "at risk"
that was identified using development phase metrics can now be linked to
errors/changes. This linkage is important because changes to code, especially
code that is high risk due to size and complexity, can lead to additional
problems and ripple effect errors. High-risk code with multiple changes
before release indicates that re-engineering of that code should strongly
be considered. Table 8 indicates a method for combining the metrics
on code modules and presenting a risk picture.
Table 8: Module Total
Risk Indication
In this table, metrics from the development
phase are combined with metrics from the testing phase to provide a more
accurate assessment of which modules pose potential risks. The size/complexity
correlation risk and comment percentage are now combined with the number
of errors detected and the criticality of those errors. The comment percentage
represents the amount of internal documentation that will provide information
to the people making the changes to the code. Minimal internal documentation
can cause indirect or "bad" fixes, or introduce new errors.
Analysis of completion rate data can
also be used to identify risks. The completion rate of tests can be used
to estimate the risk of completing all tests on time, hence completion
of testing requirements and modules. The risk is measured by use of test
report data. Table 9 is an example of data that might be compiled from
the test data and used for projecting testing results for the next 3 months
of testing (test months 6, 7, & 8). Using this information, a project
manager might conclude that if all tests must be passed by the end of the
eighth month, this project is at high risk.
Table 9: Sample Test
Data and Projections
4.4.2. Testing Risks
The metrics for this phase lend themselves
to a number of risk projections. If the project has quantified the percentage
of the errors that are to be found before the software is accepted, the
Waterman Error Trending Model can be used to project when that will happen.
If the time is significantly after the time at which the schedule requires
the software to be available, then schedule risk is high.
Another risk determination can be done
by looking at the code modules or sub-systems in which the errors occur.
Segments of code with more than the average number of errors need investigation
to see if re-engineering should be considered. Additionally, the time to
fix errors should be tracked. If this metric is increasing, then a threat
to reliability and to schedule exists and should be pointed out to the
project.
Finally, the location of faults that
caused high criticality errors should be tracked. A concentration of them
in a segment of code indicates a risk that the requirements are not well
understood, or that the design is not suitable.
4.5. Life Cycle Implementation
4.5.1. Life Cycle Implementation
Attributes and Metrics
The goal of implementation effectivity
within the life cycle activities is to maximize the effectiveness of resources
within the project scheduled activities. This is not phase specific but
starts in the requirements phase and continues to the release of the software.
The primary risk that can be assessed is to the schedule. The attributes
of this goal are:
-
Resource use - The extent resource
usage correlates to the appropriate phase of the project.
-
Completion Rates - Progress
made in completing items such as peer reviews or turnover of completed
modules to CM.
The metrics for these attributes
are shown in Table 10 below:
Table 10: Implementation
Attributes & Metrics
The purpose of collecting resource
hours is not to monitor personnel attendance but to validate schedule risks.
Figure
8 is the Rayleigh curve, which is commonly used to estimate the expected
effort expended for any people intensive activity. Projects tend to ramp
up fairly quickly then slowly decrease - the level of effort is not consistent
throughout implementation and projects need to staff accordingly [Fen91].
Projects that schedule a fixed number of personnel throughout the development
will be understaffed initially and then over staffed towards the project
conclusion.
Figure 8: Rayleigh Manpower
Effort Curve
4.5.2. Life Cycle Implementation
Risks
The risks that can be determined from
the metrics of resource use are based on the appropriateness of the tasks
to which resources are being applied at a given time during the life cycle.
To the extent that for those activities that do not match the expected
or planned activities, an element of risk may have been identified. For
example, work on prior phase activities during the current phase is a risk
indicator. If significant effort is being expended on requirement activities
during the design phase (or worse, during the implementation phase), there
is significant risk that the project will not be able to meet its schedule
objective. This is demonstrated in Figure 9 below.
Figure 9: Effort by Activity
In Figure 9 requirements are
still being written during the time allotted to coding, rippling coding
into the testing phase. This indicates that in order to maintain the schedule,
testing must decrease, putting reliability at risk, or the schedule is
at risk if the testing must be completed.
SECTION 5 - SUMMARY AND
CONCLUSIONS
As highlighted by Capers Jones
(Section 1), large software projects are very hazardous business ventures.
For projects above 10,000 function points, cancellations, delays and cost
overruns have been the norm rather than the exception. But careful analysis
of the root causes of large software project delays and disasters indicate
that most of the problems stem from inaccurate estimation, inaccurate status
reporting, and lack of historical data from similar projects.
All of these root causes can be minimized
or even eliminated by the adoption of formal estimating methods and tools,
by formal monthly status reports of both quantitative and qualitative data,
and by benchmark analysis of similar projects to provide a solid basis
of what can and cannot be accomplished.
Peter Kulik (Section
2) views risk assessments as a tool of most benefit to projects with significant
risk. Generally, a risk assessment will identify root cause strengths and
risks, and may recommend risk management actions to optimize project execution.
Risk assessment methodologies include SEI’s Taxonomy-Based Risk Identification,
Global PM’s Risk Assessment, and Kulik & Lazarus Consulting, Inc.’s
Detailed Risk AssessmentSM.
Risk assessments are typically performed by outside agents expert in Software
Risk Management.
Risk assessments generally
focus on identification of project strengths and risks, risk prioritization,
and estimation of overall project risk. For example, the Detailed Risk
AssessmentSM develops
a TDMschedule profile, analyzes project documentation, and conducts
short interviews with project staff to identify strengths and risks. The
results of these activities are used to develop an actionable framework
of risk mitigation actions based on assessor experience and individual
project characteristics. Formal risk assessment is most effective for projects
with relatively significant risk. In addition, the organization for whom
a project is being assessed needs to have sufficient project management
infrastructure to be able to take action based on the results. The organization
also needs to have a commitment to improving project execution effectiveness.
Risk management benefits range from
accelerating project completion and reducing costs, to improving organizational
capabilities and achieving customer commitments. A number of tools are
available to facilitate implementation of Software Risk Management. For
example, the Project Self-Assessment Kit from Kulik & Lazarus
Consulting, Inc. is an integrated tool which enables users to:
-
Measure 22 project metrics
-
Identify project strengths and
risk areas
-
Quantify overall project risk
-
Develop customized action plans
Using the Project Self-Assessment
Kit, these results can be achieved quickly, easily, and confidentially.
Software risk management has the most benefit for projects with 5 or more
staff members. Its practices can be implemented at any point in a project
- but the earlier in a project, the greater the impact.
On its part, the SATC (Section
3) argues that most project managers agree that risk management works,
but find difficulty in actually implementing it, even when required to
do so. The risk management plan is often hastily written and then thrown
in a corner to gather dust. The SATC has applied its metrics experience
and some concepts from theoretical models of software quality to develop
a unique model for evaluation of quality and project risks. This model
fits the needs of project managers in the NASA and GSFC environment in
the following ways:
-
The model is dynamic, not static,
in the fact that it allows the production of multiple snapshots of project
status across the development. The data is used to make projects about
specific project risks at project milestones.
-
The model uses a broad range
of measures for both software products and development processes.
-
The model is applicable across
the development life cycle. The model's metrics are derived based on aspects
of the attributes that answer questions of the project managers. The model
includes analysis guidelines for the data collected.
New metrics, especially those relating
to requirements quality, need to be validated, and a set of higher level
design quality metrics is needed to supplement those done at the code level.
The objective is to be able to establish a numerical metric scale for assessment
of all of the metrics that affect software quality. Based on the work done
to date, four conclusions can be reached:
-
Requirement metrics assist in
identifying potential project risks
-
Multiple metrics are needed
for comprehensive evaluation
-
Evaluation of requirement text
can yield risk information very early in the life cycle
-
Metric collection is cheaper,
faster and more reliable with requirement management tools
Using automated tools to track
requirements has opened the door to deriving metrics for characterizing
requirement text, stability and expansion rate. Tracking and correlating
test cases and test results to individual requirements within a database
is essential for viewing relationships not otherwise available. The use
of an automated requirements database allows the metrics program to generate
metrics for best insight into the requirements and test case interplay.
Thus one can conclude
that formal risk management analysis and formal project assessments are
effective and useful approaches that are starting to add rigor to the phrase
"software engineering". Not every risk factor is fully controllable, and
several risk factors exceed the authority of software managers, such as
the risk of excessive paperwork caused by military standards themselves.
Nonetheless, risk analysis and assessment methods are quite effective in
the identification of significant problems. Once problems are identified
and examined, solutions can often be developed.
APPENDIX A - REFERENCES
AND BIBLIOGRAPHY
[Fen91] Fenton, N.,
Software
Metrics: A Rigorous Approach, Chapman & Hall, London, UK, 1991.
[Hul96] Hulett, David,
Schedule
Risk Analysis Simplified, PM Network, July 1996.
[IEEE92] IEEE Standard
for a Software Quality Metrics Methodology, IEEE Std 1061-1992, 1992.
[Jon98] Jones, Capers,
Minimizing
the Risks of Software, May 1998.
[KP96] Kitchenham,
B. & Pfleeger, S., Software Quality: The Elusive Target, IEEE
Software, 1/96.
[Kul96] Kulik, Peter,
Team-Driven
Schedule Metrics, March 1996.
[Kul96a] Kulik, Peter,
What
is Software Risk Management?, October 1996.
[RHG99] Rosenberg,
L., Hammer, T., Gallo, A., Continuous Risk Management at NASA, May
1999.
[RH96] Rosenberg, L.,
Hyatt, L., Software Metrics Program for Risk Assessment, October
1996.
[WRH96] Wilson, W.,
Rosenberg, L., Hyatt, L., Automated Analysis of Requirement Specifications,
Fourteenth Annual Pacific Northwest Software Quality Conference, 10/96.
APPENDIX B - GLOSSARY OF RISK MANAGEMENT
TERMS USED IN EXERCISE
Assessment
A general term for a review of the
processes used in developing and maintaining software. Sometimes used more
narrowly to mean the status of a project in the context of how closely
it comes to its planned rate of progress. Also used more restrictively
in context with other terms, such as "stage assessment" or "quality
assessment".
Bottom Up Estimate
Synonymous with micro estimate, and
refers to the method of producing a software estimate by dealing with each
activity separately and then summing the results. It is the opposite concept
from top down estimating or macro estimating which generates an entire
estimate from a single equation and then divides the estimate into small
components via ratios or percentages.
Bugs
This term is one of the oldest in
the software profession. It originated in the 1950's when the late Admiral
Grace Hopper found an actual insect hamming the contacts of an electromechanical
switch. The term now refers to errors or defects that find their way into
programs and systems. Some software quality researchers find it desirable
to replace the pejorative term "bugs" with other terms such as errors,
defects, faults, and failures.
Configuration Management
This term means the methodical storage
and recording of all software components and deliverables as projects are
under development. The term also implies cross referencing among deliverables,
and the ability to trace backwards to the original requirement for any
function.
Coupling
Denotes the way parts of a program
or system interface. Some forms of coupling, such as branching directly
via "go to" statements are considered harmful.
Function Points
A unit of measure for software projects
developed by A.J. Albrecht of IBM. Function Points are derived by a formula
that multiplies the number of inputs, outputs, inquiries, logical files,
and interfaces by empirically derived weights and then adjusts these values
by a complexity multiplier. Function Points were used initially for information
systems, and have the useful attribute of remaining constant regardless
of the source language or languages used for an application.
Life Cycle
This is a very widely used term, but
it has been troubled by totally inconsistent definitions. The earliest
known definition circa the 1960's meant the sequence of events in building
and maintaining software from requirements to retirement. More recently,
CASE vendors have started advertising "full life cycle support" but have
not included defect removal, maintenance, user documentation, and many
other tasks. Exactly what any given assertion of "full life cycle support"
means is now seriously flawed.
Maturity Level
This ambiguous term originated with
Watts Humphrey of the Software Engineering Institute (SEI). It denotes
an arbitrary and artificial five-level set of plateaus of increasing sophistication
in the treatment and understanding of software. The SEI maturity metric
runs from low (1) to high (5) and is an absolute rather than relative metric.
The middle value (3) is not actually the U.S. average, but instead one
of the plateaus of increasing sophistication.
Methodology
The general meaning of this term is
a set of formal protocols followed when performing a task, such as an inspection.
A somewhat more restrictive definition refers to a recognized and sometimes
commercially marketed complex of tasks and deliverables which, it is asserted,
will give better results for software development than random techniques.
Metric
A unit of measure for expressing quantified
information in such a way that comparisons are possible. Examples of software
metrics include Function Points, McCabe complexity metrics, cost per defect,
cost per KLOC, and many others.
Outsource
A fairly new term, originating circa
1990, meaning contracting with a vendor for all of the work of an entire
software and data processing function. Outsourcing is often perceived (and
often truly is) as more cost effective than internal software and data
processing groups. Domestic outsourcing deals with companies in the same
country. International outsourcing deals with companies in other countries.
Quality
This term is extremely ambiguous for
software. It has been variously defined to mean conformance to user requirements,
high levels of customer satisfaction, reliability, and a low number of
bugs found in a given program or system. Most of the common definitions
are subjective and semi-unmeasurable, except for treating quality as the
number of bugs found. It is notable that software with a high bug count
is almost never satisfactory under any of the other definitions of quality.
Reengineering
This term is defined as the extraction
of latent information contained in the source code of an aging existing
program or system in order to develop a new and improved version of the
existing software. The technical idea of re-engineering is that much of
the missing design information that is normally lost as software ages is
actually present but hidden in the code itself. By sophisticated expert
systems methods, this hidden information can be reconstructed.
Risk
This term means the probability that
a software project will experience undesirable events, such as schedule
delays, cost overruns, or outright cancellation. Risk is proportional to
size and inversely proportional to skill and technology levels. In considering
aspects of risk for large systems, the risk of schedule slippage approaches
100% since most such systems are late. The risk of cost overruns is greater
than 50% and the risk of outright cancellation is about 10%.
Risk Analysis
This term denotes a more or less formal
study of the potential hazards that might be encountered in the course
of developing a new software system. Risk analysis is usually formal for
military projects and informal for other classes of software.
Top Down Estimate
This term means estimating a complete
project with a single equation, and then decomposing the estimating via
ratios or percentages to calculate individual tasks. The term is roughly
synonymous with macro estimating, and shares that method's intrinsic lack
of accuracy. The opposite of top down estimating is bottom up or micro
estimating.
