Requirements Consistency

-A Basis For Design Quality-

F. Scott Van Dewerker - Motorola SATCOM
Stuart T. Booth

Abstract

Large design projects have a significant challenge to maintain consistent quality specifications throughout the development life cycle. The cost of maintaining such specifications and ensuring their consistency is high, and if not performed adequately can pose a significant risk to the program by inducing errors into the design. This paper presents methods for automation in concert with a process that will effectively reduce the inconsistencies in specifications.

Introduction

Motivation. Motorola Satellite Communications Division (SATCOM) is the prime integrator for the development, deployment and operations of a complex global telecommunications system called the IRIDIUM¨ System. The IRIDIUM System is a satellite-based, wireless personal communications network designed to permit any type of telephone transmission - voice, data, fax, paging - to reach its destination anywhere on earth, at any time. The estimated cost to develop and deploy this system is $3.4 billion, which is the largest commercially financed satellite project in history. The IRIDIUM system is planned to be providing service in 1998. (Iridium Today, 1994).

In support of this development effort, Motorola SATCOM has established a formal systems engineering process for the development of the IRIDIUM system. The program has completed the conceptual, preliminary and detailed design stages, and is entering the integration and test phase. This effort has culminated in a set of specifications that establish the constraints, functional and performance requirements for each tier in the design. To date, the collection of these specifications contain about 8000 requirements. The systems engineers at Motorola SATCOM firmly believe requirements quality is an essential cost and schedule driver on the IRIDIUM System that will have a direct relation to the program success. As a result, significant resources have been allocated to establish requirements analysis and management processes, augmented with tools and training that focus on developing quality requirements.

Other authors support this quality requirements focus. B.W. Boehm states ñeconomic realities of large systems development, in particular, are such that discrepancies between the delivered system and the needs it must fulfill may cost in excess of 100 times what would have been required if the errors were discovered during the initial problem definition; in some extreme cases, discrepancies may make the entire system useless. For this reason, recent years have been marked by an increased general interest in requirements specification" (Boehm, 1981).

To avoid such consequences, there is a need to formalize the meaning of quality requirements. At Motorola SATCOM, program guidelines have been established on how to develop such requirements. In short, quality requirements must be:

• Complete
• Consistent
• Verifiable
• Valid

Requirements quality initiatives are implemented through the requirements management process. Individuals who participate in the process evaluate requirement quality based on the above criteria. The requirements must be understood within the context of the system and the engineer must make a judgment as to whether the requirement meets the criteria. With the application of RDD-100¨, requirements consistency checking can be automated because the requirements abide by a semantic structure known as the schema. The underlying model that supports the program specific schema is the Element-Relationship-Attribute-Graph (ERAG) model. A capability within RDD-100 and the ERAG is a consistency check facility that can be applied to evaluate many forms of consistency thus reducing potential errors. This paper will discuss the concepts of consistency of requirements and its application.

Definition. Consistency is defined as "the agreement or logical coherence among things or parts" (Webster's, 1984).

Putting this definitioin into the system engineering context, we would like the requirements of the system to be consistent. To be more concrete, it means the requirements conform to a well defined criteria. With this in mind, the authors have identified four categories of requirement consistency:

1. Textual Consistency - key words that will constitute the minimum criteria for a requirement. For example, every Motorola SATCOM requirement must contain the verb ñshall", which is only used once in the requirement statement.
2. Static Consistency- the external referencing of the requirement. This ensures that the requirement has a source, or some justification with other parent and child requirement relationships, and any other relationships that may be defined based on the needs of the program.
3. Dynamic Consistency- a functional model that executes. This ensures that the defined structure of the model is logically correct. It doesn't mean the model exhibits the desired behavior, but it means that the model complies with the execution rules of the simulator.
4. Composition Consistency - an engineering assessment of technical content of the requirement, within the context of the system being designed. Here the emphasis is on the requirement completeness, verifiability, and validity.

The bottom 3 categories of consistency [Figure 1] provide a conceptual foundation for the development of formal consistency checking rules. This foundation enables implementation of automation capabilities. The goal is to allow systems engineers to spend a greater percentage of their time in the development and review of technical content of the requirements (composition consistency), with the textual, static, and dynamic consistency aspects of the requirements rapidly evaluated automatically.

Figure 1.

Four Categories of Requirement Consistency

Figure 2.

System Requirement and Associations

Factors that Contribute to Inconsistencies

To gain some insight in preparing a requirements change package,and the errors that can occur,. Figure 2, represents the minimum set of relationships and target elements for a System Requirement, in accordance with the Motorola SATCOM Specification Practices Guide. In this example, an engineer is tasked with developing a change package containing 5 new System Requirements. For each of the new System Requirements, 5 - 10 relational links must be established (Figure 2). In total, this engineer would have to establish as many as 50 relational links in the example change package. For each new System Requirement, the engineer must write the requirement (in the System Requirement description field), identify a Source document, link the requirement to a Section in a specification, have a Verification Level and Method, trace to a Time Function or incorporate other System Requirements. This is a human intensive effort and without extreme discipline can lead to many errors in requirements process.

Experience has shown that the error rate for a given set of requirements prior to submittal may be as high as 30 percent. With the current SATCOM requirements baseline at 8000 requirements all linked together in a complex web of relationships, if just 5 percent of the requirements are inconsistent, there will be 400 requirement defects . Some of these defects would represent themselves as lack of functionality being flowed down to the lower tiers in the design, incomplete interface definition or no verification method or level being assigned which would impact the integration and test effort. Any of these defects could have significant impacts on the down stream development effort that could hurt cost and schedule.

There are currently 130 systems engineers involved in the requirements management and analysis process, each contributing to the design in a concurrent development environment. To minimize the existence of requirements inconsistencies, and establish a common discipline, a formal basis for consistency checking is essential. The following paragraphs will discuss how to set up consistency checks and apply them within a process.

Consistency Check Process Implementation

Establish the Rules for Consistency. The program guidelines for requirements traceability and completeness form the basis of the consistency checks. When using a design database, like RDD-100, this factors into the semantics of the database schema. The schema is largely dictated by the chosen system engineering methodology and process. Once the schema is established, rules have to be defined on how the schema is used. Some example consistency rules are: a) a requirement cannot have multiple parents, b) a functional requirement cannot be traced from multiple parent requirements, c) a message must be "input-to" and "output-from" a function. Figure 3 is an excerpt from the Motorola SATCOM Specification Practices Guide, and is a recommended approach for documenting the rules for a given schema. The table lists the RDD-100 element type with conditions, constraints, rule statements and notes. Our experience has proven this approach to be very beneficial for developing the consistency checks conceptually and allowing them to be reviewed prior to implementation in RDD-100.

Figure 3.

Consistency Check Rule Base Table

Established Change Process

A fundamental component to effective requirements management is to have a change process, and a requirements baseline under configuration control. At Motorola SATCOM, for requirements to be submitted to the baseline, each engineer must follow a process. This process requires the engineer to prepare a change package that is compliant with the program guidelines as set forth by the Motorola SATCOM Specification Practices Guide. The contents of a change package consist of requirements and related information that are in an RDD-100 format. Part of the systems engineer's task is not only to ensure technical validity of the requirements but also to meet consistency guidelines. To support compliance with the consistency guidelines, the consistency check rules (Figure 3) are codified within RDD-100, which serves as an automation aid in determining the consistency of a respective change package. The evaluation of change package consistency can be complex and very time consuming if done manually. Typically, the change package is being integrated into a complex data structure that consists of thousands of elements. The on-line consistency checks can evaluate the change package and determine if the necessary relationships and attributes of the requirements have been implemented. This process is shown in Figure 4, which describes how a change package can be evaluated against the requirements baseline to determine the relational integrity and completeness of the requirements. The same consistency checks apply to models and, in addition, the model can be executed to determine if it has logical integrity. As shown in Figure 4, an engineer can execute a consistency check report, which is a custom utility, against the changes. The report shows all of the affected elements and any inconsistencies. An example consistency check report output is shown in Figure 5. If inconsistencies are identified the engineer resolves the errors and re-runs the report.

Once the change package is deemed accurate and consistent, the next step is to conduct a requirements inspection. This activity provides closure to determine that the requirements are complete, verifiable, valid and consistent. The consistency check report provides supportive documentation for the inspection activity and lets the engineering team focus on the technical content of the change package

Figure 4.

Change Package Consistency Resolution Process

Figure 5.
i

Consistency Check Report Output

Establish Quality Metrics. Given that there is a design database that contains the requirements, and a rule set has been defined, metrics can be developed that can determine compliance with respect to the consistency check rule set. This metric is a first order determination of the quality of the design database. An example metric would be the percentage of requirements with errors with respect to the total number of requirements. Figure 5, at the bottom of the report, shows the statistics of total elements checked and the number of inconsistent elements for an example change package.

Lessons Learned and Benefits. The establishment of a program discipline to implement consistency checks initially wasn't envisioned. This discipline came about from the experience of a growing database of requirements and models without firm guidelines of schema population and model development. The problem of requirements inconsistencies came about when an audit was performed of the system level requirements. The audit revealed a number of errors that ranged from logical traceability problems to misinterpretation of schema usage. Teams of systems engineers were assigned to correct the problem so that this did not reoccur. As a result, we observed a number of repeated problems that provided a basis for understanding. This allowed us to develop a systematic process for minimizing database inconsistencies in the future. However, this lesson came at a price. It is estimated that to stem the tide of inconsistencies and eliminate legacy inconsistencies, took 4 man years of effort. From this effort, the Motorola SATCOM Specification Practices Guide was developed to incorporate the rules and guidelines for requirements development and the use of RDD-100 on program. The benefit is significantly strengthened requirements management and analysis processes. This strengthening has resulted in improved quality of requirements and reduction in change package development cycle time. The improved quality of requirements is reflected in the reduction of inconsistencies. The status of the requirements is monitored with each change package.

Conclusion

Establishing guidelines and a process for determining consistency provides a basis for evaluating the correctness of the requirements and their relevance to the problem. Further, not having a consistent database significantly reduces the effectiveness of the systems engineer to make decisions and assess impacts. Experience has shown that database technology coupled with a disciplined process provides a means for managing large sets of requirements while minimizing inconsistencies and improving the quality of the system specifications.

References

Boehm, B. W., Software Engineering Economics. Prentice-Hall, Inc., New York, 1981.
Iridium Today, Exploring the Benefits, 1994 Volume 1, Number 1.
Webster's II New Riverside Dictionary, 1984.

Biography

F. Scott Van Dewerker is a System Engineering Interface Lead with the Satellite Communications Division of Motorola. Since joining Motorola in 1993, his primary assignments have been in the development of IRIDIUM¨ System Control Segment interfaces and RDD-100¨ implementation. Prior to joining Motorola, he was a Staff Test Engineer with Martin Marietta in the Defense Systems and Special Programs product areas. He holds an M.S. from the University of Southern California and a B.S. from the University of Northern Colorado.

 

IRIDIUM is a registered trademark and service mark of Iridium, Inc.