Redesign Cost Avoidance Using System Engineering

Jim Helm - Satellite Communications Division, MOTOROLA INC.

Abstract

A system engineering approach was used for development of the Motorola space-based cellular network called IRIDIUM¨. Significant cost reductions were realized using requirements analysis and management along with system engineering team processes. Statistics show a 40 to 1 reduction in development time due to team requirements inspections. Due to their general nature, these system engineering processes are easily transportable to any project with a development phase.

Introduction

This paper describes how systems engineering has been used by Motorola to develop the IRIDIUM cellular network. Using an overall system level approach to start with, insured product conformance to customer requirements and reduced development cycle time, which equated directly to a large cost saving. It also became apparent that requirements control and overall system modeling at an early stage improved product performance and quality. Initial requirements development set the stage for integration and test of the major components and delineation of system level testing. Interface requirement development, which is generally difficult, was also made easier using a top level total system view to begin with. Such a system engineering approach, rather than a synthesis of optimized components, could be applied to automotive design, with as much success as it had during the development of the IRIDIUM system.

The Iridium System

The IRIDIUM system, shown in Figure 1, is the most complex satellite communications network ever built, consisting of 66 satellites, ground based relay stations and subscriber units capable of world wide service.

(Figure 1).

A system of such complexity would ordinarily result in very difficult interface and requirements management problems, if designed from a bottom up, component approach, i.e. optimize components first and integrate during a later phase, probably right before testing. For a system of this size and complexity, not only requirements tracking, but verification methods and procedures become a major issue that must be addressed. Considering the amount of data involved, and the interconnectivity, an error early in the development process can spawn hundreds of errors in derived requirements used to specify hardware and software. Correction of hardware during the integration and test phase is extremely costly due to the ripple effect through interfaces, and software re-write during the Operations phase is equally expensive, due to potentially linked software modules. While the automotive problem is less complex, the same problems can arise without the methodology afforded by the systems engineering process.

Requirements Inspection Teams

A major decision, during the IRIDIUM development process, was to form requirements inspection teams to cull out errors. For this type of activity to be successful it should be performed in a concurrent engineering setting. Decisions must be made by fully empowered teams comprised of representatives of all functional groups, such as scheduling, contracts, systems engineering, hardware and software engineering, the ñIlitiesî, manufacturing, Operations, field support, etc. This way expertise is available for all parts of the productsÍ total life cycle, which tends to surface problems immediately and allows a sense of co-ownership to develop among the different groups. At Motorola the requirements inspection team included Systems Engineering, Integration and Test, Software Validation and Verification. In addition each major part of the total system, deemed a ñsegmentî, supplied a representative to the team. The segments are known as the Gateway (call transfer), System Control, Space Vehicle (satellite), and Subscriber Equipment (handsets). This is analogous to the automotive chassis, body, power train, etc.

An interesting feature of the inspection process is that certain members were trained to perform specific roles - a Moderator leads the discussions, sets rules and resolves conflicts; a Producer answers technical questions; a Recorder lists requirements defects and their location in the database, and each member brings their particular technical skills to the meeting. Other methods of group organization exist and work equally well, for example, unmoderated discussions wherein decisions are made by vote. An important consideration is that the process must be agreed to by all the participants and the pre-arranged rules followed. Just these simple precepts require training, at the least, and possibly a company culture change at the other extreme.

Historical data, from other manufacturers of complex systems, indicated that these techniques are extremely effective in reducing development cycle time and overall program risk. In particular, the lessons learned by the IBM Space Flight Software effort indicated to the inspection team that early detection of requirement errors would result in a high development cycle time reduction. This was attractive due to the financial incentives for early delivery of a fully operational system.

Since there was an existing, large database of system, derived and functional requirements, the first decisions made by the primary inspection team were which requirements to inspect first and how the inspection methods could be permanently embedded into the ongoing engineering process. Using Pareto analysis and a mapping of the existing requirements change process, it was decided that all the top level system requirements would be examined in detail. This was a formidable task, since the database query of only system requirements, yielded a hardcopy document longer than 600 pages. It also became apparent that a requirements analysis process should be in place at the very beginning of any project when customer requests are first translated into system requirements.

Given the size of the data set and using Belgard-Fisher-Rayner (BFR) team tools, it was decided to dedicate five teams to conduct the inspection. Each team was equipped with guidelines for their inspection, tools to manipulate the data and completion schedules. The teams formulated their own detailed plans, trained their inspectors and also focused on collecting metrics for evaluation of their efforts and to tailor later efforts. After that each team was free to conduct their inspection as they saw fit. In order for the process to work properly, the requirements under scrutiny were frozen in the engineering database for the duration of the inspection. In practical terms, the requirements were numbered in a hierarchical fashion and the numbers controlled by the Configuration Management (CM) function. This meant that during the inspection, engineering users of the database could access the data freely, but could not change requirements, except through submission of formal change requests to CM. This approach also became the basis of the ongoing CM process for engineering data.

Cost Savings

The results of the inspection effort were excellent, due to innovative teaming and open communications, i.e. good concurrent engineering. At the outset of the inspection effort, trial cases were used to yield benchmark estimates of how long it would take to review all the requirements and approximately how many hours it would take to find each defective requirement. Using this data the teams set more stringent goals for themselves, commensurate with the overall schedule for the effort. Due to the intense focus of each team both the rate of review and the defect find rate exceeded the goals set by a wide margin, as shown in Figures 2a and 2b.

Taking this data and combining it with the benchmarked savings information, resulted in eight hours of development time saved for each minor defect corrected and one month saved for each major defect corrected. There is a cycle time reduction savings of one to two months for all minor defects corrected and two to ten months for all major defects corrected. These are illustrated in Figures 3a and 3b .

The dollar figure for estimated savings turns out to be very large. For an effort of approximately 1000 inspection hours, the identification of the major and minor requirements defects, yielded a payback of 40 engineering hours saved for each hour of inspection. In real hours and dollars, the 1000 inspection hours are equivalent to about 32 person-years of rework and development time or a cost avoidance of approximately $6.5 million for the IRIDIUM program.

This savings is not the only benefit of the requirements inspection process; the system documents ( A and B Specifications and Interface management documents) now describe the system correctly more accurately. In addition, the tools used in the inspection process have been refined and used during the routine development cycle. Two more issues have been addressed to insure the robustness of the requirements repository, that is, scrutiny of the lower level or decomposed requirements and continuous requirements processing.

Requirements Decomposition

Having examined the system level requirements, it was obvious that the other, lower level requirements derived from the system requirements must also be examined. This process will eventually merge with the normal cycle of requirements derivation and no additional special inspections will be necessary, only continuous processing which includes data examination. Inspections of the IRIDIUM data have been started on the next level of requirements by querying the requirements database for those entities, excluding the requirements already examined.

This level may be slightly more complex, since it includes functional requirements described by functional diagrams which are graphical in nature and display function logic and data transfer. In fact, some of the lower level requirements were derived using a functional analysis technique called ñthread analysis.

A typical thread will display a series of system or subsystem functions (eventually allocated to hardware or software) which perform in a certain order and transfer information during their operation. Figure 4 illustrates a typical thread (Behavior) diagram, as formulated by the systems engineering tool RDD-100, which has been effectively used throughout the IRIDIUM project.

Even a simple diagram such as this will cause discussion as to proper functionality and timing and message order and content, all of which can spawn requirements to be entered into the database. To inspect the requirements, also causes the team to scrutinize the behavior diagram as well. While this adds complexity, it often makes understanding the concepts easier. The inspection team tools and methods must consequently be expanded to accommodate the functional requirements and their diagrams.

Figure 4: Typical RDD Behavior Diagram (non-specific)

Continuous Requirements Processing

The general IRIDIUM requirements management process has been changed to include an inspection of all requirements prior to inclusion into the database. There is a comprehensive peer review of all requirementsÍ content and a data completeness checking tool which must be run before submission of all change packages to Configuration Management.

Data completeness evaluation, for example, includes information about a lack of data in fields such as descriptions, traceability, verification level and method, a responsible engineer, etc. The content of the data field cannot be checked with the software tool and is therefore the responsibility of the peer review. The object of such control is to prevent errors before they are introduced into the database.

CM controls the approved versions of all requirements and releases them as a network available database which may accessed by all of engineering. Controlled versions of all change packages and the entire database at any point in time is, of course, preserved by CM. Another by-product of the requirements inspection effort has been the development of computer tools, residing with the database, to selectively query the database, provide reports, correct errors, calibrate the database and generate metrics on requirement change activities.

Even after the inspection of the entire database requirements is accomplished, inspection training will continue for future use on other programs. The training will use the metrics generated from the current program as hard examples. The current objective is to train 100% of all system engineers as inspectors, and document the inspection process to as detailed a level as necessary to be able to repeat the process in future programs.

Requirements And Functionality Database

The IRIDIUM requirements and functionality descriptions and related data have been captured in an Element-Relation-Attribute computerized database. The database is implemented by the systems engineering tool, called RDD-100. Currently the system database exceeds 50 MB of data, which includes all the related process data, not just the requirements. Examples of such data are the critical issues raised during requirements decomposition or analysis, and the decisions made to resolve these critical issues. Data may also include problem definition, options, final solutions and identification of the engineers responsible for the decisions.

The system functionality is described using behavior models, which can act as graphical explanations, and can also be logically executed to prove the correctness of the models. System and subsystem models which execute, give the designers greater confidence in their understanding of functional requirements, at any level of the design. In fact, failure to execute in a functional model can often be traced back to a deficiency in the underlying requirements or their interpretation.

This feature was used extensively by the Call Processing Function designers to obtain proper requirements. This type of activity has also been used in other industries. An automotive company is in the process of using the concept to isolate physical system requirement problems.

Many of the activities performed during the requirements inspections were made easier by the use of other features of RDD-100, such as specialized reports, customized views of requirements and their connections in the database, metrics generation and reporting, and consistency checking reports for use by the engineering staff and Configuration Management. In fact it would have been possible to use RDD-100 to model the inspection process itself and the rule bases for each task and role, which would have made the documentation and understandability of the process easier and more concise.

Summary And Conclusions

Redesign cost avoidance using a systems engineering methodology has been extremely successful for the Motorola IRIDIUM system development. A cost benefit ratio of 40:1 was realized in reduction of design cycle time, due to inspection of system requirements, which translates to a cost avoidance of about 6.5 million dollars, after the first 1000 hours of inspection. This general technique, especially when assisted by a computerized systems engineering tool, can be applied to any project with a development phase, and result in significant savings in cost and time, and with major improvements in design integrity.

IRIDIUM is a registered trademark and service mark of Iridium, Inc.