(The views expressed are solely those of the authors and do not reflect the views of the System Safety Society or the organizations with which the authors are affiliated.

The objective for this session is to overview and discuss the principles, practices and procedures (methods) that constitute the system safety methodology. Our task is to examine the orderly thought processes and methods that are applied by system safety professionals.

PART I

WHAT IS A METHODOLOGY?

Recognition of the issues we will explore in this paper depends in part on awareness of differences between "methodologies" and "method." "

Let's take the term "methods" first. As you consider the issues, think of method as being a regular, disciplined and systematic procedure for accomplishing a task. A method is a technique. During the presentation, the term technique will be used in lieu of method, to avoid confusion with methodology.

Methodology, on the other hand, has a broader context, A methodology is a system of working concepts, principles, and body of procedures employed by a discipline for application to a specific branch of knowledge determining in large measure how that branch of knowledge is practiced. A methodology is an overall approach to a field, System safety methodology is an overall approach to safety. The broader systemic approach to the safety field, rather than individual procedures or methods, was chosen for the first workshop.

First, you are asked to consider the approach that system safety is the systematic approach to achieving acceptable safety levels through the management decision making process.

This systematic approach can be described by the following:

defining the system,
identifying the hazards associated with the system, determining the corrective action that may be implemented to either eliminate or control the hazard,
recommending corrective action or alternatives to the appropriate management level for a decision to either resolve the hazard or assume the risk, and
documenting those areas in which a decision has been made to assume the risks, including the rationale for risk assumption.

A. Defining the system

This is the process used to convert information from many sources about the system into a form so that it can be analyzed effectively. It is essential that this documentation accompany the analysis report to record exactly how the system was configured at the time the analysis was performed.

The first step is to establish the system boundaries to clearly identify the scope of the analysis. The definition process then addresses elements considered to be within the boundary (machine), the elements outside the system boundary (man, medium, management) and the transients which move across the boundary interface.

The machine has been generally defined by conceptual sketches, schematics, drawings, flow diagrams and descriptions to establish the element hierarchy which evolves from the physical and functional relationships.

The man has been generally defined by the operational procedures and technical manuals.

The medium has been generally defined by the mission profile and the normal and abnormal operating modes.

The management has been generally undefined.

What information is needed considering that the precedents are well established that accidents are caused by factors associated with machine (failure, design defect), man (human error, workload), medium (environment) and management (attitude, motivation, control)?

The definition process is essential for the system safety professional to establish and maintain professional credibility. Bring yourself up to a level where you understand the system in-depth and stay current with the changes. Establish a rapport with Systems engineering.

B. Identifying the hazards associated with the system, determining the corrective action that may be implemented to either eliminate or control the hazard.

It is essential to use all possible precedent Centers to gather previous accident experience on similar systems. There are only a few primary hazards for any system. There are many contributing hazards. There are countless accident "triggers". Gene rate and use checklists to assure compliance with base. line safety requirements and criteria from learned experiences.

The objectives for the initial safety analysis (preliminary hazard analyses) and procedural hazard analysis (operating and support hazard analysis) appropriately prescribe a life cycle approach, however, implementation requires that dynamics be assessed continually. This requires "horse trading" when changes occur dealing with program costs and schedule as well as those involving the system configuration. Plans for applying analysis techniques should reflect, not only the life cycle phase when activity is most intense, but also the iterations required to maintain the analysis product. A greater awareness is needed for real "red flags".

Inductive and deductive analysis techniques should be used in combination to provide the most effective tailoring scheme. This requires good management and engineering common sense. Formats for the preliminary hazard analysis and fault hazard analysis are examples of inductive analyses. Task analysis is an example of human factors inductive analyses. Examples of deductive analyses include hazard matrix, fault tree analysis and common cause failure analysis.

Emphasis should be on designing for minimum hazard to achieve acceptable safety levels as required by the established order of precedence for resolving hazards.

Groups of technical experts should be cultivated to achieve the best product with the minimal effort. The system safety professional should use, motivate and orchestrate the designers and system specialists to enhance the technical quality of the analysis product The pure cultist approach should be avoided.

Mastering the art of developing critical questions Is an important system safety tool. Questions must be expressed in a language that best fits the situation. Questions should be progressively more discerning so that they lead to a point where answers will help form judgments on the degree of the safety risks.

C. Recommending corrective action or alternatives to the appropriate management level for a decision to either resolve the hazard or assume the risk.

Some basic tenets (courtesy of Chuck Miller) are worth repeating because they express the problem so well:

Accidents are unplanned, but controllable combinations of events
Accidents are rare, hazard (risks) are not.
Combinations of "acceptable" hazards produce accidents.
Accidents are usually caused by a sequence of complex cause-effect relationships that may be obscured by simplistic probable/proximate cause determinations.

The acceptance of risks should be based on thorough visibility as to the nature of hazards and risks and the options and alternatives to the acceptance of the risks. The management decision to resolve the hazard or accept the risk is reached by balancing each option for hazard resolution following the prescribed order of precedence, with cost, performance, and schedule constraints.

The risk assessment may be qualitative or quantitative, depending on the hazard analysis results, in addressing the chance of certain undesired consequences such as personnel injury fatality, damage to or loss of equipment or property.

Quantitative analyses such as fault tree modeling should be performed only where the risks of parts/component failures and human errors for the operational environment are known with reasonable confidence and the criticality of alternative designs is sufficiently important to safety.

Risk assessments derived from qualitative analyses are subjective. The residual risk is accepted based on the degree of control offered by the hardware design and operating procedures for each identified hazard. Because the assessment is subjective, the management procedures and review process should provide for a free flow of information among management, engineering, test and operational personnel to evaluate the perceived risk from different points of view. The third party "jury"' is another approach that has been successfully implemented.

D. Documenting those areas in which a decision has been made to assume the risk including the rationale for risk assumption.

Closed loop procedures should be implemented to assure that management, fully apprised of the safety risks, formally approves the corrective action or risk acceptance one by one. Closed loop procedures should also be implemented to track and monitor corrective action until closeout. Drawings and technical manuals should be reviewed to monitor the implementation of corrective action. Inspections should be conducted for reviewing design and procedural controls for testing and operations to verify closeout.

PART II

The Europeans have a saying that many ideas are good, but the devil is in the details. Having made these points, let's give the devil his turn: let's look at the technical details.

System safety planning and management are only as good as system safety technology will let them be. Stated another way, system safety managers can only deliver what system safety techniques allow them to deliver, despite what they may promise. How many of you have seen your high hopes dashed by "poor work?"

A safety manager can put out the most comprehensive, grandiose system safety plan (SSP) but if the supporting concepts, principles and techniques cannot fulfill the plan, the plan creates more problems than it solves.

For example, if a safety manager or analyst promises a safety analysis to find and control the hazards in a system and then undetected serious accidents start to occur because an inappropriate concept, principle or analytical technique was used, everybody has problems. The system manager feels deceived; the safety manager feels frustrated; the analyst is viewed as inadequate; and the system safety approach itself becomes suspect in the eyes of the managers and possibly the public. One of the authors considers this the underlying problem with numerous "safety" projects in many fields; WASH 1400 is one widely known example of such problems.

Just as one would not use a hacksaw for eye surgery, so one should not establish a system safety plan based on nonsafety concepts, principles and techniques. If you accept this, what constitutes an adequate system safety methodology, or body of working concepts, principles and techniques?

CONCEPTS, PRINCIPLES AND TECHNIQUES

The first aspect that needs discussion is the group of concepts on which the practices of system safety are rounded. What are the concepts that set system safety apart from other disciplines?

These concepts might include the following:

Safe first time.
Achievement of adequate safety levels requires concrete plans
Consider life cycle safety.
Close the loop between safety prediction and safety performance.
Selection of the desired safety level should be conscious decision.
Use proof tests to demonstrate adequacy of safety measures.
Accident investigation should produce durable system improvements.
System safety considers interrelationships among man, machine, and environment.
Safety control measures have a rational order of precedence.
Hazards are less expensive to correct in the design stage of a system life cycle than later stages.
System safety facilitates rather than impedes organizations' achievements.
Documentation of safety analyses and decisions is necessary in a safety program to discipline and record predictive speculations.
System safety requires systematic safety analysis methods used by professional personnel.

How these concepts differ from other disciplines is a genuine issue for the system safety professional. For example, what distinguishes the system safety methodology from reliability methodology, common sense, engineering methodology, legal methodology, etc. Up to six differing methodologies have been observed in accident investigations, for example. How can we tell which if any of these was a genuine system safety methodology?

CRITERIA FOR SYSTEM SAFETY TECHNIQUES

To implement these concepts, system safety applies system safety techniques. How do we distinguish between system safety techniques and other techniques unsuited for system safety analysis? What are the criteria for calling a technique a "SYSTEM SAFETY" technique? What makes SYSTEM SAFETY techniques different from other techniques?

No distinguishing criteria are published. Without criteria, how can we recognize a genuine system safety technique if we want to use one, or ask someone else to use one?

Some possible criteria for identifying a system safety technique might be:

A. Reflects General Systems concepts.

1. incorporates General Systems model input, operation, output and feedback principles and procedures

2. closes loop between safety performance prediction and verification in service

3. structures speculation, data search, organization, logic and other testing, and analysis

B. satisfies safety discipline rather than other disciplines

1. originated or refined by persons attacking

2. satisfies safety analysis, prediction, verification and program needs

3. is applicable over all life cycle phases of a system

C. Permits study of the distinctive body of knowledge, concepts and principles on which system safety is based

1. Utilizes comprehensive and consistent body of SAFETY concepts and principles

2. use is distinctive from uses in other disciplines

3. requires professionals trained in the concepts, principles and techniques to produce acceptable SAFETY results.

4. requires continuing study of practicing professionals to maintain currency with the state of the art

D. Provides a basis for establishing safety plans and programs that incorporate accountability for results achieved

1. techniques are actually employed under a specific System Safety Plan

2. techniques lead to documented safety analyses, decisions, and follow-up to assure their effectiveness

3. techniques provide a way of monitoring ongoing operations as well as accidents to assess safety performance levels being attained

These criteria are necessary' to determine what techniques qualify as system safety techniques, rather than engineering techniques. Sometimes, a technique can be used for more than one purpose; when is it used as a system safety technique, and what uses disqualify it from being considered a system safety technique? Is it really possible to adapt techniques from other disciplines to the safety discipline, with or without modification? For example, one author maintains that efforts to use trend analysis and statistical probability techniques with historical "accident" data are incompatible with the safe first time principle because they must wait for sufficient accidents to support statistically valid conclusions before corrective action is taken. If this is true, what implications does this have for the use of accident data in system safety work?

Are some of the analysis techniques really only disguised "checklists", or are they genuine safety analysis tools? Are checklists acceptable safety analysis tools or not?

SYSTEM SAFETY TECHNIQUES FOR DISCUSSION

What are some System Safety techniques that both persons managing safety programs and safety professionals should recognize and understand?

Safety Plan Documentation
Safety Criteria Retrieval and Tabulating method
Simple and complex events modeling
Preliminary Hazard Analysis
Fault or Logic or Event tree analysis
Fault hazard analysis
MuItilinear Events Sequences analysis
System Hazard Analysis
. Subsystem Hazard Analysis
Operating and Support
Hazard Analysis
Time/loss analysis
Risk Analysis and Assessment
Others (suggested by participants?)

What are some analytical techniques that are often represented as system safety techniques but really may not meet the criteria above?

. Management Oversight and Risk Tree
>Technic of Operations Review
The Hartford EMP method
Change Analysis
Technique for human error rate prediction (THERP)
Engineering proof tests
Statistical inference
Trend analysis
Adversary methods
Failure mode and effects analysis
Sneak circuit analysis
Others suggested by participants

Do some of these techniques qualify as system safety techniques at some times? If so when, and in what circumstances?

What are the strengths and weaknesses of any of the above when they are applied? This will be the subject of the next workshop. x

PART III

REFERENCES FOR TECHNIQUES

The System Safety Society's recent Supplemental Statement submitted to the U.S. Department of Labor included references to methods that were useful in supporting the elements of a system safety plan. A copy of that section of the Statement is appended to this paper. The references provide general guidance about techniques for fulfilling a system safety plan. However, in one author's view, many of the references do not fully acknowledge the precautions for applying the techniques to system safety analyses. They do not explicitly address OSHA's hazardous materials problems. More needs to be done in most areas.

Appendix A To White Paper No. 1

Selected System Safety References

EXHIBIT A

REFERENCES RELATING TO SYSTEM SAFETY PROGRAM PLAN

General System Safety References

The following references are suggested for general overviews of system safety concepts and technical discussions.

American Institute of Chemical Engineers, LOSS PREVENTION, through Volume 14, AIChE, New York
Brown, D.B., SYSTEMS ANALYSIS AND DESIGN FOR SAFETY Prentice Hall Englewood Cliffs, NJ, 1976
Grimaldi, J. and Simonds, R.H., SAFETY MANAGEMENT, Richard D. Irwin, 1975
Hammer, W., PRODUCT SAFETY MANAGEMENT AND ENGINEERING, Prentice Hall, Englewood Cliffs, NJ, 1980 >
Johnson, W. THE MANAGEMENT OVERSIGHT AND RISK TREE/MORT, U.S. Atomic Energy Commission SAN 82l2, l973
6. System Safety Society, INTERNATIONAL SYSTEM SAFETY CONFERENCE PROCEEDINGS, (1973, 1975, 1977, 1979; refer to appended indexes)
U.S. Air Force Systems Command, DESIGN HANDBOOK DHl6, SYSTEM SAFETY, 3 April 1979
U.S. Department of Defense, MILITARY STANDARD, SYSTEM SAFETY REQUIREMENTS, MILSTD882A, Washington, D.C., 1977
U.S. Department of the #000080, NAVORD OD 44942, WEAPON SYSTEM SAFETY GUIDELINES HANDBOOK, PARTS 1IV, 15 January 1974
lO. U.S. National Transportation Safety Board, RISK CONCEPTS IN DANGEROUS GOODS TRANSPORTATION, Report No, NTSB STS7 11. NTIS, 1971
U.S. Nuclear Regulatory Commission, FAULT TREE HANDBOOK, NUREGO492, GPO, Washington, D.C., 1981

In addition, the journal of the System Safety Society, HAZARD PREVENTION, contains articles about many of the subjects listed below.

On the following pages, specific references to help develop and implement a hazardous material system safety program plan are presented.

EXHIBIT B

REFERENCES RELATING TO HAZARDOUS MATERIALS SSPP POLICY AND SCOPE

1. Policy and scope

References relating to hazardous materials SSPP policy and scope follow:

a. Objectives

1a1. Starr, C., SOCIAL BENEFIT VERSUS TECHNOLOGICAL RISK, Science 165, 1232, 1238, September 1969

1a2. Driver, E.T., and Benner, L., THE DILEMMA OF UNACCEPTABLE RISK, delivered to American Society of Mechanical Engineers, San Francisco, CA, 1979

1a3. See Hammer, reference B4

b. System covered by SSPP

1b1. See Reference B8, Section 1

1b2. See NAVORD OD 44942, reference B9

c. Deviations from SSPP

1c1. See MILSTD882A reference B8, Section 1.

1c2. 49 CFR 107: DOT exemption procedures

d. System Safety performance standards

e. System Safety program organization

1e1. See MILSTD882A, reference BB

1e2. See Hammer, reference B4

2. Definitions

References relating to hazardous materials System Safety definitions follow:

a. References specific to system safety application:

2a1. See MILSTD882A, reference B8

2a2. Benner, L HAZARDOUS MATERIALS EMERGENCIES, . Society of Fire Service Instructors, Hopkinton, MA, 1978, p70

2a3. See DH16, reference B7

2a4. See NAVORD OD 44942, reference B9

3. General hazardous materials SSPP requirements

References relating to hazardous materials SSPP general requirement follow:

a. General safety considerations

3a1. See Hammer, reference B4

3a2. See NTSB STS711, reference BlO

b. Types of System Safety tasks to be performed

3b1. See Hammer, reference B4

3b2. See MILSTD882A, reference B8

3b3. Refer to System Safety Conference Proceedings, reference B6.

c. MBO applied to System Safety

3c1. See MILSTD882A, reference B8

3c2. Benner, L. FOUR ACCIDENT INVESTIGATIONS GAMES INSTRUCTOR'S MANUAL, Lufred Industries, Inc., Oakton, Va. 1979

d. Relevant System Safety life cycle phases

3d1. See MILSTD882A, reference B8

3d2. See Hammer, reference B4

3d3. See DHl6, reference B7

3d4. See NAVORD OD 44942, reference B9

e. System Safety plan review and update

1e1. See Hammer, reference B4

1e2. See MILSTD882A, reference B8

4. Detailed hazardous materials SSPP requirements

References relating to hazardous materials SSPP detailed requirements and tasks follow:

a. Management of System Safety activities

4a1. See Brown, reference B2

4a2, See Hammer, reference B4

4a3. See MORT, reference B5

4a4. Smith, .et al, TASK ANALYSIS REPORT RELATIVE TO VESSEL COLLISIONS, RAMMINGS AND GROUNDINGS, Report CGD177, U.S. Coast Guard, Washington, D.C., 1976

(Purpose, goals and objectives in logic tree format)

b. Line employees' System Safety responsibilities

4b1. See proceedings, reference B6

4b2. See Loss Prevention, reference BI

4b3. See MORT, reference B5. '

c. Staff employees System Safety responsibilities

4c1. See Hammer, reference B4

4c2. See MORT, reference B5.

d. System Safety performance reviews, 4d1. See Hammer, reference B4

e. Acceptable System Safety analysis methods:

4e1. Preliminary Hazard Analysis:

a. See DH16, reference B7

b. See NAVORD OD 44942, reference B9

c. See also a 26 page report by C.A. Ericson, SYSTEM SAFETY ANALYTICAL TECHNIQUES PRELIMINARY HAZARD ANALYSIS, The Boeing Co., Aerospace Group, Aerospace Systems Div., Kent Facility

4e2. Subsystem Hazard Analysis:

a. See Hammer, reference B4

b. See DHl6, reference B7

4e3. System Hazard Analysis:

a. See NAVORD OD 44942, reference B9

b. See Hammer, reference B

4c4 Operating and Support Hazard Analysis

a. See Hammer, reference B4

b. See NAVORD OD 44942, reference B9

c. See also Dept. of Defense Form 1473, available from Defense Documents Center

4e5.. Logic Trees

a. See NUREG 0492, reference B1 1

b. Prugh, R.W. APPLICATION OF FAULT TREE ANALYSIS, Loss Prevention, Vol.14, AIChE, New York, 1981 (Fault tree application to chemicals)

c. DeGiovani, J., SAFETY OPERATIONS IN THE CHEMICAL INDUSTRY, Hercules, Inc., Cumberland, MD, 1975

d. Oliver, J.W. and Kaufman, R.A., LOGIC SYMBOLS APPLIED TO MAINTENANCE, McDonnell Douglas Corp., Long Beach, CA, 1969 (Fault tree logic applied to maintenance)

4e6. Failure Mode and Effects Analysis

a. (This is primarily a reliability analysis method; see Reliability and Maintainability Symposium Proceedings, 19561981. The System Safety Society is a cooperating sponsor for these symposia.)

4e7, Multilinear Events Sequences Analyses (MES)

a. Ferry, T.S., Modern Accident Investigation and Analysis: An Executive Guide, John Wiley & Sons, New York, 1981 (overview)

b. Benner, L., FOUR ACCIDENT INVESTIGATION GAMES, Lufred Industries, Inc., Oakton, VA 22124 (teaches MES)

c. U.S. National Transportation Safety Board, OVERVIEW OF A BULK GASOLINE DELIVERY FIRE AND EXPLOSION, Report NTSB HZM781 (example of charting to show stds/regs effects)

d. U.S. National Transportation Safety Board, SURVIVAL IN HAZARDOUS MATERIALS TRANSPORTATION ACCIDENTS, Report NTSB HZM 794(MES chart of chemical exposure/reactions)

e. System Safety order of precedence

f. See April 8 Society presentation at OSHA hearing, Figure 1.

g. System Safety subsystem analyses guidance

4g1. Hammer discusses, but no hazardous material analysis guidance is published.

h. System Safety safety tests

4h1. See DH16, reference B7

4h2. See NAVORD OD 44942, reference B9

i. System Safety accident investigations

4i1. See references 437 above

4i.2. Benner, L. ACCIDENT INVESTIGATION: A CASE FOR NEW PERCEPTIONS AND METHODOLOGIES, Society of Automotive Engineers Special Publication 461, Warrendale, PA, 1980 (details problems, alternatives)

4i3. Benner, L., 5 ACCIDENT PERCEPTIONS:, THEIR IMPLICATIONS FOR ACCIDENT INVESTIGATORS, Hazard, Prevention 16:11, September 1980

4i4. See MORT, reference BS
>

j. Operating procedure reviews

4j1. See Hammer, reference B4

4j2. See DH16, reference B7

k. System Safety emergency response analyses

4k1. Driver, E.T. and Benner, L., EVALUATING EMERGENCY RESPONSE WITH TIME/ LOSS ANALYSES, PATRAM 80, Berlin, Germany, 1980 (uses SS concepts for evaluating hazardous materials responses)

4K2. Benner, L. D.E.C.I.D.E. FOR HAZARDOUS MATERIALS EMERGENCIES, Fifth International Symposium on the Transport of DANgerous Goods by Sea and Inland Waterways, HamBURG, Germany, 1978 (describes methodological approach to analysis)

l. System Safety training

4l1. See university and continuing education program brochures, Exhibit D.

4l 2. See Hammer, reference B4

4l3. See DH16,' reference B7

m. System Safety audit procedures

4m1. E.I. DuPont de Nemours Co., RHYTHM (Remember How You Treat Hazardous Materials) program

4m2. See Hammer, reference B4

n. System Safety personnel qualifications

4n1. See proceedings, reference B6 for papers relating to these criteria

o. Hazmat regulations and standards

4o1. U.S. Code of Federal Regulations, 49 CFR 171173 (Defines Hazardous Materials; these are not to be considered "system safety standards.)

4o2. NFPA safety standards (cover many hazardous material workplace safety issues, although they would not be considered "system safety" work products.)

p. Periodic System Safety performance reports

4p1. See MILSTD882A, reference B8 9