Regulatory acceptance of software: working towards consensus

The use of software-based systems in safety-important roles on nuclear power plants is a steadily growing trend. Such systems are now almost without exception part of new plant designs and increasingly evident in the replacement of obsolescent systems in older plants. This trend is fuelled by the more sophisticated functionality per unit of cost which the technology affords, with the economic attractiveness of achieving higher plant outputs while preserving (even, arguably, enhancing) plant safety. However, until there emerges an acceptable technique for analysing the integrity of computer software (eg by numerical reliability methods), the regulatory judgement of fitness-for-purpose must be based principally on qualitative demonstration elements.

It would be of obvious value to both supplier and user if international regulatory agreement could be established as to the constituents of an acceptable safety-case demonstration for a computer system in a safety-important role on a nuclear plant.

Since about 1992, representatives of the nuclear regulatory authorities from Canada (Atomic Energy Control Board), France (Direction de la Sûreté des Installations Nucléaires / Institut de Protection et de Sûreté Nucléaire), United Kingdom (Nuclear Installations Inspectorate) and United States (Nuclear Regulatory Commission) have now met about 10 times to compare regulatory approaches, experiences and lessons in this area. This gathering presented an ideal opportunity for capitalising upon this knowledge by attempting to identify a common denominator of safety case elements which all could agree should be sought. It was recognised, of course, that this basic set could well be supplemented by the levying of additional requirements specific to individual countries. This work has now been completed and a report published.

A COMMON APPROACH

The basic requirement for a rigorously documented safety justification for a system of this kind on a nuclear power plant is apparent in all the countries, although the mechanics of its elicitation and processing differ between the separate regulatory regimes. Nevertheless, a common underlying pattern of their approaches can be discerned.

As a starting point each regulator, with reference to a licensee’s proposal, focuses on a specific set of design and operational safety principles, standards and/or criteria applicable to the type of system and the type of technology. From this the basis for acceptance is determined, after which the licensee/applicant must then provide a corresponding documented demonstration (the safety case) for the system to be licensed. It is this demonstration which the regulator reviews. The regulatory review in all countries recognises the fundamental importance of the evidence provided by a deterministic case. The added value of the evidence from a probabilistic analysis (where achievable) is also generally recognised, but the weight given to this varies from country to country.

The Report provides a concise description of the conclusions reached. The body of the consensus (Sections 4, 5 and 6) is preceded by clarifications of key terms and concepts associated with the topics (Section 2) and by a discussion of the importance that standards play in the development of systems important to safety (Section 3). The standards relevant to each country are identified more fully in an appendix (B).

The term “integrity” (rather than “reliability”) is used throughout the report to reflect the importance that is placed on the qualitative attributes of a computer-based system, as demonstrated by the total evidence assembled within the safety case. Integrity is defined as the quality of completeness, dependability and freedom from defects. The terminology in relation to “safety” and “safety-related” systems, as set out by the International Atomic Energy Agency (IAEA), is adopted in the report as the basis for distinguishing relative safety importance. Thus, the role of “safety system”, as of the highest safety importance, implies the need for a confident demonstration of (normally) high integrity performance. However, while the emphasis of the report is on safety systems, there is inclusion also of the commensurate demonstration requirements for safety-related systems, as appropriate to establish confidence in the dependable deliveries of their assigned safety functions.

In Section 4 specific attention is given to those aspects which, from experience, need to be considered, and approaches agreed, at the earliest stage of the project. These cover:

• The determination of the system’s safety importance.

• The extent to which there are claimable defence-in-depth provisions.

• Clarification of the proposed system’s boundaries and interactions.

• The identification of novel features of design and/or technology.

• The establishment of an effective licensee/regulator interface.

• The extent and form of the independent assessment. And,

• The applicability, or relevance, of demonstration evidence associated with a previous system. Hence,

• The overall basis of the safety case.

In Section 5 the report identifies fourteen general demonstration areas which are considered applicable to all systems important to safety; nevertheless for systems with higher safety importance – eg safety systems – a commensurately increased rigour of application should be evident. Aspects covered include: demonstrating the correctness of the requirements specification and of its development through to a design solution; fail-safety; system testing during development; operational testability; standards; competencies of staff and team organisation; QA; security; a controlled change process; document management; consistency with assumptions in the safety case; and the design of the human/system interfaces.

Also in Section 5, a number of additional demonstration elements are presented which specifically target safety systems. These address such matters as: the single failure criterion; common cause failures; the application of a structured development process and appropriate design standards; complete verification and validation using both testing and static analysis; and the use of valid and controlled tools.

In Section 6 there is discussion of a number of associated topics considered to be of importance:

• Independent assessment – its need, and how conducted, in terms of planning, extent, and organisation.

• The need for defence-in-depth.

• The regulatory approach to the licensing of commercial off-the-shelf systems (COTS).

• The value of formal methods. And,

• The procurement and feedback operation and performance data.

Background information is provided specifically in Appendix A to enable an appreciation to be gained of the regulatory systems applying in the four countries, and of the approaches to the computer systems’ topic which have emerged. Descriptions are given of the legal basis of the regulatory framework applying in each of the four countries, and also the regulator’s expectations of the licensee in the context of software-based safety systems. Wherever possible, information in Appendix A has been linked back to the paragraphs of the main text of the consensus report. This degree of visibility allows a better appreciation of the depth of consensus that has been achieved, and emphasises the similarities that exist between the four countries.

In summary, while the report should not be regarded as official regulatory guidance, it nevertheless does serve to identify the basic set of commonly agreed demonstration elements, compatible with the individual countries’ regulatory requirements, which all four regulatory bodies would expect to see addressed in a safety case for a software-based system intended for safety-important usage on a nuclear plant.