Defence in depth in nuclear safety30 January 2013
The concept defence-in-depth has been used in nuclear safety for over 50 years, even though a technical consensus on what it means in the context of specific designs has not been achieved. This paper offers ideas that can serve as the basis for a discussion within the nuclear technical community, with the aim of helping foster development of an international consensus standard to provide a method to answer the question "How much defence-in-depth is adequate?" By N. Prasad Kadambi
Several studies performed after the Fukushima accident have identified strengthening of defence-in-depth as one of the areas that ought to be pursued on a priority basis. However, just as happened with the accidents at Three Mile Island and Chernobyl, focus tends to be placed on design and operational attributes directly related to the events and sequences during the accident scenarios rather than more generic implications that may offer more widespread improvements and benefits. For example, for quite a while, there was unnecessary focus on spent fuel pool safety at the expense of other matters. The efforts toward a consensus standard are likely to take into consideration a wider perspective than just hardware or software issues, to include operational and organizational issues.
There is an intuitive sense in which defence-in-depth is felt to broaden and deepen safety in a way that differs from the considerations associated with performing the engineering activities required by basic reactor systems design. Basic reactor safety is generally understood to be achieved by ensuring that structures, systems, and components (SSCs) are provided in the design having the requisite performance and quality characteristics to assure that stated limits are not exceeded for a set of conditions that includes postulated events and accidents. In practice acceptance criteria associated with inspections, tests, and analyses are set such that robust safety margins exist by adopting conservative decisionmaking approaches for physical parameters like pressure and temperature. Defence-in-depth is an additional framework for safety beyond that provided by traditional engineering design methods.
At Fukushima the available information appears to show that traditional design methods were quite successful because engineered systems fared remarkably well even when challenged considerably beyond the design basis. However, the rudimentary application of defence-in-depth concepts, such as by application of the single-failure criterion (SFC), were shown to be inadequate because the tsunami became an overwhelming common cause failure mechanism. The limitations of the SFC have been recognized for a long time (they lay behind regulations against station blackout, anticipated-transient-without-scram, auxiliary feedwater systems design for some reactors) but a systematic implementation of application improvements has not happened. Hence there is justification to consider a broader and deeper application of defence-in-depth concepts so as to cope better with the most severe challenges.
Application of the SFC is emblematic of a lack of precision in the understanding of all the different aspects of measures that are taken in the name of defence-in-depth. What is lacking is a way to assess how these measures do or do not contribute to safety. SFC is the basic method for arriving at redundancy and diversity aspects of a design. It should be sufficiently important as to be applied consistently and with coherent logic. Yet the SFC is sometimes not applied in a manner that is commensurate with the safety significance of the system. Safety-insignificant single-failure event sequences are sometimes included in a plant's design basis, while some safety-significant multiple-failure sequences are not included. A broader defence-in-depth framework ought to enable safety assessments that consider safety margins and defence-in-depth features in a holistic manner to better understand whether adequate safety is being achieved.
This paper explores ideas that are felt to be important in the advancement of safety through activities that are undertaken to promote defence-in-depth. It offers ideas that could be useful in obtaining a more objective measure of defence-in-depth, even if the attainment of objectivity may not include full-scale quantification. Additionally, in the context of international consensus standards that may be considered by national regulatory authorities, suggestions are offered regarding meeting regulatory requirements through invocation of such standards. Risk-informed and performance-based approaches to assess the adequacy of defence-in-depth features are proposed to emphasize the importance of safety decisionmaking that meets practical constraints. It is hoped that a discussion within the nuclear technical community will ensue from which tangible improvements to defence-in-depth can emerge more quickly with implementation of international consensus standards.
Defence-in-depth has eluded precise definition, and it is possible that the focus on defining it has detracted from improving safety. If, instead of a definition, the focus shifts to envisioning the performance expectations from the application of the defence-in-depth concept, it may be possible to obtain more immediate safety improvements.
The US Nuclear Regulatory Commission has had a definition in place since about 1998 , which has gone through some minor evolutionary changes, but is now incorporated in the agency's Strategic Plan  as part of the Glossary. It reads:
- An element of the NRC's safety philosophy that employs successive compensatory measures to prevent accidents or lessen the effects of damage if a malfunction or accident occurs at a nuclear facility. The NRC's safety philosophy ensures that the public is adequately protected and that emergency plans surrounding a nuclear facility are well conceived and will work. Moreover, the philosophy ensures that safety will not be wholly dependent on any single element of the design, construction, maintenance, or operation of a nuclear facility.
This broad definition has been refined multiple times in the context of different applications. The characteristics of defence-in-depth have been envisioned in terms of multiple barriers, levels of defence, levels of protection, successive compensatory measures, lines of protection, multiple measures, protective barriers, echelons of defence, etc. Levels of defence have also been viewed as an approach to address accident prevention and mitigation.
The variety of formulations of defence-in-depth was addressed by an internal USNRC group that examined the near term implications of the Fukushima accident . It found that defence-in-depth is not susceptible to a rigid definition because it is a philosophy. However, and perhaps because it is a philosophy, they found that the concept as applied has much in common with a quite different construct envisioned by the International Atomic Energy Agency . The logic for finding commonality lies in envisioning the concept at multiple levels. A philosophical approach represents the highest level, and deconstructing it to lower levels may present a quite different appearance even as logical consistency is maintained.
The IAEA formulation is a much more structured definition, comprising five different levels of defence. These levels correspond to the following:
- Prevent deviations from normal operation
- Detect and control deviations
- Incorporate safety features, safety systems and procedures to prevent core damage
- Mitigate the consequences of accidents
- Mitigate radiological consequences
In the IAEA formulation, defence-in-depth is to be applied to all safety-related activities, whether organizational, behavioural or design-related, and whether in full power, low power or various shutdown states. This is to ensure that all safety-related activities are subject to independent layers of provisions, so that if a failure were to occur, it would be detected and compensated for or corrected by appropriate measures.
If defence-in-depth is conceived as applying SFC in all the activities addressed above, none of the regulatory systems in place today would likely measure up. Imposing redundancy in such an indiscriminate manner would undoubtedly represent a very significant increase in the costs of implementing nuclear technology. Besides there could be significant detrimental effects on safety by increasing the complexity of systems and practices, giving rise to unintended and unpredictable failure modes.
The economic viability of nuclear technology need not be harmed if a more graduated response is considered for the need to improve defence-in-depth. A practical way to achieve such improvements may be based on technical experts reaching consensus on how systems associated with each level are set up and monitored such that their availability and reliability measure up to safety needs. Such judgments invariably require consideration of qualitative as well as quantitative information applied in a formally structured decisionmaking process. A performance-based approach is one where such a structure and process are clearly spelled out so that accomplishment of objectives can be objectively verified. Hence, it may be better to define the levels of protection provided by defence-in-depth in such a way that there is flexibility to achieve the functional objectives for each level in a performance-based manner with sufficient assurance that the overall risk objectives are achieved.
A consensus standard on defence-in-depth would define one or more structured set(s) of functional objectives to be achieved by the safety-related and other SSCs involved in providing defence-in-depth. IAEA has a perspective in which safety is achieved by assuring three fundamental safety functions (reactivity control, core heat removal, radioactivity confinement) along with defence-in-depth. The most significant contribution to the design process from postulating design basis events and achieving a level of 'adequate protection' should be assurance that these safety functions are fulfilled to a high level of confidence. Clearly this is the realm where cost considerations hold no sway. Drawing distinctions between this domain and the design features, controls and organizational factors that are potentially subject to cost-effectiveness testing is difficult with the unstructured representation of only reactivity control, core heat removal and radioactivity confinement.
In the US, the existing approach for regulatory decisionmaking incorporates technical requirements related to the licensing, operation, and maintenance of commercial nuclear power plants along with a historical commitment to the defence-in-depth philosophy that ensures that the design basis includes multiple layers of defence. The adequacy of the design basis is intimately related to the concepts of design basis events and 'adequate protection,' which is a term of art within the US safety construct. Design-basis events are formulated with the idea of requiring safety systems to address a prescribed set of anticipated operational occurrences and postulated accidents. In addition, the design-basis requirements for nuclear power plants incorporate a set of external challenges including seismic activity and flooding from various sources. The concept of design-basis events has been equated to 'adequate protection,' and the concept of beyond-design-basis events has been equated to beyond 'adequate protection' (that is, safety enhancements).
As part of developing the agency's reactor oversight process (ROP), the USNRC prepared a structured set of functional objectives that collectively fulfill its safety mission . Accomplishment of the regulatory mission is seen to involve assuring that seven safety cornerstones be adequately fulfilled. These safety cornerstones are:
- Initiating events
- Mitigating systems
- Barrier integrity
- Emergency preparedness
- Occupational radiation safety
- Public radiation safety
These cornerstones clearly cover the three fundamental safety functions and much more. For example, reactivity control is key for dealing with initiating events. Insofar as design itself is concerned, reactor scram systems have gone beyond the minimum of being able to, say, scram with a stuck rod as the design basis (10 CFR 50, Appendix A, General Design Criteria). The need for defence-in-depth was addressed by ATWS requirements, which are different qualitatively from those that apply to the scram system. All pressurized-water reactors, for example, installed diverse means to trip the turbine and initiate auxiliary feedwater in response to an ATWS. Cost effectiveness was considered relative to requirements of the ATWS systems, but was not explicitly for scram systems to meet the GDCs. Similarly, it should be possible to develop structured objectives that would be consistent with the two perspectives represented by the IAEA view and the ROP view. The IAEA's fundamental safety requirement for radioactivity confinement is clearly addressed by the containment barrier, but the ROP does not currently have a performance indicator for the functional objective of containment. A consensus process could be implemented to separate out those aspects that are tied to SSCs required to perform satisfactorily during postulated design basis events. To a first approximation, the remaining objectives would be related to defense-in-depth.It is useful to make qualitative distinctions between the contributions to safety by the design-basis aspects of a nuclear power plant and those aspects that are primarily focused on defence-in-depth. The design-basis aspects have been dealt with deterministically with clear assessments of safety margins. There is no reason to change that because it has been demonstrated to be successful, and also because tinkering with it would likely lead to regulatory instability. This is where the direct contributions to safety from factors such as conservatism, extra QA, and organizational factors like safety culture have observable impacts. The aggregate effect is to provide substantial safety margins that can be subjected to independent inspection and enforcement. A general definition of safety margin is that it represents the difference between the capability of a system and the expected challenges. The efforts of an international consensus process should be to ensure that there exist a sufficient set of consensus standards to cover conventional design basis performance.
The safety contributions to the design, construction and operation of a nuclear power plant that are made by SSCs and operational procedures that augment deterministic safety in the interest of defence-in-depth can be quite difficult to assess with relatively simplistic tools such as probabilistic safety assessments (PSAs). A valid criticism that has been leveled at the PSA methodology is that there are so many moving parts in constructing a good PSA of a complex system that the analyst can obtain almost any desired answer. The choice of a risk metric can have inordinate significance. Instead of using a PSA as an absolute method for assessing risk, it is much more reliable to use it to make relative comparisons among options. The most effective use of a PSA is as a tool in a decisionmaking process by a body of experts in a consensus process. Defence-in-depth outcomes may not be directly observable in conventional testing protocols, and may have to be extracted through the consensus process. An appraisal of the safety contribution of defence-in-depth measures may be possible only by taking a holistic perspective wherein formal models do not play as much of an essential role as integrated assessments based on observation, experience and judgment.
The USNRC has produced guidance on a methodological complement to the PSA that was meant to be part of the new paradigm represented by a risk-informed and performance-based regulatory approach. The guidance on applying a performance-based approach for making safety decisions  focuses on formally defining and structuring objectives so that observations can be made to assess outcomes against objectives. Performance-based approaches focus primarily on results and are better able to offer integrated assessments. They can improve the objectivity and transparency of regulatory decisionmaking, promote flexibility that can reduce licencee burden, and advance safety by focusing on safety-successful outcomes. The ROP exemplifies a risk-informed and performance-based approach. Decisions regarding defence-in-depth may include proposals to use a prescriptive or a performance-based approach. According to NUREG/BR-0303, a performance-based proposal can be developed using a five-step process involving (1) defining the safety issue and its context, (2) identifying the safety functions, (3) identifying safety margins, (4) selecting performance parameters and criteria, and (5) formulating a performance-based alternative. The safety functions are extracted from a structure of goals and objectives. Safety margins can be assessed for individual objectives or for an aggregation of objectives, constituting an outcome scenario. The performance parameters and criteria may be qualitative or quantitative.
An example of a performance-based alternative may be to apply the above process to each of the five levels of defence-in-depth in the IAEA model. Each level could be subjected to the process to obtain indicators that apply separately in the assessment. Another example using the reactor oversight process of the USNRC could be to apply the process to the constituent parts of the safety cornerstones. Judgments regarding completeness and effectiveness could be made through a consensus process.
Analogies are useful when dealing with hard-to-grasp concepts such as defence-in-depth. Analogies for defence-in-depth can be found in computer science, information technologies and other fields, where strategies have been employed to characterize and assure adequate capability to tolerate or recover from unexpected conditions. In fact, the nuclear industry has long been in discussion with regulatory authorities regarding how to achieve adequacy of diversity and defence-in-depth in digital instrumentation and control systems (DI&C) for reactors. In the US, these discussions have been successfully concluded with the publication of a staff review guidance document . Having the USNRC endorse the principle of employing analogies should make it possible to seek other appropriate places for this approach.
Reference  establishes the principle of employing conceptual analogies in the section dealing with echelons of defence. A safety requirement in the DI&C domain has been promulgated requiring that sufficient information be provided to the operators to monitor (and thereby control) the following plant safety functions and conditions:
- Reactivity control
- Reactor core cooling and heat removal from the primary system
- Reactor coolant system integrity
- Radioactivity control
- Containment conditions
These key safety functions can be viewed as performance objectives that need to be accomplished by suitable means. The requirement also states that they are to be maintained within safe margins for currently operating nuclear power plants. The objectives are to be accomplished by providing adequate:
- Control systems
- Reactor trip system
- Engineered safety features actuation system
- Monitoring and indications
The USNRC's guidance sets out the acceptance criteria for these echelons of defence. The four echelons of defence are conceptual. It does not imply that these echelons of defence must be independent or diverse. The RTS and ESFAS functions may be combined into a single digital platform. Combining echelons of defence into a single software program could introduce new common cause digital system failure mechanisms that do not exist in systems that use separate software programs. Whether or not the RTS and ESFAS functions are combined into a single platform, the digital protection system should be protected against potential common-cause failures. Two design attributes that are sufficient to eliminate consideration of CCF are diversity and testability. Hence, the DI&C review criteria represents a reasonably-objective means for making decisions while considering other factors such as operational flexibility and cost.
The physical features of any major defence-in-depth provision can be represented as analogues of the above functional requirements. For example, analogous to control systems, a functional objective may be served by a non-safety system. Analogous to the RTS and ESFAS, two safety-grade systems can exist on the same platform if specified conditions are met. Analogous to the use of monitoring and indications, manual operations may be permissible if appropriate conditions are met. Establishing the appropriate ways in which to employ such analogies are effectively accomplished in a consensus process that includes international technical experts from the reactor as well as the digital I&C fields.
Assessment of adequacy in a performance-based approach requires formulation of parameters or metrics of performance that directly or indirectly serve as objective indicators of achievement of functional objectives to the desired levels. The metrics that have become associated with the application of PSAs are core damage frequency (reflecting risk of inadequate core cooling) and large release frequency (reflecting risk of failure to confine radioactivity). These are highly aggregated metrics and may not be suitable for use as proxies for assessing defence-in-depth. Aggregation of event sequences is a recognized weak point of PSA methodology. In the context of a performance-based approach, aggregation may introduce unacceptable levels of subjectivity into the data inputs for defence-in-depth features which may not be supported by the type of reliable information used in design basis analyses.
It is much more likely that use of analogies may be more helpful in finding metrics for defence-in-depth. For example, a system-wide characteristic that is closely analogous to the benefits of defence-in-depth is fault tolerance . A fault-tolerant system is one that can continue to correctly perform its specified tasks in the presence of hardware failures and software errors. For example, in the US, the response to the 2001 terrorist attacks included studies and actions which enable bringing nuclear power plants to safe conditions after loss of large areas subsequent to a postulated attack. Achieving fault tolerance involves incorporation into the system of a type of redundancy that is more than mere replication of hardware components. This type of redundancy involves addition of information, resources, or time beyond what is needed for normal system operation . Active hardware redundancy detects faults and performs actions to remove faulty hardware; that is, to reconfigure. Time redundancy uses additional time to perform functions so that fault detection and fault tolerance can be achieved.
Another tool to assess adequacy of defence-in-depth measures is the high-level guidelines for a performance-based approach to safety described in NUREG/BR-0303. These have been further elaborated on in a journal article . The guidelines have to do with identifying performance parameters, establishing objective criteria, and incorporating flexibility in a performance-based approach. Such a 'safety margin' is defined as the difference between two system states, the first of which is the expected state and the other is one in which a safety concern exists, or is an undesirable state for some other reason. If the magnitude of the safety margin is sufficient to support a performance-based approach, it may be possible to subdivide and apportion it in such a way as to consider multiple objectives. This research also describes how, under appropriate conditions, performance measures can be proposed representing margins which can be subdivided within a performance-based approach.
Safety margin can be divided into two parts, physical and temporal. Physical margin is the difference between two physical conditions, the first of which represents expected conditions and the second of which represents a performance-limiting condition. An example of a performance-limiting condition is the peak pressure capability of a pressure vessel. Physical margin in a pressure boundary is the difference between the pressure retaining capability of the vessel and the expected maximum pressure during an accident condition.
A temporal margin (or time redundancy) represents the time available to identify a concern and to take actions, such as restoring a failed safety function, implementing a corrective action programme, or initiating a regulatory response that mitigates the concern. A temporal margin in a spent-fuel pool, for example, could be the difference between the detection of an elevated water temperature and the time to boil.
The idea of finding useful analogies to characterize defence-in-depth can also be applied through the concept of "resilience", borrowed from the field of ecology. Systems engineering principles can be applied to enhance the attribute of resilience of systems so that they can withstand disruptions to a much greater degree. Resilience is the capability of a system with specific characteristics before, during and after a disruption to absorb the disruption, recover to an acceptable level of performance, and sustain that level for an acceptable period of time. In the context of a nuclear power plant, resilience would equate with, among other things, the positive contributions to safety by effective organizational factors, including safety culture.
Thus, the adequacy of defence-in-depth can be considered dynamically rather than statically by employing the concepts of resilient systems. Whereas conventional risk management approaches are based on hindsight and emphasize failure probabilities, the resilient systems approach looks for ways to enhance the ability of organizations to create processes that are robust yet flexible, to monitor and revise risk models, and to use resources proactively in the face of disruptions or ongoing production pressures .
Hence, performance measures that enable rendering judgments regarding adequacy of defence-in-depth can be based on a type of redundancy that is more than a mere replication of hardware components. For example, the USNRC imposes requirements (10 CFR 50.63) regarding coping capabilities in response to station blackout concerns. This clearly reflected an approach that recognized that merely having redundant standby diesel generators was not an adequate way to address the need to have power available for decay heat removal. In response to the Fukushima accident, the USNRC went further. US nuclear power plants have been ordered to incorporate other measures that address the same issue  by planning for the ability to add additional hardware components. Additionally, effectiveness of defence-in-depth provisions may be evaluated by considering the extra time that could become available for mitigating actions. Credit may be accorded for organizational factors by including in the assessment the effectiveness of a nuclear power plant's accident management team to take advantage of the additional time that is made available by successful defence-in-depth activities. An international consensus process sponsored by an appropriate organization may be able to obtain the participation of relevant experts to apply existing research information from a wide range of fields.
It would be appropriate for an international effort to improve defence-in-depth to consider how results of such an effort may find application within various jurisdictions. Every country can be viewed to have a nuclear safety assurance construct that operates within the state's jurisdiction, and which also comes under some form of a global nuclear safety regime. The IAEA has become the face of such a regime after the Fukushima accident because the agency has been playing a leadership role to share useful knowledge and promote information dissemination. If the international activity on defence-in-depth results in one or more consensus standards, IAEA may choose to include it in its articulation of fundamental safety principles. This allows each country to evaluate its nuclear safety assurance construct in light of IAEA high-level requirements.
An issue that needs to be addressed is that consensus standards do not automatically achieve relevance within each jurisdiction, and each authority may need to adopt or endorse such standards. Among the developed countries, the US probably has the most explicit policy regarding giving consideration to consensus standards in lieu of government-issued regulations . The USNRC also has one of the most mature safety management systems among the countries that operate nuclear power plants. As part of constantly examining the effectiveness of the regulatory system, the USNRC constituted a Risk Management Task Force in February 2011 which produced a report in April 2012 . NUREG-2150, which is currently under active consideration by the USNRC staff, calls for a performance-based regulatory system with appropriate controls and oversight. In combination with the NTTF report, the regulatory direction for the USNRC seems pointed toward a more formal and holistic framework using risk-informed and performance-based approaches.
One of the early pieces of research conducted at the USNRC in pursuit of risk-informed and performance-based regulation was a study of the single-failure criterion. The SFC is recognized to be one of the key cornerstones in the application of defence-in-depth concepts. The research resulted in preparation of a technical report . The staff report was provided to the USNRC in a paper seeking Commission direction . The Commission has not provided direction to the staff yet on this report, but clearly it will have relevance to the actions flowing from the NTTF report. Irrespective of the actions taken by the USNRC, this report is of interest to the international technical community. It points out the weaknesses in the implementation of SFC in the US, and by implication, internationally, because the requirements of the IAEA relative to SFC closely track those in the US. Were the international community to pursue a more optimal application of the SFC, and do so in a risk-informed and performance based manner, the USNRC research report provides sufficient guidance by way of choices and options.
While it is easy to perceive the benefits of international harmonization of the varied factors that affect defence-in-depth, certain key challenges must be recognized regarding setting the criteria for decisionmaking. The US vision of 'adequate protection' linked to design basis has typically led to requirements addressing beyond-design-basis concerns only when they were found to be associated with a substantial enhancement in safety and are justified in terms of cost. International consensus regarding safety decisionmaking would be promoted by acceptance of the principle that cost cannot be a factor for consideration on some decisions, but other areas of safety decisionmaking would be based on cost-effectiveness.
Cost-effectiveness is included among the broad principles promulgated by the IAEA. If defence-in-depth is to be applied as broadly as previously mentioned (to all safety-related activities, whether organizational, behavioral or design related, and whether in full power, low power or various shutdown states), it becomes exceedingly difficult to make distinctions to set safety priorities. In some countries cost considerations may not be permitted currently, while in others cost considerations may be ignored as a matter of significance in regulatory decisionmaking. However, in an international setting, procedures and processes for considering cost-effectiveness in safety matters deserves priority consideration as a means for reaching closure on some complex problems. It should be possible to reject some proposals for improving safety if the cost exceeds a threshold that is set based on consensus deliberation. Where cost considerations are given substantial significance, the decisionmaking is frequently supported by quantitative methods such as probabilistic safety assessments (PSA).
PSAs improve the objectivity in making design and operational decisions. Also, they provide a means to test whether defence-in-depth actions, such as the application of SFC, are being done optimally. Without a PSA, it might appear attractive to incorporate design features to address highly improbable scenarios while ignoring the operational complexities and multiple failure events that could actually undermine safety. Hence, there is likely to be considerable benefit from employing a risk-informed and performance based approach to incorporating and assessing the effectiveness of defence-in-depth at nuclear power facilities. However, the role of PSAs in the design, operation and regulation of nuclear safety remains quite varied among the various national safety constructs. Hence, standardization in using PSAs should be part of international consensus-building.
Summary and conclusions
Much has been discussed and written about defence-in-depth over the past fifty years. This paper takes a different perspective on this subject in order to support a proposal to prepare an international consensus standard that facilitates safety decisionmaking in the near term. A key aspect of the proposal is to employ modern safety assessment and decisionmaking methods so as to avoid the limitations of the traditional deterministic approaches. Such methods have come to be known as risk-informed and performance-based methods. Performance-based approaches focus primarily on results. They can improve the objectivity and transparency of regulatory decisionmaking, promote flexibility that can reduce licensee burden, and promote safety by focusing on safety-successful outcomes.
The safety design of nuclear power plants can be considered to be a dynamic combination of safety margins provided by design basis considerations and the resilience of plants to withstand unexpected challenges through defence-in-depth. A consensus standard on defence-in-depth would define one or more structured set(s) of functional objectives to be achieved by the safety related and other SSCs involved in providing defence-in-depth. With a generalized framework that permits accommodation of existing engineering practices with new risk management approaches, a precise definition of defence-in-depth may not be necessary. It may be more important to gain an appreciation of defence-in-depth as part of the global safety culture that learns from accidents such as Fukushima earthquake and tsunami.
N. Prasad Kadambi, Ph. D, P. E., consultant. Kadambi recently retired from the USNRC after 26 years' service.
 USNRC, "White Paper on Risk-Informed and Performance-Based Regulation," Staff Requirements Memorandum Regarding SECY-98-144, March 1, 1999.
 USNRC, "Strategic Plan: Fiscal Years 2008-2013 (Updated)", NUREG-1614, Vol. 5, February 2012
 USNRC, "The Near-Term Task Force Review Of Insights From The Fukushima Dai-ichi Accident" July 2011
 IAEA, "Safety of Nuclear Power Plants: Design" Revision DS-414 to IAEA NS-R-1
 USNRC, "Reactor Oversight Process" NUREG-1649, December 2006
 USNRC "Guidance for Performance-Based Regulation" NUREG/BR-0303, December 2002.
 USNRC, Interim Staff Guidance DI&C-ISG-02, Rev 2, "Diversity and Defence-in-Depth Issues." June 2009.
 DI&C-ISG-02, Diversity and Defense-in-Depth (D3) Revision 2, Interim Staff Guidance on Diversity and Defense-in-Depth Issues, June 5, 2009, http://pbadupws.nrc.gov/docs/ML0915/ML091590268.pdf
 Transactions American Nuclear Society, "Defence-in-Depth as an Analogue of Fault Tolerance", N.P.Kadambi , June 2006
 Barry W. Johnson, Design and Analysis of Fault-Tolerant Digital Systems, Addison-Wesley Publishing Company, 1989
 "Performance-Based (Risk-Informed) Regulation: A Regulatory Perspective", Nuclear Technology, VOL. 149, JAN. 2005
 See www.resilience-engineering.org
 Nuclear Energy Institute, "Diverse and Flexible Coping Strategies (FLEX) Implementation Guide", NEI 12-06, May 2012.
 US Public Law 104-113, "The National Technology Transfer and Advancement Act", March 1996
 USNRC, "A Proposed Risk Management Regulatory Framework,"NUREG-2150, April 2012.
 USNRC, "Technical Work To Support Evaluation Of A Broader Change To The Single-Failure Criterion"
 USNRC, "Risk-Informed and Performance-Based Alternatives to the Single-Failure Criterion," SECY-2005-0138, August 2, 2005.