The 64 000 byte question1 April 2000
The main plant computer at Hinkley Point B had operated reliably for more than 25 years, but obsolescence was becoming a real problem. British Energy had to find a suitable replacement – and planned to complete the changeover while the plant was operating at full load.
The M2140 computer system was installed at the Hinkley Point B AGR, on England’s west coast, when it started up back in 1976. Designed in the late 1960s it was, by necessity, a highly compact system, fulfilling all its monitoring and control functions using a mere 64kbytes of memory.
In 2000, 64kbytes is a laughably small system – after all, using Word to write this article takes at least 4Mbytes. But M2140 had been very successful. It was still operating at more than 99% availability well into the 1990s, testifies Hinkley Point’s Peter Caldwell, and operators and management were still satisfied with the system. But no matter what strengths the M2140 had, it was still a kilobyte system operating in a gigabyte world. Obsolete hardware was becoming more and more of a problem and the shortcomings of operating a system that still relied on punched paper tape were obvious. Obtaining spare parts was becoming more like a scavenging operation, and within the Hinkley Point B operation it was becoming more difficult to keep up the required level of expertise on the system as older members of staff moved on.
THE REPLACEMENT STORY
Preliminary investigation into a replacement system for M2140 began in 1991, when Nuclear Electric, then the owner of all the UK’s AGRs, placed a contract with Ferranti to develop new generic control and monitoring software for the group (referred to as the Ada Core Technology – ACT – contract). A further contract to produce a system-specific version for Hinkley Point B was placed in November 1993, but a month later Ferranti went into liquidation. Although work on ACT continued, it was not until 1995 that the future of Ferranti was settled. The industrial control parts of the business were bought by Syseca, a subsidiary of Thomson CSF, and it was with Syseca that the contract for the new Hinkley Point B system was finally placed in 1995.
Syseca’s contract to design and supply the new system weighed in at around £15 million, but that was only part of the story. British Energy spent at least £10 million more, in developing databases to back up the new system, and carrying out associated refurbishment work, for example on in/out (I/O) systems and uninterruptible power supplies (UPS).
British Energy originally planned to replace the I/O data acquisition equipment (commonly referred to as the ‘scanners’) along with the M2140 but when Ferranti went into receivership the I/O part of the replacement was split into a separate contract, placed with Instem. The new I/O system was installed at Hinkley Point B’s reactor 3 in 1995. (Reactors 1 and 2 at the site are Magnox units operated by BNFL Magnox; British Energy’s AGRs are R3 and R4.) The equipment ultimately planned for R4 was delivered to Syseca, where it was used in developing the M2140 replacement software. The UPS for the new system was installed in the early 1990s. This rotary stabiliser-based system with battery backup was supplied by Anton Pillar of Germany.
British Energy’s commitment in terms of manpower was also considerable: 40-50 people worked for four years to develop the database, for example. Planning, testing and executing the on-line switch to the new system took around eight months.
FUNCTIONS OLD AND NEW
M2140 was based on the GEC M2140 and was commissioned with the station. One system was dedicated to each of the two units at the site, and a third acted as a common standby.
To make the most of the strictly limited memory, M2140’s functions were tightly defined – one reason for the replacement was to provide not only more functionality but the ability to easily alter or add to the software. Its functions were:
• Handling 2560 analogue signals via five analogue scanners.
• Handling 3300 digital signals via eight digital scanners.
• Scaling and other calculations.
• Holding alarm logic.
• Generating and managing alarms.
• Reactor temperature control.
• Archive and post-incident data.
• Producing printed logs.
• Display, via six CRT displays on each reactor operating desk.
Archive data was stored on magnetic tape, but the main method of loading new software was via paper tape. An additional facility for providing M2140 data to users was provided by the Core Data Centre (CDC).
The aim in developing the new system was to retain as far as possible the good features of M2140. Visibly, this meant retaining many of the formats which were familiar to reactor operators, and in developing the system shift operations staff were closely involved in planning the new screen formats and alarm-handling formats. Towards the end of the project one shift operator was permanently assigned to the project: proposed formats for the new screens and alarms were placed in the control room at Hinkley Point B so that all shift operators would have the chance to comment on them. The new system also had to maintain the alarm logic of the M2140, along with the database definition.
Two main parts of the new system provide management and control functions to the operators.
• The digital control system (DCS) provides automatic control of the 37 control rods in the reactor that control the reactor gas output temperature. Each unit has two DCS computers, operating in master/standby roles. As the DCS has its own dedicated scanners, in the event of a total DPS failure the DCS is able to continue performing its controlling functions and also provides limited displays to the operator.
• The data processing system (DPS) provides reactor alarm and data processing functions. The DPS operates entirely separately to the DCS, using input from its own scanners, so failure of the DCS does not affect the functionality of the DPS. Each unit has two DPS computers, operating in master/standby roles.
In addition to these two systems, the unit data concentrator (UDC) fulfils the role of the M2140’s CDC, albeit greatly expanded – the old system was not able to provide trending information, for example. The new system test system (STS) is an identical, off-line, standalone replica of the on-line system, which can either take data from the operating system or from a simple plant simulator. The STS is primarily for off-line software testing. It will allow modifications to the software – new alarms, changes to event logic or general upgrades – to be tested in an agreed modifications procedure.
As part of an ongoing support contract with Syseca a group of engineers dedicated to the Hinkley Point B site has been set up and support is also provided from British Energy’s Barnwood headquarters.
The associated hardware test facility (HTF) is intended to help the maintenance engineers who will have the job of ensuring the new system is in full operation for its intended lifetime of 15 years. The HTF consists of two cubicles containing a replica of each piece of hardware contained in the new system. It will be used firstly as a source of ‘hot spares’ so that repair or replacement incurs minimum delay. Secondly, the HTF provides a simulation of every network connection existing within the system, allowing new additions or potential problems to be tested. Test facilities built-in to the HTF make it possible to test and configure all parts of the system hardware. A similar HTF has also been provided for the new I/O equipment.
The interfaces in the new system are provided by a standard ethernet. Some recabling was required, and the opportunity was taken to use fibre optics between the floors. The use of fibre optics has reduced electrical interference issues.
The system is able to identify faulty signals and provide a warning to the operator by showing them on-screen in a different colour.
A ‘BUMPLESS’ CHANGEOVER
The team at Hinkley Point B intended to complete the changeover to the new system, for both units, while the reactors were running on load. This was an ambitious programme, but as station director Les Francis explains, there are good engineering reasons to take this approach. While the reactor was on load it offered a ‘steady state’, he explains. This gave those managing the changeover a more controlled environment, whereas if the system was changed over while the reactor was shut down it would have to be implemented for the first time during reactor startup.
Work to commission the system began in earnest in December 1998. Once the hardware was in place a number of plant signals were fed into both the M2140 and the new system for the commissioning of the DCS. For the DPS live data were taken from the M2140 via the CDC and converted by computer to look like data from the new I/O. This could be input into the new system and enabled the DPS to be commissioned with ‘live’ plant data.
The systems could then be made available to the operators and test teams in parallel to the M2140. The DCS was commissioned first, in a process that took around 12 weeks. For each subsystem and system in turn, the old system was switched off and isolated. With the permission of the project implementation group the new version of the system or subsystem was then switched in and tested, before the old system was brought back on line. Over the 12 weeks the group worked through all the functions, moving from simple tests to more complicated functions at higher power levels.
Meanwhile the DPS was being commissioned, with engineers systematically checking the functions of the system. Formats were tested and compared with the M2140, for example, and alarms and event logic were checked. This system, too, was run in parallel during a three week testing phase, during which forced generation of all key alarms allowed them to be examined.
The UK regulatory authority, the Nuclear Installations Inspectorate, was not idle while commissioning work was going on but Hinkley Point used a pro-active ‘self regulating’ approach. The project implementation group had developed a structured programme for the project, so that it had close control over when commissioning work moved to the next stage. A series of ‘hold points’ was established, for example before the first control loop was switched to the new system. The project implementation group kept the NII informed of progress and of the conditions under which the project moved forward. Working together over the whole of the project lifetime meant the team was familiar with the regulator’s requirements, and in practice although the NII could have imposed hold points they never exercised this option.
The long testing phase – work specifically on the control functions lasted from March to July 1999 – eventually made it possible to change the systems over in a single day. Working in stages the inputs were disconnected from the M2140 and reconnected to the new system. The transfer was completed successfully for reactor 3 on 24 July, and reactor 4 on 1 August.
It was during the testing phase that the shift operators had their first chance to use the new system and find out whether the displays that they had seen in mockup were acceptable in practice. The M2140 had six displays on each reactor desk. These were retained during commissioning and they were joined by a display from the new system which enabled the operators to use and compare functions of the new system. The operator was able to switch control between the old and new systems as required by the tests.
Involving operators from an early stage has paid off and they have responded well to the new system. To maintain a high level of expertise the Hinkley Point B simulator has been refurbished to incorporate the changes. As the simulator was refurbished before the changeover took place at Hinkley Point B the operators were trained on the new system using the simulator, which also offered early possibilities for feedback, and it was used to ‘dry run’ some aspects of the commissioning procedure.
The ability to produce trending data easily, and to provide more detailed information to operators and increased flexibility will all contribute to improving the economic performance of the station. That economic performance is now more important is illustrated by the information available to shift operators in the control room. Key alarms for safety and for economic performance are highlighted and can be interrogated to provide more detailed information.
SETTING THE EXAMPLE
When work first began on the new computer system for Hinkley Point B at the start of the 1990s the intention was to develop a system that could be adapted for all the other AGRs. Where does the implementation at Hinkley Point B leave the other plants?
Ada Core Technologies is available for the other plants and indeed one other AGR, at Dungeness, has plans for I&C replacement in the short term. At that plant British Energy intends to employ a ‘partnership’ with its supplier, and tenders were invited for the new system in August 1999.