Abstract
Many markets use a Commercial-Off-The-Shelf (COTS)or product-based approach to engineering in order toreduce project cost, schedule and risk, take advantage ofproduct maturity and secure long-term support. Theproduct-based approach presents challenges for bothproduct developers and project engineers when applied tosafety-related applications. Project engineers are obligedto present evidence of product integrity to support theoverall safety argument. In such cases, the safety integrityrequirements for a product may not be known until asafety analysis of a specific system architecture in itstarget environment is performed. Once determined,evidence of integrity needs to be obtained and presentedto suit the customer requirements and industry standards.Concurrently, product developers need to engineerproducts and assurance evidence to support therequirements of high-integrity markets in the face ofconstant product change and the competing demands ofdifferent markets.
This paper discusses the issues involved in engineeringproducts for use in Supervisory Control and DataAcquisition (SCADA) systems in a diverse range ofapplications, both safety-related and non-safety-related.In particular, we address the issue of how to provide abase level of product assurance that can be used, if itultimately proves necessary, to support system safetycases.
Keywords: Safety-related systems, COTS, SCADA
1 Introduction
Supervisory Control and Data Acquisition (SCADA)systems are a class of control systems used in a variety ofapplications to provide centralised control over ageographically diverse area. SCADA systems aregenerally produced in a COTS-like development model,with base products customised and configured for eachcustomer.
Although they rarely directly control safety-criticalfunctions, such systems may be1 relied on to assist in hazard management and are often vital to themanagement of critical civil infrastructure.
This paper builds on a previous paper (Atchison andGriffiths 2002) and discusses the development andassurance of products for SCADA systems. Section 2provides an overview of SCADA systems and theproducts used to engineer them, and discusses SCADAsystem safety requirements in different applicationdomains. Most companies who supply SCADA systemsdevelop a number of SCADA products, which aresynthesized as required to build systems. Section 3explains why this product-based approach is unavoidablebut also points out assurance issues that arise when oneseeks to develop a product for use in a diverse range ofsystems. Section 4 discusses an approach to the “productproblem”, which does not involve product certification.The approach is outlined in more detail than previouslydiscussed and is based on the combination of strategies inproduct architecture, testing and change management.
2 SCADA Systems and Safety
This section provides an introduction to SCADA systems,outlines the products used to build such systems, anddiscusses the safety integrity requirements that can attachto SCADA systems in different application domains.
2.1 SCADA Systems – An Overview/b>
SCADA systems are generally characterised on the basisof their architecture and control paradigm (Landman2000). A typical SCADA system architecture isillustrated in Figure 1. The rounded rectangles representstandard SCADA system components, the ovals representoptional, additional SCADA components oftenconsidered to form a part of modern SCADA systems.The rectangles represent external components with whichSCADA systems sometimes interface.
In essence, a SCADA system consists of one or moremaster stations located in a control centre, withconnections to many remote terminal units (RTUs) thatare physically connected to plant. Alternatively, thesystem interfaces with third-party external devices(IEDs). The SCADA system enables an operator at thecontrol centre to monitor and control plant that is close tothe RTUs. The master stations are sophisticatedinformation systems, usually implemented on platformssuch as Unix or Windows, while the RTUs are simplercomputing devices built on custom hardware, includingcustom I/O hardware.
Figure 1 - Typical SCADA Architecture
RTUs can be spread over very large distances and areconnected to the master stations by a communicationsinfrastructure. Communications media can includededicated wide-area-networks, radio links or publictelecommunications networks. The physicalcommunications media is typically not considered to bepart of a SCADA system and a characteristic of SCADAis its ability to operate and recover in the context ofunreliable communications.
Controls are issued from the master station and reach theplant via the connecting RTU. Information from the plantis telemetered back to the master station, again via theRTU. SCADA systems are characterised by an open-loopcontrol paradigm, that is, control decisions are usuallymade by an operator based on the state of the plant, asindicated by displayed telemetered data. Operators aregenerally only present at the control centre, and one ofthe main functions the master station performs is toprovide a suitable human machine interface that allowsthe operator to rapidly assess the overall state of the plantand so to make decisions about suitable controls to issue.
Modern SCADA systems often contain features thatextend the control paradigm by providing localprocessing at the RTU and additional automation at themaster station. Support for sophisticated local processingat the RTU can be useful to distribute processing load orto allow continued operation of plant duringcommunications outages with the Master Station.Advanced master station applications such as decisionsupport systems provide advice to the operator aboutcontrols s/he should issue, based on a computer-basedanalysis of the state of the plant. Such support systemsare sometimes permitted to run in semi-automatic modein which case controls are issued directly by the system,with the operator able to intervene if necessary.
There is also a growing trend toward the integration ofbusiness enterprise systems with SCADA systems.Examples of systems to which SCADA systems mayinterface include asset management systems used tooptimise the owner's investment in their asset, andenterprise management applications, where SCADAinformation is used to run a business, for example recordelectricity or gas flows to different customers.
2.2 SCADA Products
Although every SCADA installation is unique, andsystems differ substantially between application domains,there are nonetheless a number of common buildingblocks used in every SCADA system. For this reason,most companies that supply SCADA systems developand maintain a number of SCADA products. The mainproducts are the RTU and the master station. RTU andmaster station products are typically developed separatelyand may, in turn, be comprised of modules that can beacquired separately.
The base products can be configured in a large number ofways by the systems integration activity. Each applicationwill use a different number of RTUs and master stationsto match the distribution of their field equipment andrequirements for load balancing and redundancy. Otherconfiguration parameters include:
1) selection of hardware architecture, includingmodules to connect to field equipment andcommunications infrastructure;
2) mapping of field data and controls to a data andcommunications models in the RTUs and masterstation;
3) customisation of displays and the definition of alarmconditions;
4) production of applications or RTU logic andintegration with other information systems.
The remainder of this section discusses the use ofSCADA systems in some different application domains,and the safety integrity requirements that attach to suchsystems in these different domains.
2.2.1 Rail
In the Rail Industry, uses of SCADA systems include:
1) Traction power distribution and control;
2) Monitoring and control of environmental plant onunderground railways, (e.g. chillers, smokeextraction fans, tunnel ventilation fans, etc.); and
3) An integration mechanism for many diverse systems(e.g. Passenger Address, Passenger InformationDisplay, Closed Circuit Television, miscellaneousstation plant control such as escalators, lifts, lighting,etc.)
The main hazard scenarios associated with traction powerdistribution and control are electrocution of tracksidemaintenance workers, and electrocution of emergencyservices personnel during some sort of emergency in thevicinity of high-voltage power equipment (e.g. derailmentor incursions onto the track). In such scenarios, theSCADA system is often deemed not to be safety-related,because work safety procedures involving physicalisolation and earthing of high voltage equipment arerelied on to protect trackside workers or emergencyservices personnel. Where such procedures are not used,the SCADA system must be used to isolate the relevant section and safety integrity requirements will probablyattach.
In the case of environmental plant control, the SCADAsystem may be used to ventilate tunnels when a train isstranded or to extract smoke in the event of a fire. Thesefunctions generally give rise to safety integrityrequirements. Sometimes such functions can be achievedvia a station-based, local control panel, which means thatthe safety integrity requirements may be avoided orlimited to the RTUs only.
Where SCADA systems are used to integrate otherrailway systems, the safety integrity requirements dependon the operational paradigm for the railway involved.While it is unusual for any particular function to give riseto a safety integrity requirement, it may be that theavailability of an integrated picture of the current statusof a range of railway subsystems is considered critical tothe correct handling of emergency situations in general.As such, system availability becomes a safety integrityrequirement.
In the rail industry, there is a general awareness of thepotential safety implications of such systems. This isreflected in the fact that many contracts for the supply ofSCADA systems will explicitly mandate a target safetyintegrity level (SIL) for the system (SIL 1 or 2 is typical).
2.2.2 Oil and Gas
Typical functions of SCADA systems in the oil and gasindustry include:
1) Product flow measurement and control;
2) Product quality monitoring;
3) Monitoring of pipeline control equipment, such asgas compressor and valves; and
4) Integration with business systems to managepurchase and sales transactions.
SCADA systems are generally not considered safety-related in the oil and gas industry since independentsafety and emergency shutdown systems will managepipeline hazards. Nevertheless, SCADA systems canprovide an early indication of emerging hazardoussituations, such as overpressure or pipeline leakage, andis often used as the first means of hazard control.
Triggering of an emergency pipeline shutdown is a costlyexercise and can lead to other indirect hazards stemmingfrom the loss of energy supply. The SCADA system alsoprovides an essential service in the early diagnosis ofpipeline equipment failures, for example failure of a keycompressor, so is a critical factor in the overall pipelineavailability.
2.2.3 Power Distribution
SCADA systems are used extensively to manage ourelectricity supply. Key functions include:
1) Monitoring of substation current and voltage levels;
2) Monitoring of substation equipment and automaticrecovery from some substation failure conditions;
3) Manual control of power supply;
4) Automatic regulation of transformers to achieveconstant voltage under variable load conditions;
5) Isolation of substation equipment for maintenance;
6) Indication of personnel presence on site and alarmingof emergency switch triggering; and
7) Load measurement, trending and forecasting.
Although a number of these functions are safety-related,SCADA is rarely used as the sole means for achievingthem. For example, substation maintenance personnel aretrained to physically isolate equipment beforecommencing maintenance. Despite not having directresponsibility for safety functions, it is possible that aSCADA fault or misuse, followed by a period of SCADAunavailability, could lead to insufficient power supply orloss of power.
2.2.4 Water Supply and Waste WaterTreatment
Large towns and cities use SCADA systems to managethe supply of water and treatment of waste water. Whilethe systems are different, many of the functions aresimilar and include:
1) Measurement of water reservoir and waste waterstorage tank levels;
2) Remote water flow control, including overflow ofwaste water in storm conditions and supply ofdrinking water in high demand periods;
3) Isolation of valves for pipe maintenance;
4) Monitoring of pumping station equipment and powersupplies;
5) Isolation of pumping station equipment formaintenance; and
6) Monitoring of personnel presence in pumpingstations.
Water supply and waste water utilities can generally bemanaged through manual equipment control, indeed notall parts of a utility need be automated. In some instances,large pumping stations are augmented by local closed-loop control systems that monitor and manage waterflow. The use of SCADA for large utilities allows for acentralised view of the utility and a more effective andcheaper management response.
Inappropriate operation of SCADA can lead to overflowof waste water in civil areas or to shortage of watersupply. It can increase the likelihood of burst pipes bycausing fluctuations in pipe pressure, or the stagnation ofdrinking water in reservoirs or pipes. SCADA is alsoessential in the detection of utility equipment failures andthe subsequent coordination of a management response.
2.2.5 Summary
Across different industries we see a broad range of safetyintegrity requirements that may attach to SCADA systems. In most application domains, SCADA is notconsidered safety-related due to the presence ofmechanical, procedural or independent electronicmitigations. Nevertheless, SCADA is often used as theinitial detection and response mechanism for hazardoussituations. Its advantage in providing centralisedsupervision and control of a complex system is importantand unreliability or unavailability of the SCADA systemmay increase the overall risk of failed hazardmanagement.
As our civil infrastructure grows more complex, SCADAsystems are an integral factor in their effectivemanagement. As a result, system availability is oftenbusiness critical, and system unavailability can haveindirect safety implications. As such, many contracts forthe supply of SCADA systems include clauses mandatingquantitative reliability and availability targets.
Few application industries are as mature as the RailIndustry in terms of adopting a risk-based approach tosafety integrity requirement determination (CENELEC2001). However, increasingly, contracts involve penaltyclauses for failure to meet target availability levels. Giventhat SCADA systems are heavily software-based, theissues associated with attempting to demonstratecompliance with quantitative reliability and availabilitytargets can be similar to those faced in engineeringsoftware-based systems to an appropriate safety integritylevel.
3 The Product Approach
As already mentioned most companies that supplySCADA systems maintain a number of SCADA products.Individual systems are engineered using these products.This section explains why there is no practical alternativeto the product-based development model, and then goeson to discuss assurance problems associated with thismodel. A number of possible solutions are considered,but each is shown to have one or more obstacles toovercome.
3.1 Why a Product Approach?
Despite discussion in some forums about assurancepitfalls that may be associated with using COTS and opensystems to implement safety-related functions, there isnevertheless a seemingly irreversible trend towards theuse of such systems. In the Rail Industry this trend isevident despite an awareness of the safety assuranceissues. In industries that do not acknowledge any safetyintegrity requirements for SCADA systems, the use ofCOTS products and open systems/standards is standardpractice.
The reasons why purchasers prefer COTS products arewell-documented (Lindsay and Smith 2000). In brief,using COTS can reduce development risk and canincrease reliability due to product maturity. It is verydifficult for a solution provider seeking to develop asystem “from scratch” to compete with a solutionprovider who offers a system composed of pre-existingcomponents. The lack of competitiveness exists onseveral levels, including price, and market credibility.
There is another reason why purchasers of SCADAproducts tend to prefer COTS. Most purchasers have asignificant investment in some sort of asset (e.g. powerdistribution plant). Such assets are maintained over a longperiod of time and, since the SCADA system assists inmanagement of that asset, customers require a SCADAsolution that will continue to be supported long-term.Over time, just as the purchaser will want to upgrade theirplant, so they will look to upgrade their SCADA system.As such, purchasing a system whose component productshave large user-bases means that, because a product witha large user base is likely to be upgraded, so it is likelythe purchaser will be able to upgrade their SCADAsystem by purchasing component product upgrades. Forthis reason, most purchasers would prefer a product witha broad user-base, even if many of the installations arenon-safety-related, than a product with a small user-basewhich, although it specifically targets the high-integritymarket, is nevertheless at greater risk of beingdiscontinued.
In practice, this means that most contracts for the supplyof large SCADA systems are won by companies whopossess (a suite of) SCADA products, that implementopen standards communications protocols. The businessand market forces are such that there is virtually noalternative to a product-based approach to the provisionof SCADA systems.
3.2 Product Model Assurance Issues
If a SCADA system is determined to be a safety-relatedsystem, a safety case will need to be developed for thatsystem. If the SCADA system is composed from anumber of standard SCADA products, it will beincumbent on the supplier to show that the SCADAproducts are fit for the purpose for which they areintended. Typically, this will mean showing that aproduct correctly implements, to a suitable level ofintegrity, a number of functions that are safety-related inthat system context. In this section, we consider a numberof strategies that can be used to show that a componentimplements a safety requirement to a suitable integritylevel. For each strategy, we explain why problems canarise in the context of the product model.
There are a number of COTS assurance strategies that canbe employed (O'Halloran 1999). We discuss how some ofthese strategies relate to SCADA.
3.2.1 High Quality Development Process
Many modern safety standards enable one to use evidenceof a rigorous development process to support productintegrity claims. Most companies developing SCADAproducts have generally developed them over a number ofyears and in response to an increasingly diverse set ofrequirements, including safety integrity requirements.During this period, the notion of best practice has evolvedand development practices that were considered state-of-the-art 10 years ago would probably not produce evidencesufficient to support a modern safety case. As such, safetycases based exclusively on evidence associated with theproduct development process are difficult to support.
3.2.2 Operational Evidence
Claims of product integrity can sometimes be made givensufficient evidence of use in operation. However, mostSCADA products evolve over time, with incrementalreleases, so it is difficult to build up sufficient operationaltime with each release. Furthermore, such products areconfigured differently in different systems and in anycase exhibit a different operational profile so “proven-in-use” arguments are difficult to sustain. Some industriesalso display a parochialism that sees them reluctant toaccept evidence of use in another industry as anyguarantee for correct operation in the applicationindustry.
3.2.3 Extensive Testing
Despite the bias in safety standards towards anassessment of the overall development process, manypurchasers still consider evidence of extensive validationtesting to be the best guarantee of correct performance.Extensive release testing is time-consuming, particularlybecause modern SCADA products can be extremelyfeature-rich. Moreover, SCADA products are also highlyconfigurable, which adds another dimension to the teststate space and compounds the problem further. Thismeans that test-based assurance of correct operation of allproduct features in all system configurations will not bepersuasive in most modern safety cases.
3.2.4 Safe Design Techniques
A common technique used when designing systems thatperform many functions, only some of which are critical,is to partition the system so that critical functions areimplemented in a safety kernel, and the remainingfunctions are implemented elsewhere. Safety assurancefor the non-critical functions is limited to showing thatmalfunction can not adversely impact correct andcontinuing operation of the critical functions. This designtechnique is not straightforward to apply in the case of aSCADA product, where the product must be suitable foruse in a variety of systems, and where the sets of featuresconsidered safety-related in different systems can differquite markedly.
Another common design-for-safety technique is the fail-safe approach. This approach requires a known safe stateexists, and then involves engineering the system so that inthe event of any anomaly, the system will fail to this safestate. It involves a sacrifice in the availability of somenon-critical functions, in order to ensure that systemreliability from a safety perspective is increased.Unfortunately, this strategy does not translate well toSCADA product engineering, since a state that is “safe”in one application may not be safe in another. To amplifythis, consider the issuing of controls to plant. In railtraction power distribution, where the key hazard scenariois electrocution of staff working trackside, the safetyrequirement is that no spurious controls (say to reclose acircuit breaker) occur during the period whenmaintenance is in progress. As such, in this application,“no controls”, whilst occasionally inconvenient, isnevertheless considered safe. Compare this with the environmental plant control application, where it iscritical to be able to issue controls to smoke extractionfans when required. In such applications, the possibilityof an occasional spurious control would be far preferableto the risk of non-availability of controls when required.
3.3 Possible Solutions to the Product Problem
Against the backdrop of a market preference for aproduct-based approach and the assurance problemsassociated with adopting that approach, the followinggeneric options exist.
3.3.1 Multiple Products
One solution is to maintain multiple products, eachtargeted to the functional and assurance requirements of aparticular market sector. Apart from compounding thecost of ownership, such a strategy is also unlikely to bewelcomed by purchasers who want to buy a product witha large user base, so as to be assured of continued supportfor the product line.
3.3.2 Project-based Safety Certification
This solution involves developing individual safety casesfor each deployed system. This approach has theadvantage of relieving projects that have no safetyrequirements of any safety assurance overhead. On theother hand, duplication of effort is likely to result. Also,since the safety assurance evidence is developed by andstays with the project team, future product evolution mayrender this evidence irrelevant. As such, where futuresystem upgrades involve product upgrades, and becausethe product upgrades may not have occurred with thatparticular system’s assurance requirements in mind,safety case maintenance may become extremelyexpensive.
3.3.3 Product Certification
This solution involves achieving certification of a productto a certain safety integrity level. Typically, thecertification is against a particular requirements baseline,and would involve the analysis, by an independent party,of the product development process and in-serviceperformance. This approach has the benefit ofsignificantly reducing the cost of producing system-specific safety cases (certification by a renowned thirdparty is usually very persuasive). However, it also meansthat all product upgrades would need to be re-certified bythe independent party, even those upgrades that weremotivated by requirements for systems in non-safety-related domains. This places a considerable burden on theproduct release process, which may lead to productinertia. It also represents an overhead on the cost of theproduct as a whole, which may price the product out ofcertain markets.
4 Proposed Solution
This section proposes a solution to the problem ofengineering a product that can be deployed across a diverse range of industries, with diverse functional andassurance requirements.
4.1 Organisational Structure
The basis of the solution is to distribute responsibility forthe assurance of SCADA systems across an engineeringorganisation. Responsibility for the certification ofsystems lies with the market-specific, project engineeringteams but is supported by evidence provided by theproduct development group. Requirements for evidenceare forecast by market-specific, sales and marketingteams. The distribution of responsibility is illustrated inFigure 2.
Figure 2 - SCADA Organisational Model
The proposed assurance model allows each projectengineering team to build a safety case in a form suitableto meet specific market and project demands, usingproduct assurance as necessary. This engineering groupmust fulfill two key roles in this model.
Firstly, the sales and marketing team are contributors tothe set of requirements contained in the product baseline.This will involve identification of trends in the marketrelated to safety. This can range from physicalrequirements on the product deliverable (e.g. acompulsory requirement for redundant CPUs on theRTU), through to requirements on the productdevelopment process (e.g. a trend towards the use offormal source code analysis techniques as a compulsorypre-requisite for SIL 2 certification).
Secondly, the project engineering teams are required tomake application-specific safety cases to get theirsystems accepted into service. The safety case is preparedin cooperation with the SCADA system procurer and end-users and will address aspects of the SCADA system, aswell as its operational use. Activities will typicallyinclude production of a hazard analysis to identifyproduct safety requirements, with knowledge of thesystem architecture and operational context. Asubsequent activity will then be performed to map the product and assurance evidence into a safety case thatsatisfies sector specific standards.
The model requires the product team to develop andevolve the required product assurance in line withforecasted market requirements. While engineeringgroups may fund production of the assurance, theproduct-based approach exploits the fact that the costs ofengineering the required product assurance can beamortized not only across many system sales, but alsoacross the entire life of the product.
4.2 Extending the Notion of "Product"
In line with the idea of distributed responsibility forassurance, our proposed solution entails a broaderdefinition of “product”, and hence a broader jobdescription for product development groups. Forcomputer-based systems, there is a tendency to think of aproduct as consisting solely of a physical hardwareplatform and the executable software that runs on it. Weproposed broadening this concept so that a product isconsidered to consist of three baselines, as illustrated inFigure 3.
Figure 3 - Product Baselines
The product baseline comprises the traditionallymaintained artefacts and comprises:
1) the product requirements (both functional and non-functional). This could include specific processrequirements demanded by some sectors (e.g.requirement to demonstrate 100% statementcoverage at unit testing),
2) the product design,
3) product deliverables (i.e. hardware and executablesoftware),
4) associated verification and validation infrastructure(e.g. test cases, test harnesses, etc.).
The process baseline describes how the product baselinewas developed and includes:
1) the suite of meta-processes under which developmentof the product baseline occurs, such as the SoftwareQuality Plan, the Verification & Validation Plan, theConfiguration Management Plan, etc.; and,
2) the suite of specific processes used for variousdevelopment activities, such as Coding Standards,Defect Reporting Guidelines, Change ManagementGuidelines, Technical Review Guidelines, etc.
The assurance baseline provides evidence of theintegrity of the product and process baselines, including:
1) Evidence to demonstrate that product deliverablescomply with product requirements (e.g. test results,design analyses, traceability matrices, user trials,etc.);
2) Evidence to demonstrate that the developmentoccurred in accordance with the process baseline(e.g. documented evidence of reviews, checklists,and quality audit reports);
3) Data, both quantitative and qualitative, which reflectsthe in-service performance of the productdeliverables. This would include a range ofinformation, spanning issues such as observed in-service MTBF of a hardware component, through tothe safety case used to justify inclusion of theproduct in (say) a railway traction control system.
4.3 Generating the Evidence
For a product-based approach to safety-related systemsengineering to be feasible, it must be possible to generateand maintain product assurance evidence at reasonablecost with minimal impact on delivery schedule andproduct advancement. A number of areas critical toachieve this are discussed below.
4.3.1 Product Architecture
A sound product architecture is the foundation for cost-effective product assurance. Key requirements of thearchitecture are to minimise critical functionality in boththe product itself and in the deployed project applicationsand to provide an effective means of partitioning criticaland non-critical functions. Such an architecture allows theproduct and project assurance efforts to be focussed onthose areas requiring most rigour. Modification to thecritical functions can be more tightly controlled and, asthe product matures, operational evidence of stablecritical functions can be accumulated more effectively.Similarly, non-critical functions can be developed underless stringent control in response to market demands orby market sectors themselves. This provides flexible,responsive product functionality without compromisingcritical functions.
A Master Station product with these attributes is currentlyunder study in a joint venture by Invensys Rail and Invensys SCADA Technology. The product comprises acore infrastructure that provide interprocesscommunication and redundancy management, and a set ofservices deployed to suit the target application. Eachservice controls its own data and data is exchanged viathe shared infrastructure. Services may be classified ascritical or non-critical. Data exchange from non-critical tocritical components are prohibited and data exchangefrom critical to non-critical are controlled to ensure thatprocessing load on critical services is restricted. To avoidinterference through common hardware resources, criticaland non-critical services may be physically partitioned byexecution on separate machines.
An additional feature of the architecture is the division ofthe system data into groups, with each group managed byinstances of the system services. Again, critical datagroups and services can be partitioned onto differentmachines, with restricted communication between criticaland non-critical service groups. Partitioning data in thisway allows the project engineering group to focus itsassurance efforts for activities such as system data anddisplay configuration, and communications engineering.
4.3.2 Testing
Extensive testing is fundamental to product assurancearguments but, as mentioned in Section 3.2.3, the costand time required for testing can be high for a feature-richproduct under continuous evolution. While standardsprovide details of how to test new developments, there islimited guidance on the application to legacy softwaremaintenance and evolution.
We propose that the testing be managed by a number ofcombined strategies:
1. Use a product architecture that allows impact ofchange to be minimised and well understood. Impactanalysis is applied to each change to determine thescope of testing required.
2. For clearly localised changes, manual testing ofspecific changes may be sufficient.
3. Develop automated module and integration testing toallow for rapid detailed regression testing of keysystem components. Maintain the test suites toincorporate functionality changes or tests forpreviously undetected faults.
4. Where automated test infrastructure does not exist oris not effective, conduct manual testing.
5. Conduct validation testing focused against a numberof “standard system architectures”, so as to reducethe possible configuration space needed for thoroughtesting.
4.3.3 Dealing with change
For any solution to be workable, it must be possible toevolve the product at reasonable cost. Along with theexpansion of the notion of “product” to accommodatethree distinct baselines, so change management practicesmust be sophisticated enough to cope with changes ineach baseline. Each baseline must maintain an accurate record of all product changes as well as evidence thatproduct quality is preserved. Poorly conceived or rushedchanges can easily render pre-existing test data useless.
For products with long life spans, change managementmust also address changes to the processes used todevelop the product. This is an area that is not generallyhandled in texts on configuration management andchange management. In general, applying new processrequirements to the existing product baseline wouldinvolve major re-engineering and as such would bepractically unfeasible. It is therefore necessary to find away to apply new processes to new developments andmajor product modifications, while permitting minormodifications or “bug fixes” to occur via the processesthat led to the original development. This approach to theissue of process evolution is endorsed in some standards(e.g. EN50128 (CENELEC 2001)). However, thisapproach also leads to a situation where productassurance derives from different sources both across theproduct, and over time. This needs to be carefullyexplained and justified in safety cases using the productassurance baseline.
Invensys SCADA Development are exploring the use of atool for change management developed in a collaborativeresearch project with the University of Queensland(Volzer, Atchison, Lindsay, MacDonald and Strooper:2002). The tool models a product as a hierarchy ofversioned subsystems, where each subsystem version isassociated with a set of changes to configuration items,including source code files or documentation. The tool isbuilt on existing commercial source code configurationmanagement and change management utilities.
To address product assurance change management, thetool would be extended to include references to assuranceevidence within a subsystem version. The susbsystem-based approach is useful for safety-related products sinceit allows assurance records to be tailored at a subsystemlevel depending on the assurance strategy used. Forexample, a subsystem version may reference specific testcases executed for localised changes or evidence ofcomplete automated subsystem test execution. The toolwould be useful as a vehicle for change impactassessment by relating modified configuration items toaffected subsystems and identifying invalidated assuranceevidence. It would also collect measures of subsystemchange over time as the basis for arguments of subsystemstability and reliability growth.
5 Conclusions
SCADA systems are used in many markets, but they areusually not thought of as safety-related systems. Section 2presented an overview of SCADA systems and discussedthe safety integrity requirements that may apply indifferent application domains. This analysis showed that,while SCADA systems are never solely responsible forperforming safety-related functions, they occasionallyperform safety-related functions (e.g. tunnel ventilation),and often contribute to early warning of and subsequentmanagement of hazardous situations. Also, SCADAsystems are relied on to manage the civil infrastructure used to ensure energy and water supply, and so prolongedsystem unavailability or faulty system behaviour cancompromise those supplies, which can in turn have safetyconsequences.
SCADA systems are usually composed from a number ofstandard SCADA products. Indeed, as was discussed inSection 3, companies seeking to supply SCADA systemshave virtually no commercially viable alternative but tomaintain a suite of SCADA products. This createsdifficulties from a safety assurance perspective, since theproducts need to be fit-for-purpose in the face of diversefunctional and safety integrity requirements. There aremany parallels between the issues pertaining to the use ofCOTS in safety-related systems, and to the problemsfaced by SCADA system and product suppliers.
A number of solutions to this problem were brieflyconsidered but determined to be unsatisfactory forvarious reasons. Instead, in Section 4, we proposed analternative that sees responsibility for assurancedistributed across an organisation. In this model, theproduct group’s responsibilities are expanded to involvemaintenance of three baselines, and the application-specific project engineering groups use this informationto build system and sector specific safety cases. Threeessential features of this solution were examined,including a product architecture that partitions criticalfunctionality, a testing approach that provides focussedrapid accumulation of assurance evidence and a changemanagement system that deals not only with changes in aproduct’s functional requirements, but also with changesin a product’s assurance evidence.
6 References
ATCHISON, B. and GRIFFITHS, A. (2002): EngineeringSCADA Products for Use in Safety-Related Systems.Proc. Safety Critical Systems Symposium, Southampton,UK, Springer-Verlag.
LANDMAN, R. J. (2000): Supervisory Control and DataAcquisition Systems
CENELEC (2001): European Standard ENV 50128,Railway Applications - Communications, Signalling andProcessing Systems - Software for Railway Control andProtection Systems,
LINDSAY, P. and SMITH, G. (2000): Safety Assuranceof Commercial-Off-The-Shelf Software. ProceedingsFifth Australian Workshop on Safety Critical Systems andSoftware, Melbourne, Australia, 43-51, AustralianComputer Society.
O'HALLORAN, C. (1999): Assessing Safety CriticalCOTS Systems. Journal of the System Safety Society35(2):
VOLZER, H., ATCHISON, B., LINDSAY, P.,MACDONALD, A. and STROOPER:, P. (2002): A Toolfor Subsystem Configuration Management. Proc. ICSM2002, IEEE Press.