ØMeaning of SCADA
SCADA stands for Supervisory Control and Data Acquisition. As the name indicates, it is not a full control system, but rather focuses on the supervisory level. As such, it is a purely software package that is positioned on top of hardware to which it is interfaced, in general via Programmable Logic Controllers (PLCs), or other commercial hardware modules.
SCADA systems are used not only in industrial processes: e.g. steel making, power generation (conventional and nuclear) and distribution, chemistry, but also in some experimental facilities such as nuclear fusion. The size of such plants range from a few 1000 to several 10 thousands input/output (I/O) channels. However, SCADA systems evolve rapidly and are now penetrating the market of plants with a number of I/O channels of several 100 K: we know of two cases of near to 1 M I/O channels currently under development.
SCADA systems used to run on DOS, VMS and UNIX; in recent years all SCADA vendors have moved to NT and some also to Linux.
This section describes the common features of the SCADA products that have been evaluated at CERN in view of their possible application to the control systems of the LHC detectors , .
- 8.1 Hardware Architecture
One distinguishes two basic layers in a SCADA system: the “client layer” which caters for the man machine interaction and the “data server layer” which handles most of the process data control activities. The data servers communicate with devices in the field through process controllers. Process controllers, e.g. PLCs, are connected to the data servers either directly or via networks or field buses that are proprietary (e.g. Siemens H1), or non-proprietary (e.g. Profibus). Data servers are connected to each other and to client stations via an Ethernet LAN. The data servers and client stations are NT platforms but for many products the client stations may also be W95 machines. .
- 8.2 Communications
Server-client and server-server communication is in general on a publish-subscribe and event-driven basis and uses a TCP/IP protocol, i.e., a client application subscribes to a parameter which is owned by a particular server application and only changes to that parameter are then communicated to the client application.
Access to Devices
The data servers poll the controllers at a user defined polling rate. The polling rate may be different for different parameters. The controllers pass the requested parameters to the data servers. Time stamping of the process parameters is typically performed in the controllers and this time-stamp is taken over by the data server. If the controller and communication protocol used support unsolicited data transfer then the products will support this too.
The products provide communication drivers for most of the common PLCs and widely used field-buses, e.g., Modbus. Of the three fieldbuses that are recommended at CERN, both Profibus and World flip are supported but CANbus often not . Some of the drivers are based on third party products (e.g., Applicom cards) and therefore have additional cost associated with them. VME on the other hand is generally not supported.
A single data server can support multiple communications protocols: it can generally support as many such protocols as it has slots for interface cards.
The effort required to develop new drivers is typically in the range of 2-6 weeks depending on the complexity and similarity with existing drivers, and a driver development toolkit is provided for this.
- 8.3 Interfacing
The provision of OPC client functionality for SCADA to access devices in an open and standard manner is developing. There still seems to be a lack of devices/controllers, which provide OPC server software, but this improves rapidly as most of the producers of controllers are actively involved in the development of this standard. OPC has been evaluated by the CERN-IT-CO group .
The products also provide
- An Open Data Base Connectivity (ODBC) interface to the data in the archive/logs, but not to the configuration database,
- An ASCII import/export facility for configuration data,
- A library of APIs supporting C, C++, and Visual Basic (VB) to access data in the RTDB, logs and archive. The API often does not provide access to the product’s internal features such as alarm handling, reporting, trending, etc.
The PC products provide support for the Microsoft standards such as Dynamic Data Exchange (DDE) which allows e.g. to visualize data dynamically in an EXCEL spreadsheet, Dynamic Link Library (DLL) and Object Linking and Embedding (OLE).
The configuration data are stored in a database that is logically centralized but physically distributed and that is generally of a proprietary format.
For performance reasons, the RTDB resides in the memory of the servers and is also of proprietary format.
The archive and logging format is usually also proprietary for performance reasons, but some products do support logging to a Relational Data Base Management System (RDBMS) at a slower rate either directly or via an ODBC interface.
- 8.4 Scalability
Scalability is understood as the possibility to extend the SCADA based control system by adding more process variables, more specialized servers (e.g. for alarm handling) or more clients. The products achieve scalability by having multiple data servers connected to multiple controllers. Each data server has its own configuration database and RTDB and is responsible for the handling of a sub-set of the process variables (acquisition, alarm handling, archiving).
- 8.5 Redundancy
The products often have built in software redundancy at a server level, which is normally transparent to the user. Many of the products also provide more complete redundancy solutions if required.
Ø9. Common system components
A SCADA System usually consists of the following subsystems:
- A Human-Machine Interface or HMI is the apparatus which presents process data to a human operator, and through this, the human operator monitors and controls the process.
- A supervisory (computer) system, gathering (acquiring) data on the process and sending commands (control) to the process.
- Remote Terminal Units (RTUs) connecting to sensors in the process, converting sensor signals to digital data and sending digital data to the supervisory system.
- Programmable Logic Controller (PLCs) used as field devices because they are more economical, versatile, flexible, and configurable than special-purpose RTUs.
- Communication infrastructure connecting the supervisory system to the Remote Terminal Units
·9.1 Supervision vs. control
There is, in several industries, considerable confusion over the differences between SCADA systems and Distributed control systems (DCS). Generally speaking, a SCADA system usually refers to a system that coordinates, but does not control processes in real time. The discussion on real-time control is muddied somewhat by newer telecommunications technology, enabling reliable, low latency, high speed communications over wide areas. Most differences between SCADA and DCS are culturally determined and can usually be ignored. As communication infrastructures with higher capacity become available, the difference between SCADA and DCS will fade.
·9.2 Systems concepts
The term SCADA usually refers to centralized systems which monitor and control entire sites, or complexes of systems spread out over large areas (anything between an industrial plant and a country). Most control actions are performed automatically by remote terminal units (“RTUs”) or by programmable logic controllers (“PLCs”). Host control functions are usually restricted to basic overriding or supervisory level intervention. For example, a PLC may control the flow of cooling water through part of an industrial process, but the SCADA system may allow operators to change the set points for the flow,and enable alarm conditions, such as loss of flow and high temperature, to be displayed and recorded. The feedback control loop passes through the RTU or PLC, while the SCADA system monitors the overall performance of the loop.
Data acquisition begins at the RTU or PLC level and includes meter readings and equipment status reports that are communicated to SCADA as required. Data is then compiled and formatted in such a way that a control room operator using the HMI can make supervisory decisions to adjust or override normal RTU (PLC) controls. Data may also be fed to a Historian, often built on a commodity Database Management System, to allow trending and other analytical auditing.
SCADA systems typically implement a distributed database, commonly referred to as a tag database, which contains data elements called tags or points. A point represents a single input or output value monitored or controlled by the system. Points can be either “hard” or “soft”. A hard point represents an actual input or output within the system, while a soft point results from logic and math operations applied to other points. (Most implementations conceptually remove the distinction by making every property a “soft” point expression, which may, in the simplest case, equal a single hard point.) Points are normally stored as value-timestamp pairs: a value, and the timestamp when it was recorded or calculated. A series of value-timestamp pairs gives the history of that point. It’s also common to store additional metadata with tags, such as the path to a field device or PLC register, design time comments, and alarm information.
·9.3 Human Machine Interface
Typical Basic SCADA Animations 
A Human-Machine Interface or HMI is the apparatus which presents process data to a human operator, and through which the human operator controls the process.
An HMI is usually linked to the SCADA system’s databases and software programs, to provide trending, diagnostic data, and management information such as scheduled maintenance procedures, logistic information, detailed schematics for a particular sensor or machine, and expert-system troubleshooting guides.
The HMI system usually presents the information to the operating personnel graphically, in the form of a mimic diagram. This means that the operator can see a schematic representation of the plant being controlled. For example, a picture of a pump connected to a pipe can show the operator that the pump is running and how much fluid it is pumping through the pipe at the moment. The operator can then switch the pump off. The HMI software will show the flow rate of the fluid in the pipe decrease in real time. Mimic diagrams may consist of line graphics and schematic symbols to represent process elements, or may consist of digital photographs of the process equipment overlain with animated symbols.
The HMI package for the SCADA system typically includes a drawing program that the operators or system maintenance personnel use to change the way these points are represented in the interface. These representations can be as simple as an on-screen traffic light, which represents the state of an actual traffic light in the field, or as complex as a multi-projector display representing the position of all of the elevators in a skyscraper or all of the trains on a railway.
An important part of most SCADA implementations are alarms. An alarm is a digital status point that has either the value NORMAL or ALARM. Alarms can be created in such a way that when their requirements are met, they are activated. An example of an alarm is the “fuel tank empty” light in a car. The SCADA operator’s attention is drawn to the part of the system requiring attention by the alarm. Emails and text messages are often sent along with an alarm activation alerting managers along with the SCADA operator.
·9.4 Hardware solutions
SCADA solutions often have Distributed Control System (DCS) components. Use of “smart” RTUs or PLCs, which are capable of autonomously executing simple logic processes without involving the master computer, is increasing. A functional block programming language, IEC 61131-3 (Ladder Logic), is frequently used to create programs which run on these RTUs and PLCs. Unlike a procedural language such as the C programming language or FORTRAN, IEC 61131-3 has minimal training requirements by virtue of resembling historic physical control arrays. This allows SCADA system engineers to perform both the design and implementation of a program to be executed on an RTU or PLC. A Programmable automation controller (PAC) is a compact controller that combines the features and capabilities of a PC-based control system with that of a typical PLC. PACs are deployed in SCADA systems to provide RTU and PLC functions. In many electrical substation SCADA applications, “distributed RTUs” use information processors or station computers to communicate with protective relays, PACS, and other devices for I/O, and communicate with the SCADA master in lieu of a traditional RTU.
Since about 1998, virtually all major PLC manufacturers have offered integrated HMI/SCADA systems, many of them using open and non-proprietary communications protocols. Numerous specialized third-party HMI/SCADA packages, offering built-in compatibility with most major PLCs, have also entered the market, allowing mechanical engineers, electrical engineers and technicians to configure HMIs themselves, without the need for a custom-made program written by a software developer.
Ø10. Remote Terminal Unit (RTU)
The RTU connects to physical equipment. Typically, an RTU converts the electrical signals from the equipment to digital values such as the open/closed status from a switch or a valve, or measurements such as pressure, flow, voltage or current. By converting and sending these electrical signals out to equipment the RTU can control equipment, such as opening or closing a switch or a valve, or setting the speed of a pump.
·10.1 Supervisory Station
The term “Supervisory Station” refers to the servers and software responsible for communicating with the field equipment (RTUs, PLCs, etc), and then to the HMI software running on workstations in the control room, or elsewhere. In smaller SCADA systems, the master station may be composed of a single PC. In larger SCADA systems, the master station may include multiple servers, distributed software applications, and disaster recovery sites. To increase the integrity of the system the multiple servers will often be configured in a dual-redundant or hot-standby formation providing continuous control and monitoring in the event of a server failure.
Initially, more “open” platforms such as Linux were not as widely used due to the highly dynamic development environment and because a SCADA customer that was able to afford the field hardware and devices to be controlled could usually also purchase UNIX or OpenVMS licenses. Today, all major operating systems are used for both master station servers and HMI workstations.
·10.2 Operational philosophy
For some installations, the costs that would result from the control system failing are extremely high. Possibly even lives could be lost. Hardware for some SCADA systems is ruggedized to withstand temperature, vibration, and voltage extremes, but in most critical installations reliability is enhanced by having redundant hardware and communications channels, up to the point of having multiple fully equipped control centres. A failing part can be quickly identified and its functionality automatically taken over by backup hardware. A failed part can often be replaced without interrupting the process. The reliability of such systems can be calculated statistically and is stated as the mean time to failure, which is a variant of mean time between failures. The calculated mean time to failure of such high reliability systems can be on the order of centuries.
·10.3 Communication infrastructure and methods
SCADA systems have traditionally used combinations of radio and direct serial or modem connections to meet communication requirements, although Ethernet and IP over SONET / SDH is also frequently used at large sites such as railways and power stations. The remote management or monitoring function of a SCADA system is often referred to as telemetry.
This has also come under threat with some customers wanting SCADA data to travel over their pre-established corporate networks or to share the network with other applications. The legacy of the early low-bandwidth protocols remains, though. SCADA protocols are designed to be very compact and many are designed to send information to the master station only when the master station polls the RTU. Typical legacy SCADA protocols include Modbus RTU, RP-570, Profibus and Conitel. These communication protocols are all SCADA-vendor specific but are widely adopted and used. Standard protocols are IEC 60870-5-101 or 104, IEC 61850 and DNP3. These communication protocols are standardized and recognized by all major SCADA vendors. Many of these protocols now contain extensions to operate over TCP/IP. It is good security engineering practice to avoid connecting SCADA systems to the Internet sothe attack surface is reduced.
RTUs and other automatic controller devices were being developed before the advent of industry wide standards for interoperability. The result is that developers and their management created a multitude of control protocols. Among the larger vendors, there was also the incentive to create their own protocol to “lock in” their customer base. A list of automation protocols is being compiled here.
Recently, OLE for Process Control (OPC) has become a widely accepted solution for intercommunicating different hardware and software, allowing communication even between devices originally not intended to be part of an industrial network.
Ø11. Trends in SCADA
There is a trend for plc and HMI/SCADA software to be more “mix-and-match”. In the mid 1990s, the typical DAQ I/O manufacturer supplied equipment that communicated using proprietary protocols over a suitable-distance carrier like RS-485. End users who invested in a particular vendor’s hardware solution often found themselves restricted to a limited choice of equipment when requirements changed (e.g. system expansions or performance improvement). To mitigate such problems, open communication protocols such as IEC870-5-101/104, DNP3 serial, and DNP3 LAN/WAN became increasingly popular among SCADA equipment manufacturers and solution providers alike. Open architecture SCADA systems enabled users to mix-and-match products from different vendors to develop solutions that were better than those that could be achieved when restricted to a single vendor’s product offering.
Towards the late 1990s, the shift towards open communications continued with individual I/O manufacturers as well, who adopted open message structures such as Modbus RTU and Modbus ASCII (originally both developed by Modicon) over RS-485. By 2000, most I/O makers offered completely open interfacing such as Modbus TCP over Ethernet and IP.
The North American Electric Reliability Corporation (NERC) has specified that electrical system data should be time-tagged to the nearest millisecond. Electrical system SCADA systems provide this Sequence of events recorder function, using Radio clocks to synchronize the RTU or distributed RTU clocks.
SCADA systems are coming in line with standard networking technologies. Ethernet and TCP/IP based protocols are replacing the older proprietary standards. Although certain characteristics of frame-based network communication technology (determinism, synchronization, protocol selection, environment suitability) have restricted the adoption of Ethernet in a few specialized applications, the vast majority of markets have accepted Ethernet networks for HMI/SCADA.
With the emergence of software as a service in the broader software industry, a few vendors have begun offering application specific SCADA systems hosted on remote platforms over the Internet. This removes the need to install and commission systems at the end-user’s facility and takes advantage of security features already available in Internet technology, VPNs and SSL. Some concerns include security, Internet connection reliability, and latency.
SCADA systems are becoming increasingly ubiquitous. Thin clients, web portals, and web based products are gaining popularity with most major vendors. The increased convenience of end users viewing their processes remotely introduces security considerations. While these considerations are already considered solved in other sectors of internet services, not all entities responsible for deploying SCADA systems have understood the changes in accessibility and threat scope implicit in connecting a system to the internet.
Ø12. Security issues
The move from proprietary technologies to more standardized and open solutions together with the increased number of connections between SCADA systems and office networks and the Internet has made them more vulnerable to attacks – see references. Consequently, the security of SCADA-based systems has come into question as they are increasingly seen as extremely vulnerable to cyber warfare/cyber terrorism attacks.
In particular, security researchers are concerned about:
- the lack of concern about security and authentication in the design, deployment and operation of existing SCADA networks
- the mistaken belief that SCADA systems have the benefit of security through obscurity through the use of specialized protocols and proprietary interfaces
- the mistaken belief that SCADA networks are secure because they are purportedly physically secured
- the mistaken belief that SCADA networks are secure because they are supposedly disconnected from the Internet
SCADA systems are used to control and monitor physical processes, examples of which are transmission of electricity, transportation of gas and oil in pipelines, water distribution, traffic lights, and other systems used as the basis of modern society. The security of these SCADA systems is important because compromise or destruction of these systems would impact multiple areas of society far removed from the original compromise. For example, a blackout caused by a compromised electrical SCADA system would cause financial losses to all the customers that received electricity from that source. How security will affect legacy SCADA and new deployments remains to be seen.
There are two distinct threats to a modern SCADA system. First is the threat of unauthorized access to the control software, whether it be human access or changes induced intentionally or accidentally by virus infections and other software threats residing on the control host machine. Second is the threat of packet access to the network segments hosting SCADA devices. In many cases, there is rudimentary or no security on the actual packet control protocol, so anyone who can send packets to the SCADA device can control it. In many cases SCADA users assume that a VPN is sufficient protection and are unaware that physical access to SCADA-related network jacks and switches provides the ability to totally bypass all security on the control software and fully control those SCADA networks. These kinds of physical access attacks bypass firewall and VPN security and are best addressed by endpoint-to-endpoint authentication and authorization such as are commonly provided in the non-SCADA world by in-device SSL or other cryptographic techniques.
Many vendors of SCADA and control products have begun to address these risks in a basic sense by developing lines of specialized industrial firewall and VPN solutions for TCP/IP-based SCADA networks. Additionally, application white listing solutions are being implemented because of their ability to prevent malware and unauthorized application changes without the performance impacts of traditional antivirus scans Also, the ISA Security Compliance Institute (ISCI) is emerging to formalize SCADA security testing starting as soon as 2009. ISCI is conceptually similar to private testing and certification that has been performed by vendors since 2007. Eventually, standards being defined by ISA99 WG4 will supersede the initial industry consortia efforts, but probably not before 2011 .
The increased interest in SCADA vulnerabilities has resulted in vulnerability researchers discovering vulnerabilities in commercial SCADA software and more general offensive SCADA techniques presented to the general security community. In electric and gas utility SCADA systems, the vulnerability of the large installed base of wired and wireless serial communications links is addressed in some cases by applying bump-in-the-wire devices that employ authentication and Advanced Encryption Standard encryption rather than replacing all existing nodes.
Ø13. Application Development
- 13.1 Configuration
The development of the applications is typically done in two stages. First the process parameters and associated information (e.g. relating to alarm conditions) are defined through some sort of parameter definition template and then the graphics, including trending and alarm displays are developed, and linked where appropriate to the process parameters. The products also provide an ASCII Export/Import facility for the configuration data (parameter definitions), which enables large numbers of parameters to be configured in a more efficient manner using an external editor such as Excel and then importing the data into the configuration database.
However, many of the PC tools now have a Windows Explorer type development studio. The developer then works with a number of folders, which each contains a different aspect of the configuration, including the graphics.
The facilities provided by the products for configuring very large numbers of parameters are not very strong. However, this has not really been an issue so far for most of the products to-date, as large applications are typically about 50K I/O points and database population from within an ASCII editor such as Excel is still a workable option.
On-line modifications to the configuration database and the graphics are generally possible with the appropriate level of privileges.
- 13.2 Development Tools
The following development tools are provided as standard:
- A graphics editor, with standard drawing facilities including freehand, lines, squares circles, etc. It is possible to import pictures in many formats as well as using predefined symbols including e.g. trending charts, etc. A library of generic symbols is provided that can be linked dynamically to variables and animated as they change. It is also possible to create links between views so as to ease navigation at run-time.
- A data base configuration tool (usually through parameter templates). It is in general possible to export data in ASCII files so as to be edited through an ASCII editor or Excel.
- A scripting language
- An Application Program Interface (API) supporting C, C++, VB
SCADA vendors release one major version and one to two additional minor versions once per year. These products evolve thus very rapidly so as to take advantage of new market opportunities, to meet new requirements of their customers and to take advantage of new technologies.
As was already mentioned, most of the SCADA products that were evaluated decompose the process in “atomic” parameters to which a Tag-name is associated. This is impractical in the case of very large processes when very large sets of Tags need to be configured. As the industrial applications are increasing in size, new SCADA versions are now being designed to handle devices and even entire systems as full entities (classes) that encapsulate all their specific attributes and functionality. In addition, they will also support multi-team development.
As far as new technologies are concerned, the SCADA products are now adopting:
- Web technology, ActiveX, Java, etc.
- OPC as a means for communicating internally between the client and server modules. It should thus be possible to connect OPC compliant third party modules to that SCADA product.
- 15. Engineering
Whilst one should rightly anticipate significant development and maintenance savings by adopting a SCADA product for the implementation of a control system, it does not mean a “no effort” operation. The need for proper engineering can not be sufficiently emphasized to reduce development effort and to reach a system that complies with the requirements, that is economical in development and maintenance and that is reliable and robust. Examples of engineering activities specific to the use of a SCADA system are the definition of:
- a library of objects (PLC, device, subsystem) complete with standard object behavior (script, sequences, …), graphical interface and associated scripts for animation,
- templates for different types of “panels”, e.g. alarms,
- instructions on how to control e.g. a device …,
- a mechanism to prevent conflicting controls (if not provided with the SCADA), alarm levels, behavior to be adopted in case of specific alarms.
Ø16. Potential benefits of SCADA
The benefits one can expect from adopting a SCADA system for the control of experimental physics facilities can be summarized as follows:
- A rich functionality and extensive development facilities. The amount of effort invested in SCADA product amounts to 50 to 100 p-years!
- The amount of specific development that needs to be performed by the end-user is limited, especially with suitable engineering.
- Reliability and robustness. These systems are used for mission critical industrial processes where reliability and performance are paramount. In addition, specific development is performed within a well-established framework that enhances reliability and robustness.
- Technical support and maintenance by the vendor.