Abstract

In my previous blog (Why process safety risk and cyber security risk differ) I discussed the differences between process safety risk and cyber security risk and why these two risks don’t align. In this blog I like to discuss some of the cyber security risk criteria, why they differ from process safety risk criteria and how they align with cyber security design objectives. And as usual I will challenge some of IEC 62443-3-2 misconceptions.

Cyber security can be either prescriptive or risk based. An example of a prescriptive approach is the IEC 62443-3-3, the standard is a list with security requirements to follow in order to create resilience against a specific cyber threat. For this IEC 62443 uses security levels with the following definitions:

• SL 1 – Prevent the unauthorized disclosure of information via eavesdropping or casual exposure.
• SL 2 – Prevent the unauthorized disclosure of information to an entity actively searching for it using simple means with low resources, generic skills and low motivation.
• SL 3 – Prevent the unauthorized disclosure of information to an entity actively searching for it using sophisticated means with moderate resources, IACS specific skills and moderate motivation.
• SL 4 – Prevent the unauthorized disclosure of information to an entity actively searching for it using sophisticated means with extended resources, IACS specific skills and high motivation.

I underlined the criteria that create the difference between the security levels. A few examples to translate the formal text from the standard to what is happening in the field.

An example of an SL 1 level of resilience is protection against a process operator looking over the shoulder of a process engineer to retrieve the engineer’s password. Maybe you don’t consider this a cyber attack, but formally it is a cyber attack by a hopefully non-malicious insider.

An example of an SL 2 level of resilience is protection against malware, for example a process control system that became infected by WannaCry, ransomware. WannaCry didn’t target specific asset owners and did not specifically target process control systems. Any system with an exposed Microsoft EternalBlue vulnerability could become victim.

An example of an SL 3 level of resilience is protection against the attack on Ukraine’s power power grid. This was a targeted attack, using a mixture of standard tactics like phishing to get access to the system and specially crafted software to make modifications in the control system.

An example of an SL 4 level of resilience would be the protection against Stuxnet, the attack on Iran’s uranium enrichment facilities. This attack made use of zero-day vulnerabilities, required detailed knowledge of the production process, and thoroughly testing of the attack on a “copy” of the actual system.

The difference between SL 2 and SL 3 is clear, however the difference between SL 3 and SL 4 is a bit arbitrary. For example is the Triton attack an example of SL 3 or can we say it is SL 4. The Triton attack required like Stuxnet also a significant investment for the threat actors. But where Stuxnet had clear political motives, the motives for the Triton attack are less clear for me. SL 4 is often linked to resilience against attacks by nation-states, but with all these “nation state sponsored” threat actors that difference often disappears.

The IEC 62443-3-3 standard assigns the security levels to security zones and conduits of the control system for defining the technical security requirements for the assets in the zone and communication between zones. The targets for a threat actor are either an asset or a channel (the protocols flowing through the conduit). Though the standard also attempts to address risk with the IEC 62443-3-2, there is no clear path from risk to security level. There is not an “official” transformation matrix that converts a specific risk level into a security level, apart from the many issues with such a matrix. So overall IEC 62443-3-3 is a prescriptive form of cyber security. This is common for standards and government regulations, they need clear instructions.

The issue with prescriptive cyber security is that it doesn’t address the specific differences between systems. It is kind of ready-to-wear suit, does overall a good job but a tailor-made suit fits better. The tailor-made suit for cyber security is risk based security. In my opinion risk based security is a must for process automation systems supporting the critical infrastructure. Threat actors targeting critical infrastructure are highly skilled professionals, we should not put them aside as hackers for fun. Also the offensive side is a profession, either servicing a “legal” business such as the military or an “illegal” business such as cyber crime. And this cybercrime business is big business, it is for example bigger than the drugs trade.

Protection against the examples I gave for SL 3 and SL 4 requires a detailed analysis of the attack scenarios available to launch against the process installation, analyzing which barriers are in place to stop or detect these scenarios, and estimating the residual risk after having applied the barriers to see if we meet the risk criteria. For this we need residual risk, the risk after applying the barriers, and residual risk requires quantitative risk analysis to estimate the risk reduction similar to how process safety does this with LOPA as discussed in my previous blog.

Quantitative analysis is required because we want to know the contribution in risk reduction from the various barriers. Barriers being the countermeasures with which we can reduce the likelihood that the attack succeeds or with which we reduce / take away the impact severity of the consequences of a successful cyber attack. A simple example of a likelihood reducing barrier is an antivirus solution, a simple example of an impact severity reducing barrier is a back-up. These barriers are not limited to various security packages but also include design choice in the automation system and the production facility. The list of barriers I work with during my professional time has grown to over 400 and keeps growing with every detailed risk assessment. The combination of attack scenarios (the TTP of the threat actors), barriers and consequences (the deviations of the process automation functions as result of a successful cyber attack) is the base for a risk analysis.

Where a value for consequence severity can either be a subject matter expert assigned severity level for the functional deviation (resulting in a risk priority number (RPN)) or the severity scores / target factors of the worse case process scenarios that can result from the potential functional deviation. These severity scores / target factors come from the HAZOP or LOPA sheets created by the process safety engineers during their analysis of what can go wrong with the process installation and what is required to control the impact when things go wrong. When we use LOPA / HAZOP severity scores for consequence severity we call the resulting risk loss based risk (LBR), the risk a specific financial, safety, or environmental loss occurs. For security design a risk priority number is generally sufficient to identify the required barriers, however to justify the risk mitigation (investment in those barriers) to management and regulatory organizations loss based risk is often a requirement.

To carry out a detailed risk assessment the risk analyst should oversee a wide field of skills, understanding the automation system’s working in detail, understanding the process and safety engineering options, and understanding risk. Therefore risk assessments are teamwork of different subject matter experts, however the risk analyst needs to connect all input to get the result.

But risk assessments is also a very rewarding cyber security activity because communicating risk to management is far easier than explaining them why we invest in a next generation firewall instead of the lower cost packet filter. Senior management understands risk far better than cyber security technology.

So far the intro, lets look at the risk criteria now. ISA / IEC have a tendency in their approach to cyber risk to have a So much for the intro, now let’s look at the risk criteria. ISA/IEC tend to look closely at how process security approaches the subject in their approach to cyber risk and to attempt to copy it.

However as explained in my previous blog there are major differences between cyber security risk and process safety risk making such a comparison more complex than it seems. Many of these differences also have an impact on the risk criteria and the terminology we use. I like to discuss two terms in this blog, unmitigated risk and risk tolerance. Let’s start with unmitigated risk.

Both ISA 62443-3-2 and the ISA TR 84 work group use the term unmitigated risk. As an initial action they want to determine unmitigated risk and compare this with the risk criteria. Unmitigated risk is a term used by process safety for the process safety risk prior to applying the safeguards protecting against randomly occurring events that could cause a specific worse case process impact. The safeguards protect against events like a leaking seal, a failing pump, or an operator error. The event frequency of such an initiating event (e.g. failed pump) is reduced by applying safeguards, the resulting event frequency after applying the safeguard needs to meet specified limits. (See my previous blog) Safety engineers basically reduce an event frequency gap using safeguards with a reliability that actually accomplish this. The safeguard will bring the process into a safe state by itself. Multiple safeguards can exist but each safeguard will prevent the worst case consequence by itself. It is not that safeguard 1 and safeguard 2 need to accomplish the job together, each of them individually does the job. There might be a preferred sequence but not a dependency of protection layers.

Cyber security doesn’t work that way, we might define a specific initiating event frequency for the chance on a virus infection. But after installing an antivirus engine we cannot say the problem is solved and no virus infections will occur. The reliability of a process safety barrier is an entirely different factor than the effectiveness of a cyber security barrier. Both reduce risk, the process safety barrier reduces the risk to the required limit by it self with a reliability that allows us to take the credit for the risk reduction. But the cyber security barrier most likely requires additional barriers to reach an acceptable event frequency / risk reduction.

Another difference is that in cyber security the initiating event is not occurring randomly, it is an intentional action of a threat actor with a certain propensity (A term introduced by Terry Ingoldsby from Amenaza) to target the plant, select a TTP and target a specific function. The threat actor has different TTPs available to accomplish the objective, so the initiating event frequency is not so much one point on the frequency scale but a range with a low and high limit. We cannot pick on forehand the high limit of this range (representing the highest risk) because the process control system’s cyber resilience actually determines if a specific TTP can succeed, so we need to “exercise” a range of scenarios to determine the highest event frequency (highest likelihood) applicable within the range.

A third difference is the defense in depth. In process safety defense in depth is implemented by multiple independent protection layers. For example there might be a safety instrumented function configured in a safety system, but additionally there can be physical protection layers like a dike or other forms of bunding to control the impact. In cyber security we also apply defense in depth, it is considered bad practice if security depends on a single control to protect the system. However many of these cyber security controls share a common element, the infrastructure / computer. We can have an antivirus engine and we can add application whitelisting or USB protection to further reduce the chance that malware enters the system but they all share the same computer platform offering a shared point of failure.

Returning to the original remark on unmitigated risk, in process automation cyber security risk unmitigated risk doesn’t exist. The moment we install a process control system we install some inherent protection barriers. Examples are authentication / authorization, various built-in firewalls, encrypted messages, etc. So when we start a cyber security risk analysis there is no unmitigated risk, we can’t ignore the various barriers built-in the process automation systems. A better term to use is therefore inherent risk. The risk of the system as we analyze it, the inherent risk is also the residual risk but it is not unmitigated there is a range of barriers implemented when we start a risk analysis. The question is more, does the inherent risk meet the risk criteria and if not what barriers are required that result in a residual risk that does meet the criteria.

The second term I like to discuss is risk tolerance. Both IEC 62443 and the ISA TR 84 work group pose that the cyber security must meet the risk tolerance. I fully agree with this, where we differ is that I don’t see risk tolerance as the cyber security design target where the IEC 62443 standard does. In risk theory, risk tolerance is defined as the maximum loss a plant is willing to experience. To further explain my issue with the standard I first discuss the process safety side use of risk tolerance and use an F-N diagram from my previous blog.

This F-N curve shows two limits using a red and a green line. Anything above the red line is unacceptable, anything below the green line is acceptable. The area between the two lines is considered tolerable if it meets the ALARP (As Low As Reasonably Practicable) principle. For a risk to be ALARP, it must be possible to demonstrate that the cost involved in reducing the risk further would be grossly disproportionate to the benefit gained. Determining that a risk is reduced to the ALARP level involves an assessment of the risk to be avoided, an assessment of the investment (in money, time and trouble) involved in taking measures to avoid that risk, and a comparison of the two.

In process safety where the events occur randomly this is possible, in cyber security where the events occur intentionally this is far more difficult.

Can we consider the risk for the power grid in a region with political tensions the same as in the case of a region having no conflicts at all. I worked for refineries that had as a requirement to be resilient against SL 2 level of threat actors, but also refineries that told me they wanted SL 3, and there was even a case where the requirement was SL 4. So in cyber security the ALARP principle is not really followed, there is apparently another limit. The risk tolerance limit is for all the same, but there is also a limit that sets a kind of cyber security comfort level. This level we call the risk appetite, and this level is actually should become our cyber security design target level.

Risk appetite and risk tolerance shouldn’t be the same, there should be a margin between the two that allow for the possibility to respond to failures of the security mechanisms. If risk appetite and risk tolerance are the same and this would also be our design target any issue with a security control would result in an unacceptable risk.

In principal unacceptable risk means we should stop the production (as is the case for process safety), so if the limits would be the same we have created kind of a on/off mechanism. For example if we couldn’t update our antivirus engine for some reason, risk would raise above the limit and we would need to stop the plant. Maybe an extreme example and certainly not really a situation I see happening in real life. However when risk becomes not acceptable we should have an action defined for this case. There are plants that don’t have a daily update but follow a monthly or quarterly AV signature update frequency (making AV engine effectiveness very low with all the polymorph viruses), so apparently risk appetite differs.

If we want to define clear actions / consequences for each risk level we need sufficient risk levels to do this and limits that allow us to respond to failing cyber controls. So we need two limits, risk appetite and risk tolerance. The difference between risk appetite and risk tolerance is called risk capacity, having a small risk capacity means that issues can escalate quickly. A security design must also consider the risk capacity in the selection of barriers, “defense in depth” is an important security principle here because this increases risk capacity.

Above shows an example risk matrix (different criteria as the F-N diagram above) with 4 risk levels (A, TA, TNA, NA). Typically 4 or 5 risk levels are chosen and for each level the criteria and action is specified. In above example the risk tolerance level is the line that separates TNA and NA. Risk appetite might be either the line between TA and TNA or between A and TA for a very risk averse asset owner. However the risk capacity of a security design were the risk appetite is defined at the TA / TNA border is much smaller than if our target would be the A / TA border. But in both cases a risk increase due to the failure of a cyber security control with not immediately escalate into a Non Acceptable risk condition.

If we opt for risk based security, we need to have actionable risk levels and in that case a single risk tolerance level as specified and discussed in the IEC 62443-3-2 is just not possible and therefore not practiced for cyber security risk. The ALARP principle and cyber security don’t go together very well.

Maybe a bit boring topic, maybe too much text, never the less I hope the reader understands why unmitigated risk doesn’t exist in cyber security for process automation. And I certainly hope that the reader will understand that the risk tolerance limit is a bad design target.

There is no relationship between my opinions and references to publications in this blog and the views of my employer in whatever capacity. This blog is written based on my personal opinion and knowledge build up over 43 years of work in this industry. Approximately half of the time working in engineering these automation systems, and half of the time implementing their networks and securing them, and conducting cyber security risk assessments for process installations since 2012.

Author: Sinclair Koelemij

OTcybersecurity web site

Abstract

This blog is about cyber security detection controls for cyber physical systems, process automation systems, process control systems or whatever we want to call them. The blog discusses their value in securing these systems and some of the requirements to consider when implementing them. As usual, my blogs approach the subject from a combination of the knowledge area of ​​cyber security and the knowledge area of ​​automation technology. More specific for this topic is the combination of Time Based Security and Asset Integrity Management.

It’s been a while since I started a blog, I postponed my retirement for another 2 years, so just too much to do to spend my weekends creating blogs. But sometimes the level of discomfort rises above my high alarm threshold for discomfort, forcing me to analyze the subject to better understand the cause of this discomfort. After seven months of no blogging, there are plenty of topics to discuss. For this blog I want to analyze the requirements for a cybersecurity detection function for cyber-physical systems. Do they differ from the requirements we set for information systems, and if so, what are these differences? Let’s start with the historical context.

Traditional cybersecurity concepts were based on a combination of fortress mentality and risk aversion. This resulted in the building of static defenses as described in the TCSEC and the Orange Book of the 1980s. But almost 40 years later, this no longer works in a world where the information resides increasingly at the very edge of the automation systems. Approximately 20 years ago a new security model was introduced as an answer to the increasing number of issues the traditional models were facing.

This new model was called Time Based Security, it is a dynamic view of cyber security based on protection, detection and response. Today we still use elements of this model, for example the NIST cybersecurity framework includes the TMS model by defining security as Identify, Protect, Detect, Respond and Recover. For the topic of the blog, I focus specifically on the three elements of the TMS model Protect, Detect and React. The idea behind this model is that the formula P> D + R defines a secure system. This formula expresses that the amount of time it takes for a threat actor to break through protection (P) must be greater than the time it takes to (D) detect the intrusion and the time it takes to respond ( R) on this intrusion.

In the days of the Orange Book we were building our fortresses, but also discovered that sooner or later someone found a path across, under or through the wall and we wouldn’t know about it. The TMS model kind of suggests that if we have a very fast detection and response mechanism, we can use a less strong protection mechanism. This is a very convenient idea in a world where systems become increasingly open.

This does not mean that we can throw away our protection strategies, it just means that in addition to protection, we can also implement a good detection and response strategy to compensate for a possible lack of protection. We need a balance, sometimes protection seems the logical answer, in other cases a focus on detection and response makes sense. If we compare this to physical security, it makes sense for a bank to protect a safe from the use of a torch burning through the safe’s wall. Delaying the attack increases the time we have to detect and respond. On the other hand, a counter that faces the public and is therefore relatively unprotected may benefit more from a panic button (detection) to initiate a response.

It is important to realize that detection and response are “sister processes” and that they are in series. If one fails, they both fail. Now with the concept of TMS in mind, let’s move on to a cyber attack on a physical cyber system.

To describe a cyber attack, I use the cyber kill chain defined by Lockheed Martin in combination with a so-called P-F curve. P-F curves are used in the Asset Integrity Management discipline to show equipment health over time to identify the interval between potential failure and functional failure. Since I need a relationship between time and the progress of a cyber attack, I borrowed this technique and created the following diagram.

A p-f curve has two axes, the vertical axis represents the cyber resilience of the cyber-physical system and the horizontal axis represents the progress of the attack over time. The curve shows several important moments in the time of the attack. The P0 point is where it all starts. In order for an attack to happen, we need a particular vulnerability, even more than this, the vulnerability must be exposed – the threat actor must be able to exploit it.

The threat actor who wants to attack the plant gathers information and looks for ways to penetrate the defenses. He / she can find multiple vulnerabilities, but eventually one will be selected and the threat actor will start developing an exploit. This can be malware that collects more information, but also when enough information has been collected, the code can be developed to attack the cyber-physical system (CPS).

To do this, we need a delivery method, sometimes along with a propagator module that penetrates deeper into the system to deliver the payload (the code that is doing the actual attack on the CPS) to one of the servers in the inner network segments of the system. This could be a USB stick containing malware, but it could also be a phishing email or a website infecting a user’s laptop and perhaps installing a keylogger that waits for the user to connect to the CPS from a remote location. It is not necessarily the factory it was initially aimed at, it could also be a supplier organization that builds the system or provides support services. Sensitive knowledge of the plant is stored in many places, different organizations build, maintain and operate the plant together and as such have a shared responsibility for the security of the plant. There are many scenarios here, for my example I propose that the attacker has developed malicious code with the ability to perform a scripted attack on the Basic Process Control System (BPCS). The BPCS is the automation function controlling the production process, if our target would be a chemical plant it will usually be a Distributed Control System (DCS).

The first time security is faced with the attack is on P1, where the chosen delivery method successfully penetrates the system to deliver its payload. Here we have protection mechanisms such as USB protection checks, antivirus protection, firewalls and maybe two-factor authentication against unauthorized logins. But we should also have detection controls, we can warn the moment a USB device is inserted, we can monitor who is logging into the system, we can even inspect what traffic is going through the firewall and see if it is using a suspicious amount of bandwidth , which indicates a download of a file. We can monitor many indicators to check if something suspicious is happening. All of this falls into a category defined as “early detection”, detection at a time when we can still stop the attack by an appropriate response before the attack hits the BPCS.

Nevertheless, our threat actor is skilled enough to slip through this first layer of defense and the malicious code delivers the payload to a server or desktop station of the BPCS. At this point (P2), the malicious code is installed in the BPCS and the threat actor is ready for its final attack (P3). We assume that the threat actor uses a script that describes the steps to attack the BPCS. Perhaps the script is writing digital outputs that shut down a compressor or initiate an override command to bypass a specific safety instrument function (SIF) in the safety system. Many scenarios are possible at this level in the system once the threat actor has gathered information about how the automation features are implemented. Which tag names represent which function, which parameters to change, which authorizations are required, which parameters are displayed, etc.

At this point, the threat actor passed several access controls that protect the inner security zones, passed controls that prevented malicious code from installing on the BPCS servers / desktops, and whitelisted the application controls that prevented execution of code. For detection at this level, we can use different audit events that signal access to BPCS system, we can also use anomaly detection controls that signal abnormal traffic. An important point here is that the attacker has already penetrated very deep into the system, at such a level that the reaction time to stop the attack is almost impossible without an automated response.

The problem with what we call “late detection” is that we need an automated response to be on time when the actor has an immediate malicious intent. Only when the threat actor postpones the attack until a later moment can we find the time to respond “manually”. However, an automated response requires a lot of knowledge of what to do in different situations and requires a high level of confidence in the detection system. A detection system that delivers an abundance of false positives would become a bigger threat to the production system than a threat actor.

If we look at how quickly attacks on cyber-physical systems progress, we can take Ukraine’s attack on the electricity grid as an example. The estimated time per attack here was estimated to be less than 25 minutes. In these 25 minutes – the speed of onset – the threat actors gained access to the SCADA system: opened the circuit breakers; corrupted the firmware of a protocol converter preventing the panel operators from closing the circuit breakers remotely; and wiped the SCADA hard drives. So in our theoretical TMS model, we would need a D + R time of less than 25 minutes to be effective, even if only partially effective, depending on the moment of detection. This is very short.

Can a security operations center fill this role? What would this require? Let us first look at the detection function to find an answer.

First of all we need a reliable detection mechanism, a detection mechanism that causes false positives will soon slow down any reaction. What is required for a reliable detection mechanism:

• In-depth knowledge of the various events that can happen, what is an expected event and what is a suspicious event? This can differ depending on the plant’s modes such as: normal operation, abnormal operation, or emergency. Also system behavior is quite different during a plant stop. All of these conditions can be a source of suspicious events;
• In-depth knowledge of the various protocols in use, for system’s such as DCS including a wide set of proprietary protocols not publicly documented. Also these have shown to be a considerable hurdle for anomaly detection systems;
• In-depth knowledge of process operations, what activities are normal and which not? Should we detect the modification of the travel rate of a valve, is the change of this alarm setting correct, is disabling this process alarm a suspicious event or a normal action to address a transmitter failure? This level of detection is not even attempted today, anomaly detection systems might detect the program download of a controller or PLC, but the process operator would also know this immediately without a warning from the SOC. In all cases close collaboration with process operations is essential, this would slow down the reaction.

How if the security operations center (SOC) is outsourced, what would be different from an “in-company” SOC?

• On the in depth knowledge of the various events that can happen, the outsourced SOC will have a small disadvantage because as an external organization the SOC will not always be aware of the the different plant states. But overall it will perform similarly as an in-company SOC.
• On the in-depth knowledge of the various protocols, the outsourced SOC can have a disadvantage because it will have a bigger learning curve with multiple similar customers. Never the less knowledge of the proprietary protocols would remain a hurdle.
• The in-depth knowledge of process operations is in my opinion fully out of reach for an outsourced SOC. As such control over this would remain primarily a protection task. Production processes are too specific to formulate criteria for normal and suspicious activity.

How about if the threat actor would also target the detection function? Disabling the detection function during the attack will certainly prevent the reaction. In the case of an outsourced SOC we would expect:

• A dedicated connection between SOC and plant. If the connection would be over public connections such as the Internet a denial of service attack would become feasible to disable the detection function;
• A local – in the plant – detection system would still provide information on what is going on if the connection to the SOC fails. The expertise of a SOC might be missing but the detection function would still be available;
• The detection system would be preferably “out of band” because this would make a simultaneously attack on protection and detection systems more difficult.

How about the reaction function what do we need to have a reaction that can ‘outperform” the threat actor?

Reaction is a cyber physical system is not easy because we have two parts, the programmable electronic automation functions that are attacked and the physical system controlled by the automation system. The physical system has completely independent ‘dynamics”, we can’t just stop an automation function and expect the physical process to be stable in all cases. Cooling might be required, or Even during a plant stop we often need certain functions to be active. Even after a black shutdown – ESD 0 / ESD 0.1 – some functions are still required. There are rotating reactors that need to keep rotating to prevent a runaway reaction, a cement kiln needs to rotate otherwise it just breaks under its wait, cooling might be required to prevent a thermal runaway, and we need power for many functions.

Stopping a cyber attack by disconnecting from external networks is also not always a good idea. Some malware “detonates” when it looses its connection with a command and control server and might wipe the hard disk to hide its details. Detection therefore requires a good identification on what is happening and the reaction on this needs to be aligned with the attack but also with the production process. A wrong reaction can easily increase the damage.

While a SOC can often actively contribute to the response to a breach of the security of an information system, this is in my opinion not possible for a cyber-physical system. In a cyber-physical system, the state of the production process is essential, but also which automation functions remain active. In principle, a safety system – if present – can isolate process parts, make the installation pressure-less / de-energized, but even then some functions are required. Process operations therefore play a key role in the decision-making process, it is generally not acceptable to initiate an ESD 0 when a cyber attack occurs. So several preparatory actions are needed to organize a reaction, such as:

• We need a disconnection process describing under what circumstances disconnection is allowed and what alternative processes need to startup when disconnection is activated. Without having this defined and approved before the situation occurs it is difficult to quickly respond;
• We need to have playbooks that document what to do in what situation depending on which automation function is affected. Different scenarios exist we need to be ready and not improvise at the moment the attack occurs, some guidance is required;
• We need to understand how to contain the attack, is it possible to split a redundant automation system in two halves, a not affected part and an affected part;
• We need to understand if we can contain the attack to parts of the plant if multiple “trains” exist in order to limit the overall impact;
• We need to test / train processes to make sure they work when needed.

All this would be documented in the incident handling process where containment and recovery zones, and response strategies are defined to organize an adequate and rapid response. But all this must be in accordance with the state of the production process, possible process reconstitution and the defined / selected recovery strategy.

Process operations plays a key role in the decision making during a cyber attack that impacts the physical system. As such a SOC is primarily advisor and spectator because a knowledge gap. Site knowledge is key here, detailed knowledge of the systems and their automation functions is essential for the reaction process.

Of course we should also think about recovery, but the first focus must be on detection and response, because here we can still limit the impact of the cyber attack if our response is timely and adequate. If we fail at this stage, the problems could be bigger than necessary. An OT SOC differs very much from an IT SOC and has a limited role because of the very specific requirements and unique differences per production process.

So in the early detection zone, a SOC can have a lot of value. In the late detection zone, in the case where the threat actor would act immediately, I think the SOC has a limited role and the site team should take up the incident handling process. But in all cases, with or without a SOC, detection and reaction is a key part of cyber security. It is important to realize that detection and reaction are “sister processes”, even the best detection system is rather useless if there is no proper reaction process that supports it.

There is no relationship between my opinions and references to publications in this blog and the views of my employer in whatever capacity. This blog is written based on my personal opinion and knowledge build up over 42 years of work in this industry. Approximately half of the time working in engineering these automation systems, and half of the time implementing their networks and securing them.

Author: Sinclair Koelemij

OTcybersecurity web site

Introduction

Like the Stuxnet attack in 2010 made the industry aware that targeted cyber-attacks on physical systems are a serious possibility, the TRISIS attack in 2017 targeting a safety system of a chemical plant also caused a considerable rise in awareness of the cyber-threat. Many asset owners started to reevaluate their company policies on how to connect BPCS and SIS systems.

That BPCS and SIS need to exchange information is not so much a discussion, all understand that this is required for most installations. But how to do this has become a debate within and between the asset owner and vendor organizations. The discussions have focused on the theme do we allow an integrated architecture with its advantages, or do we fallback to an interfaced architecture.

I call it a fallback because an interfaced connection between a Basic Process Control System (BPCS) and Safety Instrumented System (SIS) used to be the standard up to approximately 15-20 years ago.

Four main architectures are in use today for exchanging information between BPCS and SIS:

• Isolated / air-gapped – The two systems are not connected and limit their exchange of data through analog hard-wiring between the I/O cards of the safety controller and the I/O cards of the process controller;
• Interfaced – The process controller and the safety controller are connected through a serial interface (could also be an Ethernet interface, but most common is serial);
• Integrated – The BPCS and SIS are connected over a network connection, today the most common architecture;
• Common – Here the BPCS and SIS are no longer independent from each other and either run on a shared platform.

There are variations on above architecture, but above four categories summarize the main differences good enough for this discussion. Although security risk differs, because of differences in exposure,  we cannot say that one architecture is better than another as business requirements and their associated business / mission risk differs. Take for instance a batch plant that produces multiple different products on the same production equipment, such a plant might require changes to its SIS configuration for every new product cycle, where a continuous process of a gas treatment plant or a refinery only incidentally require changes to the safety logic. These constraints do influence decisions on which architecture best meets business requirements. For the chemical, oil & gas, and refining industries the choice is mainly between interfaced or integrated the topic of the blog.

I start the discussion by introducing some basic requirements for the interaction between BPCS and SIS using the IEC 61511 as reference.

Requirements

Following clauses of the IEC 61511 standard are describing the requirements for this separation.

Section 11.2.2:

“Where the SIS is to implement both SIFs and non-SIFs then all the hardware, embedded software and application program that can negatively affect any SIF under normal and fault conditions shall be treated as part of the SIS and comply with the requirements for the highest SIL of the SIFs it can impact.”

Section 11.2.3:

“Where the SIS is to implement SIF of different SIL, then the shared or common hardware and embedded software and application program shall conform to the highest SIL.”

A Safety Integrity Function (SIF) is the function that takes all necessary actions to bring the production system to a safe state when predefined process conditions are exceeded. A SIF can either be manually activated (a button or key on the console of the process operator) or acting when a predefined process limit (trip point) is exceeded, such as for example a high pressure limit. SIL stands for Safety Integrity Limit, it is a measure for process safety risk reduction, a level of performance for the SIF. The higher the SIL level the more criteria need to be met to reduce the probability on failure that the SIF is not able to act when demanded by the process conditions. In general a BPCS can’t meet the higher SIL (SIL 2 and above) and a SIS doesn’t provide all the control functions a BPCS has, so we end up with two systems. One system provides the control functions and one system enforces that the production process stays within a safe state, section 11.2.4 of the standard describes this. For a common architecture this is different because a common architecture has a shared hardware / software platform, therefore such an architecture is generally used for processes with no SIL requirements. Next clauses discuss the separation of the BPCS and SIS functions.

Section 11.2.4:

“If it is intended not to qualify the Basic Process Control System (BPCS) to this standard, then the basic process control system shall be designed to be separate and independent to the extent that the functional integrity of the safety instrumented system is not compromised.”

NOTE 1 Operating information may be exchanged but should not compromise the functional safety of the SIS.

NOTE 2 Devices of the SIS may also be used for functions of the basic process control system if it can be shown that a failure of the basic process control system does not compromise the safety instrumented functions of the safety instrumented system.

So BPCS and SIS should be separate and independent but it is allowed to exchange operating information (process values, status information, alarms) as long as a failure of the BPCS doesn’t compromise the safety instrumented functions. This requirement allows for instance the use process data from the SIS, reducing cost in the number of sensors installed. It also allows valves that combine a control and a safety function. And it allows for initiating safety override commands from the BPCS HMI to temporarily override a safety instrumented function to carry out maintenance activities. But functions that under normal operation fully meet the IEC 61511 requirements, can be maliciously manipulated when taking cybersecurity into account. An important requirement is that BPCS and SIS should be independent, the following sections discuss this:

Section 11.2.9:

The design of the SIS shall take into consideration all aspects of independence and dependence between the SIS and BPCS, and the SIS and other protection layers.

Section 11.2.10:

A device to perform part of a safety instrumented function shall not be used for basic process control purposes, where a failure of that device results in a failure of the basic process control function which causes a demand on the safety instrumented function, unless an analysis has been carried out to confirm that the overall risk is acceptable.

Below diagram shows the various protection layers that need to be independent. In today’s BPCS the supervisory function and control function are generally combined into a single system, as such form a single BPCS protection layer.

Figure 1 – Protection layers (Source: Center for Chemical Process Safety – CCPS)

The yellow parts in the diagram form the functional safety part implemented using what the standard calls Electrical/Electronic/Programmable Electronic Safety- related system (E/E/PES), or to be more specific systems such as: BPCS ( for the control and supervisory functions), SIS ESD ( for the preventative emergency shutdown function), SIS F&G (for the mitigative fire and gas detection function). Apart from these functions fire and alarm panels are used that interface with SIS, MIMIC panels also require interfaces, and public address general alarm systems (PAGA) require an interface. Though PAGA’s generally are hardwired connected, fire and alarm panels often use Modbus TCP to connect. These aren’t the only SIS that exist in a plant, also boiler management systems (BMS) and high integrity pressure protection systems (HIIPS) are preventative safety functions implemented in SIS.

Interfaced and integrated architectures

So far the discussion on the components and functions of the architecture let’s get into more detail to investigate these architectures. There are several reports published describing BPCS ó SIS  architectures. For example the LOGIIC organization published a report in 2018 and ISA published a report in 2017 on the subject. The ISA-TR84.00.09-2017 report ignores architectures with a dedicated isolated network for SIF related traffic. Because the major SIS vendors support and promote this architecture and in my experience it is also the most commonly used architecture my examples will follow architectures defined in the LOGIIC report on SIS. LOGIIC defines following architectures as example of integrated and interfaced structures. (Small modifications are mine)

Figure 2 – Interfaced and integrated architecture principles

In the interfaced architecture there are two networks isolated from each other. The BPCS network generally has some path to the corporate network, while the SIS network remains fully isolated. The exchange of data between the two systems makes use of Modbus RTU where the BPCS is the master and the SIS is the slave. The example shows two serial connections for example using the RS-232C protocol, but a more common alternative is a multi-drop connection using the RS-485 protocol, where a single serial interface on the BPCS side connects with multiple logic solvers (safety controllers).

An integrated architecture has a common network connecting both the BPCS and SIS equipment. Variations exist where separation between a BPCS network segment and SIS network segment is created with a firewall between the two systems (A). Or alternatively a dual-homed OPC server is sometimes used when the communication is making use of the OPC protocol (B). See the diagrams below.

Figure 3 – Some frequently used integrated architectures

Because one of the main targets is the SIS engineering function, architecture C and D were developed. In this case the SIS engineering station can be connected to the SIF network or alternatively an isolated 3rd network segment is created. Different vendors support different architectures, but the four examples above seem to cover most of the installations. Depending on the communication protocol used for the communication between BPCS and SIS there might be a need for a publishing service. This is a function mapping the SIS internal addresses into a format the BPCS understands and uses for its communication.

In the C and D architecture an additional security function can be an integrated firewall in the logic solver restricting access to a configured set of network nodes (servers, stations, controllers) for a defined set of protocols. Above architectures are just some typicals relevant for comparing interfaced and integrated architectures. Actual installations can be more complex because of additional business requirements such as extending the control and SIF networks to a remote location and the use of instrument asset management systems (IAMS) for centrally managing field equipment.

The architectures above do not show a separation between level 1 and level 2 for the process controller and the logic solver. All controllers / logic solvers are shown as connected to level 2, this is not necessarily correct. Some vendors implement a level 1 / level 2 separation using a micro firewall function for each controller with a preset series of policies that limit access to a sub set of approved protocols and allow for throttling traffic to protect against high volumes of traffic. Other vendors use a dedicated level 1 network segment where they can enforce network traffic limitations, and sometimes it is a mix depending on the technology used. There are also vendor implementations with a dedicated firewall between the level 2 and level 1 network segment. Since I want to discuss the differences independent of a specific vendor implementation I ignore these differences and consider them all connected as shown in the diagrams.

Let’s now compare the pros and cons of these architectures.

A comparison between interfaced and serial architectures

For the discussion I assume that the attacker’s objective is to create a functional deviation in the Safety Integrity Function (SIF) to either deviate from the operation intent / design intent (What I call Loss of Required Performance, for example modified logic or trip point) or making the SIS fail when it needs to act. Just tripping the plant can be done in many ways, the more dangerous threat is causing physical damage to the production system.

If we want to compare the security of an interfaced and integrated architecture the first thing is to look at the possibilities to manipulate the BPCS <=> SIS traffic. For the interfaced architecture this traffic passes the serial connection, for the integrated architecture the pink high lighted part of the network passes this traffic.

Figure 4 – BPCS <=> SIS traffic

One potential attack scenario is that the attacker attempts to create a sink hole by intercepting all traffic and dropping it. In an integrated architecture relative simple malware can do this using ARP poisoning as a mechanism to create a Man-In-The-Middle attack (MITM). The malware hijacks the IP address of one or both of the communicating BPCS/SIS nodes and just drops the message. This is possible because level 1 and level 2 traffic are normally in the same Ethernet broadcast domain, so actual messages are exchanged using Ethernet MAC addresses. A malware infection on any of the station and server equipment within the broadcast domain can perform this simple attack. Possible consequences of such a simple attack would be that SIF alarms are lost, SIS status messages are lost, SIS process data is lost, and messages to enforce an override will not arrive at the logic solver.

The noticeability of such an attack is relatively high, but this differs depending on the type of traffic and additional ingenuity added by the attacker. The loss of a SIF alarm would not be detected that quickly, but the loss process data would most likely be immediately detected. So selectively removing traffic can make the attack less noticeable. Also the protocol used is of importance for the success of this attack, a Modbus master (BPCS) would quickly detect if the slave (SIS) would not respond and generate a series of retries and ultimately raises an alarm. Similar detection mechanisms exist when using vendor proprietary protocols. But in those situations where the SIS can initiate traffic independent of the BPCS, than a SIF alarm might get lost if there is not a confirmation from the BPCS is required causing that the process operator might not receive the new alarm.

More ingenious attacks can make use of the MITM mechanism to capture and replay BPCS ó SIS traffic. Replaying process data messages would essentially freeze the operators information, replaying override messages might override or undo an override command. And of course an attacker can overload an operator this way with false alarms and status change messages causing the potential for missing an important alarm. The danger of these two attacks is that the attacker doesn’t need to know that much specific system information to be successful.

Enhancing the attack even more by intercepting and modifying traffic or injecting new messages is another option. In that scenario the attacker can attempt to make “intelligent” malicious modifications. There are various ways to reduce the likelihood of a MITM attack, but the risk remains in the absence of secure communications that authenticate, encrypt, confirms reception of a message, and includes a timestamp to validate if the message was send within a specific time window. If we analyze the protocols used today for communication between BPCS and SIS most don’t offer these functions.

Above MITM scenarios are only possible in an integrated architecture, in an interfaced architecture the communication channel is isolated from the control network by the process controller. An attacker would need physical access to the cabling to perform a similar attack.

Apart from attacking the traffic it is possible to directly attack the logic solver if it would have an exposed software vulnerability. In the integrated architecture the logic solver is always exposed to this threat and only testing, limiting network access to the logic solver, and using software validation techniques can offer protection against this threat.

In an interfaced architecture, the logic solver is no longer exposed to an external connected network but this does not offer a guarantee that such an attack can’t happen. The SIS engineering station might get infected with malware and execute an attack. Also in the Stuxnet attack the target system was on an isolated network, this didn’t stop the attackers to penetrate. But it certainly requires more skills and resources to design such an attack for an interfaced architecture.

In an integrated architecture the SIS engineering station might be exposed. In architecture A above the exposure depends very much on the firewall configuration, but there might be connections required for the antivirus engine, security patches, and in some cases for the publishing function. In architectures C and D the exposure of the SIS engineering station is equivalent to the exposure in an interfaced architecture. However it can still be infected by malware when files are transferred, for example to support this publishing function, or through some supply chain attack. But in general architecture D, where the SIS engineering station is isolated from the SIF traffic reduces the exposure for a number of possible cyber-attacks. Integrated architectures C and D are considered more secure, but how do they compare with an interfaced architecture?

Two main cyber security hazards seem to make a difference:

a)     Manipulation of BPCS <=> SIS traffic;

b)     Exploitation of a logic solver resident software vulnerability;

To mitigate / reduce the risk of scenario a) the industry needs secure communications between the two systems. Protocols used today such as Modbus TCP, Classic OPC, and most of the proprietary protocols don’t offer this.

To mitigate / reduce the risk of scenario b) we would need mechanisms that prevent or detect the insertion of malicious code. These have partially been implemented, but not completely and not by all vendors as we can learn from the TRISIS incident.

So overall we can say that the exposure in an interfaced environment is smaller than the exposure in an integrated environment. Since exposure directly correlates with likelihood we can say that the risk in an interfaced architecture is smaller than the risk in an integrated architecture. But security is not all about risk reduction, in the end the cost of security something must be weight against the benefits the business has for selecting an architecture.

I think there is also a price to pay when the industry moves back to interfaced architectures.

Possible disadvantages of a change to interfaced solutions can be:

·       Reduced communication capabilities, e.g. the bandwidth of a serial connection is lower than the bandwidth of an Ethernet connection;

·       The support for Sequence Of Event (SOE) functionality over serial connections would be hindered by not having a single clock source;

·       The cost of the architecture is higher because the need for additional components and being constrained by cable length restrictions;

·       Sharing instrument readings between SIS and BPCS is limited because of the relatively low speed serial connectivity, this can increase the number of field instruments required;

·       If Instrument Asset Management Systems (IAMS) is used these systems can no longer benefit from the HART pass-through technology and need to use the less secure HAT multiplexers;

·       The cost for engineering and maintaining a Modbus RTU implementation is higher because of the more complex address mapping;

·       Future developments w.r.t. the Industrial Internet Of Things (IIoT) would become more difficult for safety instrumentation related data. For example the new developments around Advanced Physical Layer (APL) technology are hard to combine with an interfaced architecture.

So generally serial connections were seen as a thing of the past and some vendors stopped supporting them on their newer equipment. However the TRISIS attack has put the interfaced architecture very prominent on the table again, specifically in the absence of secure communications between BPCS and SIS.

The cost of moving to an interfaced architecture must be weight against the lost benefits for the business. How high is the residual risk of an integrated architecture compared to the residual risk of an interfaced architecture. Is moving from integrated to interfaced justified by the difference and what are the alternatives?

Author: Sinclair Koelemij

Date: May 9, 2020

Introduction

There has been a lot of discussion on the relationship between cybersecurity and process safety in Industrial Control Systems (ICS). Articles have been published on the topic of safety / cybersecurity convergence, and on the topic to add safety to the cybersecurity triad for ICS. The cybersecurity community practicing securing ICS seems to be divided over this subject. This blog approaches the discussion from the process automation practice in combination with the asset integrity management practice as practiced in the chemical, refining, and oil and gas industry. The principles discussed are common for most automated processes, independent where applied. Let’s start the discussion by defining process / functional safety, asset integrity and cybersecurity before we discuss their relationships and dependencies.

Process safety is the discipline that aims at preventing loss of containment, the prevention of unintentional releases of chemicals, energy, or other potentially dangerous materials that can have a serious effect to the production process, people and the environment. The industry has developed several techniques to analyze process safety for a specific production process. Common methods to analyze the process safety risk are Process Hazard Analysis (PHA), Hazard and Operability study (HAZOP), and Layers Of Protection Analysis (LOPA). The area of interest for cybersecurity is what is called Functional Safety, the part that is implemented using programmable electronic devices. Functional safety implements the automated safety integrity functions (SIF), alarms, permissives and interlocks protecting the production process.

Figure 1 – Protection layers (Source: Center for Chemical Process Safety (CCPS))

Above diagram shows that several automation system components play a role in functional safety, there are two layers implemented by the basic process control system (BPCS), and two layers implemented by the safety instrumented system (SIS): the preventative layer implemented by the emergency shutdown function (ESD) and the mitigative layer implemented by the fire and gas system (F&G). The supervisory and basic control layers of the BPCS are generally implemented in the same system, therefore not considered independent and often shown as a single layer. Interlocks and permissives are implemented in the BPCS, where the safety integrity functions (SIF) are implemented in the SIS (ESD and F&G). Other functional safety functions exist such as the High Integrity Pressure Protection System (HIPPS) and Boiler Management System (BMS). For this discussion it is not important to make a distinction between these process safety functions. Important is too understand that the ESD safety function is responsible to act and return the production system to a safe state when the production process entered a hazardous state, where the F&G safety function acts on detection of smoke and gas and will activate mitigation systems depending on the nature, location, and severity of the detected hazard. This includes such actions as initiating warning alarms for personnel, releasing extinguishants, cutting off the process flow, isolating fuel sources, and venting equipment. The BPCS, ESD and F&G form independent layers, so their functions should not rely on each other but they don’t exist in isolation extending their ability to prevent or mitigate an incident by engaging with other systems.

Asset integrity is defined as the ability of an asset to perform its required function effectively and efficiently whilst protecting health, safety and the environment and the means of ensuring that the people, systems, processes, and resources that deliver integrity are in place, in use and will perform when required over the whole life-cycle of the asset.

Asset integrity includes elements of process safety. In this context, an asset is a process or facility that is involved in the use, storage, manufacturing, handling or transport of chemicals, but also the equipment comprising such a process or facility. Examples of process control assets include pumps, furnaces, tanks, vessels, piping systems, buildings, but also includes the BPCS and SIS among other process automation functions. As soon as a production process is started up the assets are subject to many different damage and degradation mechanisms depending on the type of asset. For electronic programmable components this can be hardware failures, software failures, but today also maliciously caused failures by cyber-attacks. From an asset integrity perspective there are two types of failure modes:

• Loss of required performance;
• Los of ability to perform as required;

The required performance of an asset is the successful functioning (of course while in service) achieving its operational / design intent as part of a larger system or process. For example in the context of a BPCS the control function adheres to the defined operating window, such as sensor ranges, data sampling rates, valve travel rates, etc. The BPCS presents the information correctly to the process operator, measured values accurately represent the actual temperatures, levels, flows, and pressures as present in the physical system. In the context of a SIS it means that the set trip points are correct, that the measured values are correct, and that the application logic acts as intended when triggered.

Loss of ability is not achieving that required performance. An example for a BPCS is loss of view or loss of control, the ability fails where in loss of required performance the ability is present but doesn’t execute correctly. Loss of ability is very much related to availability and loss of required performance to integrity. Never the less I prefer to use the terminology used by the asset integrity discipline because it more clearly defines what is missing.

The simplest definition of cybersecurity is that it is the practice of protecting computers, servers, mobile devices, electronic devices, networks, and data from malicious attacks. Typical malicious cyber-attacks are gaining unauthorized access into systems, and distributing malicious software.

In the context of ICS we often talk about Operational Technology (OT), which is defined as the hardware and software dedicated to detecting and / or causing changes in a production processes through monitoring and/or control of physical devices such as sensors, valves, pumps, etc. This is a somewhat limited definition because process automation systems contain other functions such as for example dedicated safety, diagnostic and analyzing functions.

The term OT was introduced by the IT community to differentiate between cybersecurity disciplines that protect these OT systems and those that protect the IT (corporate network) systems. There are various differences between IT and OT systems that justified to create this new category, though there is also significant overlap frequently confusing the discussion. In this document the OT systems are considered the process automation systems, the various embedded devices such as process and safety controllers, network components, computer servers running applications and stations for operators and engineers to interface with these process automation functions.

The IT security discipline defined a triad based on confidentiality, integrity, availability (CIA) as a model highlighting the three core security objectives in an information system. For OT systems ISA extended the number of security objectives by introducing 7 foundational requirement categories (Identification and authentication control, Use control, System integrity, Data confidentiality, Restricted data flow, Timely response to events, Resource availability) to group the requirements.

Now we have defined the three main disciplines I like to discuss the topic if we need to extend the triad with a fourth element safety.

Is safety a cybersecurity objective?

Based upon the above introduction to the three disciplines and taking asset integrity as the leading discipline for plant maintenance organizations, we can define three cybersecurity objectives for process automation systems:

• Protection against loss of Required Performance;
• Protection against Loss of Ability;
• And protection against Loss of Confidentiality.

If we have established these three objectives we also established functional safety. Not necessarily process safety because this also depends non-functional safety elements. But these non-functional safety elements are not vulnerable to cyber attacks other than potentially revealing these elements through loss of confidentiality. Based upon all information on the TRISIS attack against a Triconex safety system, I believe all elements in this attack have been covered. The loss of confidentiality can be linked to the 2014 attack that is suggested to be the source of the disclosure of the application logic. The aim of the attack was most likely causing a loss of required performance by modifying the application logic that is part of the SIF. The loss of ability, the crash of the safety controller, was not the objective but an “unfortunate” consequence of an error in the malicious software.

Functional safety is established by the BPCS and SIS functionality, the combination of interlocks, permissives, and the safety integrity functions contribute to overall process safety. Cybersecurity contributes indirectly to functional safety by maintaining above three objectives. Loss of required performance and loss of ability would have a direct consequence to the process safety, loss of confidentiality might lead over time to the exposure of a vulnerability or contribute to the design of the attack. Required performance is overlapping what a process engineer would call operability, operability also includes safety so also from this angle nothing is missing in the three objectives.

Based upon above reasoning I don’t see a justification for adding safety to the triad, the issue is more that the availability and integrity objectives of the triad should be aligned with the terminology used by the asset integrity discipline to include safety. Which would make OT cybersecurity different from IT cybersecurity.

Author: Sinclair Koelemij

Date: April 2020