Two words FMEA and FTA

Well acronyms really

Transport Layer Security - Part 1

Part of the Security Layer Series

Layer 4 is where the rubber meets the road as far as actual connectivity to the applications and logic of the controllers.

Layer 4 is the transport layer and for IP it typically means either TCP (Transmission control protocol) or UDP (User Datagram protocol).

I mentioned earlier that IP is inherently not deterministic and that has implications for automated control. Layer four is the first place where the compensations for this occur.

A quick run through of how TCP works will help some. I am going to grossly oversimplify here so if someone wants to correct or provide more detail feel free.

TCP establishes a session to ensure data delivery. A host initiates the communication by sending a TCP/SYN packet. The recipient of the SYN responds with a SYN/ACK with session identification information and the original host responds with an ACK/ACK establishing the session. Periodically during the communication stream the acknowledge process is repeated to ensure the communication is maintained. Checksums are included as an inherent part of the protocol. Time sent between packets received is monitored to determine if a session is lost and to initiate reestablishment of the communication stream.

What this means in a nutshell is that TCP has many mechanisms built into it that compensate (in part) for the issues introduced by the fact that IP is non deterministic. It doesn’t by any stretch of the imagination mean that TCP itself is secure in any way. There are many ways to game the system and hackers and worms use them to their full advantage. If you really want to get into the details take a look at NMAP and the lists at www.Insecure.org .

The most common one and the one I have seen cause issues on PLC’s is the syn scan. It basically works by opening up a listening port then streaming syn’s to all of the selected ports on every address that is to be inspected. Everything that responds with a syn/ack is logged. The connection is never completed with an ackack. This is where the problem is (especially for controllers with older IP stacks). The receiving host uses some resources to sit there waiting for that ack/ack. There are DoS attacks related to this but for the most part they are not that effective for newer IP stacks. (Syn floods can still cause headaches though) Unfortunately PLC’s do not always have newer stacks so they are often particularly vulnerable to this.

Aside:

This is directly relevant to the scanning discussions that have occurred with some level of passion on this blog’s comments and in the background via email. My advice here if you plan on scanning a scada system for the first time and you have done the change management it is best to start with a TCP connect scan that exits gracefully as your initial connection enumeration method. Limit the scan to a few interesting ports and don’t hit all 65k (at first at least). I wouldn’t even do fast scan ports. After you have a few under the belt for that address range then slowly expand. Do the fast scan ports then if wanted the whole 65k. After you are comfortable with this make sure you have people watching the equipment and have a recovery plan then try the syn scans. Once you have gotten past this point you can go on to the rest of your vulnerability assessment or pen test. I know this is insanely conservative for most Security professionals but the critics are not exaggerating when they want that bad things can (and will) happen. I am an advocate for scanning systems and have done so many times without significant issue on Rockwell/ABB, Honeywell, Siemens, and other vendor control systems but there is always a risk. My typical response to the DON”T SCAN crowd is “Sooner or later the systems are going to be hit by an actual attack or something that is functionally identical to one so wouldn’t you rather that happen in a controlled manner?”.

End of Aside

Many PLC vendors use TCP as their primary IP communication method to their controllers and all of them use it for their historians, MES, and control aggregation systems. I have seen a bit of an explosion in HTTP access to endpoints and I have mentioned ModBusIP in earlier posts in this series. I am not going to go into detail on what ports are used here. If you want to find out ask your vendor they will tell you. What you should do however is make sure that is possible you block access to the TCP port used as the primary PLC communication protocol at the point closest to the controllers as possible. ACL’s are acceptable if actual firewalls are not available. For vendors that use standard ports such as telnet, http, or RPC this can be somewhat more difficult to do. Take advantage of point to point and point to multipoint (subnet) rules. The key here is to not allow access to the PLC’s from an uncontrolled network. Access to the Historians and central control systems should be controlled primarily on a white list basis. For really large engagements such as regional operation centers it is often possible to isolate both the central and the local subnets and connect them via VPN tunnels. If you are doing this it is best to isolate remote sites from each other.

Enough for today

Rest of TCP and UDP continued later.

Tenable SCADA webcast

This got delayed last week but is up now.

I haven't had the time to view it yet. If anyone gets to it before I do I'll be happy to move what they say up from comments.


Update:
The link wasn't working earlier but I believe it is now fixed.

Nessus - SCADA Plugins

Tenable just plugged their SCADA plugins that DigitalBond helped develop.

Your systems will eventually be scanned. If you do it yourself and start with Passive scanning then move with proper MOC to active scanning and remediation you will be ready for the ones you don't control, plan or know about.

Boneheaded Mistake

I made a boneheaded mistake by making a statement with too little data on my PVS post. I have corrected it on the blog but the feed will still be wrong for some people.

No Excuses

You can read the corrected and updated post here.

Passive Vulnerability Scanning - SCADA

I stand behind the rest of the items on the post.

Passive Vulnerability Scanning (PVS) - SCADA

Update:
I made a huge mistake in this post. N-Circle does not offer passive vulnerability scanning. Instead of changeing the content (which I don't believe in for other than minor spelling and gramatical items) I am leaving this at both ends of the Post. Once more N-Circle does not offer PVS. Tenable does and Mu is sort of a varient.

Ok I have confession to make.

I have been ignoring an entire dimension of the SCADA scanning discussion.

I have ignored it for two reasons.

1. It just isn't that fun because it doesn't generate as much of a conundrum to argue about.
2. The thread started with a discussion of active and I wanted to stress the overall importance of scanning vs not, even if it means some risk.

Passive Vulnerability Scanning (PVS) is essentially sniffing network traffic and using known characteristics to identify systems that are likely to have vulnerabilities.

The guru's will probably jump all over me here but it is basically matching network traffic signatures to likely OS's, Patch levels, and applications and then linking that data to vulnerability information.

It isn't that new. I was following some Snort items similar to this a few years ago. They were comparing the snort rules hits against know characteristics from a variant of NMAP's fingerprints (or maybe it was QUESO... Whatever) and using that to passively identify OS and patch level.

The key here (and why it is germane to the SCADA Scanning discussion) is that passive vulnerability detection does not require you to touch the vulnerable system at all. This means that there is no realistic chance of causing a problem on it. You set up a mirror port and sniff the traffic. That is the only impact to the PCN.

Let's be clear here there are some weaknesses to doing it this way.

• You only see systems who's traffic passes the monitored port so you can miss a lot depending on where you locate the sniff point.
• There is only so much data you can acquire about a system based on watching its traffic and not interacting with it (though admittedly this is a surprisingly large amount).
• It will have false positives (though there are mechanisms to weed those out)
• It is not nearly as effective at identification and verification actual vulnerabilities.

But it is nearly risk free.

If you work in a shop that realistically has no chance of using a scan to identify your weaknesses this is a very good option.

It is also a good option as a preliminary investigation method before doing more intrusive actions.

Those actions still need to happen (as a matter of fact they probably already are without your knowledge) but this can buffer both the political and real risk.

There are a couple of companies doing this and I have already mentioned three.

N-Circle has been doing PVS for a while and integrates it neatly with the ability to match it against what active attacks are occurring. - Not True

Both MUSecurity and Tenable are developing signatures specific to SCADA.

The differentiators in this field is likely to be the quality and quantity of the matching, the collation of their library to your specific needs, and ease of monitoring/management.

Tenable probably has a lead in the SCADA side of this because of their engagement with Dale at Digitalbond but N-Circle is leading overall. - I'll stand by the first statment I was flat out wrong on the second

Update:
I made a huge mistake in this post. N-Circle does not offer passive vulnerability scanning. Instead of changeing the content (which I don't believe in for other than minor spelling and gramatical items) I am leaving this at both ends of the Post. Once more N-Circle does not offer PVS. Tenable does and Mu is sort of a varient.

Google Data - Google Desktop - Scanning SCADA

Digg This

There has been a fair amount of traffic regarding the security issues of applications like the Google desktop on the IT side in the last several months.

Within the ACS SCADA world (Update to fix spelling) you should consider the implications of the desktops but also the Google appliances. These devices are being installed by many organizations to simplify everything from Intranet Website development to E-discovery. Like most Google products they are very good at what they do.

The thing to keep in mind with them is that they are web crawlers on steroids. They don't just hit HTTP they also chase down many other file sharing and transfer mechanisms. Look at the databases they crawl as well. They will find Windows shares. They follow links and scan address ranges to index and cache data. They can be configured to limit the extent of the scan but in many cases this is haphazardly done.

Many PLC's have http interfaces now and all of the Historians I know about have some flavor of Db.

This takes on particular concern when placed in context of our recent discussions on the possible impacts of scanning.

Scanning Vs Not Scanning

More Scanning - Be careful

Ramifications of Scanning

and keep this in mind when considering what Securosis had to say.

The good news is that the vendors are getting better at designing these interfaces to be resilient.

Digg This

Digital Bond -Scanning ACS or SCADA networks

Dale has apparently been following some of the other traffic. He has his post up on scanning.

He has been hot on this for a while and has been working with Tenable on a series of plugins.

I am a bit more conservative but the white paper is pretty good.

I would work with the engineers to check against test systems first. Even in small shops there is usually one or more devices that they do their testing on before working on any others. In larger facilities sometimes it is a complete mock up (either virtural or real) of the entire system.

Then move on to the redundant/backup systems next. (often they are the same as the test systems perhaps that is why Dale didn't differentiate)

Two big items he didn't stress.

Document Document Document

and

Tell everyone exactly what you are doing and when.

Start with light scans

I use NMAP TCP connect with the fast scan to start
Add port 502, 2222 and 44818 into the list for the fast scan. (there are others dependant on your vendors)
If you are not specifically defining the timing chose the polite scan. (it really doesn't matter that much if you are only hitting a few systems)
It is almost never a good idea to scan entire subnets unless you have already successfully done dozens of other checks and know the network can handle it.

I think I will continue this when I have the time.

.

Ramifications of Scanning

Steve has some thoughts on the un intended consequences of scanning failures.


I still think it is worth the risk.

More Scanning - Be Careful

Soon it won't matter if we crash our systems in a controlled scan.

With more and more scanners rolling out SCADA modules it is more and more likely that your systems will get scanned without the operators and engineers knowing it is happening.

This would be very bad. MUSecurity is more likely be used by legitimate operators (as opposed to Nessus or metasploit with lower barriers to acquisition and use) but IT operators should be very careful about how they use their new tools.

Despite the fact that I advocate the proper use of scanning I am very conservative about how to go about doing it. Thorough pre testing and comprehensive change control are absolutely essential.

CNI Operator is right PLC's, Historians and other SCADA/ACS/DCS sub-systems respond very unpredictably to the most basic connections. There are different design criteria for the endpoints in ACS than in the normal IT world. As I have said before in many ways they are lagging the IT world by several years in terms of some forms of connectivity. These facts often make them far more sensitive to unanticipated connections and packets than other systems would be. They do indeed crash with stimuli that would be harmless to almost anything else.

NMAP can be a DOS tool for SCADA systems.

In the long run this makes it more important to scan but if you are in IT beware how you go about it. These are not the types of systems where an apology can absolve you after a screw up. It won't be a funny story to tell later.

Also don't fool yourself into believing everything is ok just because the scan doesn't find anything. It is only one piece of the puzzle.


Update:
Background

but

Ultimately Rich is right. Just be careful.

Digg it

Scanning

Awesome scanning post