30 November, 2006
He has been hot on this for a while and has been working with Tenable on a series of plugins.
I am a bit more conservative but the white paper is pretty good.
I would work with the engineers to check against test systems first. Even in small shops there is usually one or more devices that they do their testing on before working on any others. In larger facilities sometimes it is a complete mock up (either virtural or real) of the entire system.
Then move on to the redundant/backup systems next. (often they are the same as the test systems perhaps that is why Dale didn't differentiate)
Two big items he didn't stress.
Document Document Document
Tell everyone exactly what you are doing and when.
Start with light scans
I use NMAP TCP connect with the fast scan to start
Add port 502, 2222 and 44818 into the list for the fast scan. (there are others dependant on your vendors)
If you are not specifically defining the timing chose the polite scan. (it really doesn't matter that much if you are only hitting a few systems)
It is almost never a good idea to scan entire subnets unless you have already successfully done dozens of other checks and know the network can handle it.
I think I will continue this when I have the time.
29 November, 2006
Can you believe that?
Better make a new law now?!?!?!
Everyone run for your bomb shelters!!!! Cats and dogs living together!!! Oh the Humanity!!!!
Where did you get that radioactive poison Timmy? uhh... On the internet?
These articles are irritating me. The authors have no clue and are obviously too stupid to ask the right people the right question.
First of all Polonium is an alpha emitter. That basically means you have to have it in you for it to cause any problem at all.
Second the articles state that the amount available for sale is “very small”. That is the understatement of the century. I could inject 100 of the $69 samples into my jugular and if the blood loss didn’t kill me I wouldn’t have to worry about my 10% increase in the chance of cancer for another 60 years.
Ok that is an exaggeration but if they can do it so can I.
Here are the real facts.
Po-210 ‘s primary emmiter is a 5.3MeV alpha
With a penetration range of about 4 cm of normal sea level air.
It has a biological half life (the amount of time it takes to get it out of the body if ingested) of about 50 days.
The largest organ dose in that time is 100 REM to the Spleen by 1 uCi. This is bad (real bad) but not deadly at this level.
Ouch my SPLEEN!!!
Other organ exposures are an order of magnitude lower.
It would take 10 times that amount to have significant body effects (hence my 100 samples) much more (I don't know how much) to even make death likely.
It takes about 500 to 1000 REM in a relatively short period (less than a month) of time for radiation exposure to be lethal or to cause widespread organ failure. (The real nukes will note that I am fudging a bit here but it is close enough. Feel free to correct me in the comments.)
These sites are selling .1uCi in the $70 dollar range.
The Russian that got poisoned probably had to get more (well more) than 1000 times that amount for it to affect him as completely and quickly as it did.
If you had the $70 grand to blow and didn’t want to spend it on some other way to get rid of someone I suppose in a real stretch it might be possible once. Of course I imagine if you ordered a thousand samples they wouldn’t have them available (probably not all of the places that sell this added together have that much) and my guess is someone would ask why and probably inform the police. It would also be pretty trivial to track someone down who did this with a legal source.
Unless a government did it.
So why was it used?
They obviously had the money.
Even though it cost $70,000 it isn’t very big (half the size of a pin head Update: sorry pin tip. Really really small, dust).
Since it is an alpha emitter it won’t be detectible from a distance.
Once it is done it is done. There is no real way to recover.
Don't get me wrong this is pretty nasty stuff and you couldn't pay me enough to actually mainline it like I joked above but people really need to get a clue when they go implying that being able to purchase a tenth of a microcurie for 70 bucks is a problem. After all your typical smoke detector has 9 times that many microcuries of Americium.
They aren't selling grams, milligrams or even nangrams here this is picograms.
Do you want to ban them? Or make everyone register with the government to purchase them?
If you do we should start worrying about the Dirty Banana Bomb.
I like the fact that I can swap between German and English for your posts. Doubles the work for you I imagine.
Part 1 Here
More and more control systems, historians and actual control devices are adopting standard readily available operating systems, communication protocols and connection mechanisms. This subjects control systems to the same threats that plague other IT systems. It also gives them some significant advantages in both dealing with the threats and providing service. The rate of this occuring is also accelerating.
One of the largest items that conserns me about this fact is the reduced cycle times of deployments. Older Control Systems could litterally go for decades and work fine. In Jake's comment to my IT vs Control Engineer challenge he points out the breakneck speed that IT systems have to respond to new threats. In pure info security circles we have moved from hacks to worms to zero days now to less than zero day threats. (This was an interesting thread to watch develop it starts here.)
As more moves (and it is going to move wether we want it to or not) this is just going to get more pronounced.
I recently participated in an email go around about the lack of support for NT4 with a few industry heavyweights and how we communicate the risk this entails to the ACS community as a whole. One of them liked this article from 2001 about NT4 SP7.
Hell in the IT world companies are born, grow, go through a mid life crisis and either go out of business or a gobbled up and disassembled in a quarter of the time that most engineers expect their overall plant control system to last.
How many out there still have VAX, DEC and Compaq? How about IP21 systems? The list goes on.
They work fine the problem is that if you want to buy a new system you have to get Microsoft, Linux, or AIX as the OS. (yes I know the PLC's are different, I am mostly talking about the historians and control stations here, it still matters)
There really is very little choice. This means you have to be thinking about what you are going to do when Vista has been out for 4 years and MS (rightly) refuses to support 2003 let alone 2000.
There are a lot of implications to compressing the cycle time from 50 years to 20, to 10 and then to 5 or less. I think this is the biggest fact we have to prepare for but certainly not the only.
28 November, 2006
The first shot was fired by a Real Engineer (notice the lack of quotes)
Any IT "Professionals" care to respond (quotes still there:-)
The last few DCS posts and the comments related to them have brought out an old debate. Really it is my #1 Myth
Process Control systems are different from normal IT systems.
Process Control systems are the same as normal IT systems.
(two sides of the same misunderstanding)
I want to try something new here. It may not work because I am not sure I have a critical mass of readers but it just might.
I think it will work because I have a very senior group of readers from both sides of the isle.
So here goes Open thread Tuesday.
Tell me your worst stories of “IT folk” mucking in your network and insane security mechanisms implemented by “Real” engineers.
Keep it anonymous if you want to protect the guilty.
Tech Paper here.
Garrett has been doing this for a while and their products are pretty solid.
I like the way they are integrating the SSL and Firewall. I haven't seen them in action but would like to see more and hear from someone who has.
Jake Brodsky brought out some good points against the use of strictly DNS.
First, it is yet another service vulnerable to attack.
Second, despite redundancies and the like, it still represents an additional point of failure.
Third, DCS networks aren't so large that one is not able to use a host file effectively.
Fourth, the use of host files does not preclude the use of a DNS. One can configure nslookup to try host file addresses first, and then if not found, to try a DNS.
Fifth, the use of raw IP addresses can avoid the use of the nslookup latencies altogether. Thus, even if the name service infrastructure is compromized, you can still have other functional bits of software.
All of these are true and I'll add another.
In general Control systems are more static than a normal IT environment so there is less or no need for a dynamic naming environment. Where an IT server might go for months or even years without changing its address a control system is sometimes intended to go fo decades. (though I suspect this time cycle will shorten if it hasn't already)
Now it is time to flip my hat. In retrospect the points I emphasized to critisize normal naming practices were the wrong ones.
So to be more specific:
Many (most?) control systems I have looked at not only had host files, but the host files on machine A were different from the ones on machine B even though B was a backup for A. Futhermore often the names no longer match to existing IP addresses. There are certainly valid reasons for even these items to sometimes to occur but it always looked pretty sloppy to me. The same statement with some changes apply do DNS settings and direct IP entry. Honestly this is probably more of a problem with general maintenance and attention to detail than one specific to naming. So if you are an engineer perhaps it would be worth it to add an anunual maintenance item to rationalize the naming environment for your control environment.
Control Systems engineers certainly don't have a monopoly in this area. They do seem to be more inclined to it in the naming area whereas IT folk are more inclined to sloppiness in the documentation area and others. Perhaps this is a function of the Vets vs. Doctors statement that Ron mentioned in his comment.
27 November, 2006
I don’t usually put much stock in what a government “expert” says about information security but this article outlined a flaw that exists in many DCS architectures.
In most of the control systems I have seen the naming architecture is horrible. Host files are still commonly used, direct entry of IP addresses is common and in some hodge podge systems a combination of Host files, IP addresses, and DNS lookups happen. This is an area that is ripe for improvement in the SCADA arena.
Mike Murry of Episteme is starting a series of Teleseminars and podcasts relating to IT careers and job skills.
His next episode is coming up tomorrow.
The second episode of the Episteme IT/InfoSec Career Portfolio Teleseminar & Podcast Series will feature my brilliant colleague and friend Tim Keanini (aka TK) of nCircle Network Security.
In moving up the corporate laddder, he has managed to stay an incredible technologist, as well as maintaining his passion for technology
TK has an incredible ability to use "synthetic thinking" - he is as likely to pull a great technical idea out of a book on sociology or history as out of a technical book. He can use ideas from anywhere to start up his creative engine.
He has an incredible network of people around him - it's impossible not to love TK, and so he makes friends where-ever he goes. It has created a group of incredibly smart people around him who are available to help whenever he needs it.
It should be pretty interesting. I think it might be of particular value if you are currently in the military and about to get out or have just recently left the military.
You might want to tune in.
It is a bit condesending but all the facts seem straight.
One item in contrast. He mentions that very few Connections are via IP. That is certainly true but the number is getting larger. Also despite the fact that most connections are via RS 232 or other mechanisms there is usually an interface system now that is almost always IP connected. Often this is the historian but there are others as well. Rockwells RSLinx is an example. Most vendors have them.
26 November, 2006
On a strategic level the superiority of Nuclear vessels becomes even more pronounced. Even with the 212 long distance stealth is not realistically possible. At least in terms of true strategic distance. It is certainly possible for diesel boats to snorkel but every time they start they are far more subject to being located. They don't have the safety margins for under ice operations and are unable to linger. Much of the strategic value in submarine operations is related to uncertainty. The longer the period of time between last contact and the current time the less certainty there is regarding what areas are threatened or being monitored. Again the Nukes have the clear advantage. On a tactical level nukes have a slight advantage but on a strategic level the competition is not even close.
The effective range of a Nuclear vessel is unlimited. Within a week or so they can be anywhere in the oceans or seas from anywhere. They can do this surprisingly quickly. Diesel boats top speed is significantly slower regardless of their mode of operation. At most a diesels range even at slower speeds is a few thousand miles. This may sound like a lot but considering they will have to refuel at the end it is a significant limitation. There is also a significant logistic advantage to this. There is less or no supply chain to protect. Less risk to re-provision vessels and less opportunities for hostiles to harm support vessels. Again this measure goes hands down to the nukes.
Range and endurance are related but not the same. Staying time on station for a properly provisioned nuclear vessel could literally be months. The advantage this provides has significant impacts to the entire planning, logistics and cost structure of the Navy as a whole. Intelligence gathering missions are more effective and misdirection is easier. Area monitoring and denial are easier to facilitate and more effective. Once more this item goes hands down to the nuclear vessels.
In the long range speed category Nuclear vessels are several orders of magnitude superior to the best diesel vessels. Week long flank runs are possible. Diesels that do flank speed for more than a few days would have to be re-provisioned leading to the limitations mentioned above. In addition to this advantage although I don't know for absolute certain I think that the top speed for nuke vessels is likely to be quite a bit higher than diesel boats.
Area control is a function of all of the items that have already been discussed both in tactics and strategy. In the Navy as a whole a sub is much less effective at this than carriers but they do have one thing going for them. Uncertainty is the subs friend in this area. By stealthily maneuvering and positioning a significant amount of ocean can be denied to the enemy. They also are able to identify and locate objects that may not be easily located by surface vessels. If the submarine is unable to move relatively quickly or opposing forces are able to periodically locate it the entire advantage is lost. It would take several Diesel boats to effectively patrol the same area as a nuclear vessel.
One of the biggest items that advocates of diesel boats attest is their ability to operate in shallow water. Right now it is true that this is an area that nuclear boats are challenged. This is not due to any inherent weaknesses in nuclear power however this is primarily a function of the size of the vessels. It is true that the need for water flow for cooling can be problematic but designing baffles and alternate cooling mechanisms is a far less intimidating proposition than the challenges of designing mechanisms to overcome the weaknesses of Diesel boats.
Overall on a strategic level Nuclear vessels are probably 10 to 20 times as effective as a diesel boat.
Economic arguments are perhaps the most persuasive points in favor of diesel boats over Nuclear subs. I don't know what the current class construction costs are but more than 2 or 3 Billion USD per vessel would not surprise me in the least. It is unlikely that the newest diesel boats break 500 Million USD. This seems to be a very significant difference but as bubblehead points out in this post (more here) there are a number of items that would mean that the construction costs for US boats would be higher even if they are identical vessels. Furthermore if the designs are for smaller and simpler boats nuclear vessels could be cheaper.
Cost of operation is harder to determine. If the same operational tempo and deployment requirements are applied then nuclear boats are likely to be much less expensive. They don't consume nearly as much fuel (the still have a Diesel but it is only used in emergencies and drills). An additional operational cost that isn't often identified for diesel boats is the cost of the cost of the support structure to supply them away from home. More strain will be placed on oilers. Since subs often operate in different areas than surface vessels new oilers are likely to have to be bought. Over a 20 to 30 year lifespan these differences are likely to be pronounced. The need to refuel and the cost of a refueling overhaul probably evens this out however. Decommissioning of nuclear vessels is undoubtedly more expensive and probably costs nearly as much as the construction costs. This one is clearly in favor of the Diesels. Overall Nukes are probably 2 to 3 times as expensive as a diesel sub but since on a tactical level a nuke is worth 3 or more diesels and since on a strategic level they are worth 10 time or more that cost is well justified.
There are many different efforts to make diesel boats more effective. This is great sooner or later something will surpass the nukes. one of these efforts is the 212.
The 212 is a new U boat class that the German navy is building. It has some brilliant innovations that enable it to operate for extended periods of time. Most of the articles I have read place the operational time for the 212 at up to three weeks of submerged operations. This is amazing for a non nuclear vessel (even though it is an order of magnitude less than nukes). I very seriously doubt that it can do 2/3 bell or even the equivalent speed as a nuke boats 1/3 for three weeks however. As brilliant as the 212's use of fuel cells is they are still far away from true nuclear capabilities.
This brings me to my crazy idea. Combine the capabilities of either a hydrogen/oxygen fuel cell or perhaps better yet hydrogen peroxide fuel cell with decay based RTG's. The RTG's could probably be placed external to the hull simplifying cooling requirements. I doubt it is likely that you could ever design an RTG based system that on its own could achieve greater than 4 or 5 knots because their power density and power to weight ration are pretty crappy but you could use them for slow speeds and to "recharge" the fuel cell storage. I'm sure this could get close to fission overall but it might permit something that is in between and might provide a location to get rid of some of the fission "waste" that we currently are unable (by treaty) to reprocess.
Like I said Crazy idea.
25 November, 2006
There are a few items that make a Submarine an effective war fighting asset.
Cost of Construction
Cost of Operation
Cost of Decommissioning
So lets start with the tactical items
Stealth is really the main item that separates the submarine capabilities from other war machines. One of the main things that Diesel advocates espouse is that they are quieter than Nuclear Subs. This is certainly true of older SSN's but even then it is only applicable when the Diesel is running on electrical power. The newer SSN's are so much quieter than the older that there is little difference. Without going into t0o many specifics the primary consideration for stealth is a function of two items when comparing the tactical capabilities of two boat. The relative detectability range of each boat in comparison with sensor capabilities of each is the primary consideration. Both Nuclear and Diesel boats have detectability ranges that are comparable or at least so close that it makes little difference. The nuclear vessels have one significant advantage. They are able to maintain a relatively high consistent speed and energy output (within the quiet operation ranges) effectively indefinitely. The Diesel subs on the other hand can only operate for a finite period of time before they have to recharge their batteries. There is one exception to this the 212 which I will address later. This is a function of the clear advantage that nukes have in terms of endurance. Engagements between Diesels and Nukes become a game of the Nukes moving at a relatively high quiet speed (still well below the speed that they are easily detectable) while gathering a large area of data. They are able to move relatively freely when trying to gain information for solutions. If they get short but uncertain hits they have the energy available to develop more detailed data. If they feel it is advantageous they can move away and reengage from a more opportune angle without have to factor in how much time they have left. A standard diesel's range under battery is extremely limited. They have to husband it and carefully choose when and where to expend their power. This is a significant tactical advantage and a reason that despite claims to the contrary, nuclear powered vessels have the clear superiority in this arena. In engagements involving multiple hostiles this advantage is dramatically more important. In all of the games I remember being in with Diesel subs the nuke had the clear advantage and always came out on top. There were only a few exceptions to this. In an artificially tightly constricted operational area most of the nukes advantages are mitigated because they are unable to move around freely. Many people claim that this is a significant mitigate in terms of shallow water (more on this later) and harbor operations. The reality is that in a real engagement, control of area is what is important, and that will leave more than enough leeway for the nukes endurance based stealth advantages to play out on a tactical level. There was one set of games with the Brits that it was clearly a function of them having a genius for a CO. He used dozens of brilliant moves but even then it basically came to a draw after about 10 engagements over a week(I don't think the Brits have Diesels anymore). Every other hide and seek I can remember the nuke easily won.
The next significant tactical measure is speed. I don't have to spend much time on this one. The nukes easily and demonstrably win. There is no comparison.
On the surface the detection capability would seem to be even. It is primarily a function of what equipment on board. There are slight advantages in detection capability to the quieter sub but the effect only plays out for significant differences. There is one way in which the nukes have an advantage in this arena. With a nearly unlimited energy budget designers don't have to limit their choices in terms of numbers or energy use. Likewise Captains don't have to worry about energy budget in operations. This is a significant advantage. More and better equipment can be installed. Redundant operators can monitor equipment in multiple modes simultaneously increasing the likelihood of proper interpretation of gathered data. Overall this area is mostly influenced by the quality of equipment available but even here nukes have the advantage in terms of design and operability flexibility.
Weapon systems have a lot in common with with sensor capabilities in that they are largely dependant on what systems can be chosen. In this case what can be chosen is mostly independant of the power source.
Overall Nukes have a 3 to one or more advantage tactically.
This post has taken more time than I expected so I am going to break it into parts.
I'll get strategic, cost and my crazy idea later.
23 November, 2006
Did I mention that a 17 year old made fusion in has basement? In Detroit.
22 November, 2006
In any case Rich ( a different rich from the last two I quoted) from Tao Security recently linked to his reviews of the newest Hacking Exposed.
If you are new to information security then I highly recommend getting and reading the books in this series. They are easy to read, well written, detailed and their observations and facts have had an uncanny ability to remain accurate in a changing world.
21 November, 2006
With more and more scanners rolling out SCADA modules it is more and more likely that your systems will get scanned without the operators and engineers knowing it is happening.
This would be very bad. MUSecurity is more likely be used by legitimate operators (as opposed to Nessus or metasploit with lower barriers to acquisition and use) but IT operators should be very careful about how they use their new tools.
Despite the fact that I advocate the proper use of scanning I am very conservative about how to go about doing it. Thorough pre testing and comprehensive change control are absolutely essential.
CNI Operator is right PLC's, Historians and other SCADA/ACS/DCS sub-systems respond very unpredictably to the most basic connections. There are different design criteria for the endpoints in ACS than in the normal IT world. As I have said before in many ways they are lagging the IT world by several years in terms of some forms of connectivity. These facts often make them far more sensitive to unanticipated connections and packets than other systems would be. They do indeed crash with stimuli that would be harmless to almost anything else.
NMAP can be a DOS tool for SCADA systems.
In the long run this makes it more important to scan but if you are in IT beware how you go about it. These are not the types of systems where an apology can absolve you after a screw up. It won't be a funny story to tell later.
Also don't fool yourself into believing everything is ok just because the scan doesn't find anything. It is only one piece of the puzzle.
Ultimately Rich is right. Just be careful.
It is hard to tell from the article if this is for their business systems or for their operational ones. The system as described on the PDF's on the Indus site shows primarily supply and service chain optimization but also implies integration to the operation systems at at least a basic level. Either way it shows that these organizations are just like many others in that they are finding value in the interconnectivity that is increasingly available.
This stuff already happens more often than you might think even on the ACS side. Obviously there are some significant security conserns but it should be possible for it to be an improvment overall if it is done properly. Yet another area of convergence.
20 November, 2006
CNI operator said...
Jim, you already know my views on this!My view on scanning was re-enforced when I completly wiped a vendors PLC level device during a test in their lab.Before scanning, I'd need to be absolutely sure of whats on the network and be sure the devices can stand up to the scan.
20/11/06 4:03 PM
Jim C said...
I agree. I am not saying to go willy nilly and pull down Nessus and start a scan.
What I am saying is that after you make sure your systems can handle specific settings and after you have informed all of the right people and once you get the right people watching the scan live and the right operators involved.
Then you can scan.
Think of it as a test plan. Once you are comfortable with it then go ahead.
You always need change control and you always need to understand the implications if something goes wrong and be able to adjust for them.
With all of that said every security professional out there has made a mistake scanning. This is doubly true for people that haven't grown through the IT Security ranks. (and dealt with the scanning disasters there) There is a whole religion thing about if it is ok or not to scan on the IT side let alone on the CNI side.
My take is this. If it can be done properly (and it can) then if you don't scan you don't know what can go wrong. You have no idea what the environment is like.
Doing security design in that environment is like a doctor performing surgery with a blindfold and oven mits. You are lucky if you can even pick up the right tools.
Many good security professionals have gotten bitten by bad scans. In the SCADA world it makes sense to be extra careful. Especially after seeing what can happen but it doesn't mean they don't add value.
The Key point to the Myths is to make sure that CNI guys know that there is no difference between IT systems and DCS systems and so that IT guys know they are not the same.
That statement is not an oxymoron. Within context for each group it is true.
I have done hundreds of scans on PCN's successfully without problems. I wouldn't let just anyone do it but it is possible and more it is essential.
20/11/06 4:23 PM
and from Rich at Securosis
"I didn't understand a word you said in that post."
Just a sampling of the scintillating reviews of "Channeling The Ancient Submariner"
Oh Yea from Chris "you may want to tell people what a nuke is, not everyone will know a nuke is someone who works in engineering, most will probably think of the warhead. " - Quite True
Until this weekend I hadn't realized I was a polyglot. I thought I was limited to English and a few sparse phrases of German (who Austrians politely nod to then ignore).
Of course I have now discovered that after a beer and some pseudo self hypnosis I am able to speak in tongues. Now that I think of it it usually takes several beers to get to the incoherent stage for most people. Is this a gift of mine? My own Hero's power? The one Beer babbeling sublinguese guy?
Ok that was cheesy.
Here are some definitions and descriptions to explain the lingo in Channeling The Ancient Submariner and Contrasts - Submarines.
Sea Story - A genre of stories told by bored (and often boring) sailors when either bragging in a bar, (or for the newer lightweights a bookstore), Standing a long boring midwatch or in front of a class of new soon to be real sailors.
While not essential Sea Stories almost always take the following formats.
"This is a no shitter"
or for the really true to the genre ones all three.
They are frequently interrupted but the teller is never able to hear what the person interrupting has said. (this is doubly true if he is a NUB)
Something bad or stupid or preferably both always happens.
It is never that bad (well almost never) because the really bad things you don't want to dwell on.
There is almost always a moral. Kind of Darwin Awards with a specific format.
NUB - A NUB is a non useful body. Anyone who is not yet qualified. Think Plebe or Pledge but worse because the other people actually suffer because the NUB is not yet qualified. In RC Division there are only 7 people assigned to each boat. One of these 7 (the chief usually after all we took his tweaker) stands EWS so doesn't count. When underway there are at least 2 watch stations that have to be manned at all times. This is usually done in 6 hour watches. So you get up eat go stand a 6 hour watch (I'll describe these some other time) Eat again. Do Maintenance. Study for your weekly tests. Then if you are luck you get 2 to 3 hours of sleep before you get up to go on watch again. This is with 6 people. If even one of them are not qualified you go port and starboard watches. The Maintenance still has to happen so that cycle usually goes. Watch (6h), Maint (6h), Watch(6h), Easy Maint(1-3h), Short sleep(2-4h), Watch. Repeat. Similar cycles happen for every other group on the boat. So if someone isn't qualified they make everyone else have to pull their weight. So until they are qualified something they are NUBs. Guys that have been on the boat more than 3 years tend to look at everyone that is not fully qualified (every qual complete) as Nubs.
First Run 688 - 688 is the hull designation of the USS Los Angles which was the first boat of its class. They are fast attack boats. That means that their primary job is to do everything that the boomers can't or won't (not that a boomer would do anything that it can't). Basically they hunt other subs and ships and protect carriers. I'm sure they do other things but well... go read Blind Man's Bluff (I think that was based on Freedom of Information Act stuff) As Bruce says "you have no privacy get over it". I can't remember which boat is the dividing boat but the first run 88's have planes on the sail and no vertical launch tubes. There are a few other differences but I can't remember if I can talk about them. For that matter I can't remember them.
Port Vital Bus - Not worth talking about unless you are on a boat. It is an electrical bus on the port side and it is vital (really important). nuff said.
Roving Watch - Actually Shutdown Roving watch. His job is to roam around the engine room when the reactor is shutdown, take logs and make sure nothing bad is happening. Some say his real job is to stop by Maneuvering every so often and make sure the Shutdown Reactor Operator (SRO) is watching his gauges (not asleep real bad juju there) . If he gets caught in maneuvering to often he gets yelled at. But how do I say this... Have you ever been stuck in a place for 6 hours where Nothing Ever Changes . (UPDATE: Oh wait that is a cube) Over and over again for years? Unless he hates the SRO he goes and talks to him every so often. Sometimes even if they hate each other they still end up talking. Some of the greatest mysteries and problems of the universe have been solved by the SRW SRO and SEO (Shutdown Electrical Operator) and no one knows because they can't let anyone know they were talking. Same for underway for that matter.
Eng - Short for Engineer Think Scotty without the accent. In reality they are the 3rd senior officer on board usually at a Lt Cmd but sometimes a Junior one is a Lt. Always intelligent but sometimes you cannot tell because they are under so much stress that most of them freak out periodically. These guys are almost always career guys who want to be XO next time around but if someone sneezes at the wrong time during ORSE it is all over for them. We called my first Eng red because his bald pate would turn bright red when he was pissed which was pretty much all of the time. He liked to throw hard objects but never hit anyone or anything important so I think it was mostly for show. My second we called thumper because if he was aft in Maneuvering and his hand started tapping you knew a drill was about to kick off. We had another guy we called thumper (an a-ganger) but I won't go into how he got his name.
ORSE – Operational Reactor Safeguard Exam.
At least every year a bunch of Naval Reactor Experts visit every boat (and ship but who cares about skimmers). The audit they do makes a SOX review look more like a homeless guy getting his taxes reviewed by a social worker from San Francisco.
If a boat fails an ORSE the Engineers career is over and the CO's is in jeopardy. Most of the Officers and many of the enlisted personnel might never advance further in their Naval career's. At the very least they will have to pass another one very soon and forget about shore leave or seeing your family. (I always get irritated when the people on "Survivor" complain about missing their family. For god's sake it is only a month suck it up. Try going 4 months wihout even knowing for sure that they are alive let alone being able to talk to them or see them. Put a submariner on your show Jeff. Sorry sidetrack.) Failing an ORSE is Bad other people come to take over and lots of people either loose their job or might as well have. Never happened on a boat I was on but saw it afew times.
So if you are ever hiring a nuke ask him what station he stood during his ORSE’s. If it was anything other than Drill Monitor (we put the people we were worried about there) he will handle stress alright.
Nuke – Slang for engineering department staff on a sub. Not Nuclear Weapons which don’t talk and I can neither confirm nor deny the presence of on any boat I served on. Nukes were generally considered to be the RC Div (Reactor Controls) E Div (Electrical Division) M Div (Mechanical Division) and ELT’s (Engineering Laboratory Technicians)
RC Div’s responsibility was taking care of and operating the controls systems for the reactor. This included level control system, pressure control systems, the electrical and electronics portion of valve and pump controls, and nuclear instrumentation. Primary underway watch stations for RC div were Reactor Operator (RO) and Reactor Technician (RT).
E Div was responsible for all power generation and distribution systems. This included the Turbine generators (TG) Which are large 3 phase electrical generators powered by steam from the reactor plant. Each of these is big enough to power a small town. Motor Generators (MG’s) convert AC to DC power and vice verse when needed. They are large heavy motor generator’s spinning at a high rate sufficient to power a large subdivision. They were also responsible for all of the other switching and breaker systems. (Except for a few controlled by RC Div). Primary Watch stations were Electrical Operator (EO) who controls the Electric Plant from Maneuvering, Auxiliary Electrician Aft (AEA) who gets coffee for everyone.
M Div – Was responsible for all piping, pumps and valves. No long descriptions here because I didn’t mention them much in the story. The Roving Watch is usually an M Div guy.
ELT Div is responsible for maintaining the chemistry of the reactor and secondary plants. They are also responsible for anything that involves radioactive exposure or contamination. They do more paperwork than anyone on the boat and other than RC Div they spend more time than any other Division in school and taking tests. (Officers excluded of course) ELT’s often stand Engine Room Forward watch for a number of reasons. We liked to tell them they were just radiation sponges.
EWS is the engineering watch supervisor they are the senior underway enlisted watch station. Most of them are Chief’s (E-7 and above) a few are First class and a very rare second class sneaks in (usually trying to make first).
The EOOW is the Engineering Officer of the Watch. They are the senior engineering watch station. They sit in Maneuvering and direct the actions of all of the other watch stations.
CO is the Commanding Officer – The Captain of the Ship.
PD periscope depth – In the North Atlantic without ventilation it is close enough to the surface to get everyone sick.
That took longer than I thought. If you got down and are actually reading this last line you must be really bored. Next time just get a babblefish.
At the same time that many organizations are scrambling to insert protective firewalls for their Automation systems, business and operational needs are increasing the inter-connectivity of the systems that need protection. In the case of automation systems the real risk might be the inability to monitor the operation and respond to changing operational dynamics and less the improper access by a small subset of individuals. Because of these competing requirements even when strong perimeter controls are implemented they rapidly atrophy in effectiveness. Firewalls become so riddled with holes that their ability to provide control functions is severely limited. It is naive to assume that control systems can be isolated.
Look having outside connectivity sometimes provides more value than the risk it incurs. This is especially true for monitoring only systems. Say there is a rig or other high location that has a reading that has to be taken periodically. By having an RTU up there it is no longer necessary for some one to climb up. As a matter of fact they don't even have to get in a truck to drive out near it. They can read it from the comfort of the maintenance shack. That is a huge safety improvement. Now a lot of people would argue that this is simple but for many organizations they are still climbing ladders to get readings on a regular basis. Installation of a cheap and easy RTU literally can save lives here not to mention adding to accuracy and precision which will ultimately result in savings. Even 1% or 2% can mean the difference between profitability and loss for some low end sites and they cannot afford complex security arrangements.
(By the way Tofino might be an answer for them. Eric has the site up for his new company. I'll post more later after I get a chance to talk to him.)
In a different scenario you have a large complex site with thousands of variables. In a location like this the interconnection to the PCN provides many many essential functions. Many actually most significant accidents could have been avoided by having the right people know the right data earlier. Historian feeds to external aggregation points allows engineers across the world monitor and troubleshoot. Expert talent can be pooled and can always see data from major sites. Subject matter experts can see the data real time. Not only can this improve safety and efficiency in a lot of companies it is and has. Other improvements are in logistics (both supplying and planning production to feed customers), maintenance, capacity planning, and many others.
Suffice it to say that these systems need to talk to the real world and vice verse. Firewalls are a must (at least for open and closed loop controls) but just like in the IT world their utility is out of date and waining. More needs to happen. Mike at my company likes to use a statement (that he claims is several steps from its originator via the CTO of N-Circle) that fits this process.
"Firesuits not Firewalls."
I don't think anyone is advocating complete elimination of firewalls the key is that they are not enough.
Patching has to happen and has to be able to happen quickly. Access controls have to exist and be enforced. Behavior based protections have to be applied (within reason). Memory protection should be considered. It is essential that you know what your environment looks like and what it is vulnerable to using tools like Nessus and CoreImpact. Things have to be measured.
The environment has to be monitored.
Not all of these will apply to every system of course but overall all of the tired cliches need to be followed. The key is that they are essential in SCADA systems as well.
The perimeter is leaving SCADA because there is more good to be gained than bad (like it or not from a security perspective) so it is time to adapt your security strategy.
19 November, 2006
18 November, 2006
17 November, 2006
It is somewhat of a cliche but my time on the boat and in the Navy was a study of contrasts for me. There were very good and very bad moments but in the long run it was overwhelmingly a beneficial experience.
Like every navy member out there now and in the past I spent a lot of time away from my family. I basically missed the first two years of my son's life and the first year of my oldest daughter. My marriage was strained to the breaking point.
Homecomings were the greatest highs you could possibly imagine. They were the sweetest feelings and I could not possibly do them justice in words. I am not even going to try. It was a terrific and painful heat to temper my marriage that happily has lasted to this day.
Sadly many of my shipmates were not as lucky and it was not unheard of to return from a run and find a lawyer as opposed to a wife on the pier. In those times the camaraderie is what helped to take the individuals through the pain. In most cases it was a tempering experience for them as well.
It is not possible to get a true sense of comradeship without a suitably high barrier for entry. The barriers for acceptance are high and need to be. Lives rely on it and in some of my more serious memories that manifests in a direct and immediate way. If you understand the implications of the event I mentioned earlier you know that despite my flippant attitude what happened was very bad. It was treated with corresponding significance and integrity by all involved after it happened. In a very real way sea stories are a healthy part of the response to mistakes. Perhaps that deserves a post later.
The barriers to entry for the comradeship take many forms. Demonstrated integrity, time, knowledge, skill, emotional fortitude, lingo, persistence and sheer intelligence are a few of them.
All submariners develop a knowledge of the boat that is instinctive. The nukes and the officers culture a nearly eidetic memory and to this day I remember hundreds of numbers, settings, and procedures. Given a few days of thought I could probably recite them verbatim and I expect this would be the norm. The officers (even the Ensigns who I was less than properly deferential to) and many of the other nukes could quickly and accurately do complex math in their heads. This instinctive knack for the right answer doesn't leave you but it does have to be periodically exercised or it weakens.
To be honest I only knew one officer that deserved the term butter bar and the system worked and weeded him out early.
Integrity and confidence are possibly the best legacies that come from the experience.
Believe me presenting to the board of directors in multi billion dollar company is less stressful than your first final qualification board and the integrity required in an incident review is stronger than multiple SOX audits.
The treatment by senior sailors when you are a NUB (Non Useful Body) is harsh but prepares you well for the politics that exist outside and frankly if you are unable to handle that pressure you shouldn't be operating a nuclear reactor or manning a station controlling a billion dollar piece of machinery whether it be the helm, sonar of or any number of other positions.
Being underway sucked. Most of the time it was boring, occasionally during workups and particularly important runs it was slightly interesting, sometimes it was a bit frightening but overall it just really sucked. I would never voluntarily do it again but overall it was worth the experience.
More than that I wouldn't be what I am or where I am today without it.
What I described in the sea story was fairly accurate for a boat side conversation. Not pretty but true. I just should have remebered that when you get to a new boat you have to earn your quals again. You can't just take them. Now that I think of it that is applicable to where I am in life now as well.
When I first floated this one I had several people challenge the exponential piece. They were right to do so but I am standing by the statement.
There are any number of ways this can be verified but the easiest is to read the financial reports of the companies that specialize in these systems. For the last several years quarter on quarter they have shown consistent unit and revenue growth. You might not see this in individual purchasing companies but I bet in the larger ones you will see the same type of growth if the entire company is looked at. Certainly I see it in the budgets I have seen.
If you want to get away from financial indicators then look at the number of PLC's and RTU's that are being aggregated in your historians and monitoring centers. I know a couple of oil companies that are devoting whole building floors to the systems and maintaining them.
This is also backed up by power industry news that shows clear areas where growth is available.
I will make one concession though. Even though the current growth rate appears exponential we haven't yet reached the asymptote of the curve and even if we do it doesn't mean that there isn't a plateau that will occur later.
Regardless right now automation, SCADA, DCS and PCN systems are undergoing the same explosive growth that all other areas of the information industry have and are. There is a several year lag due to differences in the capital cycle and implementation and usage but the overall trend is identical. This means that we can expect more and more interconnectivity, accessible systems and most importantly more direct control of functions that previously could not be easily controlled remotely. Furthermore we should expect these cycles to shorten in length and to converge towards the technological adoption rates seen elsewhere. No more 20 year cycles 10, 8, 5 or even 2 (when looking at software portions) will begin to emerge.
This obviously has security implications but it also means great benefits are being accrued.
This is a follow on series to that one. There are a number of facts we have to deal with in the SCADA security world and these are 5 of them. Most of these look bad on the surface but there are some underlying advantages that might not be apparent so:
Pandora's box is open or the genie is out of the bottle your choice
1. Automation control systems are expanding exponentially in complexity, numbers, interconnectivity, and capability.
2. Deperimeterization is happening with DCS whether we like it or not.
3. Standardization with existing IT vendors is happening to SCADA systems and is subjecting new areas (Control systems) to old threats (Hackers and worms). This results in the creation significant risks to safety, environment and business.
4. The bad guys are now realizing that there is something here but so are the good guys.
5. Bad things have already happened and more will.
16 November, 2006
Myth # 5 - You cannot scan or update Automated Control systems.
Scanning and updates are just as essential for these systems (or more important because of the geographic and ownership distributions) as any other IT system. Scanning and updating needs to be done carefully, within change management and with good communication to the users of the systems.
The key phrase here is change management. All stake holders must know when and how the scans will occur. From the Engineering Authority to the operators (current, off going and oncoming) everyone must be informed. This also means that you need a tool to do the scanning that is able to track and log (verifiably) to the second exactly what it is doing to the end system.
The last part is why I prefer CoreImpact over Nessus.
Both are good but Core gives you verifiable CYA. (and in many cases easier granular control)
In all cases you should know what you are doing to what, when, and why and be able to explain it to the engineers and operators. If you can't then you shouldn't be doing the scan.
With the caveats made once you get the process down it becomes a non event (other than fixing the problems that are found which for a while will be many). It was a weekly event at one of the companies I was CISO at.
As for updates not only is it possible to do them it is essential that they are done. Again with proper change management not just arbitrarily.
This is a No Shitter
(… means NUB interruption so don’t worry it doesn’t matter what they said they are a nub after all)
So back on my old boat …
It was a first run 688…
So we were a couple hundred miles South East of Halifax (gotta keep an eye on those Canadians) doing some unscheduled port vital bus maintenance…
Yea at Sea…
Like I said at sea…
[Roving Watch] Stupid
Yea Ok So? Tell the eng believe me he knows now.
So anyway we were a few days from starting our work up for [pause] For [pause] What the hell do they call that coner ORSE?
Anyway I’m sitting in middle level between the HPAC’s grading tests…
Yea Yea I know I shoulda been forward. Tell that to my mutant kids… (now that I think of it that might explain my TLD reading that month)
Anyway I’m sitting between the hipacs and the EOOW announces an Electric plant shift…
Yes a shift…
[Roving Watch] That was Stupid
Yea stupid butterbar…
So I start to get up to go to maneuvering and there is this loud bang…
Real Loud and the incandescents come on and the fluorescents go off…
Yep at Sea…
A lot of loud funny noise to my right So I look over and the Starboard MG is shaking like Shakira (so I modernized this part sue me)
No I mean that sucker was moving…
(Some context for non navy guys the MG is a chunk of metal the size of a car and spinning at ungodly RPM’s )
So I see the EWS streak up the ladder to the horseshoe…
[Roving Watch] Would you stop interrupting you stupid NUB or I ain’t signing crap. What was he doing down there?
ERF where else would an ELT EWS be on a boring watch…
So I start walking around testing the tripped breakers with the back of my hand and a few seconds later a cussing engineer comes striding through…
[Roving Watch] Bet he was pissed.
Oh yea he was steaming and the CO was right after him. Probably started moving the second he heard the electric plant shift announcement…
So anyway two hours later things are getting back to normal and we are bobing at PD in the North Atlantic at 1/3 everyone starting to puke from seasickness…
[Roving Watch] Bullshit
Shut-up NUB. No really it’s a no Shitter.
So what do we learn from this?
1. A Nuclear Reactor Powered Steam Turbine will always win against an MG (even if it is the size of a car).
2. Then Breakers will open and I don’t care what the KAPL guys say you can’t always tell which ones. (Good lesson in the civy world too)
3. On the good to bad scale paralleling 120 degrees out of phase is bad. Bad bad bad
4. Never put a newly qualed butterbar on watch with half the electric plant tagged out and a Short RO, newly qualed EO and a mechanic at throttles.
5. Never let a RC Div NUB going for EO quals shift the plant at sea. Who cares what the plant status is.
6. And most importantly “The Stupid Shall be Punished”
Any questions. Not bad for a 20 year old channel huh.
Eat your heart out Iowahawk.
Oh yea if you like it Link to it.
DCSSEC [at] gmail.com
Edited slightly for readability.
15 November, 2006
A lot of the items at the NISCC site are heavily influenced by Justin Lowe at
I have worked with them extensively in the past and they are one of the few larger consulting companies that have a strong knowledge and presence in this field.
14 November, 2006
I think it is time to revive one of my pre blog writings trying to get people interested in security issues around SCADA systems. If you want to know what they are and why they matter take the time to read this. It is a big problem and getting bigger. If you are already into the topic it might give you some catch phrases and other angles for the non initiated. A little Campy but I didn't write it for a blog.
Control systems are everywhere. From nuclear plants to elevators, automobile manufacturing robots to remote surgery, multibillion dollar offshore oil facilities to children’s toys, computers are controlling more things every day. Automated control systems are not a new phenomenon. In the first decade of the 1800’s punch cards were used to operate weaving looms running relatively primitive programs using mechanical interlocks and stops to control the equipment. Digital computerized systems have been around for more than a half century and short range radio monitoring and control existed well before wireless became a common term in the IT world. These systems go by many different names, Automated Control Systems (ACS), SCADA, DCS and Process Control Systems are among the most common. Ultimately what they do is what defines them as a separate category. Whatever the name, the defining function of control systems is their ability to directly physically manipulate the real world.
In the last decade there has been a revolution in the automated control world. Mirroring the advancements of the information technology world, ACS have become more integrated, easier to connect to, and standardized.
Many systems are now directly or indirectly connected to an IP network that is ultimately connected to the internet. The key control and programming point of these systems is often run as an application on one of the common Operating Systems.
This standardization and interconnectivity has had a dramatic positive effect on the efficiency, safety and ease of implementation of these systems.
Because these systems are often more complicated than other computing systems, have a higher capital cost than other computing systems, and are tied to physical infrastructure, the adoption of the newest generation lags the IT and internet world by 8 to 10 years. This puts the ACS world right in the middle of the turn of the millennium IT environment. The same paradigms apply. There are and will be dramatic impacts on business models. Irrational exuberance abounds. A huge amount of money is being spent and saved.
Finally the security challenges of the early internet days are now being felt in systems that control our power, water distribution, oil pipelines and wastewater removal plants.
This final point cannot be overstated. The same viruses, spam, pop-ups and botnets that give the IT world and the average home PC user headaches can affect the power supply to your house and business and change the way that the natural gas pipeline in the back of the neighborhood works.
There are two key questions that define the debate about how or even whether to direct resources to protecting ACS. Can control systems be accessed and controlled by unwanted individuals? What will/can happen if they do access them?
The answer the first question is a direct and simple yes. Not only can these systems be accessed but they have been accessed. If a system is connected to any other IT or telecomm system then it can be reached and controlled.
The answer to the second is less direct. It depends. It depends on what the ACS is controlling, how much and how fast a human can get involved and most importantly how the underlying system integrates into the process being controlled. In most cases production can be stopped or efficiency impacted. In some cases people can be hurt or killed, large amounts of environmental harm can occur, and huge amounts of money can be lost.
A number of high profile incidents are easy to find.
The California power grid was compromised and service was almost interrupted, waste water has fouled beaches, David Beckham’s car was unlocked, started and stolen twice, and the slammer worm was found in the systems of a nuclear plant.
From the silly to the terrifying, compromises of automated controls systems are occurring daily. Ultimately these incidents show the public side of the impact but the real threats can be subtle. Control systems are not designed to identify abuse and hacking. Until recently identification of attacks specifically directed at ACS was not available or possible. In many organizations the control systems are not located on a segment of the network that allows easy differentiation of unwanted traffic. The result of these and other weaknesses in existing architectures is that the real level of compromise and therefore the threat and risk levels are difficult or impossible to determine for most organizations without the acquisition of greater information and understanding.
People are doing things to fix it but more needs to be done and faster.
This isn't my typical audience but welcome to the site. Feel free to snoop around.
I'll see if I can dig any of my old memories up (the ones I haven't repressed anyways. :-) ) and put them out in the next few days.
I spend a lot of time on information security and IT matters. I have a special interest in Process Control and the Security around it.
13 November, 2006
A sea Story
I have recently stumbled on some Submarine blogs. I guess being 15 or more years away from subs I didn't even think of them writing them.
Of course submariners being who they are I should have expected it as well as the quality.
The culture shines through.
One thing I don't get though.
Are todays A-Gangers really this wussified?
:) You would think they are ET's or ST's or something. (or heaven forbid Nukes like I was)
Now guys if you read this please don't tape me outside of the condensers (a fate I managed to dodge but other Nuke ET's spoke their way into).
Thanks for the launch Bubblehead has basically doubled my normal traffic today.
If you want a better idea of what this site is about go to the main page or
to this synopsis.
If you are an IT security person it can give you some indication of how things are integrating.
If you are on the SCADA side Invensys has mostly been on the leading edge of addressing the security in these systems. I'll see if I can dig up any improvements they have in this iteration.
The new focus on component and device vulnerabilities increases the exposure of DCS devices to threats.
This is emphasised by the recent Broadcom vuln that affects Dell, HP, and Gateway wireless systems.
While it is not specific to this problem (you should still be checking) my last few days of posts focuses on the growing connectivity of PCN's via different types of wireless access.
Control systems consist of several different sub-systems that act in cooperation to monitor, log and manipulate a physical process. Different vendors have different points of division and sometimes combine sub-systems but ultimately they will include all (or most) of the functions. There is a hierarchy of systems that include sensing and activation systems, data receipt, storage and transfer, presentation, and control.
At the base of the hierarchy are the PLC’s (Programmable Logic Controller) and/or the RTU’s (Remote Telemetry Unit). These devices have an interface to a physical device for either monitoring or activating purposes. Sensors typically convert mechanical or analog data into digital format then store and/or forward it. Actuators receive digital commands and convert them into actions by energizing a solenoid, activating a servo, positioning a synchro, or just turning on a switch. PLC’s act as the system interface to these devices.
PLC’s and RTU’s feed data into historians and operational control systems. The functions of these vary widely depending on requirements of the process or action being either controlled or monitored.
Typically the historian serves as an aggregation point for data from multiple different systems and subsequent actions regarding that data.
The role of this layer can usually be defined in one of three ways. Monitoring, Open Loop Control, and Closed Loop Control.
For the monitoring function the role of the system is to gather and make information from the sensors available for use in various manners. Monitoring is often also a part of open and closed loop controls.
Open loop controls require the action of an external operator to occur.
Closed loop controls are controls that occur without operator intervention.
Each of these control functions requires a slightly different approach to protection and has a different hierarchy of impact priority.