When the Cloud goes bad

Before you read too much further this is a bit of a self-serving rant, bought on indirectly by the heartbleed bug published a couple of weeks ago.

I’m a fan of cloud hosted virtual Linux servers for hosting and messing about of the geeky kind.

The ability to spin up a box and play with it to your hearts content then just shut it off again for cents an hour has to be one of the greatest enablers of growth in online services since caffeine laden fizzy drinks hit the market.

I evangelised the use of cloud boxes for web hosting a while ago with some of my geeky friends, which of course rubbed off onto some of the not-so-geeky ones as well.

That’s where the cloud goes bad, right there, just at the end of that last sentence.

I even went as far as to help a couple of the not-so-geeky folks set up cloud servers and migrate their websites in a flurry of command line goodness with bash scripting that is in fact a native language for sandal wearers in many countries.

That was 2009 and skipping forward five years I now seem to have become the Linux agony aunt for a few of the converts and even some of the ones I considered to have strong bash-foo had dug a hole so deep that apt-get and yum can’t rescue them.

One whole day, almost to the minute, after CVE-2014-0160 was published and about four hours after I’d finished the initial patching of all *ix boxes for work my inbox was showing signs of the darker side of the cloud.

The questions were varied, to be fair, but all symptoms of the same underlying issue. “Is my server vulnerable?”, “My server wont update, how do I upgrade it?”, “I googled it but apt-get gives me xxxx error”.

Some of these machines had not been updated by their well meaning owners since they were spun up, in some cases over five years ago. The distribution in the case of Debian 5 now unsupported. This is my problem how?

The next morning I was due to start a three day holiday in the form of being parent helper on a school camp so I really didn’t want to be a part of someone else’s IT dilemma. For work I would keep an eye on things while away, but the other folks? Hmmm.

I replied with a pretty generic message to all of them, along the lines of ‘Patch OpenSSL, replace your SSH keys and if you’re using https get the certificate re-issued’.

So, instead of just relying on the trusty iPhone to keep up I ignored the rules for school camps and packed my netbook along with my toothbrush, socks and bug repellent. My plan being to get online tethered to the phone at least a couple of times during the break.

Well, you could have knocked me down with a feather. About midnight on the Wednesday I get online and there’s about forty emails waiting for my weary eyes.

Half of them unhappy replies to my apparently useless advice and the other half from more people who clearly shouldn’t run public facing servers on the internet without first putting on their overcoats.

Where I had credentials I made a half-hearted effort to get on and update OpenSSL in the wee hours between mountain biking, making hundreds of filled rolls and hut building in the rain but generally I just deffered them as my enthusiasm for cloud computing was being sorely tested. I’m not a very good agony aunt it seems.

And here’s the rant, in a nutshell:

If you don’t know how to keep a server secure and up to date you should not be running your own virtual servers online. Windows or Linux I don’t care. Just don’t do it.

If the concept of migrating applications or websites between releases of Linux Distro is foreign, or you think a public key is something used by kids to sneak into the school pool, get yourself some cheap shared hosting. Virtual servers are not for you.

If you think the scope for a Windows firewall is a tube with mirrors at each end allowing you to peer over the wall…. Well, you get the idea.

I hereby retract all of my former enthusiasm for cloud hosted virtual Linux servers.

</rant>

Security in the cloud, KISS

The idea of keeping things simple when it comes to server security is not at all radical and cloud servers provide the ability to reach the not so lofty goal of keeping your servers simple and secure without breaking the bank.

The theory is simple: The smaller the number of processes you have running on your box the less there is to go wrong, or attack. This is one area where Windows based servers are immediately at a disadvantage over a *ix server, but I digress.

When I was pretending to be a hosting provider a few years ago I ran colocated discrete servers. They weren’t cheap to own or run, not by a long shot. That cost was a huge enemy of the KISS security concept.

In the process of trying to squeeze every last cent of value from the boxes I overloaded them with every obscure daemon and process I could think of. Subsequently the configuration of the servers became complex and difficult to manage, while applying patches became a cause of sleepless nights and caffeine abuse.

With the cost to deliver a virtual server in the cents per hour and the ability to build a new server in a matter of minutes the barrier to building complex applications with a robust security architecture is all but vanished.

The mySQL server behind this blog site is a base install of Debian Lenny with mySQL, nullmailer, knockd and an iptables firewall script. That’s it. Simple to build, simple to configure, simple to backup and simple to manage. KISS.

A little bit of searching around on hardening up a linux box and you’ll quickly find information on changing default settings for sshd and iptables rulesets which you can combine with small targeted cloud servers to reduce the sleepless nights.

I can’t help with the coffee addiction though, I’m still trying to kick that habit myself!

Working on a cloud

This blog is now coming to you from a cloud. A rackspace cloud server that is. Two of them in fact, the front end server running the CMS, and the back-end MySQL server.

The concept of cloud computing really isn’t all that new, but if you’re all at sea when it comes to clouds you might want to toodle over to Wikipedia and read about it there.

“This is the pointy end of the geek scale where crontabs are complex and the preferred editors have two letter names.”
The service I’m using is probably better described as cloud provisioning, in that I’ve got two virtual servers living somewhere in the bowels of the Rackspace data centre. I don’t have to care about memory sizing, disk space, network infrastructure, or anything else for that matter, I’m just renting some resources out of the cloud.

I picked how much memory and disk space I wanted in a few clicks then before the kettle had time to boil the server was on line and ready for configuration. If this service was available back when I was running a hosting business I’d probably still be running a hosting business, although I’d also be stark raving bonkers.

At this point I should say that I’m talking about virtual Linux servers here, not cloud hosting or full service shared hosting. This is the pointy end of the geek scale where crontabs are complex and the preferred editors have two letter names.

I’ve moved the blog onto the fluffy stuff to get a feeling for the service before I shift my work-in-progress link shrinker into the cloud as well. What I want to achieve with the lngz.org is simply not possible on a shared platform, as I want to build a tiered application which can scale quickly.

The traditional way of achieving this goal would be to slap your gold card down on the counter of a hosting company and then proceed to the bank to arrange a second mortgage on your house. Virtualised ‘cloud’ server services such as rackspace cloud, Amazon EC2 or gogrid lets you do the same things for a fraction of the cost and with amazing flexibility.

note: I’m not affiliated with Rackspace, I just think they provide a nifty service. 🙂