Freelock status on Bash ShellShock vulnerability

The short answer: Not Vulnerable.

We've been asked by several customers about whether they or we are affected by the recently discovered Bash ShellShock vulnerability. And to the best of our knowledge, we are completely unaffected.

What is Bash?

Bash is a "shell," the main environment that most Linux and Unix systems use to run software, especially in server environments. There are many different shells, but Bash is one of the oldest and has perhaps the most widespread usage.

In particular, when one program talks to another program in Linux or Unix, it uses the "system" shell, which is generally set to Bash unless somebody has changed it.

What is ShellShock?

ShellShock is the name given to a vulnerability that has been in Bash since its earliest days, over 22 years ago. It takes advantage of a couple of Bash features that were not properly locked down -- the ability to set environment variables before executing a script, and improper variable handling that allows an attacker to piggyback a command after setting a variable.

Combined, this allows a remote attacker that can reach anything that loads a system shell to run any arbitrary command they want. It's trivial to exploit.

What is affected?

The main thing that has so many people so worried about this exploit is that a very common way of running web applications is through a standard called "CGI," short for Common Gateway Interface. Many, many web applications use CGI to live on the web -- a web request goes to the web server, which spawns a system shell to execute the application, and sends the output back to the browser.

This describes a huge, huge number of applications on the web.

But... in spite of lots of commentary about this being worse than Heartbleed, there's one big, big saving factor that prevents this from being catastrophic: A large part of the web no longer uses CGI.

CGI is perhaps the original way of doing web applications, the oldest architecture for running programs to make web pages. But it's far from the only way. CGI has a big challenge with performance, because every request involves creating a new system process and spawning the language run-times and everything else. This doesn't necessarily scale very well, so many (most?) modern web-facing applications haven't used CGI for a long, long time.

The standard way of running PHP (which is what powers Drupal, WordPress, Joomla, and a huge number of very popular applications) is using something called mod_php, which entirely skips system shells and runs inside Apache itself. While it's possible to run PHP under CGI and some places do, it is not a very standard way of running it. There are newer replacements (Fast CGI) that improve upon CGI, but these also stay running for long periods of time -- I have not analyzed whether or not FastCGI is vulnerable, because we don't (yet) use it, but it would surprise me if it is vulnerable.

PHP is generally not affected

It's certainly possible to run PHP in a shell, and it's certainly possible to use PHP with CGI, and have a vulnerable system. Many maintenance tasks, and our favorite Drupal tool (drush) make heavy use of the shell -- but they do not provide any kind of external access to the shell.

So unless you're running PHP on a CGI server, you probably have nothing to fear from this vulnerability, at least where your Drupal site is concerned.

None of Freelock's servers are affected by the most widespread attack vector

... or the servers we manage for customers.

We have standardized on Ubuntu Long-Term Support releases for our production servers. And... Ubuntu does not use Bash as the system shell! Several years ago they replaced Bash with a lighter, revamped version called "dash". And it turns out dash is not vulnerable. Bash is installed on Ubuntu servers, it's just not the default system shell.

We also don't run CGI on most of our production servers, so the vast majority of our customer sites are completely unaffected. The only place we use CGI is in some back-office systems -- our bookkeeping software, our secondary backup system, some statistics packages, and a few other small, one-off systems.

Steps we've taken

When we learned about this vulnerability, the first thing we did was analyze our exposure. We identified that our exposure was minimal if it existed at all, and we ran some preliminary tests to see if we could find any affected systems. We found nothing vulnerable in the first couple hours of testing.

We then applied the security updates for Bash to all of our servers and the servers we manage for our customers (you are on our server maintenance plan, aren't you?). So by 2pm PDT yesterday (9/25) we had applied the available vendor patches across all our managed servers. An updated patch was released by Ubuntu last night for the latest 14.04 release, which we've now applied.

We also developed our own quick testing tool based on the code here: http://shellshock.iecra.org/ , and have tested all of our routers and network infrastructure. We have not found any of our systems to be vulnerable.

Safety Nets

One of our customers asked a very important question: "I wanted to make sure we're getting backed up..."

That's the very first line of defense to have in place, if this were a vulnerability that affected us. The ironic thing about this is that our secondary backup system does actually run under CGI, so if this had affected a primary system, it could also conceivably wipe out the secondary backups. Which is why one set of backups is not enough.

We always recommend at least two backup systems/redundant copies of any site. In addition to source code management, developer copies, Dropbox, etc. These days the vast majority of our clients are on cloud-based servers, and the primary backup for those are snapshot images taken nightly. These allow for quick recovery on new hardware of the entire machine.

We generally then suggest a secondary, file-based backup system that keeps snapshots around for months to protect against deletions, file-corruptions, etc that might contaminate your primary backups without you noticing.

So even if the worst case happens and you lose your server entirely, you can recover without it being a big deal.

Conclusion

While all the hue and outcry over this vulnerability is definitely justified, because it is a very severe vulnerability, it is not one that affects Freelock or any of the servers or sites we manage for our customers.

For us, Heartbleed was far worse because it did affect roughly 1/2 of our production servers. ShellShock, by comparison, involved a simple update to code that is installed on our servers, but as best we can tell, has never actually been remotely attackable.

If you're on our server maintenance plan, you're already completely covered. If you're not, we'd be happy to add you to our managed systems, where we can analyze and handle future incidents for you! Contact us if you're interested...