At Freelock, we don't think one backup is enough. All kinds of things can, and often do go wrong. Murphy was an optimist, after all.
We make sure that all of our code and customers' data is backed up not just once but multiple times, in multiple places. Perhaps we go overboard with this, but over the course of nearly a decade of business, we have not lost more than a few hours of data for any of our customers we cover. And we've recovered data from hardware failures, hacked servers, accidental deletion, and more. Our backup systems have not just been tested, but they have successfully been used to recover failed systems. We have years of experience helping customers prevent and recover from all sorts of failures.
We make sure there are at least two backups of everything we do. For production sites, we keep an image backup of each server at the host where it can be spun up in a matter of minutes on brand new hardware, and a file-based backup of everything on the server on our backup server in our office. For development sites, we have the same file-based backup server in house, and we mirror all our data out to a dedicated backup service on the Internet -- if our office burned down, servers along with it, it would only take a few days to restore. Our customers would not lose any data or have to start over.
We have a sophisticated backup tool that allows us to cram huge amounts of data into relatively small disk space. By using compression and a sophisticated de-duplication algorithm, we're able to store some 3 terabytes of data in around 300 gigabytes of disk space -- and keep particular snapshots of every file on every server for up to 16 months. Without a historical backup, you can be screwed if you accidentally delete something or if an attacker breaks in and you don't notice for a few days.
In addition to all of these backups, often our developers will have local copies of the sites they are working on, with the full project history. We get this by using the same code management tools that Linux itself uses, storing every change in the entire project on every instance of the site.
Backups are just the safety net, though. They help you when you have a problem. Some problems are unavoidable, so you cannot do without backups -- but many can be avoided fairly easy. Read on to find out how.