No-one likes to loose digital data, especially precious memories in photos that can’t be replaced. I’ve had two pretty major hard drive failures in the last year and there’s nothing like the thought of having lost photos or other important information to give you the kick in the arse you need to make sure you have a backup strategy that is being followed.
About 9 months ago I had a 1TB hard drive start to fail. It had 360+ identified reallocated sectors and had numerous I/O errors while trying to access specific areas of the drive. That drive had all our photos on it. Luckily I got most of the data off it and, combined with a simple manual backup I was doing to an external drive, I didn’t loose a thing. It was a painful and somewhat stressful process restoring and checking the recovered data and could have easily resulted in significant data loss. I needed a more robust backup process. Our entire home computing system is based on Linux - Ubuntu Lucid Lynx 10.04. This includes a desktop computer and a HTPC that triples as NAS, VDR using MythTV, and media centre using XBMC Media Centre.
After review of the backup options available I finally settled on using backupninja. It’s a piece of cake to install as it’s available in the Ubuntu repositories, is easy to setup, has good documentation, and has just the right amount of notification related bells and whistles to let me know that things are running when they should, and when things don’t go as they should. The backup strategy I use now is pretty straight forward, with all the important information on those 2 machines being backed up to 3 places using backupninja:
- Automatically to the other computer on a daily basis
- Manually to an external drive stored in our detached garage (in case the house burns down or we are burgled) on a weekly basis
- Manually to an external drive stored at work (in case both the house and garage burn down) on a weekly (turning out to be more like monthly) basis
- Future: Automatically to an Amazon S3 account in the cloud (only for the very most important information since bandwidth is an issue here in New Zealand)
Backupninja provides built in support for all the types of information I needed backing up and includes support for setting them up using the curses based ninjahelper interface. Configuring the necessary files manually is also ridiculously easy.
- System Information dump - e.g. hard drive partition information, installed packages
- MySQL database dump - for all of the MythTV recorded TV show information
- File backup using rdiff-backup over SSH or directly to a locally mounted drive - for all our photos and other multimedia
- Future: File backup using duplicity to Amazon S3 - for all our photos
To install backupninja and the other utilities that it can leverage, use the following command line:
sudo apt-get install backupninja, debconf-utils, hwinfo, rdiff-backup
Below is part of a sample configuration file for an rdiff-backup configuration to show how easy it is to configure:
include = /etc include = /home include = /root include = /mnt/drive1/multimedia include = /usr/local/*bin include = /var/backups include = /var/lib/dpkg/status* include = /var/spool/cron/crontabs exclude = **/.ccache exclude = **/.local/share/Trash exclude = **/.thumbnails wxclude = **/.Trash*
There is oodles of other useful information in the Backupninja Wiki so go and have a look at that if you want to know more.
I’m also using smartmontools to monitor the SMART status of the hard drives in all our machines totalling ~3TB of data across 5 drives. Getting a heads up that you might need to move data off a drive onto a new one before it fails is sometimes the easier option than restoring from a backup. Putting a backup strategy in place is only something most people consider once they’ve already lost data - the same way that most people only get a house alarm after they have been burgled. If you need to go into your house to access any important digital information, then you are at risk of loosing something. Be smart, do something before you lose something.