Using Rsync to do local snapshotting/backups

Using Rsync to do local snapshotting/backups January 9, 2010

Jeff Atwood's recent data-loss story is a good reminder of why you should do off-site backups. But what if you just accidentally rm'ed a file, or saved over it's contents with bad data? It's a lot of work to do a full backup recovery of your entire system just to get back one file. Isn't there an easier way?

Local Snapshotting: not perfect, but super usefull

Here's what I do: run rsync with cron to copy all your important files to a local /backup directory.
First, create a new file called crontab.txt with contents like:

@hourly  rsync -a --inplace --max-size=1MB /www /etc /backup/hourly/
@daily   rsync -a --inplace --max-size=1MB /www /etc /backup/daily/
@weekly  rsync -a --inplace --max-size=1MB /www /etc /backup/weekly/
@monthly rsync -a --inplace /www /etc /backup/monthly/

Let me explain:

@hourly (and the others) is a special interval understood by cron
/www and /etc are specific directories I want to keep snapshotted. You might want to include /home as well
The -a option to rsync tells it to do "archive" mode (preserving permissions, etc...)
The --inplace option is just an optimization so that files in /backup are overwritten inplace, as opposed to rsync creating an intermediate temp file.
The --max-size=1MB tells rsync to ignore files greater than 1MB in size. I do this so that I don't bother making lots of copies of big log files and videos and other stuff that isn't that important and doesn't change that often.

Now you install this crontab by doing:

crontab crontab.txt

I do this as the root user, but you can do this as any user that can read all the directories you need to backup, and can write to /backup. (Warning: the above command will overwrite any other crontabs you already have installed. Do a crontab -l first, to see what's installed.) Now you can just sit back and relax -- copies of your local files are being copied every hour, day, week, and month to the /backup directory. If you want to make sure you have a full (monthly) backup right away, then you should execute:

rsync -a --inplace /www /etc /backup/monthly/

right now.

How does this help me?

Let's say you just accidentally removed a local file

rm /etc/lighttpd/lighttpd.conf
# Oh shit, I didn't mean to do that!

Not to fear, you can recover it by doing

cp /backup/hourly/etc/lighttpd/lighttpd.conf /etc/lighttpd/lighttpd.conf

Why do hourly, daily, etc...?

For mistakes that you catch right away, the /backup/hourly directory is what you'll goto most often to get a recent version. But sometimes you don't realize something is wrong until days (or weeks) later. In that case, the hourly backup is of no use, since it has already mirrored the mistake. If you were really paranoid, you could add a @yearly line to the crontab above.