Rsync - What to backup?

General Interworx Control Panel Info General Inquries
1

Hi Guys,

We are Rsync-ing our servers now for backup.
I was wondering:
What do I need to backup, so that I can restore the backup using rsync to a fresh installed iworx server (if the old server catches fire, gets destroyed or whatever)?

I’m currently backing up:
/usr/local/interworx
/chroot/home

But I guess that we need to backup more things. (we also use the iworx database, mail and structure only backup, but that is still a lot of work to restore this all if your server gets destroyed or is gone.)

This is also nice to know when we have to migrate a server.

2

I backup…

/etc/
/home/
/var/lib/mysql

3

Hi

For those who might be interested we have created a script that does :

  • Structure only backups
  • rotation

The script use rsnapshot and backup all you need (you can set what you want to be backuped)

  • All rsync with rsnaphot are done under a data/fs directory
    reWe also use a exlude_file listing to exclude some dir/files

  • All iworx backup structure only are done under a structrure/IP directory (IP of the domain, it will help for restoring account)

  • All mysql are done under the data/mysql/ directory

By default we set ? 2 days rotation, 1 week, 1 month

here is our rsnaphot conf file


# LOCALHOST
backup_script   /root/scripts/backupv3/so_backup.sh   structure/   ([B]see[/B] 1- backup_script	/root/scripts/backupv3/so_backup.sh  structure/)
backup  /chroot/home/	   data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /etc/passwd          data/fs/	 ([B]see [/B]2- Data, filesystem backups)
backup  /etc/group            data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /etc/shadow          data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /etc/hosts.allow     data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /etc/hosts.deny     data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /etc/resolv.conf     data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /etc/hosts             data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /etc/proftpd.conf   data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /etc/httpd/conf/     data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /etc/httpd/conf.d/  data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /root/  		 data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /var/spool/cron/    data/fs/    ([B]see [/B]2- Data, filesystem backups)
backup  /usr/src/redhat/RPMS/   data/fs/  	([B]see [/B]2- Data, filesystem backups)
backup  /usr/src/redhat/SRPMS/  data/fs/	([B]see [/B]2- Data, filesystem backups)
backup_script   /root/scripts/backupv3/backup_iworx_db.sh       data/mysql/iworx/	([B]see [/B]3- Mysql backups)
backup_script   /root/scripts/backupv3/backup_users_db.sh       data/mysql/users/	([B]see [/B]3- Mysql backups)

1- backup_script /root/scripts/backupv3/so_backup.sh structure/

It is the iworx structure only backup script.
It looks for all siteworx accounts, find the IP and create a directory structure based on the IP. It backups all iworx account structure only backups in the good IP directory

see here for the script http://www.interworx.com/forums/showpost.php?p=13390&postcount=5

here the result
for example

ls /backups/daily.0/

data structure

ls -al /backups/daily.0/structure/

ip-65.110.xx.xx0 ip-65.110.xx.xx1 ip-65.110.xx.xx2 ip-65.110.xx.xx3 ip-65.110.xx.xx4
etc…

ls -al /backups/daily.0/structure/ip-65.110.xx.xx5/

domain-01.com-structure-Jun.08.2007-01.20.42.tgz domaine-02.com-structure-Jun.08.2007-01.21.01.tgz
etc…

2- Data, filesystem backups
We then do a rsync of /home, some important config files (
passwd, group, host_allow, etc), the cron file of the users (/var/spool/cron/ ) etc…

You can set what you want here !

These data are rsynced in a /data/fs/ directory

ls /backups/daily.0/data/fs/

chroot etc root usr var

ls /backups/daily.0/data/fs/chroot/home/

account01 account02 account03 interworx …
etc

ls /backups/daily.0/data/fs/etc/

group hosts hosts.allow hosts.deny httpd passwd proftpd.conf resolv.conf shadow

etc

The needed files for iworx only are :
/home/account/*

But us, we do all /home !

There is also a “exclude file” that list all directory that you want to exclude
exclude_file /root/scripts/snapshot_exclude

Today we exclude some /home directory, the client iworx backups and all stats data oldest than this year

#Exclude rsync dir
/chroot/home/qscand
/chroot/home///iworx-backup
/chroot/home//var//stats/analog/daily/2005*
/chroot/home//var//stats/analog/daily/2005
/chroot/home//var//stats/analog/daily/2006*
/chroot/home//var//stats/analog/daily/2006
/chroot/home//var//stats/analog/weekly/2005*
/chroot/home//var//stats/analog/weekly/2005
/chroot/home//var//stats/analog/weekly/2006*
/chroot/home//var//stats/analog/weekly/2006
/chroot/home//var//stats/analog/monthly/2005*
/chroot/home//var//stats/analog/monthly/2006*
/chroot/home//var//stats/awstats/daily/awstats05
/chroot/home//var//stats/awstats/daily/awstats06

3- Mysql backups
backup_script /root/scripts/backupv3/backup_iworx_db.sh data/mysql/iworx/
backup_script /root/scripts/backupv3/backup_users_db.sh data/mysql/users/

These two scripts does a mysqldump of the iworx database and users databases and then create a tar.gz of these files !

see he for the script http://www.interworx.com/forums/showpost.php?p=13390&postcount=5

The sql backup file are in data/mysql/iworx/ and data/mysql/users/

ls /backups/daily.0/data/mysql/

iworx users

ls /backups/daily.0/data/mysql/iworx/

iworx_db.tar.gz

ls /backups/daily.0/data/mysql/users/

users_db.tar.gz

With our 3 scripts ([B]see here for these scripts http://www.interworx.com/forums/showpost.php?p=13390&postcount=5[/B])

  • rsnaphot we have a full backup solution, ready for easy restore/import/migration processes

We have the last days, the previous one, one week ago and 1 month ago backup data (iworx structure backups and filesystem data)
Even if we keep this history It takes only just a little more than only oen backup (du to the hard link used by rsnapshot)

We are going to create a SRPM of this solution, which will install everything.

You’ll then be able to set what you want by editing two configs file

If some of you are interested, we’ll give this script asap

Pascal

4

Wow great pascal!

I am doing a more simple backup using rsync directly vs rsnapshot but I really like the way rsnapshot takes care of the hard links to not waste space. My current system is good so I think I will wait to see what you come up with before moving forward.

Thanks for this!

Justin

5

Hi Justec

Our script+rsnapshot are in production for more than 10 days. It works like a charm !

If you want I give you our 3 scripts here and you’ll able to test it / use it

There is already the rsnapshot config in my previous post

The /root/scripts/backupv3/so_backup.sh script :

#!/bin/bash
SO_PATH=pwd
SO_PDIR=$SO_PATH"/ip-"
IPLIST=“/you_path_to_iplist/so_backup_iplist”

for MYIP in cat $IPLIST | awk '{print $1}'
do
SO_FULLDIR=$SO_PDIR$MYIP
[ -d $SO_FULLDIR ] || mkdir $SO_FULLDIR 2>&1
for DOMAIN in ~iworx/bin/listaccounts.pex --domain --ip-address | grep $MYIP | awk '{print $1}'
do
~iworx/bin/backup.pex --domains=$DOMAIN --structure-only --output-dir=$SO_FULLDIR
done
done

Configure the list of your existing IP in a file, one per line.
IPLIST="/you_path_to_iplist/so_backup_iplist

This point might be better. We choosed to define the IPLIST and not to search which IP has an account. We could do this with something like :
~iworx/bin/listaccounts.pex --domain --ip-address | awk ‘{print $2}’

But with this solution it will have duplicate entry, so we’d had to remove the duplicated IP. It is possible to do, but… no more time yet !

Here is the 2 mysqldump script
backup_script /root/scripts/backupv3/backup_iworx_db.sh backup_script /root/scripts/backupv3/backup_users_db.sh

backup_iworx_db.sh

#!/bin/sh

backup the users database

mysqldump -uroot -pYOURPASSWD --socket=“/home/interworx/var/run/mysql.sock” --port=“2406” --opt --all-databases > iworx_db.sql
tar -czf iworx_db.tar.gz ./iworx_db.sql
if [ ! $? -gt 0 ]; then
rm -f ./iworx_db.sql
chmod 770 iworx_db.tar.gz
chown root.iworx iworx_db.tar.gz
fi

backup_users_db.sh

#!/bin/sh

mysqldump -uroot -pYOURPASWD --opt --all-databases > users_db.sql
tar -czf users_db.tar.gz users_db.sql
if [ ! $? -gt 0 ]; then
rm -f ./users_db.sql
chmod 770 users_db.tar.gz
chown root.iworx users_db.tar.gz
fi

Hope it helps

Pascal

6

I’m going to try something like what you have done on my new server. One quick questions that I have noticed doing structure backups is they all get a new filename based on the date. How do you manage this structure “history”?

7

I’m not exactly sure what you mean by “manage this structure history”? Can you elaborate?

8

Basically every time you run a structure only backup it creates a new file. Unlink rsync where only changed files are updated, but you can’t really do that with a tgz. So after a few days of doing nightly structure backups they being to pile up. I would have to go in an manually delete the older ones.

Just wonder if there is a nicer way to handle this or maybe just some other script I can run that delete files older than X date.

9

A new question… Is there a way to make rsnap use rsync with links? For example right now I have to do /chroot/home b/c all rsnap does is copy the link “home -> /chroot/home” so it isn’t actually backing anything up. I’m think it might do this for all symbolic links. I think there is a command for rsync to make it follow the sym links and copy them, but also copy the symlinked path. Is there a way to pass that through rsnap?

I’m looking at a backup in progress now and /chroot/home/interworx backup is interworx -> /usr/local/interworx, but that points to the main file so that is basically not being backed up

I see you can put commands for rsync so you can change it to use L instead l. But not even sure if thats better or not, might be better just to make a backup of the where the symlink is pointing to and just leave the symlink in the backup. Only thing is if you are screwing around with a backup folder and mess with a symlink you are messing with files on the live server.

Another thought…
Symlinks with relative paths like backups -> domain.com/iworx-backup make perfect sense like this since the data will be backed up anyway and you dont need it twice, its just the absolute path symlinks that would be a problem.

10

[QUOTE=Justec;14214]Basically every time you run a structure only backup it creates a new file. Unlink rsync where only changed files are updated, but you can’t really do that with a tgz. So after a few days of doing nightly structure backups they being to pile up. I would have to go in an manually delete the older ones.

Just wonder if there is a nicer way to handle this or maybe just some other script I can run that delete files older than X date.[/QUOTE]

You can change the backup filename using the --filename-format parameter:

You could do something like: --filename-format “%D”

which would create filenames like domain1.com.tgz, foobar.net.tgz, etc.

11

Thanks Socheat, I figured I could do something like this. Would be interested to see how Pascal is doing this now.

Also, anyone have any insight on a best practices with regards to symbolic links and backups?

12

Cron issues…

 *   	  */6   	  *   	  *   	  *   	nice -n 10 rsnapshot hourly

So this should run ever 6 hours and it seems to work, but for some reason it tries to run over and over and over once it fires.

So I get a bunch of emails from CRON. I get 1 email every min starting at xx:01 for one hour.

It is running, but not sure why it tires to run it again.

ps shows:

/usr/bin/perl -w /usr/bin/rsnapshot hourly

I’m obviously doing something with the CRON, but not sure what it is?

Any ideas?

EDIT
Ooops :o

 [B][SIZE="3"]0  [/SIZE][/B] 	  */6   	  *   	  *   	  *   	nice -n 10 rsnapshot hourly
13

[QUOTE=Justec;14218]Thanks Socheat, I figured I could do something like this. Would be interested to see how Pascal is doing this now.

Also, anyone have any insight on a best practices with regards to symbolic links and backups?[/QUOTE]

Hi

Not sure to understand, so it is difficult to answer you :wink:

For the link, there is an option in rsync to copy the content of the link or just the link (see man rsync and the -l and -L option). But I think it is a good think to just copy the link.
Indeed in /chroot/home there is for example the vpopmail symlink which points to /var/vpopmail/

I dont rsync this directory as when I"ll restore the accounts these data will be created, so the symlink will point to the right place.

Interworx team, could yu please confirm that with a data only backup when you restore the account all informations needed in /var/qmail and /var/vpopmail are created well and so we don’t have to backup these data ?

Pascal

14

[QUOTE=Justec;14214]Basically every time you run a structure only backup it creates a new file. Unlink rsync where only changed files are updated, but you can’t really do that with a tgz. So after a few days of doing nightly structure backups they being to pile up. I would have to go in an manually delete the older ones.

Just wonder if there is a nicer way to handle this or maybe just some other script I can run that delete files older than X date.[/QUOTE]

Ok in fact as we use rsync with rsnapshot we don’t have this problem. We do a rsnapshot for 2 days (daily.0 & daily.1) so all rotation are done by rsnapshot (using hard links)

We also copy the datas of the accounts in different directories. Every datas go in a directory which has the IP of the account as name. This is very usefull when you do tje restore. Like this you’ll be able to restore all accounts of one IP to a new IP. With this solution you can do a transfert/restore of all accounts IP by IP (which allows us for example to easily tranfert all accounts of the IP xx to a new box for example and allows us to migrate all accounts to a new box with exactly the same IP schems )

We use our scripts for 5 months now and are very happy !
Before it took us 8 hours to backup all our accounts when it take us only 3 hours with this solutions and the data only backup option :slight_smile:

Pascal

15

First a question for Iworx team. When you backup / restore an account using a structure or a full iworx backup does it backup the users blacklist, spam settings, learn spam/ham history for the domain, etc?

I recently moved and one of my clients told me they are getting a lot more spam not marked as spam. Also, there drop score was reset back to the default instead of what I set it to previously.

[QUOTE=pascal;14253]For the link, there is an option in rsync to copy the content of the link or just the link (see man rsync and the -l and -L option). But I think it is a good think to just copy the link.[/quote]Yeah, and you can force rsnapshot to use the -l option. My question is what’s the better thing to do. There are some links that are relative like public_html which if you use the -l you would get the same data twice which isn’t good. But there are some situation where you would not get data you need.

Right now I believe what I’m going to do is just copy links as links and then if I need the data the link is pointing to, to just back it up directly.

The other thing I mention is when you backup a symlink as a link for example “/var/whatever” if you are messing around in your backup directory you also run the risk of messing up the live directory since /var/whatever is a absolute path. But Im sure your backup is for root only access so not a problem.

[QUOTE=pascal;14253]Indeed in /chroot/home there is for example the vpopmail symlink which points to /var/vpopmail/

I dont rsync this directory as when I"ll restore the accounts these data will be created, so the symlink will point to the right place. [/QUOTE]I believe the structure only would just have the email accounts and maybe the settings (spam, blacklist, etc) but not the email. But all of these should be under their home directory. The /var/vpopmail/domains just links back to the home directory so that should be fine.

[QUOTE=pascal;14254]Ok in fact as we use rsync with rsnapshot we don’t have this problem. We do a rsnapshot for 2 days (daily.0 & daily.1) so all rotation are done by rsnapshot (using hard links)[/quote]Yes, I am also using rsnapshot now. My question was in reference not to the rsnapshot data backup, but the Iworx structure backup. By default it creates the filename based on the date/time so it would never delete the old structure backup and would eventually fill up the backup disc.

[QUOTE=pascal;14253]We also copy the datas of the accounts in different directories. Every datas go in a directory which has the IP of the account as name.[/quote]Yes, I’m doing this too with your script for structure (thanks :slight_smile: ), but the problem above still remains. The only difference is you call the outside struct backup through rsnap so maybe that does something different?

16

I believe the structure only would just have the email accounts and maybe the settings (spam, blacklist, etc) but not the email. But all of these should be under their home directory. The /var/vpopmail/domains just links back to the home directory so that should be fine.

Yes it is !

[QUOTE]We also copy the datas of the accounts in different directories. Every datas go in a directory which has the IP of the account as name.

Yes, I’m doing this too with your script for structure (thanks ), but the problem above still remains. The only difference is you call the outside struct backup through rsnap so maybe that does something different?[/QUOTE]

It is indeed !

I have added the "outside struct backup (iworx one) through rsnapshot to have the exact same rotation shems thna the data I backup through rsync with rsnap. If you don’t you might have a difference betxeen the data accounts stored in iworx and the data you backup through rsync. Launching the struct only backup in rsnaphot prevents to have a difference between the struct only backup and the rsync one and also allow to have the same rotation shems between these two backups methods. Witht this solution everytime I rsnapshot is launched, all the date are rotated and are syncs (strcut only and rsync)
In one word, launchng the iworx struct only backup in rsnapshot resolve your problem as the dbackup filename are rotated !

Pascal

17

Pascal how do you call your so_backups.sh from rsnapshot conf?

I get the following error when trying to call this from rsnapshot.

ERROR: /etc/rsnapshot.conf on line 199:
ERROR: backup_script /backup/scripts/so_backup.sh /backup/so_backups/ \
         - no destination path specified
18

Use this

LOCALHOST

backup_script /root/scripts/backupv3/so_backup.sh structure/

YOU HAVE TO SET A DESTINATION !
Here it is

structure/

19

1- Be sure to have a TAB space between every entries
2- The destination can’t be /xx/ but rather xx/

Paul

20

Thanks for the tips Paul!

  1. I had a space between the last 2 entries

  2. I was trying to put it to a path outside of the path where the other snapshots are. So I created my structure_backup directory inside the snapshot directory and did a relative path ./structure_backup