Backup options

I found this in my drafts folder of the mail client. May be too late for 3.0 (as I see “Improved Backup System” in the 3.0 features), but as I said I just found it in my drafts folder and I thought I could paste here.

  • deleting previous backup files for scheduled backups with option to how much (say 2 or 3) backups to keep
  • option to exclude "stats" in backup
  • option to compression level for the backups (this may help reducing the load/time-taken)

[QUOTE=tiger;9620]

  • deleting previous backup files for scheduled backups with option to how much (say 2 or 3) backups to keep
  • option to exclude "stats" in backup
  • option to compression level for the backups (this may help reducing the load/time-taken)
[/QUOTE]

You’ll have much more control over the filenaming of the backups, so it’ll be much, much easier for you to setup your own custom backup rotation system. Also, excluding different “portions” of an iworx account is also easier now as well. And, adding control for compression may or may not make it into this next release, we’ll keep you posted. :slight_smile:

Socheat

[QUOTE=IWorx-Socheat;9621]You’ll have much more control over the filenaming of the backups, so it’ll be much, much easier for you to setup your own custom backup rotation system. Also, excluding different “portions” of an iworx account is also easier now as well. And, adding control for compression may or may not make it into this next release, we’ll keep you posted. :slight_smile:

Socheat[/QUOTE]

cool :slight_smile:

I’m not sure to understand why there is the structure only option ?
It means a backup of home&vpopmail&qmail with rsync or a special tool + your backup, right ?

I know the tar process is intensive usage cpu/mem but to do this without tar you need a second HDD, not in raid or a NAS to rsync the data of the iworx accounts. It means you duplicate the data. I know the GB is not so more expansive but… a NAS is still expensive, so…

Socheat do you know why this structure only will be used to ?

Pascal

[QUOTE=tiger;9620]

  • option to exclude "stats" in backup
[/QUOTE]

An option to exclude certain directories would be great :smiley: - I’m mirroring a small ISO of a new Linux distribution on my website, which results in my backups being enormous, instead of the usual ~50MB or so.

[QUOTE=pascal;9684]cool :slight_smile:

I’m not sure to understand why there is the structure only option ?
It means a backup of home&vpopmail&qmail with rsync or a special tool + your backup, right ?

I know the tar process is intensive usage cpu/mem but to do this without tar you need a second HDD, not in raid or a NAS to rsync the data of the iworx accounts. It means you duplicate the data. I know the GB is not so more expansive but… a NAS is still expensive, so…

Socheat do you know why this structure only will be used to ?

Pascal[/QUOTE]

Think of “structure” only as backing up the core information of the SiteWorx account, the information that’s stored in the internal InterWorx database, which you normally don’t have access to.

For example, we’ve personally found it very useful for sites with extremely large databases (2GB or more). We can quickly backup and import the “structure” of a SiteWorx account (email accounts, database names and passwords, ftp accounts, quota limits, etc), and then we can just copy the database data directory (/var/lib/mysql/uniqname_dbname) over to the new server.

Socheat

Already done. :slight_smile:

Woo! Cool :smiley:

[QUOTE=IWorx-Socheat;9686]Think of “structure” only as backing up the core information of the SiteWorx account, the information that’s stored in the internal InterWorx database, which you normally don’t have access to.

For example, we’ve personally found it very useful for sites with extremely large databases (2GB or more). We can quickly backup and import the “structure” of a SiteWorx account (email accounts, database names and passwords, ftp accounts, quota limits, etc), and then we can just copy the database data directory (/var/lib/mysql/uniqname_dbname) over to the new server.

Socheat[/QUOTE]

Yes it might be usefull, but maybe not for a complete backup and recovery plan.
I’d do some tests to see if doing a structure only backup for a all accounts + rsync/tar to a NAS all needed directories (home;vpopmai;qmail,var/spool/cron) is speedest than just doing a full backup of all accounts… Not sure

I still looking for the best backup/recovery plan for cases like :

  • 1 hdd crash --> raid is one solution
  • 2 hdd crash --> full backup is needed *
  • kernel panic --> full backup is needed *
  • files corruption --> partial backup is ok
  • full backup = siteworx accounts backups (or home;users db; vpopmail; etc); interworx db, iworx ini; some etc files

Today we do two kinds of backups:

1- We tar.gz some important directories

  • Interworx DB ( /usr/local/interworx/var/lib/mysql/* )
  • Interworx config ( /usr/local/interworx/iworx.ini )
  • Users DB (/var/lib/mysql/*)
  • Some ETC files (etc/passwd /etc/group /etc/shadow /etc/hosts.allow /etc/hosts.deny /etc/resolv.conf /etc/hosts /etc/proftpd.conf)
  • Qmail files (/var/qmail/*)
  • Vpopmail files (/var/vpopmail/*)

2- we do a full siteworx accounts backups, then tar.gz thz dir

3 - copy all these tar.gz files to a NAS location

Like this we have all siteworx accounts backuped, so we can do a simple restore of one or more siteworx accounts if needed. If we have a migration to do we have all data and we can choose to import all siteworx accounts from the full backup or to manually copy the iworx db, iworx ini, user db then construct the home dir and data from the full siteworx account backup.

Indeed we do not tar.gz the home dir it really take to long, so we use the full siteworx backup to create it (not so cool, but…)

It is not the best backup and recovery plan, but it did already help us.

If you have better solution, do not hesitate to share with us what is the best backup/recovery plan for you

Pascal

I won’t argue with you about the usefulness, as that is something each client will determine for themselves based on their recovery plan. :slight_smile:

This is exactly the reason why we created the structure-only importer. You can’t simply dump the internal iworx database and restore it, and have an instant, working version of InterWorx. In most cases, there’s a bit of work that must be done before things are happy again, as this method assumes too much. And, of course, we could never offer much support for people who tried to recover using this method.

If you did a structure-only backup of every site on the box, you have essentially backed up all the SiteWorx related data in the internal iworx database. Additionally, we can/will support this method of recovering. :slight_smile:

This brings up another good point. We’ve had a few clients (including you Pascal, I believe), who have either many accounts on a single box or accounts that are very large in size, or both. Clients have found that doing a full siteworx backup of all accounts on boxes like that can take a whole day or longer. Doing a structure-only backup of all accounts, by comparison, will be much faster, especially if you already have some other mechanism for backing up your user/client data. In other words, if you’re already rsyncing or mirroring data to another drive, having the SiteWorx backup tar up and compress that same data is using up additional time and resources.

Not trying to sell you on structure-only backup/restore, because again, it will all depend on each business’ recovery plan. Just wanted to give you some insight into what we were thinking when we added this feature. :slight_smile:

Socheat

I’m sure it might be usefull but do you think it will be more speed ?

You’ll have to :

  • structure only backup
  • tar home
  • tar vpopmail
  • others ?

then scp to a NAS

people who habe a second hard drive might only do a rsync, but for those whom have a NAS, I do not want to rsync 150GB to my nas. A lot of bandwidth and time. so I’ll have to tar.gz home and I’m pretty sure it will take same time than full iworx backup.

But I’m pretty sure it is a good thing to have a mutliple choice backup tool. It is great.

And I’m sure I could fuind a way to finally have a good backup/recovery plan with this option (I think to buy a new server and connect it to a private lan so, rsync could be fine)

Thanks again for all

Pascal

When InterWorx does a backup of a domain, the only directory we tar up is /home/uniqname, since mail is stored in /home/uniqname/var/domain.com/mail. And yes, I agree, if you choose to tar and compress the data yourself, you probably won’t be much faster than doing a full backup.

Socheat