I began to think how much cleaner and easier it would be to do backups through NodeWorx instead of doing the kinda hack i was thinking about in the thread above.
So no I have decided to make this an official feature request. I’m pretty sure there is a Secure FTP and Rsync option for the current backup system but I assume this still does zip’d tar files correct?
Would it be possible for you to have another option for not zip’n the files so that you can do an rsync with the option to only update the changed files? I think this would be a great option to have especially when doing off site backups. This would offer the main benefit of saving time on the transfer and on the total amount of GB used. On top of this there would be a slightly lower server load because of the no-zip’n part of it.
I know you have a long list, but maybe you can sneak this one in there if it isn’t too much work and someone else thinks it could be useful
We’re one step ahead of you Justec The new backup system has a Structure-Only backup option, which allows you to create a backup that only has account information (e.g., email users, email passwords, database names, database users/passwords, ftp users, etc). However, no backup data (email data files, website data, or database data) is stored in the backup. This way, you can use whatever method you want to back up the actual data, but will still be able to restore the “structure” of the SiteWorx account.
Will produce a valid SiteWorx backup archive for ‘mydomain.com’, but it will only contain the structure information, no actual data files. You would be responsible for backing up the user’s home directory (which includes emails and web files), and the user’s database files. Obviously, depending on your needs, you can choose to leave any of these out.
To restore, you would first restore the backup just like any other SiteWorx backup, either using the import.pex from the command line or the NodeWorx interface. After the restore completes, you would copy back the account data (/home/uniq/ and the database data in /var/lib/mysql/).
When my hardrive failed a few months ago, I just restored the drive from my weekly full backup of “/” and then my nightly incremental backup of “/”.
However we did have to rebuild the boot partition first and then restore the filesystem, but all in all the entire restore took less than a 4 hours when we started it.
The hosting provider really screwed the pooch on it for a couple of days before I got someone on the phone and we proceed to do the restore from my backups and in the interim they tried several things that prolonged the outage considerably.
As a result, they provided me with a mirror hard drive (at no cost) that is rsysnc’d every night.
The server can be switched to the backup drive and be up and running in a matter of minutes, we’ve tested it a couple of times and it works great.
I still backup everything with “backup” nightly also, allowing me to restore data files pretty easily should something need to be restored…
Why use rysnc? When I hear mirror i think if RAID?
In my new server I will be using a software RAID mirror in addtion to the daily rsync backup of /etc, /home, /var/lib/mysql.
Socheat,
Is doing a structure backup any different than back’n up the mysql files that are under /home/interworx/var/lib/mysql?
If i restored all the files in that directory on a fresh server that should bring back all my siteworx accounts and settings? If so is there any beneifit to doing a NodeWorx (or command line) structure backup versus back’n up the interworx mysql directory?
A structure only backup is on a per account basis, and only gets the data relevant to that SiteWorx account out of InterWorx databases. You can obviously create a structure-only backup of multiple accounts, but it will create a separate backup archive for each account, rather than putting them all in the same backup.
Backing up the entire InterWorx-CP mysql directory will get the data on all your SiteWorx accounts, plus all NodeWorx structure related information (site packages, nodeworx user passwords, etc).
Benefits? Depends on what you need, I guess. It’s definitely useful if your InterWorx-CP database gets corrupted, or if someone ran goiworx.pex on an already running machine (everyone remembers that’s a no-no, right? ) and hosed your database. You also get a backup of all NodeWorx related information (again, no data, just “structure”).
If you want to be able to bring up the exact same machine/setup (all the same accounts) from a fresh install, restoring the InterWorx-CP database might work. I’m sure there are a few things that will need to be done after restoring the database. However, as we’ve never actually tried it, and in general we don’t condone messing with the InterWorx-CP database directly, we can’t support this method. This method assumes too much, and we’re not sure what would happen if one of those assumptions were wrong.
Doing structure only backups of all the accounts on your box gives you a little more flexibility (and our support). You can easily restore all of the backups or you can only restore a subset of them, and restore the others on different boxes.
What you don’t get is a NodeWorx-structure backup (a backup of all your NodeWorx settings), which we’ve thought about before and is on our somewhere-down-the-road-todo list
The first time it will take a pretty long time to run but after that it runs in a matter of minutes…
This is a really easy and cheap method for having a failover drive in case of a hd failure, I additionally backup the entire system to a third drive weekly using linux backup and I backup changed files (level 1) nightly, and take backups of all mysql databases 4 times a day.
This method works very well and only takes a few minutes a day to backup, its easier for me to manage the backups at this level rather than have my users do it. Of course they are welcome to if they prefer…
Thanks for the explanation. I dont have any crazy setting in NodeWorx that I can’t redo manually so I will just stick to the support’d side of things so when I do something stupid I can ask for help
Don’t get me wrong, a backup of the InterWorx MySQL dir is very useful in certain situations, and you should probably keep doing it. But it’s definitely for one of those specific cases where the InterWorx database becomes corrupt.
That sounds great. I’ve had to rebuild my server using the mass transfer (with a temporary second InterWorx server) and the whole process takes 18 to 24 hours. My box has an empty hard drive bay and I plan to set up like yours.
I already use open source software called BackupPC to pull backups to a local box. That solution is great for restoring particular files or sites but I’d hate to try to rebuild the server from those backups.
Justin – My preference for Rsync over RAID is that the Rsync method described backs up only once per night, whereas RAID is constantly synchronizing. For the problems I’ve had it would be great to be able to go back to last night’s backup, instead of having OS issues or infections replicated instantaneously. Plus, my server doesn’t have an RAID controller (FYI: I looked at your site - very clean - and noticed that the word “quarterly” was spelled “quaterly”).
:o Opps, guess I should have caught that, thanks for let’n me know!
Thanks for the postive comments on my website, thats exactly the look we were going for.
Yeah, that makes sense for file protection (non-hardware failure) which is probably more often the situation rather than complete hard drive failure. I think even if I go with raid on my new server I will continue to do a rsync backup (even if its to the same hard drive) so that I will have a history of X amount of days of files (I do a copy first of the last nights Rsync to another directory before rsync the current days).
Socheat,
I think the Iworx database backup is good for a restore back onto the same server, like you said if the IW DB gets corrupt or hard drive failes, etc. But I decided I will not try to do that moving to a new server b/c I will have a new license number and new IPs, etc. so I dont think it would make sense.
Actually, that brings up another quick question, when importing a SiteWorx structure do you assign it an IP on restore? I have like 6 static IP account then the rest are on a shared IP.
It’s been a while since I’ve done any of this from the command like so bear with me if I’m wring but the install wizard should ask you what IP to assign the domain to.
I just re-read your post, Justec. A “structure-only” backup is still a SiteWorx backup, it just doesn’t have any data files in it. It gets restored the same way as a full backup would, it will need an IP address when being restored, and the end result is a new SiteWorx account. The only thing different is the restored/imported SiteWorx account won’t have any user files in it (no emails, no website data, no databases or database tables).
The new backup system has a Structure-Only backup option, which allows you to create a backup that only has account information (e.g., email users, email passwords, database names, database users/passwords, ftp users, etc). However, no backup data (email data files, website data, or database data) is stored in the backup. This way, you can use whatever method you want to back up the actual data, but will still be able to restore the “structure” of the SiteWorx account.
) and hosed your database. You also get a backup of all NodeWorx related information (again, no data, just “structure”).