I think this could be another alternative for rsync backups, as this doesn’t require the daemon part of the rsync. So, the clients could do rsync like backups without having their iworx host/server running rsyncd. (a windows binary of zsync clinet is not available on their site; a cygwin port might be available, don’t know about that)
zsync currently works only on single files. And in compressed files only for .gz files. So, ideal for siteworx backups? In the site (http://zsync.moria.org.uk/server), it is said that creating the .gz file using “zsyncmake -z” is more efficient than to have the normal already compressed .gz file.
Could this be (easily) built into iworx?
It’s just a thought, I don’t know you will be considering this or not.
I apologize Tiger, this one slipped through. No, we don’t plan on adding zsync support to the backup system. We don’t even have rsync in the backup system. Since our backups are tar-gzipped, it doesn’t really make sense to rsync/zsync the archive file. However, if you use the new, upcoming, structure-only feature, you are free to use whatever method you want to backup your site data yourself, be it rsync, zsync, cpio, or whatever.
Yes, I know it and I didn’t request that. zsync seems to be a simple program (runs by itself) and the backup is just doing a download.
Since our backups are tar-gzipped, it doesn’t really make sense to rsync/zsync the archive file.
At zsync Server-Side , it is said that zsync works with .gz files (then why not for tar.gz?)
However, if you use the new, upcoming, structure-only feature, you are free to use whatever method you want to backup your site data yourself, be it rsync, zsync, cpio, or whatever.
It is fine if clients could run zsyncmake on their backups to download(zsync) the backups.
Why bzip2 instead of gz? Due to better compression?
Better if you could provide an option for it to be in bzip2 or gz (or simply tar).
So that users have the option to rsync (the tar) or zsync (the tar.gz).
As you said users can create the tar/tar.gz of (mail & web data) themselves along with structure-only backup. But, what is the case for database backups? Need to do a mysqldump and then compressing it?
Instead of all the above, better to have an option for tar.gz/tar/tar.bzip2
Let the user decide, and I think it won’t be much difficult to implement this.
(may be not in 3.0, but for a future release)
I agree, that would be the ideal situation. However, for various technical reasons, we had to choose a different compression than gzip.
We contemplated an “importer auto-detect” feature, where you no longer had to pass --control-panel= to the import.pex command. This would allow you to do what you propose, let the user decide the compression type. However, it was complex enough that we decided to table it for a future release.
I mean the advantage of rsync is to detect which files have been updated between source and destination and to upload only the updated files (differences between source and destination). With a tar file it will be always the case ? no ? the tar file will always be updated after a siteworx account backup.
I mean the advantage of rsync is to detect which files have been updated between source and destination and to upload only the updated files (differences between source and destination).
Not only that.
the tar file will always be updated after a siteworx account backup.
Yes, it would. But, as rsync transfers only the changes within the file, there is no problem.
And another thing is, I have read that the memory requirements of rsync directly depends on the number of files it operates on. Being a single (tar) file, you are not affected with that (assume you are low/(not high) on memory). But, not sure the tar (a single file) could go beyond 2GB on some servers.
Of course, you could delete the tar, after you rsync it, to gain disk space. But the problem is you need 2 times the space that of your site’s disk requirements, at least temporarily.
I guess TARing the files won’t make the backup not rsyncable. But, I am not sure. (as long as the order of files getting added to the tar is not changed, it should be OK; even in that case the subsequent backup should get back to the optimal transfer)
Anyways, InterWorx people are not going to choose tar/tar.gz and they are with tar.bz2 , so you are on your won (Luckily, they haven’t chosen .7z/.zip which is not yet widely known/used in the linux/unix world).
I think we need to be clear on our terms here, I meant tar – tar is a packaging format which gathers all the files up into one huge file. THIS (as opposed to gzip, bzip or zip) is used to preserve the file perms and ownership, but doesn’t really reduce the overall size (at least not that much). gzip (currently) and bzip (in 3.0) are used to compress the file size to something more manageable. Even thouth they are generally used together tar and g/bzip are two different things.
OK, I just wanted to make sure you didn’t mean simple .tar as the backup format (i.e without gz/bz2) as Socheat said that tar.bz2 for 3.0.
I know that gzip/bzip2 can compress only one file and 7zip could add multiple files. I guess .7z on Unix doesn’t yet save the file owner/perms (it wasn’t some time ago). Anyways, 7zip would require more memory/time to compress (of course depends on options) than the others (but better compression) and also not widely know as gzip/bzip2. Another one is RAR available for all OSes, but that isn’t free and ( also not widely know ).
tar is a packaging format which gathers all the files up into one huge file. THIS (as opposed to gzip, bzip or zip) is used to preserve the file perms and ownership, but doesn’t really reduce the overall size.
I knew this.
gzip (currently) and bzip (in 3.0) are used to compress the file size to something more manageable. Even thouth they are generally used together tar and g/bzip are two different things.
