FTP backups not completing intermittently

I’m using the following script in the ‘iworx’ user’s crontab to perform a weekly backup of all domains on the server via ftp to another server:

/home/interworx/bin/backup.pex --domains all --backup-options all --xfer-method ftp --xfer-ini /home/interworx/transfer.ini --compression 9 --email larryh@angelichost.net --quiet

This was working for quite some time, but recently the backups have become intermittent. Occasionally I get all domains, but more often than not, I’m only getting a couple of dozen domains backed up.

I get emails for those that successfully completed, but nothing else for any of the other domains (i.e. no failure messages). For example, this morning’s backups only included 20 domains. If I log into the backup server, there are 20 gzipped tarballs, with timestamps spread over about 10 minutes.

Spot checking through 5 weeks of history, I found that the domains that are being backed up or missed are not consistent. A domains that is missed one week might be backed up the next, and vice versa.

Any direction would be appreciated.

Larry Hiscock

Nothing? Anyone?

Additional info: I just attempted to backup 8 domains locally (before deleting them), through the Nodeworx control panel, as the administrator. Six completed successfully, one started but never completed, and the last one never started. The one that didn’t complete had a “.tgz” file in the iworx-backup directory, along with a “.pid” file, but the pid it referenced didn’t exist.

I cleared the .tgz and .pid files from the iworx-backup directory, and re-ran the backup for the two domains that didn’t complete, and they went through fine the 2nd time.

Curioser and curioser …

Sounds to me like you may be running out of disk space on the partition housing the backup directory. That’s just a guess, though - I’d have to look inside the machine to know for sure, and for that you’d have to open up a ticket.

Thank for the response Robert, but not much chance of that. The server that the domains are on has 195GB free out of 250GB, and the server that the backup files are being sent to has almost 2TB available.

Even if you have lots of space on the server, I found out you need plenty of space in /tmp for big backups - I had a similar problem with iworx backups on a server with lots of disk space which turned out to be /tmp (on its own dedicated partition in this case) running out of space. It was solved by changing the location of tmp in iworx.ini in the [iworx.dir] section to sowewhere with lots of free space.

May not help, but worth a mention, I thought…