Announcement

Collapse
No announcement yet.

Considering the switch...

Collapse
X
  • Filter
  • Time
  • Show
Clear All
new posts

  • #31
    Is there a way to direct these backups to a 2nd hard drive automatically instead of to the /home/siteworx folder?

    I guess I could just run a script that scanned all the home folder and then moved the back files to the 2nd drive, but would be much nicer to do it in one shot.
    [ JUSTIN ]
    [ OFF unit ]
    [ WEB DESIGN / DEVELOPMENT, GRAPHIC DESIGN, OTHER STUFF
    ]

    Comment


    • #32
      Originally posted by Justec
      Is there a way to direct these backups to a 2nd hard drive automatically instead of to the /home/siteworx folder?

      I guess I could just run a script that scanned all the home folder and then moved the back files to the 2nd drive, but would be much nicer to do it in one shot.

      Here's how I do my backups:
      (/dev/hdc1 is my secondary drive, mounted at /mnt/backup)

      [/etc/cron.daily/backup, which is a symbolic link to my backup script /usr/local/sbin/backups.sh]
      Code:
      #!/bin/bash
      
      rsync -av /etc /mnt/backup
      rsync -av /home /mnt/backup
      rsync -av /service /mnt/backup
      rsync -av /var /mnt/backup
      This backs up everything. All the logs, all the configurations, all the email, site data. Because it's rsync, it only copies the files which are new or changed. So I'm not doing huge copies every night, just "incremental" (okay, pseudo-incremental).

      Comment


      • #33
        Originally posted by cdemain
        Here's how I do my backups:
        (/dev/hdc1 is my secondary drive, mounted at /mnt/backup)

        [/etc/cron.daily/backup, which is a symbolic link to my backup script /usr/local/sbin/backups.sh]
        Code:
        #!/bin/bash
        
        rsync -av /etc /mnt/backup
        rsync -av /home /mnt/backup
        rsync -av /service /mnt/backup
        rsync -av /var /mnt/backup
        This backs up everything. All the logs, all the configurations, all the email, site data. Because it's rsync, it only copies the files which are new or changed. So I'm not doing huge copies every night, just "incremental" (okay, pseudo-incremental).
        I am currently doing something similar. My Rsync is a little different as I include the --delete which removes files from the backup which are no longer on the disc, but I do keep 10 days worth of seperate backups.

        Here is one of the lines from my script:
        rsync -ax --delete /home/ /backup/0/home

        All of this is great if you need to recover a file here or database there, but if you have a dead hard drive you will have to recreate all the siteworx accounts again (emails, etc.) and then copy the data back.

        I basically want to do both, have the incremental file backup and also have SiteWorx backups. I could do the SiteWorx once a week and would be used to restore the accounts since once they are setup they shouldn't change too much. Then I'll have the daily file backup to restore files to the latest ones if needed.

        Then I also want to create another cron to SSH-rsync the files to a remote linux box for offsite storage.

        My question is still using the command line InterWorx backup feature, is it possible to set the backup destination directory?

        I would try to figure it out myself, but the backup.php (which is run by backup.pex) is protected (as it should be :D ) so I don't know what the other options are. Maybe you guys (Chris, Paul) could add a --help to some of the command line PEX (PHP) files to help us out
        Last edited by Justec; 03-09-2005, 03:32 PM.
        [ JUSTIN ]
        [ OFF unit ]
        [ WEB DESIGN / DEVELOPMENT, GRAPHIC DESIGN, OTHER STUFF
        ]

        Comment


        • #34
          Now that I thought about it more, the rsync may be enough.

          If you get the /home directory this include the /home/interworx directory which contains all the InterWorx setup info (database, config files, etc). Now this wouldn't be very useful if you need to restore a single account it should work in the "doom's day" situation.

          I am really just guessing here so I would like to here from someone from InterWorx, but it does make sense (although I'm sure I have missed something)
          [ JUSTIN ]
          [ OFF unit ]
          [ WEB DESIGN / DEVELOPMENT, GRAPHIC DESIGN, OTHER STUFF
          ]

          Comment


          • #35
            Thanks everyone, this is exactly the type of info I need. I can wait on a scheduled backup, I manage everything on my sites from top down and am not available to log into the system each and every day to backup via a mouseclick, thus the need for a scheduled one. I ended up using webmin to back up the data because ensim's backup program failed most of the time. In addition they write out a file for each backup each day which meant that pretty soon if I did not clean them up very often I had hundreds of files out there pretty quickly.

            Thus any solution that avoids that would be nice. Of course any solution that makes work avoidance by me easier is always nice, but I digress.

            I'm just beginning the migration process, sort of stalling to see if an ensim script finds its way into my beta testing hands providing for a means for that work avoidance thingy.

            Comment


            • #36
              Justec...

              If that's the case, I could use webmin to just backup everything at /home and use webmin to restore a single site if needed by restoring /home/sitename.

              I'm guessing that a problem might arise if something about the config of the site that interworx keeps up with changes. In addition, what about sql databases, etc? Should I backup /home/interworx a few times a day for safekeeping?

              I just tested that with webmin and it only dumped 129 meg and took less than 15 seconds, of course this is on an empty system with only one site on it right now, but taking a backup like that 4 times a day is certainly in the picture.

              Comment


              • #37
                Originally posted by bluesin
                Justec...

                If that's the case, I could use webmin to just backup everything at /home and use webmin to restore a single site if needed by restoring /home/sitename.
                I've never used webmin, so I dont know how that works. But InterWorx keeps a lot of information in its own Mysql database (and instance of MySQL seperate from where all the SiteWorx user database are stored).

                What I was thinking is if you backup the the entire /home directory (which is something every backup should do) you will get a copy of the InterWorx config and database which may be able to restore the SiteWorx accounts to a new hard drive. This would NOT give you the ability to restore one though, it would be all or none kind of thing.

                And even then, everything I've said above may not even work. I have never tried that so I can not say if that would work or not, just trying to throw out ideas
                [ JUSTIN ]
                [ OFF unit ]
                [ WEB DESIGN / DEVELOPMENT, GRAPHIC DESIGN, OTHER STUFF
                ]

                Comment


                • #38
                  webmin will backup the directory /home to a specified file name. You can specify level of dumps with 0 being a full backup, 1 new or changed since a previous dump level etc, all the way up to 9. This allows for incremental backups at any interval really.

                  I take two backups to two different drives, a full backup once a week and a level one dump daily at midnight, all to the same file name, meaning that there is always only one backup file with a mirror on another drive. When you want to restore you just specify the backup filename, the directory or file you want to restore and it takes care of the rest.

                  Thus I was thinking that if I backed up the /home directory each night, this would capture everything for the interworx and all of the sites, and if I backed up the /home/interworx directory say 4 times a day I could keep up with any changes to interworx itself to within 6 hours.

                  I'm wondering if it'll work also, one of the important considerations is sql data...

                  Comment


                  • #39
                    It will not work, you will have your basic web and email files, but you will miss your database files located here /var/lib/mysql and your httpd configuration files located here /etc/httpd and below (you need EVERYTHING IN /etc/httpd/conf.d/ ).

                    In addition you may have file permission problems during restore. InterWorx has very specific file permissions that it needs in order to work. Certain files need to be owned by the individual siteworx users, some by iworx, and some by root (and possibly httpd, I don't recall exactly).-- I know I've screwed them up before ;-) )

                    Comment


                    • #40
                      Originally posted by Justec
                      Is there a way to direct these backups to a 2nd hard drive automatically instead of to the /home/siteworx folder?

                      I guess I could just run a script that scanned all the home folder and then moved the back files to the 2nd drive, but would be much nicer to do it in one shot.
                      Right now InterWorx itself only recognizes one hard drive, so youneed to do soimething outside of Iworx to do this suck as the others have mentioned.

                      Comment


                      • #41
                        Originally posted by Justec
                        My question is still using the command line InterWorx backup feature, is it possible to set the backup destination directory?

                        I would try to figure it out myself, but the backup.php (which is run by backup.pex) is protected (as it should be :D ) so I don't know what the other options are. Maybe you guys (Chris, Paul) could add a --help to some of the command line PEX (PHP) files to help us out
                        When you add "--siteworx" to the backup.pex param list, that automatically sets the filename to a standard filename, <domain>[_backup-type]_<timestamp>.tar.gz, and a standard location, /home/<uniqname>/backups. things in <> are required, [] are optional. _backup-type indicates what type of partial backup was made, and is omitted on a full backup.

                        However, if you replace --siteworx with --file=<path/and/filename.tar.gz> you can specify any filename and location you want. However, if you do not place this file in /home/<uniqname>/backups, it won't show up in the NodeWorx/SiteWorx backup management interface.

                        We can add a --help. :)

                        Originally posted by timryberg
                        It will not work, you will have your basic web and email files, but you will miss your database files located here /var/lib/mysql and your httpd configuration files located here /etc/httpd and below (you need EVERYTHING IN /etc/httpd/conf.d/ ).

                        In addition you may have file permission problems during restore. InterWorx has very specific file permissions that it needs in order to work. Certain files need to be owned by the individual siteworx users, some by iworx, and some by root (and possibly httpd, I don't recall exactly).-- I know I've screwed them up before ;-) )
                        This is correct. Even if you backed up all that information, it won't be easy to restore all of NodeWorx (though obviously easier if you did manage to have *all* backed up somewhere).

                        As for permissions, there are two scripts that can fix most of them (but not all), and must be run as root: cvspermsfix.pex and varpermsfix.pex.

                        cvspermsfix.pex has no additional parameters, and will fix all the permissions in the Interworx install directory.

                        varpermsfix.pex has one paramater --siteworx=<domainname.com>, and will set all the var directory permissions of the account belonging to domainname.com back to the defaults. Specifically, everything in /home/<uniqname>/var/. This does *not* touch web data, since those can be set to anything you want, so we wouldn't even know what to set them to.
                        Last edited by IWorx-Socheat; 03-10-2005, 08:25 AM.
                        Socheat Sou
                        InterWorx-CP | http://interworx.com
                        InterWorx Control Panel

                        Comment


                        • #42
                          Most of my stuff is database driven and I back up my mysql databases hourly. The pressing need is to be able to restore everything as is to a single point in time and then update dynamic data such as database data to within the hour if possible.

                          Webmins restore routines don't mess with permissions at all, the file will be replaced as they were, permissions intact, restoring is always messy to some degree no matter what you use to do it, the desire it to minimize it as much as possible.

                          The onus is on me to make sure I get everything, which is what this discussion is about, I use webmin because Its capabilities are very sophisticated, I'm very familiar with the product and because it works, something that could not be said about Ensim at all. I do realize that I am taking a chance by using it which is the reason for hashing this out, however it'll only be until a better method comes about for doing it through Interworx.

                          As you can probably tell, backups are one of the most important things on my list, just my long experience in IT playing itself out:>)

                          I'm inpressed with the support over here, glad I'm making the switch!

                          Comment


                          • #43
                            Originally posted by timryberg
                            It will not work, you will have your basic web and email files, but you will miss your database files located here /var/lib/mysql and your httpd configuration files located here /etc/httpd and below (you need EVERYTHING IN /etc/httpd/conf.d/ ).

                            In addition you may have file permission problems during restore. InterWorx has very specific file permissions that it needs in order to work. Certain files need to be owned by the individual siteworx users, some by iworx, and some by root (and possibly httpd, I don't recall exactly).-- I know I've screwed them up before ;-) )
                            This is why I backup what I do on my box. I'm concerned mostly about the DoomsDay event. I can re-create a domain or two, and recover files and email during an hour's outage. However, if the server dies, I want hdc to have a complete backup for the event of a bare-metal reinstall to hda.

                            /etc/ is backed up because it contains server configs, including the entire httpd/ config tree.
                            /var/ is backed up because it has logs, qmail configs, dbases, etc.
                            /home/ is backed up for user mail and site files, as well as iworx specific data
                            /service/ is for various djb software junk.

                            the -a option in rsync is a shortcut for a huge amount of options (equivalent to -rlptgoD) which, among other things, ensures that permissions are preserved (which will match new UIDs since /etc/passwd is backed up).

                            Incidentally, since interworx doesn't recognize more than one disk, the -x option isn't necessary. I hadn't thought about the --delete option, but I think I'll add it to mine; however, you should consider using --delete-after instead, since by default rsync deletes files before transferring. Also, -v is annoying in the logwatch reports. This leaves me with:
                            Code:
                            #!/bin/bash
                            rsync -a --delete-after /etc /mnt/backup
                            rsync -a --delete-after /home /mnt/backup
                            rsync -a --delete-after /service /mnt/backup
                            rsync -a --delete-after /var /mnt/backup
                            Can anyone from Interworx comment on whether I'm missing anything in my backups? I know that there's no click-a-button-restore from my method, but I should be able to rebuild it, databases and all.

                            Comment


                            • #44
                              Originally posted by bluesin

                              Webmins restore routines don't mess with permissions at all, the file will be replaced as they were, permissions intact, restoring is always messy to some degree no matter what you use to do it, the desire it to minimize it as much as possible.
                              Not sure if you've tested InterWorx's restore yet, but it isn't messay at all. It's literally "one click" if the restore file is in the right place.

                              Socheat, can that shell command be set up to just back up databases?

                              Comment


                              • #45
                                Yea, I'm more referring to a situation of a hd failure or a scenario where I had a rootkit infect the server and had to trash it .

                                I guess I should have qualified that, in the scenario of the rootkit, once the rootkit was discovered, they give you a small amount ot time before you get to wake up with a clean server with your old drive accessable for a period of time, as I suspect anyone like EV1 would do.

                                So my process is to login to the new server, wget the webmin rpm, install it, login to webmin and access the restore file and tell it to restore /home from last nights dump then restore all the config files, then restore live database data from the latest hourly dump, etc.

                                Before I decided to make the switch EV1 would have set me up with a server with Ensim on it already, which is what happend last time and when I found out about how bad their backup/restore process was as it literally took me 2.5 days to fully recover, thus I went the webmin route and tested restoring to a new server and the entire process took me less than two hours from the time the new server was handed to me.

                                I have not tested Interworx's restore yet, but it looks like I'd have to have a method of accessing the system. Accessing the secondary mounted drive (FWIW, I now have a mirror drive I backup to), navigatingto the domain name, identifying the file name of the latest backup and then typing all of that into the restore box, restoring the site and then moving on to the next one.

                                And that's after I've gone through the non-painless process of reinstalling Interworx, of course Interworx is going to have to be reinstalled anyway, thus I'm assuming that I'd install it first after webmin anyway, then restore /home, but I'm actually unsure of the process.

                                the rootkit was nasty, really upset my clients that I one got infected with one and two that it took me so long to recover, something that I guaranteed then would never happen again.

                                It was my fault, I had left an easy crack of a password on the server along with root login, thus I got what I deserved. Me and whoever it was were actually attempting to lock each other out for a time, I won because I had webmin and was able to quickly install a firewall with only the webmin port open, this allowed me to find out where the rk was, this is important because you want to insure you don't restore it, and 2 take some last minute backups before I trashed the server.

                                I'd rather deal with a HD failure personally:>)

                                So based on this what would be the process with Interworx, if we were to awaken to a brand new server starting from scratch?

                                Comment

                                Working...
                                X