Hey guys. After a server fart, I rebooted the IWorx box, and the InterWorx instances of MySQL isn’t starting.
First problem: Its not starting OK, but the init script says it does.
Reading through the script, I found out why its not starting. Its looking for /home/interworx/iworx-db
Second problem: It doesn’t exist. It also doesn’t exist in my backups for at least 2 weeks. I haven’t check through the logs yet to see what dates the machine was rebooted, but I’m thinking it hasn’t been awhile, because if the file is necessary to use NodeWorx, but hasn’t been there for at least 2 weeks (still combing through 1.5 months of backups.
Does it need to be there? What is it? One of the really old backups said it was a symlink to mysqld_safe. Well, that didn’t work out so hot when I tried that.
(Yes, some of this is support ticket worthy, but not all. So I’m doing that too.)
We’re aware of this problem but it should have been fixed by a recent mysql-iworx package update. You can fix this by running the command above, and I’ll look into why it didn’t fix itself for you.
Just wanted to let you guys know that the symlink is missing again. I"m trying to look through logs to see if I can find any trace of what’s happened to it but haven’t turned up a thing.
As a related note, DNS for the box blew up and despite the fact that InterWorx is up again, the DB isn’t quite responding just yet. What may be useful is some kind of intermediate record storage. Say IWorx handles DNS records in the database, validates, and pushes out to a flat file. That way when InterWorx blows up, at least the hosted web sites will still work. Just a thought
Oh, and the djbdns init script is saying OK, when the daemon actualy isn’t starting. So its that and the iworx init script. It would be really helpful if they failed when there was any sort of problem and spit the program’s STDOUT and ERR to a failures log somewhere.
Re: the iworx-db symlink disappearing… I can’t think of any reason that would happen. But check your /var/log/yum.log file for any updates that happened between the last time you fixed it and now, particularly any mysql-iworx updates.
re: djbdns not starting, run
ps auxf
and look for the line that starts with
readproctitle service errors:
possible problems starting DNS will be described there.
The last mysql-iworx update was on Oct 9 a little before 10 AM, which was 4 days after the last fix. So at least we know with some certanty what blew it up this time. This is going to be ugly, but for now I’ll have the symlink re-made nightly.
