Understanding Clustering & Website Requirements

I am new to clustering. I am used to traditional hosting of single servers (aka cPanel). A few of my friends are starting two projects/websites that we expect to grow and have high demand. I was considering InterWorx heavily due to the growth as I am not happy with any cloud solutions.

I envisioned that we will start with a single server. We are unsure if our first upgrade will be disk space or dedicated MySQL server. We will assume the later. Obviously this should be pretty simple of a move and I am not concerned with this. Now if we need to upgrade disk space, I see that InterWorx supports a file server (with some considerations).

So with the implementation of the file server, how does that look to the website? If your /home/account/www directory contains the files for the internet, does that mean it is stored on this file server and the app servers just keep a copy?

Should I be using a NFS instead of a file server?

I am sure my post is full of bad assumptions, etc.


I’d imagine “file server” in the clustering docs is either FC, iscsi, or some sort of NAS solution like NFS. And, yes, I think the idea is that multiple hosts all have access to the same remotely stored home directory structure - that would be how I’d attack it at least. I’m the furthest thing from a storage expert, but IME, NAS is much simpler to implement since it takes care of things like file locking, etc.

So technically the NFS or File Server is the home directory? I guess all of these clusters make it appears as one big machine?

Hi Frank,
The “file server” refers to where your client’s www data, mail, and rrd data live. By default this is the cluster manager’s /home/ directory (the main interworx server) and the cluster manager then shares this data to the nodes via NFS. The nodes mount the NFS share as their own home directory and interworx ensures that unix users/groups are in sync with the cluster manager. The load balancer will feed the incoming http/smtp/imap/ftp request to the nodes and the nodes (since they have the nfs share mounted as their home directory) handle the request as if they were a standalone server with all the data stored locally. So, yes websites and apache just think the web data is in /home/user/domain.tld/html on all the cluster servers.

The external file server that interworx supports means that instead of serving files from the cluster manager, you use another NFS file server (can be a NAS appliance) that the cluster manager will mount as it’s own home directory. When setting up the cluster, the cluster manager will detect that it’s home directory is an NFS mount and instruct the nodes to directly mount the external file server instead of trying to mount the cluster manager’s home directory.

Alternatively, I’ve also seen an iSCSI share mounted as /home/ on the cluster manager and then it operates much like a conventional storage disk would, being used as an NFS export to the nodes.

If you were to need more space and just wanted to increase the /home partition size, the procedure would mean first de-clustering, moving the data from the cluster manager to a file server, mounting the data back as the cluster manager’s home directory, and re-clustering. It would probably be best if you thought about your storage situation up front (minus storage needed for databases since that can be moved more easily than web/mail) to try avoiding having to do mass data migration later.

Another way of increasing space would be to do it for a single domain/account that needed a lot of disk space. It would entail manually setting up an NFS or iscsi share on an external server or NAS appliance and mounting it to each server in the cluster to something like /mnt/share. Then, adding a symlink in /home/user/domain.tld/html/SYMLINK->/mnt/share and then all the nodes would have that symlinking pointing to the same share that they have mounted. That would essentially give the user more space in that symlink’d directory for things like video or images (which tend to take up the bulk of space).

Thanks Dan! That was excellent and really helped.

This is going to be a perfect solution for us. The growth is hard to be hard to predict.

So for your last paragraph, this is the part we just don’t know how it will progress. So lets say we had a /home/user/domain.tld/html/media which contained the websites images and video. We then migrate it off the cluster manager to a NAS and setup a SYMLINK to the /mnt/share. That would be acceptable, correct? Would we just do that on the cluster manager and InterWorx would setup the appropriate mounts on the clusters?

Since social websites can be very unpredictable, we have to scale smartly. I think we will probably do a 2U server with 8 HDD capability. I am thinking 3 HDD in RAID 0 for MySQL, 5 HDD in RAID 5E for the /home. I will throw the rest of CentOS/InterWorx in another HDD. As we need to, we will throw in more memory and additional CPU. If we offload, MySQL, we can move /home to the RAID 0 with the RAID 5E staying present for the /home/user/domain.tld/media.


For that sort of thing you’d have to manually set up the mounts on all the nodes. InterWorx does not yet have the capability to set this up automatically.
Since the symlink will be added within the “main” NFS share (/home), you will not have to add the symlink multiple times.

If you do not add the mounts to the nodes, http requests served from the nodes will 404 on data that is located within the auxiliary share.