Need better management of IPs for Mass Import (iworx to iworx)

[Justec]

I just moved to a new server last week and I’ve run into a few issues with some IPv6 sites I had setup. On the import it just pulled from the pool (same pool as on previous) server, but pulled different IPs and not matched up to previous ones. So i’ve had to manually delete the IPs, releasing them to the “static pool” of IPv6 and then assign them to the right site (or change the DNS).

I have run into this before on IPv4, but with just a few dedicated sites and the rest on a single shared IP it wasn’t much of an issue. But static IPv4s also weren’t able to match up with the sites by domain (keeping same IPs on both server and just having them routed differently in the datacenter). The static IPs were just handed out in order of how they came up on the import list, for any sites that had SSL setup on the previous server. No big deal with just a few sites, but once I go more to using IPv6 it will be an issue the next time I upgrade servers.

Would be nice if there was an option to check off (or just by default) if the IPs on both servers are the same on both server, where it will use the same IP for the same SiteWorx account (IPv6 and IPv4)

I know this sounds a lot simpler on paper than real world. I’m sure there are a 100 things that I’m not thinking of that would make this more complex, but I have to believe it could be pulled off by the interworx team.

[d2d4j]

Hi Justin

Interesting post
I did a quick test from here, and the issue which I thought might happen, did and that was how to use the same IP addresses on both servers, at the same time (see pic).
I set the same IP on a test server from a live server
from SSH, I tried to connect to IP address, but was only connecting back to self
I then adjusted a secondary locked IP address on live server, not on test server and tried SSH using the secondary IP address, which failed due to no route
Only when deleting the duplicate IP address from test server, was a SSH connection successful
I do understand your post though, but I think it will be a big job to do make it so it can retain all IP settings. It may even just be better to have a clone option, but this I can see would have big issues as well, due to differences in distro, software versions etc…
Perhaps others may post their thoughts or insights as to how it maybe achieved
Many thanks
John

[Justec]

Hey John, thanks for the reply. I think you were just experimenting, but my situation wasn’t to have the same IP on two active servers.

Basically I think my case, which probably applies to a large group of people, is moving from an old server to a new server, with the same hosting company (in the same data center).
This type of move is very convenient and actually pretty easy with the mass import tool. Yes, there are a lot of little things to take care of as well, but the moving of the sites is very smooth, even if it takes a while for large sites (which I’m going to make a new post on for a feature request option on this as well).

So the way this move works is I setup the same secondary IP address on the New server as the Current server. Both Current and New server have their own main IP which is unique and active. So then I move all my sites via NodeWorx Mass Import by pointing to the old server main IP in the import tool. Once all the sites are moved over to the new server, then I can just have my datacenter / hosting company route my secondary IPs inside their network to my New server and that’s it. I’m live on the New server.

Here is a simple example of what I want to fix for IPv6, looking at it in the IPv4 world since it’s easier:
I just have one secondary IPv4 setup as shared and the rest of my secondary IPs set as dedicated. So since most sites right now are on the shared IP and there is only one shared IP, those all get assigned correctly during the move. The few sites with dedicated / SSL get messed up usually because they are assigned in order of how they come up on import list. I just have to go back in and remove / add IPs to those sites (or import them one by one in the order of their IPs).

What I’d like an option for is during the import, if the same IPv4 dedicated secondary IP from the Current server exist on the New for a particular SiteWorx account, it will assign that one to it. Versus just assigned a static IPv4 at random.

In the case of IPv6 instead of just 10 or so dedicated IP sites having to be fixed after the move like IPv4, it’s every site on the server. As every site gets a static IPv6.

[d2d4j]

Hi Justin
Many thanks, and sorry, I think I misunderstood, thinking you were wanting to keep the same IP address on both servers.
There is a CLI for mass changing IP addresses, which is usually used if changing a shared ip, but could be used to change just 1 ip address
Also, I think I better understand, and have just checked on a test server, but you can preselect an IPV4 or IPV6 (single or range), when using mass import.Apologies for the pic, there is not a lot of IPV4 or 6 setup (on test server we have a full /64 IPV6 live), but hopefully you get the idea… and these are shown to the right of the import screen, which on my browsers, are usually not shown unless you schroll right.
Is this on the right path to what you were wanting. I do appreciate you were also meaning not to have ot manually select an IPV address, but use an autoselect in order, which I think you mean either as ifcfg-eth0, ifcfg-eth0:0, ifcfg-eth0:1 etc or to use an IPV4 range, 5.5.5.5/48, 5.5.5.8, 5.5.5.9, 5.5.5.10 etc…
I’m sorry if I am still not fully understanding, must be an age thing, but also, I would be interested in your next post over extremely large site importing. Hopefully, you’ll post some avergae large/extremely large site stats for importing
Many thanks
John

[Justec]

I think we are on the same page now, but a couple more example to help clarify further.

Example One
One thing you said though, was you can pick whatever IPs in the dropdown. This wasn’t the case for me in regards to my IPv4 dedicated IPs.

For me this is how it worked:
Site 1 has dedicated IPv4 of 1.1.1.3
Site 2 has dedicated IPv4 of 1.1.1.1
Site 3 has dedicated IPv4 of 1.1.1.2

When importing the mass import tool brings this in alphabetic (numeric) order:
Site 1 linked automatically to 1.1.1.1
Site 2 linked automatically to 1.1.1.2
Site 3 linked automatically to 1.1.1.3

So Site1 should be on 1.1.1.3, but it gets 1.1.1.1 because it’s the first alphabetically and gets the first dedicated IPv4 on the system.
You can’t change it to 1.1.1.3 because it’s not available in the dropdown. This is because it is already in the drop down for Site 3.

Example 2
This is general example of my overall idea.

Site 1:
Shared IPv4 1.1.1.2
Dedicated IPv6 FFFF:FFFF:FFFF:FFF0

Site 2:
Shared IPv4 1.1.1.2
Dedicated IPv6 FFFF:FFFF:FFFF:FFF1

Site 3:
Dedicated IPv4 1.1.1.1
Dedicated IPv6 FFFF:FFFF:FFFF:FFF2

On the new server I added my secondary IPv4s of 1.1.1.1 and 1.1.1.2 and (for ease of explaining, but in reality it would be the same pool setup) FFFF:FFFF:FFFF:FFF0, FFFF:FFFF:FFFF:FFF1, FFFF:FFFF:FFFF:FFF2.
One note about this IPv6 simplification, the way the IPv6 pools work, the IPv6 IPs don’t technically exist on the system as an IP you can’t select to add to a SiteWorx account. They only come into existence when adding an IPv6 to a site from the pool. Once they are are pulled from the pool, you can remove the IPv6 from that site and now it’s in the list of available IPs to add to an account just like the IPv4 IPs. So this new import method would have to be smart enough to pull the right IPv6 address from the pool, make them as just useable dedicated IPs on the system, then match it up to the correct site. A bit more involved than the matching up of IPv4s.

Technically at this point in the example we have the same IPs on both servers, at the same time, but only one is active in the datacenter routing.

So what I’m looking for when running this mass import tool, that the window show up like this picture below. Showing these same IPs in the drop downs, automatically.

[d2d4j]

Hi Justin
Many thanks, and sorry, I am still struggling to fully understand sorry.
I have added IPV6 addresses, shared and dedicated, but these are not assigned to any siteworx account. I can now assign to mass import as I need to
IS your IW-CP latest version, or sorry if I have misunderstood
I do take your point over IPV6 handling on mass transfer though, as nearly all I would expect to be dedicated, with hundreds of sites, you would not want to change or select manually
Many thanks
John

[Justec]

Technically I don’t think an IPv6 should ever need to be set to shared. By nature, they should be dedicated.

But I think I complicated things by bringing IPv6 into this. The reason I want this to be improved is for IPv6, but I think it might be easier to explain with just IPv4.

The only difference between IPv4 dedicated and IPv6 dedicated for interworx and the mass import is when you create a new IPv6 pool on your server, there are not IPv6 dedicated IPs available. I see in your image above you have several IPv6 showing in the list, but when you first start fresh there are none because they are created on the fly as they are needed and pulled from the IPv6 Pool. If you create one on the fly for a siteworx account, then remove it from that siteworx account, it will then show up as an IPv6 available to the system (like your image), but not before that.

Ok, so let’s talk dedicated IPv4 and mass import.
If you have a couple test servers you could try this yourself.

Create three sites with these domain names so they are sorted alphabetically on the mass import screen of the import server.
Give them each a dedicated IP.

Source Server:
bbb.com -> 1.1.1.1 (dedicated)
aaa.com -> 1.1.1.3 (dedicated)
ccc.com -> 1.1.1.2 (dedicated)

Make sure the same exact IPv4s (1.1.1.1, 1.1.1.3, and 1.1.1.2) are created and ready to use on the import server, as dedicated.

Run mass import on the Mass Import Server and it will show something like this:

aaa.com -> 1.1.1.1 (dedicated)
bbb.com -> 1.1.1.2 (dedicated)
ccc.com -> 1.1.1.3 (dedicated)

Basically the issue is that the domains show up in the mass import list domains (siteworx accounts) are sorted alphabetically and are assigned dedicated IPv4s in numerical order.

Because A comes first, it gets the first IPv4 of 1.1.1.1 instead of it’s correct IP of 1.1.1.3.

The only way I’ve come up with to fix this now is to import them wrong and then manage IP on each SiteWorx account to move them around after the import. This does get tricky because in order to give site aaa.com it’s correct IP, you first have to set ccc.com to another IP (like a shared IP on the system) and then remove that dedicated IP from it to get the process started. Then you can switch aaa.com, freeing up the 1.1.1.1 IP which can then be assinged to bbb.com. Then that frees up 1.1.1.2 to be set on ccc.com.

So basically the same idea applies to IPv6, but instead of having to manually fix a handful of SSL sites that need a static IPv4, it will be all your sites. So the workload to fix this, and not mess something up, is amplified 100x when dealing with IPv6. Then throw on top of that the way IPv6 are pulled out of the pool, and it gets even tricker. Because maybe all your sites are given complete different IPv6 from the pool than on the previous server, so you can’t even move them around like my fix for IPv4 above.

[d2d4j]

Hi Justin

Many thanks, got it now thanks.

Sorry for been slow

Many thanks

John

[Justec]

Haha, no worries. It gets very confusing with all the IPs and stuff and trying to explain this over text in a forum.

Hopefully someone from interworx can explain if this is a possibility or if there is some behind the scenes reason that this wouldn’t work.