Out of Memory - System reboot

Hello,

For the last several weeks, we have been experiencing one of our Interworx machines running out of memory. The system currently has 64GB assigned, plenty of disk space (all SSD), plenty of CPU, running standard Interworx, ConfigServer Firewall.

Usually things will start with websites loading slowly or not at all, then eventually, if you’re looking at shell, we see hundreds, if not thousands of entries of Out of Memory: killed process xxxx. All kinds of processes are killed, tcpserver, anvil, scdaemon, rotatelogs, supervise, multilog, systemd-cgroups, etc. until such time the entire box just reboots itself.

After the reboot, the server runs fine as if nothing happened. Haven’t really been able to find anything helpful in the logs as to what is causing this. Everything is up to date. We are new to the Interworx platform from cPanel and we are a bit of a loss on why this is occurring.

Any insight/thoughts would be greatly appreciated and helpful as if this keeps up, I’m not sure we can stay with the Interworx platform.

Hi

Welcome to the forums

You have not stated what distro/os or IW version

I would open a support ticket but please be aware it’s the weekend so unless urgent would be Monday and support is USA es time

Many thanks

John

Probably would have been helpful wouldnt it? ha!

It’s AlmaLinux 8 AlmaLinux release 8.9 (Midnight Oncilla) and the latest release of IW, InterWorx-CP v7.13.25

Hello–

Does your server have swap? That is a requirement for InterWorx, which some customers do tend to miss on the minimum requirement page. If the 64GB you mention is RAM (it is unclear if that is RAM or storage space), it would be less likely that a lack of swap would be an issue, but that is the most common cause I tend to see for repeated OOM issues.

If you do have swap, please submit a ticket and I can take a look for you on Monday. I will need to know some general times the issues last occurred: InterWorx Direct Support Guide — InterWorx documentation

Thanks,
-Jenna
Friendly Neighborhood InterWorx Support Manager

64GB Ram not disk space. We currently have just 500GB disk space assigned to the VM. We have plenty of additional ram we can dedicate to this VM if needed.


Uploaded is the last screen grab I was able to get right before the entire server itself rebooted. This occurred on 1/6/2024 at about 0117am.

I will ask our upstream vendor to reach out.

Thank you.

*Edit: for confirmation, we have a 30GB swap partition configured.

Hello–

That is very odd. Definitely submit a ticket, or have your provider do so, and we can take a look.

Thanks,
-Jenna

Try disabling swappiness.
You can disable OOM on some OS’s but I usually dont recommend this.

Lastly check via top what is using all the memory. You could limit MySQL or Nginx etc to not use all the servers memory if you use those.