httpd log : sh:convert --> how to find who ?

General Interworx Control Panel Info General Inquries
1

Hello,

Somebody on our box is tryng to use convert. A bin part of imagemagick. We don’t know who and it causes us trouble.

In the /var/log/httpd/error_log we have huge
sh: convert: command not found
sh: convert: command not found
sh: convert: command not found
sh: convert: command not found
sh: convert: command not found
sh: convert: command not found
sh: convert: command not found
sh: convert: command not found
sh: convert: command not found
sh: convert: command not found

We encounter down services and finaly server down. Our sentiment is that the convert is an attack, take all ram, cpu then finish to switch down the box.

We’ve tried to secure the httpd.conf by adding

#-------------------------------------------------

ADD FROM CARAT TO TRY TO BLOCK ALL PBMS

#-------------------------------------------------

suppress version info in ‘Server:’ string

and in document signatures

ServerTokens ProductOnly
ServerSignature Off

limit scripts to roughly 96Mbytes ram

RLimitMEM 101145600

limit cpu time for scripts to 120 seconds

RLimitCPU 120

limit HTTP request body to 8Mbytes

LimitRequestBody 8192000

------------------------------------------------

and

MaxRequestsPerChild 10000

So now we have after few sh:convert

Allowed memory size of 8388608 bytes exhausted (tried to allocate 0 bytes)
Allowed memory size of 8388608 bytes exhausted (tried to allocate 72 bytes)
Allowed memory size of 8388608 bytes exhausted (tried to allocate 131559 bytes)

We also added mod_evasive with apf ban command inside.

It seems to be better, but we steal have a lot of SH:convert

So the question is :

Do you know a way to find which user, siteworx account, try to use convert ? is there a way to find this ?

Thanks

Pascal

2

Hi Pascal,

In the ticket you have open with us, you mentioned one of your sites was running a gallery2 script that was causing the following message to appear in your error.log:

Allowed memory size of 12582912 bytes exhausted (tried to allocate 0 bytes)

I’d take a look at that account, as it’s very plausible that an image gallery would try to use imagemagick binaries.

Socheat

3

Thanks Socheat

Pascal

4

I used to receive similar error messages to that one - The problem can be solved (for me anyway) by finding the memory limit in php.ini, and changing it into bytes rather than having it say “xM”, where x is the number of megabytes.

I’m guessing yours probably says “12M” in your php.ini… try changing it to say “16777216” (which should be 16MB in bytes)

Sorry if the above is completely irrelavent, but it’s worth a try :slight_smile:

5

do you know what are these errors in httpd/error_log

[Fri Mar 24 16:01:59 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “emule-zenzone.com”) failed in watchLogHash()
[Fri Mar 24 16:01:59 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchLogHash()
[Fri Mar 24 16:01:59 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchCleanUpHash()
[Fri Mar 24 16:01:59 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “emule-zenzone.com”) failed in watchCleanUpHash()
[Fri Mar 24 16:02:03 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “emule-zenzone.com”) failed in watchLogHash()
[Fri Mar 24 16:02:03 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchLogHash()
[Fri Mar 24 16:02:03 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchCleanUpHash()
[Fri Mar 24 16:02:03 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “emule-zenzone.com”) failed in watchCleanUpHash()
[Fri Mar 24 16:11:33 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “drum-bass.net”) failed in watchLogHash()
[Fri Mar 24 16:11:33 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchLogHash()
[Fri Mar 24 16:11:35 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchCleanUpHash()
[Fri Mar 24 16:11:35 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “drum-bass.net”) failed in watchCleanUpHash()

Pascal

6

Hello

I also have this error log. Httpd restart after a glibc

[Fri Mar 24 18:33:51 2006] [notice] Apache configured – resuming normal operations
*** glibc detected *** double free or corruption (out): 0x000000000109e0a0 ***
[Fri Mar 24 18:34:53 2006] [notice] caught SIGTERM, shutting down

If you have an idea it 'll be really welcome

Pascal

7

Thanks fr3d,

About the memory limit I arealdy done this change in php indeed. And it seems to be much netter. But I still have errors about sh:convert (ok I have to find who) but much bad others errors as told eraly.

the last one about glibc happens rarely.

If someboy has already solved these sort of pbms thanks to share your tips and tricks :wink:

Pascal

8

Hello,

It seems it is mod_watch which do this.

I still have a lot of erros like

[Fri Mar 24 16:01:59 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “emule-zenzone.com”) failed in watchLogHash()
[Fri Mar 24 16:01:59 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchLogHash()
[Fri Mar 24 16:01:59 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchCleanUpHash()
[Fri Mar 24 16:01:59 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “emule-zenzone.com”) failed in watchCleanUpHash()
[Fri Mar 24 16:02:03 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “emule-zenzone.com”) failed in watchLogHash()
[Fri Mar 24 16:02:03 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchLogHash()
[Fri Mar 24 16:02:03 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchCleanUpHash()
[Fri Mar 24 16:02:03 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “emule-zenzone.com”) failed in watchCleanUpHash()
[Fri Mar 24 16:11:33 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “drum-bass.net”) failed in watchLogHash()
[Fri Mar 24 16:11:33 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchLogHash()
[Fri Mar 24 16:11:35 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “SERVER”) failed in watchCleanUpHash()
[Fri Mar 24 16:11:35 2006] [crit] (20014)Error string not specified yet: shGetLockedEntry(b98278, “drum-bass.net”) failed in watchCleanUpHash(

…

After few, httpd restart. But some times it is more bad…

I try to see what is the log in access_log at the same time. Of course in var/log I only have local access log. So all are watchxxxx. My idea is to try to merge all vhost access log file for this same time and see what could cause this pbm (which request if it is outside local). Is there a simple way to have a global access log for all vhosts ?

Thanks

Pascal

9

Pascal, I’m not sure these errors are actually casuing any problems. I found this thread from December, and it sounded like those messages may have been a side effect of high load problems, but it never was clear. Here’s the thread:

http://interworx.com/forums/showthread.php?t=956

While they are listed there as a “critical” error the worst that could happen iworx-cp wise is that the webserver graph on the system graphs page might be off for a reading.

10

Thanks Paul,

I’m not so sure it is not hurt anything. Everytime I’ve got them the httpd reboot and without having a high server usage : CPU = 10% of use and load average = 0.80 0.45 0.34

On google I’ve seen that there was some fix and they all tell to download the last version of mod_watch. But you already distribute the last mod_watch version : 4.3

I’m continuing to investigate, and let you know if I find something new.

Thanks for you update

Pascal