Optimize the fively iworx.pex cron job

Hello,

On our cluster manager the iworx.pex --fively cron job eat a lot of ressources (CPU, load-average, …)

Is there a way to optimize it ? delay it ? do only some portion of it (like the stat) ?

It could be great to be able to set what might be run or not in this cron job. I’m thinking about all the stats that are generated for example in this script. If we could set the cron time of all high ressources intensive tasks it could be great (like tell I want the stats every 30mn not 5)

Thanks
Pascal

You can change when the “realtime” stats are processed (bandwidth, etc), by editing the runtime=“fively” value in the iworx.ini. Changing it to “fifteenly” will cause those calculations to happen during the fifteenly, instead of the fively. The other options are quad_daily and daily. However, anything other than “fively” will cause the “realtime” graphs to disappear (since it’s no longer realtime).

tsss I knew that :\
I should think before ask !

5mn is realtime 15mn is not ? no way to have a graph based on 15mn and not every 5mn ?

Also if I set this on 15mn there will be more data to treat in one time ? right ?

So basicly I’ll certainly have a long 15mn cron job that will cause more ressources usages than those used with a 5mn one ?

What about the chunksize ?
chunksize=“200”

Is it best to lower or set an higher value ?
Could you please explain what does Interworx with this and what performance gain I could have ?

Thanks a lot :slight_smile:
Pascal

ha ha so basicly it only means set on or set off the graphs :slight_smile:
Set it to something else than “fively” continue to treat the stats but doesn’t show them : Maybe in a future version this could be set differently ?

Option 1 : Set on/off the reatime stats
Option 2 : Set On realtime stats every X time (fively, fifteenly, …)

Pascal

[QUOTE=pascal;13148]Also if I set this on 15mn there will be more data to treat in one time ? right ?

So basicly I’ll certainly have a long 15mn cron job that will cause more ressources usages than those used with a 5mn one ?[/quote]

Depends on your hardware setup, how many accounts you have, etc. I don’t have statistical numbers, but it’s not necessarily true that it will take 3 times longer to process when set to “fifteenly”. It may only take a minute or two longer than when it was set to “fively”, which means a longer period of “rest” time between the processing.

[QUOTE=pascal;13148]
What about the chunksize ?
chunksize=“200”

Is it best to lower or set an higher value ?
Could you please explain what does Interworx with this and what performance gain I could have ?[/QUOTE]

Chunksize is more about InterWorx memory usage. For example, say you have 450 accounts, and a chunksize of 200. When the bandwidth.pex runs (iworx.pex --fively runs bandwidth.pex), it will run 3 times:

bandwidth.pex --chunk 1 --chunksize 200
^ processes the first 200 accounts. then exits, freeing up memory and resources

bandwidth.pex --chunk 2 --chunksize 200
^ processes the second 200 accounts. then exits, freeing up memory and resources

bandwidth.pex --chunk 3 --chunksize 200
^ processes the last 50 accounts. then exits, freeing up memory and resources

For people who don’t have a lot of memory on their server, smaller chunksize is better.

Also if I set this on 15mn there will be more data to treat in one time ? right ?

So basicly I’ll certainly have a long 15mn cron job that will cause more ressources usages than those used with a 5mn one ?

The extra data will have very little, if any affect at all, on the resource usage pascal.

Paul

[QUOTE=IWorx-Paul;13153]The extra data will have very little, if any affect at all, on the resource usage pascal.

Paul[/QUOTE]

cool

So here it is what we’ve done.

Our need is more during night when the full backups is processing.
We decided that during night siteworx clients wouldn’t need realtime stats.
So the idea was to be able to set a different runtime based on the time
07:00 AM – 11:00 PM --> fively
11:05 PM – 06:55 AM --> fifteenly

2 crons calls 2 scripts

at 11:05PM go from 5 to 15

cp -f /home/interworx/iworx.ini /home/interworx/iworx.last
sed -r -e ‘s|runtime=“fively”|runtime=“fifteenly”|’ -i /home/interworx/iworx.ini

at 07:00 go from 15 to 5

cp -f /home/interworx/iworx.ini /home/interworx/iworx.last
sed -r -e ‘s|runtime=“fively”|runtime=“fifteenly”|’ -i /home/interworx/iworx.ini

Thanks to all for the advises

PAscal

ouahou !!!

Since I set the runtime=“fifteenly” my average loadaverage is downed to less than 1 !

0.89 0.78 0.75

Rather than
3.25 2.86 2.78

So the REALTIME stats was really ressources intesensives !

Pascal