High Server Load During/After FTP Uploads

Read 19626 times
The server load gets very high when I try to upload mp3's while a stream is running, this happens with either web based or ftp uploads - this does not happen when streams are not running, in order to upload I must stop the server and then upload. Datacenter has said they will suspend my VPS if it keeps happening, but I do currently have 2 streams running and it is doing great.
Can't wait to start selling this, I see you have added even more features!
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
You'd need to check your process list and find out what process is causing the high load.  If it's being caused merely by updating the media library, then that definitely has nothing to do with whether or not the stream is running... they're entirely separate processes that have no knowledge of one-another.
No updating media library is fine it's when I upload mp3's either FTP or web based
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
Right.  In v3, media library updates take place automatically, immediately after web-based upload or within approximately 1 minute after FTP uploads.  You never need to update it manually unless you introduce files by hand (via ssh, scp, etc.)

Anyway, the only thing in common between FTP and web-based uploads is the media library update that automatically takes place afterwards, so if you experience high CPU load after both, then it's likely something to do with the media library update.

That explains a lot, I wondered why I could ftp other files with no problem, it was only when I uploaded mp3's

 I have a couple auto dj's that have been running fine for a week, I uploaded 3 mp3's and about mid-way of the 2nd upload the server load started getting high again. I ran /etc/init.d/centovacast restart and it went down. So I now know what the problem is, but how do I correct it? Thanks for any help you can give!

My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
Sorry for the slow reply.. per my earlier post the first step would be to check the process list when the CPU load spikes to find out exactly what process is doing it.

Once we know what process it is, I can provide exact instructions for figuring out what's causing the load issue.
 BEFORE TOP COMMAND

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    1 root      15   0 10352  212  180 S  0.0  0.0   0:00.08 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 migration/0
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.02 ksoftirqd/0
    4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    5 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 events/0
    6 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 khelper
    7 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 kthread
    9 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 xenwatch
   10 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 xenbus
   19 root      RT  -5     0    0    0 S  0.0  0.0   0:00.01 migration/1
   20 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/1
   21 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/1
   22 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 events/1
   39 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kblockd/0
   40 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kblockd/1
   41 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/0
   42 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/1







About 30 seconds after I start a upload here is what I get when I run TOP command


 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 3546 nobody    25   0 28768 1256  588 R 93.9  0.1   0:05.52 cc-imaged
 3544 nobody    18   0 1436m 668m  608 R  5.1 59.2   0:02.01 cc-imaged
  118 root      10  -5     0    0    0 D  3.2  0.0   0:04.59 kswapd0
  299 root      10  -5     0    0    0 S  0.3  0.0   0:00.05 kjournald
 1506 root      15   0 74812  272  212 S  0.3  0.0   0:00.01 crond
    1 root      15   0 10352  200  176 S  0.0  0.0   0:00.08 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 migration/0
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.02 ksoftirqd/0
    4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    5 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 events/0
    6 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 khelper
    7 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 kthread
    9 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 xenwatch
   10 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 xenbus
   19 root      RT  -5     0    0    0 S  0.0  0.0   0:00.01 migration/1
   20 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/1
   21 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/1
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
Thanks, just to let you know, another member of our staff has been able to reproduce this and a fix will be included in the next release as soon as I track down the bug in cc-imaged.  Thanks again for the bug report.

That's great thanks Steve, can you please let me know when this is ready!
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
Ah, sorry, it's been ready for a couple of days now.  Just use /usr/local/centovacast/sbin/update to pull down the fix. :)
Thanks but now I am not able to create accounts, get a license error when i try. but I run a License test in Centova Admin and get Test passed, I have installed the same method several times with no errors and I know the key is good.



Internal Error
We apologize for the inconvenience, but an internal error has occurred. Please try again later.
Unhandled exception: Cluster host authentication failure for Local server: Licensing error in /usr/local/centovacast/system/class_RPCDaemonClient.php:900



Test result: PASSED - Centova Cast can successfully communicate with the Centova Technologies license servers.

Details:

Testing server license1.centova.com ...
Host license1.centova.com resolves to XXXXXXXXX
Connection successful and correct response received




My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
The license test just checks that your server can communicate with the licensing servers, it doesn't check your key. You need to check your logs (or just run /etc/init.d/centovacast restart) to see the details of the licensing error.
Yes so sorry I reissued the key and it's fine now -- uploading mp3's and no high load!!! Thanks Steve!

I have the trial but looks like I will be upgrading to a paid license soon, can you tell me the command to change a license please?
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
Great job Steve, been uploading and testing --  I'm putting packages up for sale this afternoon!


root@server [~]# uptime
 20:08:34 up 29 min,  1 user,  load average: 0.01, 0.06, 0.02
root@server [~]# uptime
 20:09:58 up 30 min,  1 user,  load average: 0.03, 0.05, 0.02
root@server [~]# uptime
 20:12:25 up 32 min,  1 user,  load average: 0.00, 0.03, 0.00
root@server [~]# uptime
 20:13:44 up 34 min,  1 user,  load average: 0.00, 0.02, 0.00
root@server [~]# uptime
 20:15:49 up 36 min,  1 user,  load average: 0.00, 0.01, 0.00
root@server [~]# uptime
 20:18:58 up 39 min,  1 user,  load average: 0.30, 0.09, 0.03
root@server [~]# uptime
 20:19:03 up 39 min,  1 user,  load average: 0.27, 0.08, 0.03
root@server [~]# uptime
 20:19:06 up 39 min,  1 user,  load average: 0.25, 0.08, 0.02
root@server [~]# uptime
 20:19:13 up 39 min,  1 user,  load average: 0.23, 0.08, 0.02
root@server [~]# uptime
 20:20:14 up 40 min,  1 user,  load average: 0.08, 0.06, 0.02
root@server [~]# uptime
 20:21:20 up 41 min,  1 user,  load average: 0.03, 0.05, 0.01
root@server [~]# uptime
 20:21:58 up 42 min,  1 user,  load average: 0.01, 0.04, 0.01
root@server [~]# uptime
 20:27:53 up 48 min,  1 user,  load average: 0.00, 0.03, 0.01
root@server [~]# uptime
 20:30:05 up 50 min,  1 user,  load average: 0.00, 0.02, 0.00
root@server [~]# uptime
 20:54:34 up  1:15,  1 user,  load average: 0.61, 0.72, 0.39
root@server [~]# uptime
 20:55:01 up  1:15,  1 user,  load average: 0.37, 0.65, 0.37
root@server [~]# uptime
 20:55:08 up  1:15,  1 user,  load average: 0.34, 0.64, 0.37
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
You just need to edit /usr/local/centovacast/etc/license.conf and put your new key in there.  Then run "/etc/init.d/centovacast restart" and you're all set.  (If the license you're installing isn't brand new, you'll also need to reissue it at centova.com first, btw.)

Glad to hear the new imaged is working well. :)