Centova Technologies Forum

Centova Cast v3 => Bugs and issues => Topic started by: My Auto DJ on April 11, 2012, 11:15:12 am

Title: High Server Load During/After FTP Uploads
Post by: My Auto DJ on April 11, 2012, 11:15:12 am
The server load gets very high when I try to upload mp3's while a stream is running, this happens with either web based or ftp uploads - this does not happen when streams are not running, in order to upload I must stop the server and then upload. Datacenter has said they will suspend my VPS if it keeps happening, but I do currently have 2 streams running and it is doing great.
Can't wait to start selling this, I see you have added even more features!
Title: Re: High Server Load During/After FTP Uploads
Post by: Centova - Steve B. on April 18, 2012, 05:55:58 pm
You'd need to check your process list and find out what process is causing the high load.  If it's being caused merely by updating the media library, then that definitely has nothing to do with whether or not the stream is running... they're entirely separate processes that have no knowledge of one-another.
Title: Re: High Server Load During/After FTP Uploads
Post by: My Auto DJ on April 18, 2012, 05:58:03 pm
No updating media library is fine it's when I upload mp3's either FTP or web based
Title: Re: High Server Load During/After FTP Uploads
Post by: Centova - Steve B. on April 18, 2012, 06:00:50 pm
Right.  In v3, media library updates take place automatically, immediately after web-based upload or within approximately 1 minute after FTP uploads.  You never need to update it manually unless you introduce files by hand (via ssh, scp, etc.)

Anyway, the only thing in common between FTP and web-based uploads is the media library update that automatically takes place afterwards, so if you experience high CPU load after both, then it's likely something to do with the media library update.

Title: Re: High Server Load During/After FTP Uploads
Post by: My Auto DJ on April 20, 2012, 05:46:28 am
That explains a lot, I wondered why I could ftp other files with no problem, it was only when I uploaded mp3's

 I have a couple auto dj's that have been running fine for a week, I uploaded 3 mp3's and about mid-way of the 2nd upload the server load started getting high again. I ran /etc/init.d/centovacast restart and it went down. So I now know what the problem is, but how do I correct it? Thanks for any help you can give!

Title: Re: High Server Load During/After FTP Uploads
Post by: Centova - Steve B. on May 08, 2012, 12:06:20 am
Sorry for the slow reply.. per my earlier post the first step would be to check the process list when the CPU load spikes to find out exactly what process is doing it.

Once we know what process it is, I can provide exact instructions for figuring out what's causing the load issue.
Title: Re: High Server Load During/After FTP Uploads
Post by: My Auto DJ on May 18, 2012, 12:35:28 pm
 BEFORE TOP COMMAND

PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
    1 root      15   0 10352  212  180 S  0.0  0.0   0:00.08 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 migration/0
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.02 ksoftirqd/0
    4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    5 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 events/0
    6 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 khelper
    7 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 kthread
    9 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 xenwatch
   10 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 xenbus
   19 root      RT  -5     0    0    0 S  0.0  0.0   0:00.01 migration/1
   20 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/1
   21 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/1
   22 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 events/1
   39 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kblockd/0
   40 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 kblockd/1
   41 root      20  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/0
   42 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 cqueue/1







About 30 seconds after I start a upload here is what I get when I run TOP command


 PID USER      PR  NI  VIRT  RES  SHR S %CPU %MEM    TIME+  COMMAND
 3546 nobody    25   0 28768 1256  588 R 93.9  0.1   0:05.52 cc-imaged
 3544 nobody    18   0 1436m 668m  608 R  5.1 59.2   0:02.01 cc-imaged
  118 root      10  -5     0    0    0 D  3.2  0.0   0:04.59 kswapd0
  299 root      10  -5     0    0    0 S  0.3  0.0   0:00.05 kjournald
 1506 root      15   0 74812  272  212 S  0.3  0.0   0:00.01 crond
    1 root      15   0 10352  200  176 S  0.0  0.0   0:00.08 init
    2 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 migration/0
    3 root      34  19     0    0    0 S  0.0  0.0   0:00.02 ksoftirqd/0
    4 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/0
    5 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 events/0
    6 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 khelper
    7 root      12  -5     0    0    0 S  0.0  0.0   0:00.00 kthread
    9 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 xenwatch
   10 root      10  -5     0    0    0 S  0.0  0.0   0:00.00 xenbus
   19 root      RT  -5     0    0    0 S  0.0  0.0   0:00.01 migration/1
   20 root      34  19     0    0    0 S  0.0  0.0   0:00.00 ksoftirqd/1
   21 root      RT  -5     0    0    0 S  0.0  0.0   0:00.00 watchdog/1
Title: Re: High Server Load During/After FTP Uploads
Post by: Centova - Steve B. on June 01, 2012, 05:05:52 pm
Thanks, just to let you know, another member of our staff has been able to reproduce this and a fix will be included in the next release as soon as I track down the bug in cc-imaged.  Thanks again for the bug report.

Title: Re: High Server Load During/After FTP Uploads
Post by: My Auto DJ on June 07, 2012, 07:18:43 am
That's great thanks Steve, can you please let me know when this is ready!
Title: Re: High Server Load During/After FTP Uploads
Post by: Centova - Steve B. on June 08, 2012, 12:24:38 pm
Ah, sorry, it's been ready for a couple of days now.  Just use /usr/local/centovacast/sbin/update to pull down the fix. :)
Title: Re: High Server Load During/After FTP Uploads
Post by: My Auto DJ on June 08, 2012, 01:04:44 pm
Thanks but now I am not able to create accounts, get a license error when i try. but I run a License test in Centova Admin and get Test passed, I have installed the same method several times with no errors and I know the key is good.



Internal Error
We apologize for the inconvenience, but an internal error has occurred. Please try again later.
Unhandled exception: Cluster host authentication failure for Local server: Licensing error in /usr/local/centovacast/system/class_RPCDaemonClient.php:900



Test result: PASSED - Centova Cast can successfully communicate with the Centova Technologies license servers.

Details:

Testing server license1.centova.com ...
Host license1.centova.com resolves to XXXXXXXXX
Connection successful and correct response received




Title: Re: High Server Load During/After FTP Uploads
Post by: Centova - Steve B. on June 08, 2012, 01:18:29 pm
The license test just checks that your server can communicate with the licensing servers, it doesn't check your key. You need to check your logs (or just run /etc/init.d/centovacast restart) to see the details of the licensing error.
Title: Re: High Server Load During/After FTP Uploads
Post by: My Auto DJ on June 08, 2012, 01:20:12 pm
Yes so sorry I reissued the key and it's fine now -- uploading mp3's and no high load!!! Thanks Steve!

I have the trial but looks like I will be upgrading to a paid license soon, can you tell me the command to change a license please?
Title: Re: High Server Load During/After FTP Uploads
Post by: My Auto DJ on June 08, 2012, 02:20:15 pm
Great job Steve, been uploading and testing --  I'm putting packages up for sale this afternoon!


root@server [~]# uptime
 20:08:34 up 29 min,  1 user,  load average: 0.01, 0.06, 0.02
root@server [~]# uptime
 20:09:58 up 30 min,  1 user,  load average: 0.03, 0.05, 0.02
root@server [~]# uptime
 20:12:25 up 32 min,  1 user,  load average: 0.00, 0.03, 0.00
root@server [~]# uptime
 20:13:44 up 34 min,  1 user,  load average: 0.00, 0.02, 0.00
root@server [~]# uptime
 20:15:49 up 36 min,  1 user,  load average: 0.00, 0.01, 0.00
root@server [~]# uptime
 20:18:58 up 39 min,  1 user,  load average: 0.30, 0.09, 0.03
root@server [~]# uptime
 20:19:03 up 39 min,  1 user,  load average: 0.27, 0.08, 0.03
root@server [~]# uptime
 20:19:06 up 39 min,  1 user,  load average: 0.25, 0.08, 0.02
root@server [~]# uptime
 20:19:13 up 39 min,  1 user,  load average: 0.23, 0.08, 0.02
root@server [~]# uptime
 20:20:14 up 40 min,  1 user,  load average: 0.08, 0.06, 0.02
root@server [~]# uptime
 20:21:20 up 41 min,  1 user,  load average: 0.03, 0.05, 0.01
root@server [~]# uptime
 20:21:58 up 42 min,  1 user,  load average: 0.01, 0.04, 0.01
root@server [~]# uptime
 20:27:53 up 48 min,  1 user,  load average: 0.00, 0.03, 0.01
root@server [~]# uptime
 20:30:05 up 50 min,  1 user,  load average: 0.00, 0.02, 0.00
root@server [~]# uptime
 20:54:34 up  1:15,  1 user,  load average: 0.61, 0.72, 0.39
root@server [~]# uptime
 20:55:01 up  1:15,  1 user,  load average: 0.37, 0.65, 0.37
root@server [~]# uptime
 20:55:08 up  1:15,  1 user,  load average: 0.34, 0.64, 0.37
Title: Re: High Server Load During/After FTP Uploads
Post by: Centova - Steve B. on June 08, 2012, 02:53:49 pm
You just need to edit /usr/local/centovacast/etc/license.conf and put your new key in there.  Then run "/etc/init.d/centovacast restart" and you're all set.  (If the license you're installing isn't brand new, you'll also need to reissue it at centova.com first, btw.)

Glad to hear the new imaged is working well. :)
Title: Re: High Server Load During/After FTP Uploads
Post by: My Auto DJ on June 10, 2012, 04:39:29 pm
I've had several streams/auto dj's running for a couple days now, not a single problem with stability, in fact it seems we will be able to cram more accounts on each server, I guess due to sc_trans (which uses a lot less server resources vs ices_cc) and the fact SHOUTcast v2 also uses less resources -- I don't know exactly why but this is great, the few clients I have on this love it and most have already canceled their old v2 accounts for v3 -- it took a while but at last you all got it!!! Thanks Steve!!!

http://173.255.141.34:8012/index.html?sid=1

http://173.255.141.34:2199/start/twok/