Seems sc_trans is nearly twice as efficient as LiquidSoap.  I have the same issue on other servers here is a screenshot of one, both CPU and Memory are nearly twice as much

I have a feeling there is nothing wrong with LiquidSoap, rather that DrO and his co-workers (the ones that developed sctrans)  know what they are doing and have yet to be beat!
I like the ogg format option and the fade options in LiquidSoap, but I will most likely have to charge more because it looks like I would be able to run 1/2 the amount of auto dj's on a server now vs. sc_trans

Is anyone else having the same issues?

Seems sc_trans is nearly twice as efficient as LiquidSoap.  I have the same issue on other servers here is a screenshot of one, both CPU and Memory are nearly twice as much

Yes, that is to be expected as you are actually running several components written in different languages talking to each-other as opposed to sc_trans which runs as a single process doing everything internally.

The trade-off of course is that you have considerably more features using a modular approach as liquidsoap does.

To elaborate on Roger's post, it's important to bear in mind that sctrans2 and Liquidsoap are completely different beasts.  sctrans2 is a source application, written in C++, and designed for a single purpose.  Liquidsoap, on the other hand, is an interpreter for an entire programming language dedicated to audio processing.  The purpose for which we use Liquidsoap with Centova Cast only just scratches the surface of what Liquidsoap is capable of... if you learn the Liquidsoap scripting language, you can do some jaw-dropping stuff with Liquidsoap that would be entirely inconceivable with sctrans2.

That said, while Liquidsoap is indeed a bit more resource-hungry, the results you're seeing on your particular server are atypical of what I've seen on most of our clients' machines and on our development servers.  It's important to make sure you're comparing apple to apples.  Make sure you've configured both Liquidsoap and sctrans2 with identical encoder settings (stereo MP3 at 128kbps/44KHz on both, for example) and an identical number of mount points.  Remember that adding a second mount point doubles the amount of work the application has to do, so if you're comparing a 2-mount point Liquidsoap instance to a 1-mount point sc_trans2 instance, that's not a fair fight.

Here's a few lines of `top` output from a dev server that are more representative of what I've seen:

  PID USER      PR  NI  VIRT  RES  SHR S  %CPU %MEM    TIME+  COMMAND
19181 ccuser    20   0  212m  36m 9364 S   5.6  7.3   0:06.36 liquidsoap
19107 ccuser    20   0  229m 7584 3204 R   4.6  1.5   0:08.10 sc_trans
19265 ccuser    20   0 29924 5252 1772 S   4.0  1.0   0:01.80 ices
 
It's my experience that the CPU utilization difference between Liquidsoap and sc_trans is typically within a percentage point or so -- certainly not double!  And that's reflected above.  And ices uses even less than sc_trans.

As for memory, yes, liquidsoap has a substantially higher baseline memory usage.  (Be sure you're looking at the "RES" column.)  But you'll find that sc_trans' memory utilization grows substantially with each additional mount point, whereas with Liquidsoap you generally don't see much difference per-mount point. 

Unfortunately that's the cost of all of the extra functionality offered by Liquidsoap, but really, memory is cheap these days.  Besides, I see *so* many stream hosting providers deploying high-end servers with 16GB or 32GB of RAM, and then actually using perhaps 1GB.  Memory is probably the most underutilized resource on stream hosting servers, so for most of our clients I don't expect Liquidsoap's higher resource utilization will cause much if any real-world impact.

seems funny seeing sc_trans claimed as less resource intensive to something else as we often got complaints that sc_trans was using too many resources (which was mainly due to re-encoding everything even when it wasn't needing to be done - which is also the reason for higher resources as you re-encode more streams via sc_trans).

and like is noted, the main difference is most likely because it's all a contained / optimised(ish) c++ application compared to what liquidsoap is implemented as / via.


really smart re-encoding is the ideal way to go where input sources are only re-encoded when needed. otherwise any viable auto-dj solution just needs to just pass the audio straight-through without re-encoding with it at most just checking frames and ensuring they are aligned to work around client player issues when sources change / drop (as that's what causes clients to fail in most cases and they can cope with incomplete audio as long as it's all aligned on frame boundaries correctly).


either way, this has tickled me

seems funny seeing sc_trans claimed as less resource intensive to something else as we often got complaints that sc_trans was using too many resources (which was mainly due to re-encoding everything even when it wasn't needing to be done - which is also the reason for higher resources as you re-encode more streams via sc_trans).

We heard this a lot when sc_trans2 was first released, but once end-users got ahold of it and realized the huge amount of new functionality it offered vs. sc_trans1 or ices the complaints more-or-less dried up.

The other issue I've seen is the widespread misunderstanding of how processes use memory on Linux... Both Liquidsoap and sc_trans2 use a fair bit of shared memory/libs, and when folks check their process list and see the virtual memory size of sc_trans2/liquidsoap up in the 250MB range (vs ~32MB for ices), they believe it's using 250MB per process.  The number of times I've had to explain that the RSS column is the per-process memory utilization.....

and like is noted, the main difference is most likely because it's all a contained / optimised(ish) c++ application compared to what liquidsoap is implemented as / via.

Yep, Liquidsoap is implemented in ocaml.  I have virtually no experience with ocaml so I'm not sure what kind of overhead it adds (as a language or via its libraries) but I presume the extra CPU utilization is due to a combination of the ocaml factor and the overall script-based architecture of Liquidsoap.

really smart re-encoding is the ideal way to go where input sources are only re-encoded when needed.

ices does this, and it's amusing to watch the CPU utilization drop to near-nil when the source media is correctly pre-encoded.   Unfortunately Liquidsoap doesn't seem to support this, likely because of the way audio is pipelined through the application -- I believe it's arbitrarily decoded during input processing, then the raw audio data makes its way through any processing performed by the user's script, and Liquidsoap internally has no real way of knowing what the output format is going to be until the raw audio data hits the output(s).

For what it's worth after upgrading to shoutcast v2 and liquidsoap today. I'm also seeing up to twice CPU % utilisation from liquidsoap v ices-cc. Disabling replaygain support seems to make no difference unfortunately. Regardless from the documentation... 

---
Usage optimisations for liquidsoap.

This page lists various optimizations that can be used for running liquidsoap under various conditions and hardware.

Load optimization.

On machine with several cores, liquidsoap's load can be reduced by splitting computations into several processes. This will ease the operating system to balance the load of the processes to the various cores.

For communicating between the instances, you can use the {input,output}.jack operators, but this does not communicate metadata and breaks (end of tracks). You can also give a try to the experimental {input,output}.marshal operators, but using them can be troublesome.

Also, using an external encoder helps reducing the load. However, it is not recommended for any format using the ogg format.

Memory usage.

Liquidsoap uses caml's float to represent raw audio data. While this is a great optimization for CPU usage, this may consume a lot of memory, in particular when buffering a lot of data.

If you want to optimize memory usage in this kind of situations, the global setting set("buffering.kind","raw") will force data buffering to be done in PCM S16LE format, which representation in memory is smaller.

However, in this case, data will be translated back and forth for each buffering, leading to some computational cost.

Other possible values for this operator are:

    "default": standard representation.
    "disk": will store temporary data in several big files.
    "disk_manyfiles": will store temporary data in a lot of small files. This option is less CPU-consuming than "disk".

For data stored on disk, the temporary directory can be set using the set("buffering.path", "/new/path") command.

The disk caching can also be used along with ramfs or tmpfs. In this case, files are also put in the RAM however, the temporary data is no longer managed by liquidsoap. This may help to reduce the load generated by the garbage collector when dealing with important amount of allocated memory.

http://savonet.sourceforge.net/doc-0.9.2/optimization.html
---

Party Vibe Radio