When playing songs from a general playlist on random, does the system:

1. play each song only once until all songs in that playlist are played and then start a new random sequence?

or

2. rely only on the 6 hour repeat limit to make sure songs aren't played twice and can result in some songs played more often than others in the same playlist?

I wonder because I want to limit songs repeating over longer periods of time as well as over the short term and knowing this will help me in how to proceed.

rely only on the 6 hour repeat limit to make sure songs aren't played twice

Yes, that's correct.

So, does that mean that any playlist is limited for 6 hours?
So If I have a bunch of songs/hours, Do I create bunch of playlists of 6 hours each in order to play all the files?

I've replied to that question here:
http://forums.centova.com/index.php?topic=3447.msg10337#msg10337

In short, no you are not limited to only 6 hours of music, but Centova Cast will simply do its best not to play the same song over the course of 6 hours.