Backups

Read 24069 times
I think I read somewhere there was a backup script? If not how would you recommend we run backups, rsync ?   
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
Yup, check /usr/local/centovacast/extra/backup.sh. :)
Whoops, that hasn't been updated for v3 yet.  It just needs /home/centovacast/ replaced with /usr/local/centovacast/ and /vhosts/ replaced with /var/vhosts/ -- do a search/replace and it should work fine with v3.

I'll update this script for the next build but we have a proper backup system in the works that is able to provide per-account backups/restoration (as well as backing up an account and restoring it to a different Centova Cast installation) which will eventually replace this.  It's unfortunately a very complicated procedure behind the scenes (mainly due to the fact that ID numbers have to be stripped at backup time, then unique ID numbers reintroduced at restoration time, and everything has to maintain its referential integrity) so it won't make the initial 3.0 release, but hopefully will be available at least for testing in one of the subsequent point releases.


Yes I do understand that all my other Centova servers all used cPanel/WHM so I was able to set up automated backups.
I would be willing (and I am sure others) to a  pay-per-month or even a one time fee for some type of addon-on for a automated script that would allow me and my clients to set up backups, download backups, schedule, send to local or remote server, and manage their backups right in their Centova Panel, that would be a great selling feature and also save lots of time. That is one valuable lesson I have learned in the past, backups are very important and you just sleep better @ night! Guess this should be moved to feature request.
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
Can we know what files or directories are backup and where ?
Any updates on this Steve?
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
I'm actually just going to post this here for now, rather than in the manual, because technically it's still experimental.

So to make this clear: YMMV, this may break, you may lose all your data, your cat may become impregnated, errors may occur, etc.

That said, the v2-to-v3 import utility is actually built on top of the backup/restore functionality in v3, so it has indirectly received a fair bit of testing and should be reasonably usable.

Creating a Backup

The command to back up an account is:
Code: [Select]
/usr/local/centovacast/bin/ccmanage backup --username=USERNAME
USERNAME is, of course, the username of the account to backup.

The available parameters are documented in the backup API reference.  (For example, to use the "nocontent" argument documented in the API reference, pass the --nocontent=1 parameter to the backup command).

Restoring a Backup

To restore a backup, use:
Code: [Select]
/usr/local/centovacast/bin/ccmanage restore --filename=FILENAME --rpchostid=RPCHOSTID
FILENAME is the full pathname to the backup file to restore, and RPCHOSTID is the ID number of the server you want to restore it onto.  If you only have one server, use --rpchostid=1.  If you have multiple servers, check the Management->Hosts page in the admin area for the ID numbers.  (Hover over the "Edit" link -- the ID is at the end of the URL.)

The available parameters are documented in the restore API reference.

General Information

An account backup is just a zip file.  No crazy proprietary formats.  A backup contains all the user's files (all of them, including the server/source configuration files, media files, logs, etc.), a database dump in a custom format internal to Centova Cast (which is necessary since a standard SQL dump would cause key conflicts upon restoration), and some metadata files.

Backups contain ALL data relevant to the account -- the account configuration, bandwidth limits/disk quota settings, statistics, media files, playlists, etc.

If the user has a lot of files, the backup will be huge and will take ages to create.  You may consider using the --nocontent and/or --nologs parameters to omit the var/spool/ and var/log/ directories from the backups, and then use rsync or another efficient mechanism to externally backup the users' media files to another machine.  You'll find that that's a LOT faster than including them in the backups -- ZIP compression of media files (or any general-purpose compression, really) is very slow and provides a very poor compression ratio.

Backups can be created on one machine and restored to the same machine, or copied to another machine and restored there.  During restoration, Centova Cast will change the IP address to the local machine's IP address, and will change the port (if needed) to an available port, but everything else will remain unchanged and should yield a copy of the account on the new server that is completely identical to the original.

Backups are cryptographically signed using a key that's unique to your server.  By default, the "restore" command will reject any backups that weren't created by your server.  This is so that (in future, when we put the backup/restore functionality into the UI) users won't be able to restore backups created on other servers that have configurations not permitted on your server.  You can bypass this (as admin) with the --novalidate parameter to restore a backup from another server.

Enjoy, and if you encounter bugs, report them. :)
Last Edit: March 19, 2013, 01:57:44 am by Centova - Steve B.
Thanks. But isn't any way to make a global backup to move all the data to another server? For example, in v2, backup the vhost dir and export the full database...
Last Edit: March 19, 2013, 06:31:02 am by mikels
We can right now just go into the database by clicking on the manage database under utilities, then export / save the entire DB, but a more complete backup option, or even as MIKELS said, using something like RSYNC to move the backups to another server right in the panel would really help me sleep better @ night, even automate them using a cron -- I think I already recommended it in the feature request though
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
Thanks. But isn't any way to make a global backup to move all the data to another server? For example, in v2, backup the vhost dir and export the full database...
Well, yes, you can of course do the same thing under v3.  Dump the database, and take a snapshot of /usr/local/centovacast/var/vhosts.

And yes, there is already an automated script to make a full backup and rsync it to a remote server:
Code: [Select]
/usr/local/centovacast/sbin/simplebackup --host=remote.example.com --port=22 --username=root --path=/path/to/store/backups/
You'll probably also want the --delete option, but run it with --help first to see what that does and understand the risks.
This backup utility does work right?

When I try to do a backup on an account, I get the following:

Code: [Select]
[INF] Creating database backup
[INF] Creating database dump /usr/local/centovacast/var/tmp/cc-app/database-usernamehere_20130601-114749_51aa41d539b5f.ccsql
[INF] Creating backup manifest
[INF] Creating backup archive
Result: ERROR Socket error while receiving command response

What would be causing that socket error ?
Last Edit: June 01, 2013, 11:51:29 am by DJFire_CFR
CrossFire-Hosting LLC.
Co-Owner
I have not been able to get it to work either, it's been a while since I tried but I think I had the same error messages
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com
Well, for some reason it seem to have fixed itself..

When I went looking around to see if a file was created somewhere, I did find one.. under 

/usr/local/centovacast/var/vhosts/username/.backup

then after transfering it to another server.. I tried restoring, but seems i did something wrong, cause it said the zip file wasnt found.. So I went back over to the original server, ran the back up again, and its going through the process now..

Now I just got to figure out how to get this backup to restore on another server.  I did read the posts about that said rsync.. but im not keen on how to use rsync.. =\


After it completes the backup fully, it puts it here:

/usr/local/centovacast/var/backups/vhosts/
Last Edit: June 01, 2013, 12:41:31 pm by DJFire_CFR
CrossFire-Hosting LLC.
Co-Owner
Well, I got closer with this...

but when using  --novalidate=0

I get a mismatch error, as if its not even acknowledging it

Code: [Select]
/usr/local/centovacast/bin/ccmanage restore --filename=/usr/local/centovacast/var/backups/vhosts/backup-username_20130601-123518_51aa4cf6da626.zip --rpchostid=1 --username=username --novalidate=0


Results in:
Code: [Select]
[INF] Extracting backup archive ...
Result: ERROR Could not unpack backup file: Signature verification failed: Signature mismatch
CrossFire-Hosting LLC.
Co-Owner
I messed with it over the weekend a little, seemed to just hang, tried multiple Centova servers and remote servers never got it to work
My Auto DJ
Orlando, FL USA
Quality SHOUTcast Hosting http://myautodj.com
SHOUTcast Widgets http://shoutcastwidgets.com