You don't have to restart autodj after you've made changes to the playlist, they are picked up without a restart. Just be sure to run 'update media library' after uploading/deleting mp3s..