Backupskript mit rsync für zentrale Backup-Server

Lesenswerte Artikel, Anleitungen und Diskussionen
elfrico
Posts: 9
Joined: 2004-03-05 19:04

Backupskript mit rsync für zentrale Backup-Server

Post by elfrico » 2004-03-05 20:28

Hi,

bin neu hier, möchte euch daher als Wilkommengeschenk mein Shell-Skript vorstellen, mit welchem ich meine Backups vom Server ziehen.
Ich verwende dazu rsync, um inkrementelle Backups zu machen.
Die Backups rotieren dann von Tag zu Tag, es werden Links zum vorangegangenen Tag erzeugt, der letzte Backup-Tag wird rausgeschmissen.

In einem Konfigurationsfile werden die zu sichernden Pfade verschiedener Hosts folgendermaßen angegeben

Code: Select all

####################################################################
#
# /etc/backup.conf
#
####################################################################
#
# localhost
#
####################################################################
localhost /home
localhost /etc
localhost /var
localhost /boot
localhost /boot2
localhost /root
localhost /local
####################################################################
#
# remote.tld
#
####################################################################
remote.tld /bin
remote.tld /boot
remote.tld /etc
remote.tld /lib
remote.tld /opt
remote.tld /root
remote.tld /sbin
remote.tld /usr
remote.tld /srv
remote.tld /home
remote.tld /var
und hier das Skript:

Code: Select all

#!/bin/bash

# /usr/local/sbin/backup

####################################################################
#
# Global variables
#
####################################################################

BACKUP_PATH=/backup             # Default backup path
CURRENT_PATH=`pwd`                      # Store the current path
BACKUP_DAYS=30                          # Default number of backup days
LOG_FILE=/var/log/backup.log        # Default log file path
CONFIG_FILE=/etc/backup.conf    # Default config file path
LOCAL_RSYNC_PATH=/usr/bin/rsync         # Default local rsync path
REMOTE_RSYNC_PATH=/usr/bin/rsync  # Default remote rsync path
DELETE_LOG=FALSE         # Default value if logfile will be deleted or not
RESTORE=FALSE            # Switch, do not change!
RESTORE_DAY=0            # Default day for restores
ERROR_TEXT=""            # Do not change!
MIN_KBYTES=1000000       # Define minimal amount of free diskspace
DISK_SPACE_LOW=FALSE     # Do not change
SAVE_BROKEN_BACKUP=FALSE # Do not change
SENDMAIL=TRUE            # Send Email if backup fails
MAIL_RECEIPT="email_1 email_2 email_n"

####################################################################
#
# FUNCTION log message
#
# DESCRIPTION
#   Writes messages to the log file. If the script is not run
#   as cron job, it also prints the messages at the console
#
####################################################################
function log()
{
  echo $1>>$LOG_FILE
  if [ $TERMINAL = TRUE ]; then
    echo $1
  fi
}

####################################################################
#
# FUNCTION error message
#
# DESCRIPTION
#   Appends error messages to an error message string
#
####################################################################
function error()
{
  message=$1
  log "ERROR: $message"               # Do also log and print the messages
  ERROR_TEXT="$ERROR_TEXT $message"   # Append error msg to previous msgs
  ERROR_TEXT=`echo $ERROR_TEXT $'r'` # Append newline after each msg
}


####################################################################
#
# FUNCTION rsyncIt source destination
#
# DESCRIPTION
#   Calls the rsync program with the given parameters.
#
####################################################################
function rsyncIt()
{
   HOST=$1
   DIR=$2

   RSYNC_OPTS=" -av --delete --rsync-path=${REMOTE_RSYNC_PATH} --relative"
   if [ $# -ne 2 ]; then
     log "wrong number of arguments in rsyncIt()"
     error "rsyncIt: wrong number of arguments in $0"
     # exit
   fi

   # rsync runs out of memory if a very large number of files is transfered.
   # Therefore the file list is splitted into smaller parts from top of the
   # source rsync directory
   # dirlist=`rsh $HOST ls -a $DIR` # first the content of the directory is
                                  # required
   #if [ -n $dirlist ]; then
   #   error "rsyncIt: could not get directory list on $HOST:/$DIR."
   #fi

   # process all parts of the complete rsync file list step by step
   #for i in $dirlist; do
   #  if [ "$i" != "." -a "$i" != ".." ]; then
        backupDir=${DIR}
        log "${LOCAL_RSYNC_PATH} $RSYNC_OPTS $HOST:$backupDir ${BACKUP_PATH}/0/${HOST}"
        ${LOCAL_RSYNC_PATH} $RSYNC_OPTS $HOST:$backupDir ${BACKUP_PATH}/0/${HOST}/
        rsync_exit_code=$?
        if [ $rsync_exit_code -ne 0 ]; then
           error "rsyncIt: rsync failed with exit code $rsync_exit_code while backing up ${HOST}:/${backupDir}"
        fi
   #  fi
   #done
}

####################################################################
#
# FUNCTION check()
#
# DESCRIPTION
#    1. checks if script runs under user root
#    2. checks if config file exists
#    3. checks if all directories for incremental backups exists.
#       If they don't exist, the directories will be created
#
####################################################################
function check()
{
   if [ `whoami` != "root" ]; then
     log "You have to be logged in as root to execute the script. Exiting..."
     error "check: You have to be logged in as root to execute the script. Exiting..."
     exit 1
   fi

   if [ ! -e $CONFIG_FILE ]; then
     log "ERROR: Configuration $CONFIG_FILE does not exist. Exiting..."
     error "check: Configuration $CONFIG_FILE does not exist. Exiting..."
     exit 1
   fi

   if [ ! -d $BACKUP_PATH ]; then
     mkdir $BACKUP_PATH
     if [ $? -ne 0 ]; then
        error "check: Could not create directory $BACKUP_PATH."
     fi
     log "Directory $BACKUP_PATH created."
   fi

   if [ ! -d ${BACKUP_PATH}/0 ]; then
     mkdir ${BACKUP_PATH}/0
     if [ $? -ne 0 ]; then
        error "check: Could not create directory ${BACKUP_PATH}/0."
     fi
     log "Directory ${BACKUP_PATH}/0 created."
   fi

   # Check if enough diskspace is available
   FREE_KBYTES=$( df ${BACKUP_PATH} | awk '/^//{print $4}' )
   if [ -n "$FREE_KBYTES" ]; then

     if [ $FREE_KBYTES -lt $MIN_KBYTES ]; then
       error "check: Disk space low!!!"
       error "There are only ${FREE_KBYTES}KB avilable on ${BACKUP_PATH}"
       DISK_SPACE_LOW=TRUE
     fi
   else
     error "check: I was not able to do a free diskspace check"
   fi

}

####################################################################
#
# FUNCTION copyBackup source destination
#
# DESCRIPTION
#   Copies the source directory to the destination directory.
#   Instead of a simply copy from source to destination,
#   there are created hard links  from source to destination.
#
####################################################################
function copyBackup()
{
   source=$1
   destination=$2
   if [ $# -gt 2 ]; then
     if [ $3 = "asLink" ]; then
        log "Creating links from $source TO $destination."
        cp -al ${source} ${destination}
        if [ $? -ne 0 ]; then
           error "copyBackup: Could not create links from ${source} to ${destination}."
        fi
         #If cp -al is not available on the system, replace with the following lines
         #cd ${source}
         #find . -print | cpio -dplu ${destination}
     fi
   else
     if [ -d ${source} ]; then
       log "Moving ${source} to ${destination}"
       mv ${source} ${destination}
       if [ $? -ne 0 ]; then
          error "copyBackup: Could not move ${source} to ${destination}."
       fi
     fi
   fi
}

###################################################################
#
# FUNCTION incrementBackup
#
# DESCRIPTION
#   Rotates and increments the backup
#   1. Delete the oldest backup directory
#   2. Move each backup directory representing one backup day to
#      one backup day further
#   3. The latest backup directory will be linked to the backup of
#      one day before
#
###################################################################
function incrementBackups()
{
   log "INCREMENTING BACKUPS"
   counter=$((${BACKUP_DAYS}-2))

   # Delete oldest backup directory
   if [ -d ${BACKUP_PATH}/$((${BACKUP_DAYS}-1)) ]; then
     log "deleting directory ${BACKUP_PATH}/$((${BACKUP_DAYS}-1))"
     rm -fr ${BACKUP_PATH}/$((${BACKUP_DAYS}-1))
     if [ $? -ne 0 ]; then
        error "incrementBackups: Could not delete latest backup directory ${BACKUP_PATH}/$((${BACKUP_DAYS}-1))"
     fi
   fi

   while [ $counter -gt 0 ]; do
     copyBackup ${BACKUP_PATH}/${counter} ${BACKUP_PATH}/$((${counter}+1))
     counter=$((${counter}-1))
   done

   copyBackup ${BACKUP_PATH}/${counter} ${BACKUP_PATH}/$((${counter}+1)) asLink
}

###################################################################
#
# FUNCTION executeConfig filename
#
# DESCRIPTION
#   Reads file $filename and rsyncs the specified path.
#   All letters behind an '#' will be ignored.
#
###################################################################
function executeConfig
{
   log "EXECUTING CONFIG"

   while read LINE
   do
     # log "read line: $LINE"
     validLine=${LINE/%#*/ }
     if [ "$validLine" != " " ]; then
       # log "$validLine is a valid target for rsync."
       rsyncIt $validLine
     fi
   done <$1
}


###################################################################
#
# FUNCTION getEnvironment
#
# DESCRIPTION
#   If the script runs under cron, the environment will be set.
#
###################################################################
function getEnvironment()
{
   if [ $TERMINAL = FALSE ]; then
     export PATH="/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin"
     log "Environment is set: $PATH"
   fi
}


###################################################################
#
# FUNCTION checkTerminal
#
# DESCRIPTION
#   Checks if the script runs on a tty (cron or human)
#
###################################################################
function checkTerminal()
{
   myTerm=`tty`
   # log "Terminal is: $myTerm"
   if [ "$myTerm" = "not a tty" ]; then
     TERMINAL=FALSE
   else
     TERMINAL=TRUE
   fi
   # log "Terminal is $TERMINAL"
}


###################################################################
#
# FUNCTION deleteLog
#
# DESCRIPTION
#   Deletes the log file.
#
###################################################################
function deleteLog()
{
   if [ -e $LOG_FILE ]; then
      rm -f $LOG_FILE
      if [ $? -ne 0 ]; then
        error "deleteLog: Could not delete log file $LOG_FILE."
      fi
   fi
}




###################################################################
#
# FUNCTION printUsage
#
# DESCRIPTION
#   If the user pased wrong arguments or the script is called
#   with the -h option, then the usage is printed out.
#
###################################################################
function printUsage()
{
   echo
   echo "BACKUP"
   echo
   echo "USAGE: backup [ -lhpLBCDRLOTM ]"
   echo
   echo "OPTIONS:"
   echo "    -l    Do not delete log file."
   echo "    -h    Print Usage."
   echo "    -L    specify log file (default: /backup.log)."
   echo "    -B    specify backup path (default: /backup)."
   echo "    -C    specify config file (default: /etc/backup.conf)."
   echo "    -D    specify amount of backup days (default: 7)."
   echo "    -R    specify remote rsync path (default: /usr/local/bin/rsync)."
   echo "    -L    specify local rsync path (default: /usr/bin/rsync)."
   echo "    -O    specify directory to restore. Format: host:/directory"
   echo "    -T    specify day of restore (default: 0). "
   echo "          '0' is the latest backup. The maximum number of backup days"
   echo "          can be specified with the -D option."
   echo "    -M    specify Mail address in case of error notification"
   echo "    -s    save broken backups"
   echo
   exit 0
}

###################################################################
#
# FUNCTION parseOpt $*
#
# DESCRIPTION
#   Parse all command line options and decide what to do.
#
###################################################################


function parseOpt()
{
   while getopts shlL:B:C:D:R:O:L:T:M: OPTS
   do
      case $OPTS in
      h) printUsage;;
      l) log "Log file will not be deleted."; DELETE_LOG=FALSE;;
      L) log "Log file saved at $OPTARG"; LOG_FILE=$OPTARG;;
      B) log "Backup path set to $OPTARG"; BACKUP_PATH=$OPTARG;;
      C) log "Using configuration file $OPTARG"; CONFIG_FILE=$OPTARG;;
      D) log "$OPTARG backup days selected"; BACKUP_DAYS=$OPTARG;;
      L) log "local rsync path $OPTARG selected"; LOCAL_RSYNC_PATH=$OPTARG;;
      R) log "remote rsync path $OPTARG selected"; REMOTE_RSYNC_PATH=$OPTARG;;
      O) log "Restoring backup $OPTARG"; RESTORE=TRUE; RESTORE_PATH=$OPTARG;;
      T) log "Restoring backup of day $OPTARG"; RESTORE_DAY=$OPTARG;;
      M) log "Mail will go to $OPTARG"; MAIL_RECEIPT=$OPTARG; SENDMAIL=TRUE;;
      s) log "If the backup is broken, save it"; SAVE_BROKEN_BACKUP=TRUE;;
      ?) printUsage; shift;;
      esac
   done
   shift $(($OPTIND - 1))
}

###################################################################
#
# FUNCTION restore
#
# DESCRIPTION
#   Restores a backup from the backup server to the destination
#   host.
#
###################################################################
function restore()
{
   log "Restoring $RESTORE_PATH"
   DIRECTORY_FROM=${RESTORE_PATH#*:/}
   HOST=${RESTORE_PATH%:/*}
   DIRECTORY_TO=${DIRECTORY_FROM%*/*}
   RSYNC_OPTS=" -av --delete --rsync-path=${REMOTE_RSYNC_PATH}"
   ${LOCAL_RSYNC_PATH} ${RSYNC_OPTS} ${BACKUP_PATH}/${RESTORE_DAY}/${HOST}/${DIRECTORY_FROM} ${HOST}:/${DIRECTORY_TO}
   if [ $? -ne 0 ]; then
     log "ERROR: Restoring failed. Please refer to rsync log directory."
   fi
   exit 0
}




###################################################################
#
# FUNCTION sendErrorMail
#
# DESCRIPTION
#   If error notification is turned on, a mail with the
#   error message content will be sent to the specified receipient
#
###################################################################
function sendErrorMail()
{
   mailtext=$'From: backupr'
   mailtext="$mailtext $ERROR_TEXT"
   echo $mailtext | mailx -s "Error in daily backups" -a "From: backup"  $MAIL_RECEIPT
}


###################################################################
#
# FUNCTION saveBrokenBackup
#
# DESCRIPTION
#   If an error has occured, it's a good idea to save the current
#   backup directory aside.
#
###################################################################
function saveBrokenBackup()
{
   dateSuffix=`date +%y%m%d`
   cp -R ${BACKUP_PATH}/0 ${BACKUP_PATH}/brokenBackup.${dateSuffix}
   log "Saved backup tree to ${BACKUP_PATH}/brokenBackup.${dateSuffix}"
   error "Saved backup tree to ${BACKUP_PATH}/brokenBackup.${dateSuffix}"
}




###################################################################
#
# FUNCTION main
#
# DESCRIPTION
#   Main program
#
###################################################################
function main()
{
   checkTerminal
   getEnvironment
   parseOpt $*

   if [ $DELETE_LOG = TRUE ]; then
     deleteLog
   fi

   if [ $RESTORE = TRUE ]; then
     restore
   fi

   log "-----------------------------------------------"
   log "BACKUP STARTED: `date`"

   cd /
   check
   incrementBackups
   executeConfig $CONFIG_FILE
   cd $CURRENT_PATH

   log "BACKUP FINISHED"
   log "-----------------------------------------------"

   if [ "$ERROR_TEXT" != "" -a $SENDMAIL = TRUE ]; then
     log "$ERROR_TEXT"
     if [ $DISK_SPACE_LOW = FALSE -a $SAVE_BROKEN_BACKUP = TRUE ]; then
        saveBrokenBackup
     fi
     sendErrorMail
   fi

   exit
}

# Start the script
main $*

Ihr solltet natürlich die globalen Variablen ganz oben nach Euren Bedürfnissen anpassen 8O

Anschliessend einen Cronjob mit crontab -e erstellen:

Code: Select all

############################################################################
#
# C R O N   T A B
#
# min hour day day day job
############################################################################

############################################################################
#
# Daily Backups
#

55 23 * * *  /bin/bash -c /usr/sbin/backup
Hier ist es nun möglich, innerhalb von 30 Tagen auf verschiedene Snapshots der einzelnen Server zuzugreifen.

Wenn Ihr das Skript richtig konfiguriert habt, sollte es so auf einem zentralen Backup-Server laufen (hier: Debian). Ansonsten einfach mal rumprobieren. 8O
Ich musste vorher mit ssh-keygen zunächst Schlüssel generieren, um die Daten via rsync vom Remoteserver abholen zu können.

Nicht vergessen:
- Auch der Backup-Server benötigt Backups auf anderen Medien (es kann ja mal brennen). Die Medien am Besten an einem anderen Ort aufbewahren.
- Am besten RAID mit redundanter Spiegelung für Backup-Server verwenden.

Ansonsten bin ich natürlich für jegliche Verbesserungsvorschläge dankbar.

Habs leider noch nicht geschafft, hierüber mal ein ordentliches Tutorial aufzusetzten, geschweige denn von einer Download-Site, sorry.