Once you get your system up and running they way you want it, it's only a matter of time before you either make a serious mistake, get HD corruption, or a HD dies all together. COUNT ON IT!
What can you do? Back it up!
So you are probably asking "what should I back up", "how to back things up", etc. Starting out, it's a good idea to backup the STATE of the system onto floppy (or USB flash, etc.). What do I mean by "state"? This small backup will just keep a copy of the primary configuration files, a listing of the binaries installed on your machine, etc. This backup will at least let you get a new system running again with a minor amount of work after re-installing the OS manually. A pain but much better than nothing.
After creating a state config backup, I really recommend need to backup everything. Everything can mean different things to different people. For me, I want a FULL backup where I can restore the entire system onto a new or replacement HD with as little work as possible. To other people, they just want a DATA backup where they just want to backup their various word processing files, pictures, etc. to a safe place.
Both styles of backups can take up a LOT of space which can be a problem. The backup industry used to only have tape drives as the solution. The problem with tape drives is that they can be slow, require multiple tapes, can be very expensive, and unforutunately be unreliable. All of these factors have made hard drive or CD/DVD backups very appealing.
TrinityOS covers backups via:
- STATE backup to a floppy
- FULL backups to a HD
* Data being either local to the backup server as well as
and remote data via NFS / Samba shares
- Tape backups using the commercial tool Bru for local backups
Copying files to floppies is EASY. All you need to do is:
- Format the floppy diskette:
mke2fs /dev/fd0
- Mount the floppy
mount -t ext2 /dev/fd0 /mnt/floppy
- Copy at least the following files to the floppy:
Recommended:
OPTIONAL (recommended but only if you use these files):
- I would also recommend to record a full file listing of your system as well:
ls -laR / | gzip -9 > /mnt/floppy/file-list-`date +'%b%d'`.lst.gz
- Another GREAT idea comes from the Config-HOWTO to make a backup of your HD's Master Boot Record (MBR). So, instead of manually having to recreate it from your updated details in Section 4, simply copy the MBR to a file:
Example:
this will backup /dev/hda's table:
dd if=/dev/hda of=/boot/mbr.dd bs=512 count=1
cp /boot/mbr.dd of=/mnt/floppy
Use this to restore the table:
dd if=/mnt/floppy/mbr.dd of=/dev/hda bs=512 count=1
You can find more info about the parition table layout at: http://www.win.tue.nl/~aeb/partitions/partition_tables-2.html
** You will need to redo this backup every time you:
Backing systems up to a HD has finally become easy and affordable. Not only are large HDs cheap but you can put them into Firewire/USB enclosures for portability and hot-plug abilities. The same can be said for CD/DVD backups but I find that I /don't/ want to constantly shovel discs in / out and even with compression, backing up 100GB of data requires is a LOT of DVDs.
Here is the TrinityOS "backup-to-disk" script. What this script brings to the table that I haven't seen before is:
Please read through the script's comments to understand how it works but here are some highlights:
There are some known limitations with this script that might not work for you. In the future, I plan to make the script support simultaneous NFS backups, use BASH functions, etc.
If you have ideas, URLs for similar backup solutions, or you'd like to see a specific feature added, let me know.
<backup-to-disk START>
#!/bin/sh
# TrinityOS HD Backup Script - Supports LOCAL and Remote NFS/SAMBA file systems
#
# Part of the TrinityOS(tm) documentation
# Written by David Ranch
# dranch@trinnet.net
#Version of the TrinityOS backup script
VERSION=v4.8
# v4.8 - 031404 - Initial release on TrinityOS
# v4.7 - 081403 - Added comments to add FULL and differential support
# v4.6 - 050803 - Removed the dos-c volume from dranch-lt-minidock
# - Added EXTHOST system as some new DNS servers give
# hostnames instead of IPs
# v4.5 - 032203 - More comments, moved some things around
# v4.4 - 011603 - made the backup destination more generic
# - moved away from hosts being IPs back to names. Very
# ugly and the df issue was due to line wrapping
# - Added the compression of the log files (10.5M to 1M)
# v4.3 - 011003 - Added verbage when NFS mount checks hang
# - Attempt to use df -P to fix parsing problems
# v4.2 - 122602 - Moved to using IP addresses vs. hostnames to help with
# df parsing issues
# v4.1 - 122402 - Updated Trinity directory list
# - removed a lame if/then that would stop remounting NFS
# if ANY nfs mounts existed for that specific remote client
# UNFORTUNATELY, Linux will allow duplicate NFS mounts...
# v4.0 - 112802 - Fixed the estimation phase for Samba clients
# v3.9 - 112502 - Added the backup of the backup-to-disk to the dest disk
# v3.8 - 090602 - Corrected the estimated backup size for local backups
# v3.7 - 090602 - Added additional text for firewall situations
# - moved -check termination point
# v3.6 - 090402 - Added additional formatting to improve backup output
# - Removed unneeded souce backup estimation
# - Added the ability to disable file-by-file logging
# - Changed the colors of the backup window
# - added the "check" option to check for minimium disk space
# v3.5 - 090302 - Added more FSs on Trinity
# v3.4 - 070702 - Added the spawning of a logging window
# - Added more comments
# v3.3 - 061802 - Added some more comments
# v3.2 - 060102 - Fixed some tail information errors
# - Deleted the use of restarting CRON as it is already dynamic
# - Fixed the problem where NFS couldn't umount at the end
# v3.1 - 053002 - Added some more comments
# v3.0 - 040202 - changes some mount points, more formatting, etc.
# v2.9 - 031902 - fixed the BACKUPPATH for Trinity to watch for sub-mounted dirs
# v2.8 - Added the capture of an error log
# v2.7 - Added addition error checking, more debug statements, etc.
# v2.6 - Only backup one physical FS at a time
# v2.5 - Added compression and HOT backups
# v2.4 - added Samba support
# v2.3 - Fixed backup paths to be more normal instead of overly nested
# v2.2 - added support for multiple NFS mountpoints
# v2.1 - changed to backup machine at home with additional testing
# v2.0 - added lots of network availablity testing
# v1.0 - Initial version
#NOTES
#-----
# - This backup script is intended to be run on the backup SERVER and not on
# the backup CLIENT
#
# - For remote NFS backups, the backup client needs to be the NFS server.
# The backup server is only an NFS client.
#
# - Remote backups are done using RELATIVE domain names. ie host names like
# "roadrunner" vs. "roadrunner.acme123.com". If you cannot ping just the hostname
# from the backup server, you need to fix this via the /etc/resolv.conf file
#
#
# - NFS users:
#
# No need to check if CDROMS are mounted on the client as they are seperate
# file systems that are not exported to NFS. If they are exported, just make sure
# they aren't included in the BACKUPPATH variable below
#
# This does NOT apply to backups via SMB !!
#
#
# - Samba users
#
# Nothing has to be loaded for things to work properly
#
#
# - Compression
#
# Compression isn't currently functional. Id like to do this via one pass
# but I don't see how that will be possible with using TAR
#
#
# - Seti
#
# This script looks to see if the Seti program is running. If you arent
# running seti or dont know what it is, dont worry about it.
#
# TO DO
# -----
#
# 1. Re-write the script to exensively use Bash functions instead. Put the
# unmounting into a function so when -check is used, it cleans up
#
# 2. update the logic to avoid duplicate NFS mounts
#
# 3. run a check to make sure the partition table and MBR are imaged
#
# 4. make the script multi-instance aware so if say multiple NFS backups are
# running, additional run scripts won't clobber the first run NFS backup
#
# 5. add command line support for FULL vs. DIFFERENTIAL support
#HOW TO USE THIS SCRIPT
#----------------------
#
# 1. Edit the BACKUP variables below to reflect the desired CLIENT machines,
# method for backup, etc.
#
#
# 2. Mount the local BACKUP disk
#
# For example:
#
# IDE BUS: mount /dev/hdc1 /mnt/backup-disk
#
# FireWire BUS: mount /dev/sdd1 /mnt/backup-disk
#
#
# -------------------------------------------------------------------------
# NOTE: if the file "/mnt/backup-disk/backup-drive-ready" doesn't exist
# on the backup drive, the backup will abort. This is just to make
# sure that not just any HD will be used for the backup
# -------------------------------------------------------------------------
#
#
# 3. NFS Users: Start up **REMOTE** NFS daemons
#
# This is not needed for LOCAL or SMB backups
#
# LOCAL: start the NFS client (OPTIONAL as this is done automatically)
# /etc/rc.d/init.d/portmap start
#
#
# REMOTE: start the NFS server
#
# /etc/rc.d/init.d/portmap start
# /etc/rc.d/init.d/nfs start
#
# NOTE #1: make sure that the backup clients IP addr is in
# its /etc/exports file
#
# NOTE #2: some hosts might need their IPCHAINS/IPTABLES
# firewall removed before NFS will work
#
#
# 4. Delete old CLIENT data directory on /mnt/backup-disk
#
# 5. Start new backup by running this script with the given host:
#
# ./backup-to-disk coyote
#
# You can also run "./backup-to-disk coyote -check"
# to understand the backup requirements (runs the estimation
# phase and then exits.
#
#Setup the BACKUP variables
#-------------------------------------------------------------------------------------
clear
if [ "$1" == "" ]; then
echo -e "\n\n** ERROR **: Backup source not specified "
echo -e "\nbackup-to-disk usage: \n"
echo -e " backup-to-disk < roadrunner | coyote | wile | acme > <-check>"
echo -e "\n -check : determine client disk requirements then exit\n\n"
exit 1
fi
case $1 in
roadrunner)
# Backup via NFS
#How to back things up
BACKUPMETHOD=NFS
#The machine to be backed up
CLIENT=roadrunner
#Backup SOURCE on the REMOTE machine
SOURCEMOUNT="/mnt/nfs"
#What files are being backed up from the SOURCE
MOUNTLIST="/ /var /home/johndoe /home/johndoe/pictures /home/johndoe/movies /tmp"
UNMOUNTLIST="/tmp /home/johndoemovies /home/johndoe/pictures /home/johndoe /var /"
#Backup Path
BACKUPPATH="bin boot bru dev dosc etc home home/johndoe \
home/johndoe/pictures home/johndoe/movies lib misc mnt opt root sbin tmp usr var"
#Backup destination
BACKUPDEST="/mnt/backup-disk"
DEST_PATH="/mnt/backup-disk"
#Do we want to do compression
COMPRESSION=no
#Backup options for NFS
NFSOPTIONS="rsize=8192,wsize=8192"
#Enable logging of every backed up file to output file
LOGGING=yes
;;
coyote)
#Backup via Samba
#How to back things up
BACKUPMETHOD=SAMBA
#The machine to be backed up
# SAMBA wants short names (NetBIOS)
CLIENT=coyote
#Backup SOURCE on the REMOTE machine
SOURCEMOUNT="/mnt/samba"
#What files are being backed up from the SOURCE
MOUNTLIST="coyote-c coyote-d"
UNMOUNTLIST="coyote-d coyote-c"
#Backup Path
BACKUPPATH="coyote-c coyote-d"
#Backup destination
BACKUPDEST="/mnt/backup-disk"
DEST_PATH="/mnt/backup-disk"
#Do we want to do compression
COMPRESSION=no
#Backup options for SAMBA
SMBOPTIONS="username=johndoe,password=<your-password-here>"
#Enable logging of every backed up file to output file
LOGGING=yes
;;
wile|wilee)
#Backup via local
#How to back things up
BACKUPMETHOD=LOCAL
#The machine to be backed up
CLIENT=wile
#Backup SOURCE on the LOCAL machine
SOURCEMOUNT="/"
#What files are being backed up from the SOURCE
MOUNTLIST=""
UNMOUNTLIST=""
#Backup Path
LOCALMOUNT="/dev/sdb3 /dev/sdc2 /dev/sda1 /dev/sdb1 /dev/sdc1"
BACKUPPATH="/ /usr/src /mnt/dos-c /mnt/dos-d /mnt/dos-e"
#Backup destination
BACKUPDEST="/mnt/backup-disk"
DEST_PATH="/mnt/backup-disk"
#Do we want to do compression
COMPRESSION=no
#Enable logging of every backed up file to output file
LOGGING=yes
;;
acme | acme-corp)
# Backup via NFS
#How to back things up
BACKUPMETHOD=NFS
#The machine to be backed up -- USE IP address to avoid "df" parsing iss.
CLIENT=acme
#Backup SOURCE on the REMOTE machine
SOURCEMOUNT="/mnt/nfs"
#What files are being backed up from the SOURCE
MOUNTLIST="/"
UNMOUNTLIST="/"
#Backup Path
BACKUPPATH="/"
#Backup destination
BACKUPDEST="/mnt/backup-disk"
DEST_PATH="/mnt/backup-disk"
#Do we want to do compression
COMPRESSION=no
#Backup options for NFS
NFSOPTIONS="rsize=8192,wsize=8192"
#Enable logging of every backed up file to output file
LOGGING=yes
;;
-h)
echo -e "\n\n ** ERROR: Hostname $1 not recognized. Aborting\n\n."
exit 1
;;
*)
echo -e "\n\n ** ERROR: Hostname $1 not recognized.\n"
echo -e "Usage: \n"
echo -e " backup-to-disk \[roadrunner | coyote | wile | acme\]\
<-check>\n"
echo -e " -check - calculates requires disk for remote host and exits\n\n"
exit 1
;;
esac
#LOCAL machine's network interface name
EXTIF=eth0
#----------------------------------------------------------------------------------
#-- DO NOT EDIT BELOW THIS LINE UNLESS YOU KNOW WHAT YOU ARE DOING ----------------
#----------------------------------------------------------------------------------
echo -e "\nWelcome to the TrinityOS HD backup script $VERSION"
echo -e "------------------------------------------------\n\n"
#Calculate the SERVERs IP address
#
EXTIP=`/sbin/ifconfig | grep -A 4 $EXTIF | awk '/inet/ { print $2 } ' \
| sed -e s/addr://`
EXTHOST=`host $EXTIP | awk '{print $5}'`
#Backup DESTINATION on the LOCAL machine - should be a LARGE disk
DEST_DIR="`date "+%m%d%y"`"
DEST="$DEST_PATH/$CLIENT-$DEST_DIR"
#Automatic backup time determination - do not edit
START=`date`
if [ "$LOGGING" == "yes" ]; then
#Override the variable contents now with the logging destination
LOGGING="$DEST/$CLIENT-backup.log"
else
LOGGING="/dev/null"
fi
if [ "$BACKUPMETHOD" == "NFS" ]; then
echo -e "\nMake sure that you have enabled the following on [ $CLIENT ] \n"
echo -e "echo 262144 > /proc/sys/net/core/rmem_default"
echo -e "echo 262144 > /proc/sys/net/core/rmem_max\n\n"
echo -e "\nPAUSING for 10 seconds\n"
sleep 10
fi
if [ "$BACKUPMETHOD" == "SAMBA" ]; then
echo -e "\nMake sure that you have disabled any Anti-Virus softeware on the backup"
echo -e "source. If you don't do this, the remote system can and will do wierd"
echo -e "things such as report file size changes during backup, etc."
echo -e "\nPAUSING for 10 seconds\n"
sleep 10
fi
#If we are using compression, make sure that Seti is NOT running
if [ "$COMPRESSION" == "yes" ]; then
GZIP="z"
if [ -f /usr/local/sbin/start-seti ]; then
SETIPID=`ps ax | grep "seti" | grep -v "grep" | awk '{print $1}'`
if [ -n "$SETIPID" ]; then
echo -e " ** Stopping SETI.."
kill $SETIPID
SETIPID=`ps ax | grep "seti" | grep -v "grep" | awk '{print $1}'`
if [ -n "$SETIPID" ]; then
echo -e " ** ERROR: Could not stop SETI"
exit 1
fi
fi
mv /etc/cron.hourly/start-seti /etc/cron.hourly.disabled/
echo -e " ** Warning: Restarting cron to then disable seti from starting"
/etc/rc.d/init.d/crond restart
fi
else
GZIP=""
fi
echo -e "\nPreparing to backup [ $CLIENT ] to [ $EXTIP ] via [ $BACKUPMETHOD ]"
if [ "$BACKUPMETHOD" == "NFS" ] || [ "$BACKUPMETHOD" == "SAMBA" ]; then
# Verify the required NETWORK subsystem is running..
if [ ! -n "`ping -c 1 $CLIENT | grep icmp_seq`" ]; then
echo -n " ** ERROR - ICMP: Cannot reach $CLIENT Aborting.\n\n"
exit 1
fi
echo -e "\n ICMP: [ $CLIENT ] is reachable.."
fi
#Do tests based upon the backup method
#
if [ "$BACKUPMETHOD" == "NFS" ]; then
echo -e " NFS: checking PORTMAP.."
if [ ! -n "`ps ax | grep portmap | grep -v "grep portmap"`" ]; then
echo -e "\n WARNING - NFS: PORTMAP not running. Attempting to start it.."
/etc/rc.d/init.d/portmap start
echo -e "\n"
if [ ! -n "`ps ax | grep portmap | grep -v "grep portmap"`" ]; then
echo -e " ** ERROR - NFS: Could NOT start PORTMAP. Aborting."
exit 1
fi
fi
echo -e " NFS: PORTMAP is running.."
echo -e " NFS: checking exports [any hangs at this point are due to remote FWs]"
echo -e " or the remote host is not running NFS at this point"
#Make sure we can mount the remote machine
#
# Newer NFS servers export the DNS name and not the IP
#
if [ -z "`showmount -e $CLIENT | grep "/" | awk '{print $2}' | grep "$EXTHOST"`" ] -o \
[ -z "`showmount -e $CLIENT | grep "/" | awk '{print $2}' | grep "$EXTIP"`" ]; then
echo -e "\n ** ERROR - NFS: Local machine not in $CLIENT export list. Aborting."
echo -e "\nExports list was:"
echo -e "----------------"
showmount -e $CLIENT
echo -e "----------------"
echo -e "\nExpected EXPORTed IP: $EXTIP (old NFS servers)"
echo -e "Expected EXPORTed DNS name: $EXTHOST (new NFS servers)"
exit 1
fi
echo -e " NFS: Remote machine [ $CLIENT ] is properly exporting to our IP"
echo -e " NFS: Starting to run NFS mounts.."
#Mounting the remote file systems
# BUG:
#
# WRONG: Linux allows duplicate NFS mounts, fix this logic to test for
# each specific mount
#
# if [ ! -n "`df | grep $CLIENT`" ]; then
echo -e " NFS: Mounting [ $CLIENT ] with options: [ $NFSOPTIONS ]"
for I in $MOUNTLIST; do
echo " Mounting: [ $SOURCEMOUNT$I ] "
mount -t nfs -o $NFSOPTIONS $CLIENT:$I $SOURCEMOUNT$I
done
if [ ! -n "`df | grep $CLIENT`" ]; then
echo -e " ** ERROR - NFS: Cound not mount [ $CLIENT ]"
exit 1
fi
# fi
echo -e " NFS: [ $CLIENT ] successfully mounted."
fi
if [ "$BACKUPMETHOD" == "SAMBA" ]; then
echo " SMB: Checking status of remote SMB host.."
#Make sure that the remote machine is responding to SAMBA requests
if [ -z "`smbclient -L //$CLIENT -N | grep -i "disk"`" ]; then
echo -e " ** ERROR: [ $CLIENT ] is not responding to SAMBA requests"
exit 1
fi
echo " Host [ $CLIENT ] is reponding to SMB requests.."
#Samba - Mount things up
echo -e " SMB: Starting to run SMB mounts.."
for I in $MOUNTLIST; do
if [ ! -d $SOURCEMOUNT/$I ]; then
echo -e " ** ERROR: destination mount [ $SOURCEMOUNT/$I ] point does not exist"
exit 1
fi
echo " [ $I ] mount point already exists. Continuing.."
if [ -z "`df | grep $I`" ]; then
echo " Mounting: [ $I ]"
echo " Mounting [ $SOURCEMOUNT/$I ] - Please provide required passwords"
/usr/bin/smbmount //$CLIENT/$I $SOURCEMOUNT/$I -o $SMBOPTIONS
else
echo " Samba mount [ $I ] already mounted. Continuing.."
fi
done
if [ ! -n "`df | grep $CLIENT`" ]; then
echo -e " ** ERROR - SAMBA: Cound not mount [ $CLIENT ]"
exit 1
fi
echo -e " SAMBA: [ $CLIENT ] successfully mounted."
fi
# Must run this AFTER the network is up to get CLIENT info
#
#Is the backup media really present
# This looks for a file called "backup-drive-ready" on the backup DESTINATION
#
if [ ! -f $BACKUPDEST/backup-drive-ready ]; then
echo -e "\n ** ERROR ** Backup media isn't present. Make sure the dest \
backup drive"
echo -e " is installed and mounted.\n"
echo -e " If the media IS mounted properly, make sure the file"
echo -e " $BACKUPDESK/backup-drive-ready exists. Until then..\n\n"
echo -e " Aborting.\n\n"
exit 1
fi
echo -e "\n Backup destination media is present"
#Does the backup destination have enough space?
#How big is the REMOTE backup
if [ "$BACKUPMETHOD" == "NFS" ] || [ "$BACKUPMETHOD" == "SAMBA" ]; then
TOTAL=0
# The issue must be the use of the ":"
# #coyote wants awk var3 and not var2
# #roadrunner needs awk var2
#coyote
#coyote:/ 18951536 11212792 6776048 62% /mnt/nfs
#roadrunner
#
#acme
#//acme/acme-c
# 2096832 1974688 122144 94% /mnt/samba/acme-c
if [ "$BACKUPMETHOD" == "SAMBA" ]; then
#Samba's use of screws up awk
echo " Calcing Samba size"
for I in `df -P | grep "$SOURCEMOUNT" | awk '{print $3}'`; do
TOTAL=$(($TOTAL + $I))
done
else
echo -e "\n Calcing NFS size"
# 122502 - moving from $3 to $2 though I dont know why
# -- maybe something in the src nfs hostname
# awk-3 is good for coyote
# awk-2 is good for roadrunner
# must change this to do it via the mount point and not the sourcemount
# parse for /mnt/nfs/dos-c and not 192.168.0.7:/dos-c
# because the lines wrap on long lines. also use df -Tk to
# help parsing
#
# I need to either parse from the RIGHT to the left or use
# some other feature of awk
for I in `df -Pk | grep "$SOURCEMOUNT" | awk '{print $3}'`; do
TOTAL=$(($TOTAL + $I))
done
fi
echo -e "\n ESTIMATED Backup size : $TOTAL"
BACKUPDESTDU=`df -Pk | grep $BACKUPDEST | awk '{print $4}'`
echo " Backup DESTINATION capacity: $BACKUPDESTDU"
fi
#How big is the LOCAL backup
if [ "$BACKUPMETHOD" == "LOCAL" ]; then
TOTAL=0
for I in $LOCALMOUNT; do
#acme
#/dev/sdb3 7302300 2240072 4691284 32% /
J=`df -P | grep "$I" | awk '{print $3}'`
echo " - Checking mount: $I - SIZE: $J"
TOTAL=$(($TOTAL + $J))
done
echo -e "\n ESTIMATED Backup size : $TOTAL"
BACKUPDESTDU=`df -P | grep $BACKUPDEST | awk '{print $4}'`
echo " Backup DESTINATION capacity: $BACKUPDESTDU"
fi
if [ $TOTAL -ge $BACKUPDESTDU ]; then
echo -e "\n ** ERROR ** NOT ENOUGH DISK SPACE on backup device. Aborting.\n\n"
exit 1
fi
echo -e "\n [ $BACKUPDEST ] has enough diskspace to backup host [ $CLIENT ]"
if [ "$2" = "-check" ]; then
echo -e "\n********************************************************"
echo -e "** ABORT: **"
echo -e "** **"
echo -e "** -check command line option specified. Exiting. **"
echo -e "********************************************************\n\n"
exit 0
fi
echo -e "\n Backup Destination is: [ $DEST ] "
mkdir $DEST > /dev/null
if [ ! -d $DEST ]; then
echo " ** ERROR: Could not create destination directory"
exit 1
fi
echo " Created the destination directory.."
#Get the backup size - dont use -c but use -s instead since you will
# match on multiple "total" lines
#
echo -e "\n---------------------------------------------------------------------" \
> $DEST/$CLIENT-backup.log
echo -e "Auto-generated by the TrinityOS backup script $VERSION" >> $DEST/$CLIENT-backup.log
echo -e "\nThis is a FULL backup of host: $CLIENT" >> $DEST/$CLIENT-backup.log
echo -e "\nRun from machine: `uname -a`" >> $DEST/$CLIENT-backup.log
echo -e "\nBackup START: $START" >> $DEST/$CLIENT-backup.log
echo " ESTIMATED backup size: $TOTAL"
echo -e "\nESTIMATED backup size: $TOTAL" >> $DEST/$CLIENT-backup.log
# This section is not required as the $TOTAL calculation above is accurate enough
#
# THIS SECTION WILL BE REMOVED SHORTLY
#
#if [ "$BACKUPMETHOD" == "LOCAL" ]; then
# #Calc space for local volumes since du does't do what we expect
# CALCEDSIZE=0
# echo " Calculating actual backup space requirements. Please wait."
# for I in $BACKUPPATH; do
# J=`du -s -x $I | awk '{print $1}'`
# #echo "$I"
# CALCEDSIZE=$(($CALCEDSIZE + $J))
# done
# echo " Initial backup size: $CALCEDSIZE"
# echo -e "\nINITIAL backup size: $CALCEDSIZE" >> $DEST/$CLIENT-backup.log
# else
# #Calc space for NFS and SMB
# echo " Calculating actual backup space requirements. Please wait.."
# CALCEDSIZE="`du -s --exclude /mnt/mnt $SOURCEMOUNT | awk '{print $1}'`"
# echo -e "\n Calculated backup size: $CALCEDSIZE"
# echo -e "\nCalculated backup size: $CALCEDSIZE" >> $DEST/$CLIENT-backup.log
#fi
if [ "$BACKUPMETHOD" == "NFS" ]; then
#Create placeholder dirs
mkdir -p $DEST/mnt/floppy > /dev/null
mkdir -p $DEST/mnt/cdrom > /dev/null
mkdir -p $DEST/lost+found > /dev/null
mkdir -p $DEST/proc > /dev/null
fi
#Put of a copy of the backup script on the backup drive
cp /root/backup-to-disk $DEST/backup-to-disk
echo -e "\n\nSpawning logging window..\n"
/usr/X11R6/bin/xterm -fg white -bg darkblue -title "$CLIENT backup-to-disk=log-window" \
-e tail -f $DEST/$CLIENT-backup.log &
echo -e "\nBacking up data on host $CLIENT with permissions, ownerships, etc"
echo -e "=============================================================================="
echo -e "\n\n-------------------------------------------------------------------------------"
echo -e "Full backup logs can be monitored by running:\n"
echo -e " tail -f $DEST/$CLIENT-backup.log"
echo -e "\n-------------------------------------------------------------------------------\n\n"
echo -e "\n-------------------------------------------------------------------------------" >> $DEST/$CLIENT-backup.log
echo -e "Full backup logs can be monitored by running:\n" >> $DEST/$CLIENT-backup.log
echo -e " tail -f /mnt/$BACKUPDEST/$CLIENT-backup.log" >> $DEST/$CLIENT-backup.log
echo -e "\n-------------------------------------------------------------------------------" >> $DEST/$CLIENT-backup.log
for I in $BACKUPPATH; do
echo -e "\n---------------------------------------------------"
echo -e "Messages below are due to ERRORS encountered during"
echo -e "the backup:"
echo -e "---------------------------------------------------"
echo -e "\n------------------------------------------------------" >> $DEST/$CLIENT-backup.log
echo -e "Messages below are due to ERRORS encountered during" >> $DEST/$CLIENT-backup.log
echo -e "the backup:" >> $DEST/$CLIENT-backup.log
echo -e "------------------------------------------------------" >> $DEST/$CLIENT-backup.log
echo -e "Backing up : [ $I ]\n"
echo -e "Backing up : [ $I ]\n" >> $DEST/$CLIENT-backup.log
#do this manually to not create bakups with /mnt/mnt/backup/mnt/nfs/bin
cd $SOURCEMOUNT/$I
mkdir $DEST/$I > /dev/null
if [ ! -d $DEST/$I ]; then
echo " ** ERROR: Could not create destination directory"
exit 1
fi
# *** HEAVY LIFTING ***
#
#tar cpsf - $SOURCEMOUNT/$I | (cd $DEST; tar xvpvf - )
#Be sure to NOT to backup anything other than the local filesystem
tar clpsf - . | (cd $DEST/$I; tar xpvf - ) 2>> $DEST/$CLIENT-backup-errs.log >> $LOGGING
echo -e "DONE backing up: $I"
echo -e "DONE backing up: $I" >> $DEST/$CLIENT-backup.log
echo -e "------------------------------------------------------"
echo -e "------------------------------------------------------" >> $DEST/$CLIENT-backup.log
done
echo -e "\n\n=============================================================================="
echo -e "\n\n==============================================================================" \
>> $DEST/$CLIENT-backup.log
echo -e "Backup COMPLETED.\n\n"
echo -e "Backup COMPLETED.\n\n" >> $DEST/$CLIENT-backup.log
#Get the final backup size - dont use -c but use -s instead since you will
# match on multiple "total" lines
#
echo "Calculating FINAL backup size.. [ please wait.. ]"
echo "Calculating FINAL backup size.. [ please wait.. ]" >> $DEST/$CLIENT-backup.log
CLOSING=`du -s $DEST | awk '{print $1}'`
echo -e " ESTIMATED backup size: $TOTAL"
echo -e " ESTIMATED backup size: $TOTAL" >> $DEST/$CLIENT-backup.log
echo -e " FINAL backup size : $CLOSING"
echo -e " FINAL backup size : $CLOSING" >> $DEST/$CLIENT-backup.log
#get out of any existing NFS/SAMBA partions
cd /root
if [ "$BACKUPMETHOD" == "NFS" ] || [ "$BACKUPMETHOD" == "SAMBA" ]; then
echo -e "\nUnmounting [ $CLIENT ] "
for I in $UNMOUNTLIST; do
echo " UNMounting: [ $SOURCEMOUNT/$I ] "
umount $SOURCEMOUNT/$I
done
fi
if [ "$BACKUPMETHOD" == "NFS" ]; then
echo -e "\nUnloading PORTMAP"
/etc/rc.d/init.d/portmap stop
if [ -n "`ps ax | grep "portmap" | grep -v "grep portmap"`" ]; then
echo -n "\nCould NOT stop PORTMAP. Aborting."
exit 1
fi
fi
#If we were using compression and seti is on this machine, restart it
if [ "$COMPRESSION" == "yes" ]; then
if [ -f /usr/local/sbin/start-seti ]; then
echo -e " ** Starting SETI.."
/usr/local/sbin/start-seti
SETIPID=`ps ax | grep "seti" | grep -v "grep" | awk '{print $1}'`
if [ -z "$SETIPID" ]; then
echo -e " ** ERROR: Could not start SETI"
exit 1
fi
mv /etc/cron.hourly.disabled/start-seti /etc/cron.hourly
fi
fi
#WILL BE REMOVED
#tail --lines 16 $DEST/$CLIENT-backup.log
echo -e "\nBackup STARTed: $START"
echo -e "\nBackup STARTed: $START" >> $DEST/$CLIENT-backup.log
echo -e "Backup STOPped: `date`\n\n"
echo -e "Backup STOPped: `date`\n\n" >> $DEST/$CLIENT-backup.log
if [ "$LOGGING" != "/dev/null" ]; then
echo -e "Compressing all log files"
gzip -9 $DEST/$CLIENT-backup.log
gzip -9 $DEST/$CLIENT-backup-errs.log
else
echo -e "Logging NOT enabled. Log Compression stopped."
fi
echo -e "\nEnd of TrinityOS HD backup script $VERSION"
echo -e "==============================================================================\n\n"
To get the script, download it from the TrinityOS-archives.tar.gz file on Dranch's web site. PLEASE, don't try to cut and paste this into a new file:
http://www.ecst.csuchico.edu/~dranch/LINUX/TrinityOS-security/TrinityOS-security.tar.gz
Once you have the script, put it in the ROOT user's directory. Why root? Well, you'll need to be root to mount the remote or local backup HD. You'll need to be root to backup all the local file systems. Etc.
To make it executable, run:
chmod 700 /root/backup-to-disk
To run it, simply type something like:
/root/backup-to-disk coyote
+-----------------------------------------------------------------------------+
| //// Prerequisites: \\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\\|
+-----------------------------------------------------------------------------+
| |
| + Bru (tape software is installed). Check by using this command: |
| |
| whereis bru |
| |
| |
| + Compiled a kernel to either support (at MINIMUM). Please see the |
| Kernel Compiling Section for more details on how to do the following: |
| |
| * IDE tape drives |
| |
| Enhanced IDE/MFM/RLL disk/cdrom/tape/floppy support (CONFIG_BLK_DEV_IDE) |
| Include IDE/ATAPI TAPE support (CONFIG_BLK_DEV_IDETAPE) |
| |
| or |
| |
| * your specific SCSI controller with SCSI tape support |
| |
| SCSI support (CONFIG_SCSI) |
| SCSI tape support (CONFIG_CHR_DEV_ST) |
| Verbose SCSI error reporting (kernel size +=12K) (CONFIG_SCSI_CONSTANTS)|
| |
| .....and for example, the Adaptec 1522 SCSI controller: |
| Adaptec AHA152X/2825 support (CONFIG_SCSI_AHA152X) |
| |
| |
| + A properly installed IDE (master/slave) or a SCSI tape drive |
| (with proper SCSI IDs and termination) |
| |
| |
| + Files created/edited: |
| |
| /usr/local/sbin/bru-fullbackup |
| /etc/brutab |
| /etc/bruxpa |
| |
+-----------------------------------------------------------------------------+
(Bru isn't free if you don't install Redhat or Caldera but it's the best Linux backup software out there. This is one place you just CAN'T skimp!) If you don't want to use Bru, at least use CPIO instead of TAR. Tar does work fine UNTIL you hit an error on the tape. After that, tar will shutdown and you'll be screwed since it can't do data recovery. CPIO on the other hand can at least skip the bad file.
NOTE: I've noticed that the behavior of BRU between v14.3 and 15.0 (Bru2000) is quite different. Still works though!)
+-----------------------------------------------------+
| All the BRU documentation is available at: |
| |
| http://www.estinc.com/brumanual/toc.html |
+-----------------------------------------------------+
**NOTE**: This is ONLY for users running anything LESS than Glibc-2.0.7-19:
- To check , run "rpm -q glibc"
- Edit /etc/profile and add your appropriate time zone above the "export" command (this is for the Pacific time zone):
TZ=PDT
Next, find the line that starts with "export" and add "TZ" to the end of it. Here is my "export" line:
export PATH PS1 HOSTNAME HISTSIZE HISTFILESIZE USER LOGNAME MAIL NNTPSERVER TZ
Next, you need to setup BRU to understand your tape drive. Personally, I would recommend to use ESTINC's setups at:
http://www.estinc.com/brutabs.html
Or, startup Xwindows and run "bruconfig" and configure it this way.
--< /etc/brutab START>--
# BRUTAB Globals
#+MAXWRITES=1000
#+RAWZBUFSIZE=500
#+RECYCLEDAYS=0
#+OVERWRITEPROTECT=YES
#+ZBUFSIZE=5M
#
# Changed Zbufsize from 500k to 2M
# Changes size from 4000MT to 8000MT
# Changed bufsize from 32k to 64k
#### NOTE!!! BRU tracks the size of uncompressed files by design.
####
#### So, when using either software or hardware compression, simply set
#### the tape drive capacity size to ZERO in /etc/brutab (size=0).
# Devices
/dev/st0 devname="NS-8 Drive, 8GB, rewind" \
size=0MT bufsize=16k \
shmseg=10 shmmax=200k \
rawtape tape shmcopy rewind autoscan \
fmtcmd="mt -f /dev/st0 erase" \
rfmcmd="mt -f /dev/st0 fsf" \
bfmcmd="mt -f /dev/st0 bsf" \
retencmd="mt -f /dev/st0 reten" \
rewindcmd="mt -f /dev/st0 rewind" \
eodcmd="mt -f /dev/st0 seod" \
/dev/nst0 devname="NS-8 Drive, 4GB, norewind" \
size=0MT bufsize=16k \
shmseg=10 shmmax=200k \
rawtape tape shmcopy norewind noautoscan # # # # # # \
fmtcmd="mt -f /dev/st0 erase" \
rfmcmd="mt -f /dev/nst0 fsf 1" \
bfmcmd="mt -f /dev/nst0 bsf 1" \
retencmd="mt -f /dev/st0 retension" \
rewindcmd="mt -f /dev/st0 rewind" \
eodcmd="mt -f /dev/nst0 eod" \
# /dev/null device, useful for testing
/dev/null devname="Bit Bucket" \
size=0 bufsize=20k \
norewind noautoscan
- devname="stdin/stdout" \
size=0 bufsize=20k \
norewind noautoscan
--< /etc/brutab END>--
Now we need to setup an exclude file so you don't backup things like CD-ROM drives or compress ZIP files, etc. First, backup the original file by doing "mv /etc/bruxpat /etc/bruxpat.orig" and then create this file and edit it to fit your needs:
--< /etc/bruxpat Start>--
# Updated 03/09/99 to change the tape drive capacity to "0" for compression reasons
# Updated 11/25/98 to add no compression of RAR files --dranch
# Updated 7/23/98 to add Cdrom2-8 exclusion --dranch
# Updated 6/14/98 to add [aA] for the ARJ multivolume stuff --dranch
#
# This file is used by -X option to provide an inclusion/exclusion
# list. For each pathname of a file selected for backup, each line
# of this file is examined for a pattern, and that pattern is applied
# to the pathname. If the pattern matches, the appropriate action
# is taken (the pathname is accepted or rejected). If the pathname
# makes it through all the patterns it is accepted.
#
# These patterns will ONLY be applied to filenames that are part
# of directories that are specified on the bru command line (or
# the current directory, if none are specified).
#
#
# Each command line in the bruxpat file (the file you are now reading)
# consists of a control field and a pattern. The pattern
# is separated from the control field by whitespace. Control field
# characters are:
#
# i Include this pathname if pattern matches. The
# pathname is accepted and no further patterns are
# applied.
# *** NOTE ****
# It stops trying on the first pattern match found
# and passes the filename. Since it scans patterns
# in the order listed, "include" patterns normally
# should be listed before any "exclude" patterns.
#
# x Exclude this pathname if pattern matches. The
# pathname is rejected and no further patterns are
# applied.
#
# z Exclude this pathname from compression if pattern
# matches (if the -Z option is specified).
#
# s The pattern is a shell style wildcard pattern except
# that '/' characters are not treated as special characters.
#
# r The pattern is a regular expression (same as used by the "grep"
# command).
#
# l The pattern is a literal string.
#
# Exclude all core files
xs */core
xs core
# Don't try to get the stuff in /proc
xs /proc/*
xs ./proc/*
# Don't backup the CD-Rom
xs /home/hpe/CDROMs/Cdrom0/*
xs ./home/hpe/CDROMs/Cdrom0/*
xs /home/hpe/CDROMs/Cdrom1/*
xs ./home/hpe/CDROMs/Cdrom1/*
xs /home/hpe/CDROMs/Cdrom2/*
xs ./home/hpe/CDROMs/Cdrom2/*
xs /home/hpe/CDROMs/Cdrom2/*
xs ./home/hpe/CDROMs/Cdrom2/*
xs /home/hpe/CDROMs/Cdrom3/*
xs ./home/hpe/CDROMs/Cdrom3/*
xs /home/hpe/CDROMs/Cdrom4/*
xs ./home/hpe/CDROMs/Cdrom4/*
xs /home/hpe/CDROMs/Cdrom5/*
xs ./home/hpe/CDROMs/Cdrom5/*
xs /home/hpe/CDROMs/Cdrom6/*
xs ./home/hpe/CDROMs/Cdrom6/*
xs /home/hpe/CDROMs/Cdrom7/*
xs ./home/hpe/CDROMs/Cdrom7/*
# Exclude all files and subdirectories in the temporary directories.
# Handle files specified with relative and absolute pathnames
#
# -- NOTE -- the actual directory names will still be backed up,
# only the files within the directories will be
# excluded.
#xs ./usr/tmp/*
#xs /usr/tmp/*
#xs ./tmp/*
#xs /tmp/*
# Don't compress files that end in ".z" or ".Z"
zs *.[Zz]
zs *.zip
zs *.ZIP
zs *.arj
zs *.ARJ
zs *.[Aa][0-9][0-9]
zs *.[Rr][Aa][Rr]
zs *.[Ra][0-9][0-9]
zs *.[0-99]
zs *.gz
zs *.GZ
zs *.gzip
zs *.GZIP
zs *.bz2
zs *.BZ2
zs *.tgz
zs *.TGZ
zs *.tar.gz
zs *.tar.bz2
zs *.rpm
zs *.RPM
zs *.iso
zs *.ISO
zs *.mp3
zs *.MP3
zs *.asf
zs *.ASF
zs *.[Gg][Ii][Ff]
zs *.[Jj][Pp][Gg]
zs *.[Mm][Pp][Gg]
--
Create the file /usr/local/sbin/bru-fullbackup with the following in it. NOTE: You might want to change the label field to your tape drive and proper date
--< /usr/local/sbin/bru-fullbackup >--
#!/bin/sh
clear
# Edited 08/25/98
#HP TR4 SCSI Internal, 2.0.36, 486/160Mz/40MB, 4)IDE 3)RAID0, AHA1542 SCSI
#------------------------------------------------------------------------
#02/09/99: wrote (3904000 KBytes), 3:28:00, 330 Kb/sec (effective)
#02/09/99: autoscan (3904000 kbytes), 2:16:54, 475 Kb/sec
echo "Setting environment vars"
export BUFSIZE=16k
export BRUTMPDIR=/tmp
export BRUMAXWARNINGS=20000
#Only needed for old Glibc users
#export TZ=PDT
echo "Compressing old log files. This might take a while.."
mv /var/log/bruexeclog /var/log/bruexeclog.`date +'%b%d'`
mv /var/log/bru-log /var/log/bru-log.`date +'%b%d'`
bzip2 -9f /var/log/bru-log.`date +'%b%d'`
echo "Starting BRU full backup with exclusions, compression, user intervention"
# Do not use -j, -m,
bru -c -vvvv -V -X -Z -G -L "Hp Tr4 11/27/98 - FULL" -f /dev/st0 / > /var/log/bru-log
#Only needed for old Glibc users
#export TZ=PST8PDT
# v8.8.98
# See /etc/bruhelp for A LOT of more details
#
# Defaults to backing up "/"
#
# -c : create (autoscan verification on by default)
# : - if you specify -i or -d, autoverify is disabled
#
# -d : file comparison (normal)
# -dd : file comparison access mod, lengths, symlinks, ID groups
# -dddd : file comparison - hard core
#
# -e : Estimate archive size
#
# -f : select regular input device (same as -r)
#
# -g : Read : Dumps the header block
# -gg : Read : Generates ted cmd line, label, date, time, release,
#
# -h : Print this help information
#
# -i : inspect a archive *checksum of a directory)
# : Not needed with "-v"
#
# -r : Backup a raw partition
#
# -t : List archive table of contents for files
#
# -u - use selected files
# a - all files
# b - block special files
# c - character (special files)
# d - dirs
# l - syms
# p - fifos
# r - reg
#
# -vvvv : Level 4 verbosity
#
# -w : confirmation of each file
#
# : wildcard expantion [must be placed in double quotes]
# -x : restore
#
# -G : Write a archive list (header block) at beginning of
# -L : Label the tape
# -B : disabled user intervention
# -D : Enabled double buffering for faster throughput
# -Z : compression
# -V : execution summary w/o volume
# -X : Exclude specific files
#
# bru -gg -f /dev/st0 : Display archive contents if written
#
#bru -vv -t -f /dev/st0 : Display entire contents of archive tape
#
#bru -x -vvvv /user/dranch/*
#
# Also, these environment variables are available in /etc/brutab
#
# Global BRU settings
#
#+OVERWRITEPROTECT=YES
#+RECYCLEDAYS=180
#+MAXWRITES=200
#+ZBUFSIZE=512k
#+SHELL=/bin/sh
#+BRUTABONLY=no
#+DEVNAMECHECK=no
#+MATCHLEVEL=2
#+MAXFILENAMELEN=255
#+READCHECKLEVEL=1
#+BRUHELP=/bru/bruhelp
#+BRUMAXWARNINGS=1000
#+BRUMAXERRORS=500
#+BRUXPAT=/etc/bruxpat
#+BRURAW=/etc/bruraw
#+BRUSMARTREST=/etc/brusmartrest
#+BRUREMOVELOG=/var/adm/bruremovelog
#+BRUTMPDIR=/tmp
--< /usr/local/sbin/bru-fullbackup End.>
- Ok, go ahead and insert a tape in the tape drive and run
"/usr/local/sbin/bru-fullbackup"
I usually also run "tail -f /var/log/bru-log" in another TTY to watch the progress of the backup.
- Once your backup is completed, you need to verify that you can read the files OFF the tape, restore files to different places, and also restore files back to their ORIGINAL location:
-- Based on an email from the BRU mailing list:
The techniques differ depending on how the backup was created (absolute [/] or relative [.]). If you used "I" use "/" as a backup point, we are using absolute paths so (assuming you have a tape with full backups as well):
- Restore the /etc/passwd file to a different location (/tmp):
cd /tmp
bru -xvf /dev/st0 -PA /etc/passwd
* the trick is "-PA" which translates absolute to relative
Now test that the files are the same:
diff /etc/passwd /tmp/passwd
- Restore the /bin/fullbru file to the same location (/bin):
mv /bin/fullbru /bin/fullbru.save
bru -xvf /dev/st0 /bin/fullbru
- Now test that the files are the same:
diff /bin/fullbru.save /bin/fullbru
- Once you are convinced that you have a good backup, now its time to create a rescue diskette.
- Download the BRU rescue diskette from:
ftp://ftp.estinc.com/pub/linux/Bootkit-1.01.tar.gz
- Here are a few other scripts that I find useful with Bru:
--< /usr/local/sbin/bru-viewtape >--
#!/bin/sh
clear
#echo "Starting BRU to view tape contents"
bru -gg -f /dev/st0 > /var/log/bru-tape-contents.`date +'%b%d'` 2>&1
--<end.>--
--< /usr/local/sbin/bru-find-changes >--
#!/bin/sh
clear
# Edited 01/06/99
echo "Setting environment vars"
export BUFSIZE=16k
export BRUTMPDIR=/tmp
export BRUMAXWARNINGS=20000
#export TZ=PDT
echo "Starting BRU to find all changed/missing files between tape and disk.."
bru -dd -f /dev/st0 / > /var/log/bru-diff-del-find-log.`date +'%b%d'` 2>&1
--<end.>--
--< /usr/local/sbin/bru-restore >--
#!/bin/sh
clear
# Edited 03/09/99
#
# NOTE: This script is run as: "/usr/local/sbin/bru-restore /home/username"
# where the "/home/username" is the path and/or the full path and filename
# of the data you want to restore. Bru will then find this data on the
# tape and restore it to its original location. If you want to restore
# the file to a DIFFERENT location, please consult the manual for
# "absolute to relative path translation"
#
echo "Setting environment vars"
export BUFSIZE=16k
export BRUTMPDIR=/tmp
export BRUMAXWARNINGS=20000
#export TZ=PDT
echo "Compressing old log files. This might take a while.."
mv /var/log/bru-restore-log /var/log/bru-restore-log.`date +'%b%d'`
mv /var/log/bruexeclog /var/log/bruexeclog.`date +'%b%d'`
bzip2 -9f /var/log/bru-restore-log.`date +'%b%d'`
echo "Starting BRU partial restore "
# Do not use -j, -m,
bru -x -vvvv -f /dev/st0 $1 > /var/log/bru-restore-log
--<end.>--
See Section 39 for full details.