At work I have been given the task of coming up with a procedure of bare metal restore of redhat servers. There were two requirments.
1) Hot backing up
2) Remote restore
After some researching this is the procedure I came up with. Please bear in mind this procedure has only been tested in model enviroment and do your own homework before using this as your DR plan.
A) Backup
Backup the mbr and the bootloader. First 512 bytes of the primary HD contains the partion table and the bootloader. Here is the command for that.
dd if=/dev/sda of=/backup/mbr.img bs=512 count=1
Ensure that you have the backup directory and correct HD.
Now we need to backup the filesystem. Linux is dynamically creating the following directories. So no need to take the backup of them.
1) /sys
2) /proc
3) /var/run
4) /lost+found
5) /tmp
You need to exclude the backup directories. You can use tar or rsync to take the backup, however if you use rsync ensure you preserve symlinks and file permissions. Here is the command to create the tar archive.
tar cvfpz /backup/root.tar.gz –exclude=sys –exclude=proc –exclude=var/run –exclude=tmp –exclude=lost+found –exclude=backup /
Take a backup of the /etc/fstab and partion table info from the following command.
fdisk -l > /backup/fdisk.txt
B) Restore
Okay before restore lets crash the HD. Please beware this command shall ruin your HD, do not run this in the production enviroment.
dd if=/dev/random of=/dev/sda bs=512 count=100
Okay let’s start restoring.
You need a lInux live cd. Well I prefer a knoppix since it has all the useful tools.
Boot the server with the knoppix cd and restore the partion table. Since I am soundf this remotely I will be using iLo virtual media to mount the knoppix live cd.
dd if=/backup/mbr.img of=/dev/sda bs=512 count=1
Linux will not identify the HD until you reboot, however you can do the following as well. Run fdisk and write the partion table by pressing ‘w’.
Okay now let’s format the partions. This is where the copy of fstab helps. You can run the following command for each partion.
mkfs.ext3 /dev/sda2 -L /
mkfs.ext3 /dev/sda1 -L /boot
Now your partions are ready for the data. Let’s mount the devices. First create the mount point by running the following command. I am killing two birds with one command. I am creating rootfs and boot directories.
mkdir -p /rootfs/boot
Okay change directories, into cd /rootfs
Let’s extract the archive
tar xvzp /backup/root.tar.gz
Okay now we need to create the following directories.
mkdir /sys; mkdir /tmp; mkdir /lost+found; mkdir /proc; mkdir /var/run
Okay now you need to chroot into the /rootfs
mount -t proc /proc /rootfs/proc
mount -o loop /dev /rootfs/dev
chroot /rootfs -c /bin/bash
Now you have mounted the filesystem. Run the following command to reinstall grub
grub-install /dev/sda
Okay fingers cross you have recovered your crashed server. Reboot the box you have resurrected your Linux box.
Things you need to know. This won’t work if you have a LVM. If your fstab has uid rather than block devices, you need to change it to block devices.
If you have grub2 grub reinstallation might change a bit.
I will cover that in another post.
Okay something you might notice that I didn’t cover the backup directory. You have several options here. You can use a nfs or samba mount for that. Please keep in mind if you are using rsync as your backup method you cannot have the symlinks in a samba mount. So you need to use tar if you are using the samba option.
If you have a better way or see a mistake please comment.
Writings on the Train 