Booting RHEL from LVM snapshots
By: Date: 05/08/2020 Categories: azure

Red Hat Enterprise Linux (RHEL) version 7.5 introduced the “boom” utility for managing LVM snapshot and image boot entries.

This new functionality could be very helpful for system administrators, especially those responsible for RHEL servers running directly on physical hardware, where booting from snapshots has previously been more difficult.  

This post will cover the basics of how to use boom, including an example where a snapshot is created, boom is configured to add a boot entry for the snapshot, and the system is booted from the snapshot.  

Demo Environment

In this demo environment, we have a RHEL 7.5 host that is using LVM (except for the /boot filesystem, which is not using LVM). The volume group is named rhel_rhel7, and contains a logical volume (LV) for the root filesystem named root, and a logical volume for swap space named swap:

# vgs
 VG      #PV #LV #SN Attr   VSize VFree
 rhel_rhel7   2 2 0 wz--n- 48.99g <30.00g
#
# lvs
 LV   VG   Attr    LSize   Pool Origin Data%  Meta% Move Log Cpy%Sync Convert
 root rhel_rhel7 -wi-ao---- <17.00g                                               
 swap rhel_rhel7 -wi-ao----   2.00g

The rhel_rhel7 volume group has enough available free space to create a snapshot logical volume for the root logical volume. Remember to properly size your snapshots because if the snapshot fills up, it becomes invalid.  

Getting Started with Boom

The first step is to install the lvm2-python-boom package, which provides the boom utility. This can be installed by running yum install lvm2-python-boom.

Next, we will create a snapshot of the root logical volume in the rhel_rhel7 volume group by running:

# lvcreate -s rhel_rhel7/root -n root_snapshot_071018 -L 17G
 Logical volume "root_snapshot_071018" created.

The “-s” specifies the volume group/logical volume name of the logical volume we want to create a snapshot of. The “-n” specifies a name for the snapshot logical volume (in our case the name will be “root_snapshot_071018”), and the “-L” specifies the size of the snapshot logical volume, which in our example is 17 GB.  

Then, we will use boom to create a boot entry for this snapshot by running:

# boom create --title "Root Snapshot - 07/10/18" --rootlv rhel_rhel7/root_snapshot_071018
WARNING - Boom configuration not found in grub.cfg
WARNING - Run 'grub2-mkconfig > /boot/grub2/grub.cfg' to enable
Created entry with boot_id d2c8369:
 title Root Snapshot - 07/10/18
 machine-id 1181b7e44d6845e38a9dc3257af5d56b
 version 3.10.0-862.6.3.el7.x86_64
 linux /vmlinuz-3.10.0-862.6.3.el7.x86_64
 initrd /initramfs-3.10.0-862.6.3.el7.x86_64.img
 options root=/dev/rhel_rhel7/root_snapshot_071018 ro rd.lvm.lv=rhel_rhel7/root_snapshot_071018 rhgb quiet

The “--title” option on the boom command specifies what title should be shown from the grub2 menu, and the “--rootlv” specifies the volume group/snapshot logical volume name, which in our example is rhel_rhel7/root_snapshot_071018.

Notice that when we ran the boom create command, it came back with a warning that the boom configuration was not found in grub.cfg, and recommended the solution to this.   We will go ahead and run the command it specified in the warning:

# grub2-mkconfig > /boot/grub2/grub.cfg
Generating grub configuration file ...
Found linux image: /boot/vmlinuz-3.10.0-862.6.3.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-862.6.3.el7.x86_64.img
Found linux image: /boot/vmlinuz-3.10.0-862.el7.x86_64
Found initrd image: /boot/initramfs-3.10.0-862.el7.x86_64.img
Found linux image: /boot/vmlinuz-0-rescue-1181b7e44d6845e38a9dc3257af5d56b
Found initrd image: /boot/initramfs-0-rescue-1181b7e44d6845e38a9dc3257af5d56b.img
done

At this point, we have created a snapshot of the root logical volume, and configured boom to make the snapshot bootable via the grub2 menu.

We can see what snapshots boom is configured to use by running boom list:

# boom list
BootID  Version               Name                         RootDevice                     
d2c8369 3.10.0-862.6.3.el7.x86_64 Red Hat Enterprise Linux Server /dev/rhel_rhel7/root_snapshot_071018

Next, we will make some changes to the system, then reboot from the snapshot. When we boot from the snapshot, any changes made to the root logical volume after the snapshot was taken should not be visible. 

We will install some additional packages, create a user, and update the /etc/motd file:

# yum install tmux screen
# useradd newuser
# echo “Updated motd file, after snapshot was taken” > /etc/motd

At this point, we will reboot the server and see what it looks like from the grub2 menu:

Fig. 1: Boom LVM Snapshot Boot

We now have a “Snapshots” menu option within the grub2 menu. If we press enter on Snapshots, we will see the snapshot we configured boom with:

Fig. 2: Boom Guide

Once we press enter here, the system will boot from this snapshot.  

Once logged in to the system, we can see that the changes we made in the root logical volume after the snapshot creation are not present:

The tmux and screen packages are not installed:

# yum list installed | egrep "screen|tmux"
#

The “newuser” account doesn’t exist:

# id newuser
id: newuser: no such user

And the /etc/motd file doesn’t have the text we added to it:

# cat /etc/motd
#

And if we look at /proc/cmdline, we can see that the system was booted from the /dev/rhel_rhel7/root_snapshot_071018 snapshot:

# cat /proc/cmdline

BOOT_IMAGE=(hd0,msdos1)/vmlinuz-3.10.0-862.6.3.el7.x86_64 root=/dev/rhel_rhel7/root_snapshot_071018 ro rd.lvm.lv=rhel_rhel7/root_snapshot_071018 rhgb quiet

Next, we will reboot, and let the system boot normally (not from the snapshot) and clean up the snapshot.

Once the server is rebooted, we can see that all previously made changes in the root logical volume are present again:

# yum list installed | egrep "screen|tmux"
screen.x86_64                     4.1.0-0.25.20120314git3c2946.el7
tmux.x86_64                       1.8-4.el7            @rhel-7-server-rpms
# id newuser
uid=1000(newuser) gid=1000(newuser) groups=1000(newuser)
# cat /etc/motd
Updated motd file, after snapshot was taken

To remove the snapshot from boom, first run boom list to get the BootID for the snapshot:

# boom list
BootID  Version               Name                         RootDevice                     
d2c8369 3.10.0-862.6.3.el7.x86_64 Red Hat Enterprise Linux Server /dev/rhel_rhel7/root_snapshot_071018

In this example, the BootID is d2c8369 as shown in the above output.  To delete the snapshot from boom, run the command below, replacing “d2c8369” with the BootID shown in the previous command:

# boom entry delete d2c8369
Deleted 1 entry

Then run “boom list” to verify the entry was removed:

# boom list
BootID  Version              Name                  RootDevice

Next, delete the snapshot logical volume with the lvremove command:

# lvremove rhel_rhel7/root_snapshot_071018
Do you really want to remove active logical volume rhel_rhel7/root_snapshot_071018? [y/n]: y
 Logical volume "root_snapshot_071018" successfully removed

Merging the snapshot back into the Original Logical Volume

A common workflow is to make a snapshot before a change, then make the change. If the change was successful, usually the snapshot is deleted after some time (once we are confident that the change is working as expected). This is what was done in the previous example.  

The other scenario is the change was not successful, and we need to merge the snapshot in to the original logical volume in order to permanently rollback the unsuccessful changes that were made after the snapshot was created. When this is done, the snapshot is removed, but first, the snapshot is merged in to the original logical volume, and the end result is a rollback of the logical volume to its state at the time the snapshot was created.  

In this next example, we will assume all the steps in the previous example have been completed again, up to the point where we have used Boom to boot from the snapshot. At this point however, we have determined that the changes made after the snapshot was taken (installing tmux/screen, creating the user, and updating the motd file) caused an issue on the system, and we want to merge the snapshot in to the original root logical volume in order to permanently rollback the changes that were made.  

To do this, once we have used Boom to boot from the snapshot, we can then run the following command to merge the snapshot in to the original root logical volume:

# lvconvert --merge /dev/rhel_rhel7/root_snapshot_071018 
 Delaying merge since snapshot is open.
 Merging of snapshot rhel_rhel7/root_snapshot_071018 will occur on next activation of rhel_rhel7/root.

The lvconvert command reports that it cannot merge the snapshot since it is open and in use, and that it will be completed on the next activation. So we’ll go ahead and reboot the server with the reboot command.  

The system merges the snapshot during boot, and in the end, we are left with the root logical volume back in its original state at the time at which the snapshot was created (thus rolling back the changes made to the root logical volume after the snapshot had been created). We can confirm that the snapshot logical volume was removed by running the lvs command which will no longer show the snapshot logical volume.  

Final Thoughts

LVM snapshots are a powerful tool, and the new boom utility in RHEL 7.5 (and later) provides an easy to use method to boot from these LVM snapshots. Booting from a snapshot has long been available for RHEL hosts running under most virtualization hypervisors, and boom helps provide a similar ability for RHEL servers running on physical hardware to more easily boot from snapshots.