7 Volume Management

The entire storage management, including the root file system, is done using LVM2 (Local Volume Manager) which is included in Linux for some time. LVM has the benefit that it allows very flexible dynamic disk, volume group, and partition management without being limited to older disk partition formats. More background information on LVM is available at: en.wikipedia.org/wiki/Logical_Volume_Manager_(Linux). It offers similar features as IBM AIX or Oracle Solaris ZFS volume management.

As LVM is very powerful, it uses many tools to manage it. HELIOS has developed the HELIOS Configuration Assistant to manage all interactive storage management needs, see 4.2.3 “Storage setup (data volumes)”.

The HELIOS Virtual Server Appliance uses basically two volume groups:

rootvg (“root volume group”) for the operating system and swapping memory. “rootvg” is also LVM-based, however there is an additional ext3 partition (“/dev/sda1”) which is used for the GRUB boot system. The LVM manages the system disk partition (“/dev/sda2”) as a physical volume.
datavg (“data volume group”) for all customer data that is separated from the operating system to allow upgrading the OS without changing the customers' data. The LVM handles all disks of the “datavg“ volume group directly and without any traditional partition table.

7.1 LVM overview

This section describes special characteristics of the individual components of the LVM storage. It gives you an overview of the HELIOS default storage setup. The illustration below shows an example LVM setup:

Fig. 7.1: Logical Volume Manager (LVM) with individual disks

Note:: There is no need to work manually with disks, physical volumes, volume groups, or logical volumes because the HELIOS Configuration Assistant will do this for you.

7.2 Physical volumes

Physical volumes represent disks that have been assigned to a volume group. All disks or RAID combined disks are managed by LVM. Disks are called physical volumes if used with LVM. We do not use any DOS partition table or GUID partition table (GPT) for disks, to avoid size and resizing limitations. LVM uses each disk directly as a physical volume to overcome limitations and to allow increasing existing disks, which can be done in the hypervisor. Another option is adding additional disks to the hypervisor, which are used automatically.

Important:: Traditional partitioning tools, e.g. “fdisk” or “parted”/“GParted”, do not recognize disks without partition table and will therefore display the LVM disk as empty. Do not partition these disks!

Disks can be scanned in the Configuration Assistant (see “Show physical disk overview” in Manage other volume groups (Advanced storage setup)) or via the LVM command: “lvmdiskscan”.

Physical volumes can be listed in the Configuration Assistant (see List physical volumes (disks used)) or via the LVM commands: “pvs”, “pvdisplay”, “pvscan”.

7.3 Volume groups

A volume group (e.g. “datavg”) is a container combining multiple physical volumes. Disks can be added to or removed from a volume group, assuming there is space left over to remove a physical volume. Volume groups combine physical volumes and supply logical volumes (partitions with file systems) with storage blocks.

Volume groups can be listed in the Configuration Assistant (see “List volume groups and volumes” in Manage other volume groups (Advanced storage setup)) or via the LVM commands: “vgs”, “vgdisplay”.

7.4 Logical volumes (file systems)

Logical volumes are created in a volume group and represent a block device to be used with a file system. LVM supports snapshots of a logical volume. The HELIOS Virtual Server Appliance uses ext4 based file systems which are created with the mkfs.ext4 command on the block device. A block device can be enlarged, which requires the file system to be converted in order to support the new size. The HELIOS Configuration Assistant will do this automatically, and manage the storage in a secure way.

Logical volumes can be listed in the Configuration Assistant (see List logical volumes) or via the LVM commands: “lvs”, “lvdisplay”, “lvscan”.

Note:

Once disks are managed by LVM there is no configuration file containing them because LVM manages the disks directly by scanning all devices enabling physical volumes, volume groups, and logical volumes. This is also independent from the disk slot position or SCSI IDs, which makes it very flexible compared to static configurations.

Data security (RAID or mirroring) is not provided by LVM which means that, if multiple disks are added to a volume group, one failing disk will possibly make the entire volume group unusable. Therefore a hardware mirror or RAID solution is required to provide safe disks to be used by LVM. Another option is to re-initialize a volume group and its logical volumes, and restore all data from a backup.

7.5 Snapshots of logical volumes

Snapshots allow a read-only or read/write copy of logical volumes, which can be used to create a backup while the real logical volumes are still being used. The Configuration Assistant creates only read/write snapshots. A snapshot reflects the state of a logical volume at a certain point in time.

LVM supports multiple snapshots per logical volume.

7.5.1 The benefit of snapshots

While the snapshot exists, the original data can be modified without affecting the snapshot data. The snapshot serves as a temporary backup in a consistent state, and can be used as a source to create a full backup of the file system. During this process, you can continue to work with and modify the original data.

Once the full backup is completed the snapshot can be removed.

7.5.2 How a snapshot works

A snapshot requires free disk space in the volume group where the logical volume is located. It merely stores the differences to the source logical volume so that the complete size of the logical volume is not needed a second time. Instead, the maximum size of differences is defined as the snapshot size. We recommend at least 10% of the source volume size. This means that 10% of the logical volume can be modified until the snapshot size is used up and the snapshot will be disabled.

Example:

The snapshot “data1-snap1” for a logical volume “data1” from the “datavg” volume group saves all data that is modified in “data1” while the snapshot is active.

The snapshot itself will contain the differences (“undo information”) of the “data1” file changes. It will use this “undo information” to roll back all file changes, additions, and deletions subsequent to the creation of the snapshot, and hence will show only the state of files as of when the snapshot was created.

Snapshot management is a cheap operation which can be done within seconds. Create and delete snapshots by use of the HELIOS Configuration Assistant (see Manage other volume groups (Advanced storage setup) in chapter 4.2.3 “Storage setup (data volumes)”).

Important:: If the space of the running snapshot is exceeded, the snapshot copy (snapshot file system) does not work anymore and cannot be used because it includes false data. The original file system continues to work without any problems.

7.5.3 Performance considerations when using snapshots

Snapshots will have a negative effect on the disk read/write performance and will make the system significantly slower. This is because all modified data is saved multiple times, depending on the number of snapshots; to the original file system and to the undo data in the snapshot partitions.

Also, when volume groups are manually activated, or automatically at system startup, all existing snapshots are initialized. Therefore all snapshot data must be read-in to create the LVM internal structures. Depending on the disk or RAID performance, this can take a significant amount of time, up to hours when hundreds of gigabytes have been modified since creating the snapshot(s).

For this reason, we strongly recommend to activate snapshots on a temporary base only, e.g. for creating backups. When not needed anymore, snapshots should be removed to prevent poor performance or long booting times.

7.5.4 Activate HELIOS volumes on a snapshot

By default, all snapshots are mounted in “/snapshots”, e.g. “/snapshots/data1-snap1”. As snapshots are read/write, it is possible to create a HELIOS volume pointing to the snapshot copy of the file system. The volume could be made available for certain users, to allow easy access to the previous data version when the snapshot was taken.

7.5.5 Example snapshot description

The illustration below shows three example snapshots of the “/data1” file system in the “datavg” volume group.

Fig. 7.2: Snapshots of logical volume “data1”

As seen in the figure, file system snapshots can be a very powerful solution to create stable backups from a running system, i.e. while the system is still modifying data. Another option is to use snapshots to permit access to data from an earlier point in time, e.g. from 11am today (see “Snapshot 1” in the illustration above).

7.6 A typical “datavg” setup with a RAID

The illustration below shows a RAID 5 (with disk fail protection) which is included as a physical volume into the “datavg” volume group, and on the right hand the file systems “/data1” … “/data4” which use storage space from “datavg”.

Fig. 7.3: Logical Volume Manager (LVM) in a RAID system

7.7 Replace data disks or a RAID system

Disks can be swapped without interrupting the server. Reasons for doing so could be an upgrade to larger disks or an upgrade to a new RAID system.

7.7.1 Upgrade to a new RAID system

To upgrade to a completely new RAID system, the new RAID must be added to the “datavg” volume group. This is best done via the HELIOS Configuration Assistant in the Storage setup menu (see 4.2.3 “Storage setup (data volumes)”).

: Connect the new RAID system to the hypervisor that is used to assign the RAID to the VSA.

: In the Configuration Assistant select Create or extend data pool (add disks) which will detect the new RAID (in the following referred to as “disk”).

: Select the desired disk by entering the device name(s). (If you are not sure which disk is the correct one compare the SCSI ID with your hypervisor configuration!)

The Configuration Assistant will add the disk(s) to the “datavg” volume group.

: Back to the Storage setup menu, select the menu item Reduce data pool (remove disks).

: Enter the disk device name(s) of the old RAID and confirm the removal.

This will take a while because all blocks in use will be moved to the new RAID system to free up the old RAID.

: Disconnect the old RAID from the hypervisor.

7.7.2 Replace disks in an existing RAID

Replacing one or more disks in the RAID system (e.g. to increase the storage capacity) works similarly to the description in 7.7.1 “Upgrade to a new RAID system”. If the storage capacity of the remaining disks is not sufficient to temporarily store the whole RAID data you will need a temporary disk or RAID system, e.g. a USB disk drive or any other kind of disk system.

: Connect this temporary disk to the hypervisor like you would attach a new RAID (as described in 7.7.1 “Upgrade to a new RAID system”).

: Via the menu item Reduce data pool (remove disks) remove the old disk from the volume group.

: Now remove the old physical disk from the RAID and replace it with the new disk.

: Add the new disk via the Create or extend data pool (add disks) menu.

: Via the menu item Reduce data pool (remove disks) remove the temporary disk from the volume group.

: Finally, remove the temporary disk from the hypervisor.

Note:: Before doing this kind of operation, a backup of the entire server data is mandatory. Although we expect that the backup is never needed, every customer must have backups of all their server data to ensure that there is a restore option if something goes wrong.

7.8 Increase the existing “datavg” volume group size

There are two ways to increase the size of a volume group:

Add disk to the volume group
Grow the size of an existing disk

The described tasks are done quickly within a minute.

7.8.1 Add disk to the volume group

: In the hypervisor configuration create a new virtual disk. This can be an image or a hardware disk attached to the hypervisor.

7.8.2 Grow an existing disk

In the hypervisor configuration expand the disk image of one of the physical volumes assigned to the volume group.

Note:

VMware ESX Server allows growing the disk size on-the-fly without any service interrupts for the HELIOS Virtual Server Appliance. Also the HELIOS Configuration Assistant will automatically detect the grown disks and update the affected physical volume and the volume group behind it. However, some hypervisors require a shutdown of the VM before you can grow the disk size. In such a case shut down the VM before doing changes to the disk images.

Growing a disk image can only be done if the disk is not partitioned, i.e. completely used as LVM physical volume. See also 7.2 “Physical volumes”. This means that if you wish to increase the size of the volume group “rootvg”, you should not grow the disk “/dev/sda” but add a new virtual disk to the volume group instead.

Important:: Never shrink the disk on the host because this will destroy the data!

7.8.3 Add new or grown disk(s) to the volume group

: In the Configuration Assistant Storage setup menu choose Create or extend data pool.

The Configuration Assistant will scan for new or grown disks and offers to add the additional storage space to the volume group. Both new and grown disks can be added at the same time.

7.9 Use a RAID 1 (mirroring) on the system disk

Installations may require a mirrored system disk. This feature is not included in the HELIOS Virtual Server Appliance. System disk mirroring must be configured via a hardware RAID system. This is called RAID 1 (mirroring without parity or striping) and is supported by most RAID controllers. We recommend to keep the system disk image on a separate physical disk to isolate the operating system from the customer data. This will make things much easier in case of problems and also make OS disk performance independent of the data disk (RAID) performance.

7.10 Use a RAID 5 on the system disk

from: http://en.wikipedia.org/wiki/RAID
A RAID 5 (block-level striping with distributed parity) distributes parity along with the data and requires all drives but one to be present to operate; the array is not destroyed by a single drive failure. Upon drive failure, any subsequent reads can be calculated from the distributed parity such that the drive failure is masked from the end user. However, a single drive failure results in reduced performance of the entire array until the failed drive has been replaced and the associated data rebuilt. … RAID 5 requires at least three disks.

7.11 Restore the operating system disk

In case a system disk is defective, a new HELIOS Virtual Server Appliance image can be reimported and used as new system disk. Then the following files/directories must be restored from a backup:

– “/etc/fstab”
– “/etc/passwd”
– “/etc/shadow”
– “/usr/local/helios/var/”
– “/home/”

In addition, custom configuration settings like date/time and network setup must be done anew using the HELIOS Configuration Assistant.

Note:: We recommend that you define all users' home directories in a logical volume on “datavg”. The advantage is that “rootvg” is not loaded with variable amount of data stored in the home folders. This limits the variable data on “rootvg” to a few MB only, which can be saved and restored quickly.

All volume groups and their connected logical volumes (partitions), e.g.: “/data1” from the volume group “datavg” are automatically detected when a new system disk is attached or existing disks are added into a new server. During the system boot process, Linux will create the proper device entries in “/dev/mapper” with their names, e.g.: “datavg-data1”. Setting “/etc/fstab” entries (if missing) allows mounting these existing file systems.

7.12 Summary

The HELIOS Configuration Assistant makes the LVM management very easy. The LVM benefit is that it handles disks, volume groups and partitions (logical volumes) in a very flexible and efficient way. This saves the customer an enormous amount of time when adding, changing or modifying storage needs. Doing this manually without LVM would take many days of work, compared to hours or sometimes even minutes.

HELIOS Manuals August 10, 2015