In this article, we will see how logical volumes can be created and used on EC2 instances. We will make use of the Logical Volume Management (LVM) feature available on Linux operating systems.
VMware Training – Resources (Intense)
LVM allows us to abstract the interaction between operating system and hard drives. It allows the user to configure the hard drives to act as a single logical drive. The block devices that are present on Linux (/dev/*) are combined to a new block device that can be mounted and in which data can be stored. The data will be written on any of the physical block devices transparent to the user.
The use of LVM allows dynamic (on the fly) resize and creation of disks and partitions.
The same principle can be used on EC2 instances. The physical drives in this case will be the EBS volumes and we will create a logical volume out of them. Later we will extend the logical volume by adding another EBS volume.
In this article, we will do the following:
-
Create a logical volume.
-
Format and mount the new logical volume to be used by the operating system.
-
Extend the logical volume.
These are the steps to create the logical volume:
-
Initialize the disks that will be part of the logical volume.
-
Create the volume group.
-
Create the logical volume.
So let’s start checking the EC2 instance:
The EC2 instance has three volumes attached. The 8GB (/dev/xvda) volume is the root volume and the other two volumes will be merged in a logical volume. You just need to remember that /dev/sdb will be converted to /dev/xvdb, /dev/sdc will be converted to /dev/xvdc, and so on.
Let’s connect to the EC2 instance:
[mbp:~/Downloads] lab% ssh -i "kp_lvm.pem" ec2-user@52.11.251.208 The authenticity of host '52.11.251.208 (52.11.251.208)' can't be established. RSA key fingerprint is ae:d3:bb:b2:56:6a:80:ee:b3:80:ed:b1:2d:ea:ad:cb. Are you sure you want to continue connecting (yes/no)? yes Warning: Permanently added '52.11.251.208' (RSA) to the list of known hosts. __| __|_ ) _| ( / Amazon Linux AMI ___|\___|___| https://aws.amazon.com/amazon-linux-ami/2015.09-release-notes/ [ec2-user@ip-172-31-25-17 ~]$
We have these three block devices:
[root@ip-172-31-25-17 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 8G 0 disk └─xvda1 202:1 0 8G 0 part / xvdb 202:16 0 10G 0 disk xvdc 202:32 0 12G 0 disk [root@ip-172-31-25-17 ~]#
Now we can start the process of creating the logical volume.
First, we need to initialize the disks that will be used as part of the logical volume:
[root@ip-172-31-25-17 ~]# pvcreate /dev/xvdb /dev/xvdc Physical volume "/dev/sdb" successfully created Physical volume "/dev/sdc" successfully created [root@ip-172-31-25-17 ~]#
We can check the disks that will be used:
[root@ip-172-31-25-17 ~]# pvdisplay "/dev/sdb" is a new physical volume of "10.00 GiB" --- NEW Physical volume --- PV Name /dev/sdb VG Name PV Size 10.00 GiB Allocatable NO PE Size 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID eXAGSw-xYw3-hBpY-h6ks-Oiit-lCXR-Ukk5ES "/dev/sdc" is a new physical volume of "12.00 GiB" --- NEW Physical volume --- PV Name /dev/sdc VG Name PV Size 12.00 GiB Allocatable NO PE Size 0 Total PE 0 Free PE 0 Allocated PE 0 PV UUID C8LcHS-LkTk-rn0J-u5YK-cOlj-w0Wl-x7nOpQ [root@ip-172-31-25-17 ~]#
The next step is to create the volume group. In this case, it will be called “test-lvm” and it will contain the previously initialized disks:
[root@ip-172-31-25-17 ~]# vgcreate test-lvm /dev/xvdb /dev/xvdc Volume group "test-lvm" successfully created [root@ip-172-31-25-17 ~]#
We can check the status of the disks again and now you can see there is a difference in the output once we added them to the volume:
[root@ip-172-31-25-17 ~]# pvdisplay --- Physical volume --- PV Name /dev/sdb VG Name test-lvm PV Size 10.00 GiB / not usable 4.00 MiB Allocatable yes PE Size 4.00 MiB Total PE 2559 Free PE 2559 Allocated PE 0 PV UUID eXAGSw-xYw3-hBpY-h6ks-Oiit-lCXR-Ukk5ES --- Physical volume --- PV Name /dev/sdc VG Name test-lvm PV Size 12.00 GiB / not usable 4.00 MiB Allocatable yes PE Size 4.00 MiB Total PE 3071 Free PE 3071 Allocated PE 0 PV UUID C8LcHS-LkTk-rn0J-u5YK-cOlj-w0Wl-x7nOpQ [root@ip-172-31-25-17 ~]#
You can also check the volume group details:
[root@ip-172-31-25-17 ~]# vgdisplay --- Volume group --- VG Name test-lvm System ID Format lvm2 Metadata Areas 2 Metadata Sequence No 1 VG Access read/write VG Status resizable MAX LV 0 Cur LV 0 Open LV 0 Max PV 0 Cur PV 2 Act PV 2 VG Size 21.99 GiB PE Size 4.00 MiB Total PE 5630 Alloc PE / Size 0 / 0 Free PE / Size 5630 / 21.99 GiB VG UUID iIpbXE-hOLj-xMIa-1S6l-PmqR-iLKo-U2pzjs [root@ip-172-31-25-17 ~]#
Before we can create the logical volume, we need to know how big it can be. For now, we know that the logical group can be as much as 21.99GB.
We can create the logical volume by specifying as parameter the number of PEs that we want to allocate. In this case, we want to allocate everything and we will create the logical volume called “logical-volume”:
[root@ip-172-31-25-17 ~]# vgdisplay test-lvm | grep "Total PE" Total PE 5630 [root@ip-172-31-25-17 ~]# lvcreate -l 5630 test-lvm -n logical-volume Logical volume "logical-volume" created. [root@ip-172-31-25-17 ~]#
This is the logical volume where you can see a few of its details, like path, size and others:
[root@ip-172-31-25-17 ~]# lvdisplay --- Logical volume --- LV Path /dev/test-lvm/logical-volume LV Name logical-volume VG Name test-lvm LV UUID 7hlhN3-93Q3-Ac49-iCgg-Vl1e-VIXn-2YyU0S LV Write Access read/write LV Creation host, time ip-172-31-25-17, 2015-11-13 11:55:39 +0000 LV Status available # open 0 LV Size 21.99 GiB Current LE 5630 Segments 2 Allocation inherit Read ahead sectors auto - currently set to 256 Block device 253:0 [root@ip-172-31-25-17 ~]#
To make the logical volume usable, we need to format the filesystem, ext3, in this case:
[root@ip-172-31-25-17 ~]# mkfs -t ext3 /dev/test-lvm/logical-volume mke2fs 1.42.12 (29-Aug-2014) Creating filesystem with 5765120 4k blocks and 1441792 inodes Filesystem UUID: a82cfb90-be32-4b07-8248-ae9ca63975ee Superblock backups stored on blocks: 32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 4096000 Allocating group tables: done Writing inode tables: done Creating journal (32768 blocks): done Writing superblocks and filesystem accounting information: done [root@ip-172-31-25-17 ~]#
Next, we mount the logical volume:
[root@ip-172-31-25-17 ~]# mkdir /mnt/LVM [root@ip-172-31-25-17 ~]# mount -t ext3 /dev/test-lvm/logical-volume /mnt/LVM/ [root@ip-172-31-25-17 ~]# mount proc on /proc type proc (rw,relatime) sysfs on /sys type sysfs (rw,relatime) /dev/xvda1 on / type ext4 (rw,noatime,data=ordered) devtmpfs on /dev type devtmpfs (rw,relatime,size=500712k,nr_inodes=125178,mode=755) devpts on /dev/pts type devpts (rw,relatime,gid=5,mode=620,ptmxmode=000) tmpfs on /dev/shm type tmpfs (rw,relatime) none on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime) /dev/mapper/test--lvm-logical--volume on /mnt/LVM type ext3 (rw,relatime,errors=continue,user_xattr,acl,barrier=1,data=ordered) [root@ip-172-31-25-17 ~]#
And now the EBS volumes shows as part of the logical volume that has a size of 22GB:
[root@ip-172-31-25-17 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 8G 0 disk └─xvda1 202:1 0 8G 0 part / xvdb 202:16 0 10G 0 disk └─test--lvm-logical--volume 253:0 0 22G 0 lvm /mnt/LVM xvdc 202:32 0 12G 0 disk └─test--lvm-logical--volume 253:0 0 22G 0 lvm /mnt/LVM [root@ip-172-31-25-17 ~]#
This is how you create a logical volume out of multiple physical disks.
Let’s say that you want to add additional EBS volumes to the EC2 instance like this:
Remember that you need to stop the EC2 instance before you can add another volume.
As you can see, /dev/xvdf is the new EBS volume:
[ec2-user@ip-172-31-25-17 ~]$ lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 8G 0 disk └─xvda1 202:1 0 8G 0 part / xvdb 202:16 0 10G 0 disk └─test--lvm-logical--volume 253:0 0 22G 0 lvm /mnt/LVM xvdc 202:32 0 12G 0 disk └─test--lvm-logical--volume 253:0 0 22G 0 lvm /mnt/LVM xvdf 202:80 0 30G 0 disk [ec2-user@ip-172-31-25-17 ~]$
We need to extend the volume group by adding the new EBS volume:
[root@ip-172-31-25-17 ec2-user]# vgextend test-lvm /dev/xvdf Physical volume "/dev/sdf" successfully created Volume group "test-lvm" successfully extended [root@ip-172-31-25-17 ec2-user]#
Next, we want the logical volume to be extended to 51.98GB. If you remember, the logical volume was 22GB and we want to add another 30GB. In total that would be 52GB, but it is actually just a little bit lower than this:
[root@ip-172-31-25-17 ec2-user]# lvextend -L51G /dev/test-lvm/logical-volume Size of logical volume test-lvm/logical-volume changed from 21.99 GiB (5630 extents) to 51.00 GiB (13056 extents). Logical volume logical-volume successfully resized [root@ip-172-31-25-17 ec2-user]# lvextend -L+0.98G /dev/test-lvm/logical-volume Rounding size to boundary between physical extents: 1004.00 MiB Size of logical volume test-lvm/logical-volume changed from 51.00 GiB (13056 extents) to 51.98 GiB (13307 extents). Logical volume logical-volume successfully resized [root@ip-172-31-25-17 ec2-user]#
It’s time to extend the filesystem:
[root@ip-172-31-25-17 ec2-user]# resize2fs /dev/test-lvm/logical-volume resize2fs 1.42.12 (29-Aug-2014) Filesystem at /dev/test-lvm/logical-volume is mounted on /mnt/LVM; on-line resizing required old_desc_blocks = 2, new_desc_blocks = 4 Performing an on-line resize of /dev/test-lvm/logical-volume to 13626368 (4k) blocks. The filesystem on /dev/test-lvm/logical-volume is now 13626368 (4k) blocks long. [root@ip-172-31-25-17 ec2-user]#
Now the logical volume of 51.98GB can be used by the operating system:
[root@ip-172-31-25-17 ec2-user]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT xvda 202:0 0 8G 0 disk └─xvda1 202:1 0 8G 0 part / xvdb 202:16 0 10G 0 disk └─test--lvm-logical--volume 253:0 0 52G 0 lvm /mnt/LVM xvdc 202:32 0 12G 0 disk └─test--lvm-logical--volume 253:0 0 52G 0 lvm /mnt/LVM xvdf 202:80 0 30G 0 disk └─test--lvm-logical--volume 253:0 0 52G 0 lvm /mnt/LVM [root@ip-172-31-25-17 ec2-user]#
In the future, you might need to remove the logical volume. This is certainly possible. You can use the references below to find more information about how to do so.
By reaching this point in the article, you now know that you can use the LVM feature on EC2 instances from AWS like on any other physical server.
You should know how to create the logical volume, how to extend it and how to make use of it so that it can store your data.
References