Increase/Decrease the size of the Hadoop cluster on the fly using LVM
To increase or decrease the size of the Hadoop cluster on the fly it is required to understand the concept of logical volume management also known as LVM.
using LVM multiple logical volumes can be created combining all available harddisk so whenever it is required to increase the size of Hadoop using the lvm size of logical volume can be increased on the fly.
So let’s get started.
Step 1: Attach Hard Disk to Virtual Machine and To see run the following command
Step 2: Convert the hard disk to Physical Volume
pvcreate command initializes these disks so that they can be a part of forming volume groups.
Step 3: Create a volume group
Physical volumes are combined into volume groups (VGs). It creates a pool of disk space out of which logical volumes can be allocated.
Step4: Create Logical Volume
It takes the space from the volume group and creates a partition of the capacity mentioned in the command
lvcreate --size 10G --name mylv myvg
To display logical volume
Step 5: Format the logical Partition
Step 6: mount logical volume with datanode shared directory
Step 7: To see storage gain by Hadoop cluster
Step 8: Extend the logical volume
here the size of the partition is increased by 9G but the file system still is of older 10G. To resize the file system
Step 9: Resize the file system
Step 10: Hadoop cluster report
after resizing the lv partition we don’t have to do anything Hadoop automatically picks the updated size.