RAID command “mdadm”

After we builded up partition at article “Using “gdisk” to manage HDD partition“, we’re gonna to build a RAID. How many disks you need and size of raid depends on your scenario.

In my situation, I installed CentOS on SSD. I also get another HDDs, larger space for saving data such as /home or /var directories. I would use my 3 disks and 2 partition each disk to build up 2 volume for home and var directories.

After we installed CentOS7 and boot up system, how do we build up software RAID on Linux? I would explain it in this article.

 

Prerequisites

It depends on your RAID type, you need to prepare enough HDDs. 

You can refer to wiki website below: 
Standard RAID levels[1] or Nested RAID levels[2]


[1] https://en.wikipedia.org/wiki/Standard_RAID_levels
[2] https://en.wikipedia.org/wiki/Nested_RAID_levels

Let’s start.

use “lsblk” to list out devices’ status.

[nathaniel@CentOS7 /]$ lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda       8:0    0 167.7G  0 disk
├─sda1    8:1    0   512M  0 part  /boot/efi
├─sda2    8:2    0    20G  0 part  /boot
├─sda3    8:3    0 143.2G  0 part  /
└─sda4    8:4    0     4G  0 part  [SWAP]
sdb       8:16   0   1.8T  0 disk
├─sdb1    8:17   0   1.7T  0 part
└─sdb2    8:18   0   113G  0 part
sdc       8:32   0   1.8T  0 disk
├─sdc1    8:33   0   1.7T  0 part
└─sdc2    8:34   0   113G  0 part
sdd       8:48   0   1.8T  0 disk
├─sdd1    8:49   0   1.7T  0 part
└─sdd2    8:50   0   113G  0 part

How do we build a software RAID on CentOS 7? Easy way. Using a “mdadm” command.

 

#mdadm --create /dev/md0 --auto=yes --level=5 --raid-devices=4 --spare-devices=1 /dev/sd{b,c,d,e}1

–level=5 is meant RAID 5. –raid-devices equals to how many partitions we use. –spare-devices are some disks for auto rebuild but that disk is also needed to count in –raid-devices parameter. In example above, I have 4 disks or partitions but I just want 3 of them for RAID 5. Remaining one is for spare.

 

In my case, I should change my parameter as below. This volume is for home directory.

#mdadm --create /dev/md0 --auto=yes --level=5 --raid-devices=3 --spare-devices=0 /dev/sd{b,c,d}1

And this on is for var directory.

#mdadm --create /dev/md1 --auto=yes --level=5 --raid-devices=3 --spare-devices=0 /dev/sd{b,c,d}2

Then, run command and you would see a picture as below in your terminal.

#cat /proc/mdstat

#top

Sync usage is around 30% of CPU.

After RAID volume is sync, check the /dev/md0 RAID volume status and should be shown like this.

#mdadm --detail /dev/md0
[nathaniel@CentOS7 ~]$ sudo mdadm --detail /dev/md0
[sudo] password for nathaniel:
/dev/md0:
           Version : 1.2
     Creation Time : Tue May 28 00:35:25 2019
        Raid Level : raid5
        Array Size : 3669751808 (3499.75 GiB 3757.83 GB)
     Used Dev Size : 1834875904 (1749.87 GiB 1878.91 GB)
      Raid Devices : 3
     Total Devices : 3
       Persistence : Superblock is persistent

     Intent Bitmap : Internal

       Update Time : Thu Jun  6 18:20:55 2019
             State : clean     #<-------epsecially here, clean.
    Active Devices : 3
   Working Devices : 3
    Failed Devices : 0
     Spare Devices : 0

            Layout : left-symmetric
        Chunk Size : 512K

Consistency Policy : bitmap

              Name : CentOS7:0  (local to host CentOS7)
              UUID : b0025773:aa726537:b28ec4a3:b385b60d
            Events : 3788

    Number   Major   Minor   RaidDevice State
       0       8       17        0      active sync   /dev/sdb1
       1       8       33        1      active sync   /dev/sdc1
       3       8       49        2      active sync   /dev/sdd1

We’ve done our software RAID configuration on CentOS7. 😀

We watch devices again via “lsblk”.

[nathaniel@CentOS7 ~]$ lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda       8:0    0 167.7G  0 disk
├─sda1    8:1    0   512M  0 part  /boot/efi
├─sda2    8:2    0    20G  0 part  /boot
├─sda3    8:3    0 143.2G  0 part  /
└─sda4    8:4    0     4G  0 part  [SWAP]
sdb       8:16   0   1.8T  0 disk
├─sdb1    8:17   0   1.7T  0 part
│ └─md0   9:0    0   3.4T  0 raid5 
└─sdb2    8:18   0   113G  0 part
  └─md1   9:1    0 225.9G  0 raid5 
sdc       8:32   0   1.8T  0 disk
├─sdc1    8:33   0   1.7T  0 part
│ └─md0   9:0    0   3.4T  0 raid5 
└─sdc2    8:34   0   113G  0 part
  └─md1   9:1    0 225.9G  0 raid5 
sdd       8:48   0   1.8T  0 disk
├─sdd1    8:49   0   1.7T  0 part
│ └─md0   9:0    0   3.4T  0 raid5 
└─sdd2    8:50   0   113G  0 part
  └─md1   9:1    0 225.9G  0 raid5 

 

Using “gdisk” to manage HDD partition

Let’s start. I would use 3 HDDs to build up a RAID 5. Run “lsblk” command to show out what devices I connect now. 

[nathaniel@CentOS7 ~]$ lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda       8:0    0 167.7G  0 disk
├─sda1    8:1    0   512M  0 part  /boot/efi
├─sda2    8:2    0    20G  0 part  /boot
├─sda3    8:3    0 143.2G  0 part  /
└─sda4    8:4    0     4G  0 part  [SWAP]
sdb       8:16   0   1.8T  0 disk
sdc       8:32   0   1.8T  0 disk
sdd       8:48   0   1.8T  0 disk

From terminal, we see that there are 3 HDDs connected to server. They are “sdb”, “sdc” and “sdd”. Before we start to make new partition on our HDDs, we should decide what partition table we use. MBR or GPT? In my opinion, GPT is newer and supports over 2TB HDD. So, use “gdisk” command to manager partition.

[nathaniel@CentOS7 ~]$ sudo gdisk /dev/sdb
GPT fdisk (gdisk) version 0.8.10

Partition table scan:
  MBR: protective
  BSD: not present
  APM: not present
  GPT: present

Found valid GPT with protective MBR; using GPT.

Command (? for help): ?
b       back up GPT data to a file
c       change a partition's name
d       delete a partition
i       show detailed information on a partition
l       list known partition types
n       add a new partition
o       create a new empty GUID partition table (GPT)
p       print the partition table
q       quit without saving changes
r       recovery and transformation options (experts only)
s       sort partitions
t       change a partition's type code
v       verify disk
w       write table to disk and exit
x       extra functionality (experts only)
?       print this menu

use “?” to print out command. on the list, we see that “q” for exit without any changes. Sometimes, you operate wrong but you are not familiar with gdisk operation. Then, we just use “q” to exit gdisk program. It wouldn’t effect anythings to disk.

Now, we start to make new partition to disk. I decide 1.75TB for first partition and left for another one.

Command (? for help): n
Partition number (1-128, default 1):        #leave it blank here
First sector (34-3907029134, default = 2048) or {+-}size{KMGTP}:     #leave it blank here also
Last sector (2048-3907029134, default = 3907029134) or {+-}size{KMGTP}: 1750G
#####################################################################
#Beware of here!!I made 1750G for my first part so I fill in "1750G"#
#####################################################################
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): fd00 
########################################################################
#if you want to learn more type, you can type "L" to show out.         #
#But now we're gonna build up a RAID, so we type "fd00" for Linux RAID"#
#By default 8300, It's Linux file system                               #
########################################################################
Changed type of partition to 'Linux RAID'

Then, another other partition

Command (? for help): n
Partition number (2-128, default 2):
First sector (34-3907029134, default = 3670018048) or {+-}size{KMGTP}:
Last sector (3670018048-3907029134, default = 3907029134) or {+-}size{KMGTP}: #I leave it blank here this time because I would use whole space.
Current type is 'Linux filesystem'
Hex code or GUID (L to show codes, Enter = 8300): fd00
Changed type of partition to 'Linux RAID'

Finally, we use “p” to print out partitions of disk and check it.

Command (? for help): p
Disk /dev/sdb: 3907029168 sectors, 1.8 TiB
Logical sector size: 512 bytes
Disk identifier (GUID): DBDE959D-606B-44FE-9844-C0DA7340FB2D
Partition table holds up to 128 entries
First usable sector is 34, last usable sector is 3907029134
Partitions will be aligned on 2048-sector boundaries
Total free space is 4061 sectors (2.0 MiB)

Number  Start (sector)    End (sector)  Size       Code  Name
   1            2048      3670016000   1.7 TiB     FD00  Linux RAID
   2      3670018048      3907029134   113.0 GiB   FD00  Linux RAID

After confirmation, we use “w” to save changes to disk and quit.

Well done. We have just finished first HDD. Then, we do the same opeartion to others 2 HDDs via command below.

[nathaniel@CentOS7 ~]$ sudo gdisk /dev/sdc
[nathaniel@CentOS7 ~]$ sudo gdisk /dev/sdd

Then we use “lsblk” to list out our devices again to view changes.

[nathaniel@CentOS7 ~]$ lsblk
NAME    MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
sda       8:0    0 167.7G  0 disk
├─sda1    8:1    0   512M  0 part  /boot/efi
├─sda2    8:2    0    20G  0 part  /boot
├─sda3    8:3    0 143.2G  0 part  /
└─sda4    8:4    0     4G  0 part  [SWAP]
sdb       8:16   0   1.8T  0 disk
├─sdb1    8:17   0   1.7T  0 part
└─sdb2    8:18   0   113G  0 part
sdc       8:32   0   1.8T  0 disk
├─sdc1    8:33   0   1.7T  0 part
└─sdc2    8:34   0   113G  0 part
sdd       8:48   0   1.8T  0 disk
├─sdd1    8:49   0   1.7T  0 part
└─sdd2    8:50   0   113G  0 part

I will teach how we format a partition as ext4 or xfs file system in other article for saving data.

Server is back again :D

All data at old database was disappeared again because some hardware configuration conflicted. What’s happened in fact? HDD data lose when I was changing my RAID 1 to RAID 5. I tried to rescue data from them but unfortunately I am not familiar with LVM architecture. 

Luckily, I saved most of data from Samba server at least. Almost much data such as my photos, videos, educated tutorials for server are saved at Samba server directories. I backed up them then I was going to rebuild entire server.

Finally, server goes fine again. This time, I gave up Intel “faked RAID” configuration and installed OS to SSD. Then I moved /home and /var directories to 3 HDDs after I configured to RAID 5 via Linux software RAID. I will jot down those procedures in coming articles.