Adding & Replacing Drives

This is a guide on how to add/replace a disk on a Software RAID. In the event of a missing drive or a single drive failure. You can follow this guide as reference to replace/add a new disk and put your RAID configuration back in a healthy state.


Status of current RAID can be obtained using:

root@Software-Raid-Recovery-Test:~# cat /proc/mdstat  


A sample output would be similar to: 

Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md126 : active raid1 sdb31 sda3[0]
      466720576 blocks super 1.2 [2/1] [U_]
      bitmap: 3/4 pages [12KB], 65536KB chunk
md127 : active (auto-read-only) raid1 sda2[0] sdb2[1]
      1995776 blocks super 1.2 [2/2] [UU]    

unused devices: <none>


You can also review messages in syslog: 

root@Software-Raid-Recovery-Test:~# tail /var/log/syslog

Mar  9 05:17:09 Software-Raid-Recovery-Test mdadm[1557]: Fail event detected on md device /dev/md126, component device /dev/sdb3Mar  9 05:17:09 Software-Raid-Recovery-Test kernel: [75455.286573] md/raid1:md126: Disk failure on sdb3, disabling device.
Mar  9 05:17:09 Software-Raid-Recovery-Test kernel: [75455.286573] md/raid1:md126: Operation continuing on 1 devices.
Mar  9 05:17:09 Software-Raid-Recovery-Test kernel: [75455.300175] RAID1 conf printout:
Mar  9 05:17:09 Software-Raid-Recovery-Test kernel: [75455.300181]  --- wd:1 rd:2
Mar  9 05:17:09 Software-Raid-Recovery-Test kernel: [75455.300185]  disk 0, wo:0, o:1, dev:sda3
Mar  9 05:17:09 Software-Raid-Recovery-Test kernel: [75455.300188]  disk 1, wo:1,o:0, dev:sdb3
Mar  9 05:17:09 Software-Raid-Recovery-Test kernel: [75455.323552] RAID1 conf printout:
Mar  9 05:17:09 Software-Raid-Recovery-Test kernel: [75455.323557]  --- wd:1 rd:2
Mar  9 05:17:09 Software-Raid-Recovery-Test kernel: [75455.323561]  disk 0, wo:0, o:1, dev:sda3


Remove the device from RAID

root@Software-Raid-Recovery-Test:~# mdadm --remove /dev/md126 /dev/sdb3

mdadm: hot removed /dev/sdb3 from /dev/md126

unused devices: <none>


You can see /dev/sdb2 is not seen in /proc/mdstat 

Also if we check mdadm --detail  command we can see that /dev/sdb3 has been removed.

root@Software-Raid-Recovery-Test:~# mdadm --detail /dev/md126

/dev/md126:
        Version : 1.2
  Creation Time : Wed Mar  8 08:15:07 2017
     Raid Level : raid1
     Array Size : 466720576 (445.10 GiB 477.92 GB)
  Used Dev Size : 466720576 (445.10 GiB 477.92 GB)
   Raid Devices : 2
  Total Devices : 1
    Persistence : Superblock is persistent
  Intent Bitmap : Internal
    Update Time : Thu Mar  9 05:43:10 2017
          State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0
           Name : 232491c29715:ROOT
           UUID : 61f43f57:7fe803b5:61156f1d:c2e7ea0d
         Events : 588
    Number   Major   Minor   RaidDevice State
       0       8        3        0      active sync   /dev/sda3
       2       0        0        2      removed


Add a new device:

root@Software-Raid-Recovery-Test:~# mdadm --add /dev/md126 /dev/sdb3

mdadm: re-added /dev/sdb3


The partition should now be added and you can view the status of the Raid Config on the commands below.

root@Software-Raid-Recovery-Test:~# cat /proc/mdstat

Personalities : [raid1] [linear] [multipath] [raid0] [raid6] [raid5] [raid4] [raid10]
md126 : active raid1 sdb3[1] sda3[0]
      466720576 blocks super 1.2 [2/2] [UU]
      bitmap: 3/4 pages [12KB], 65536KB chunk
md127 : active (auto-read-only) raid1 sda2[0] sdb2[1]
      1995776 blocks super 1.2 [2/2] [UU]

unused devices: <none>      


root@Software-Raid-Recovery-Test:~# mdadm --detail /dev/md126

/dev/md126:
        Version : 1.2
  Creation Time : Wed Mar  8 08:15:07 2017
     Raid Level : raid1
     Array Size : 466720576 (445.10 GiB 477.92 GB)
  Used Dev Size : 466720576 (445.10 GiB 477.92 GB)
  Raid Devices : 2
  Total Devices : 2
  Persistence : Superblock is persistent
  Intent Bitmap : Internal
  Update Time : Thu Mar  9 05:46:42 2017
          State : clean
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
           Name : 232491c29715:ROOT
           UUID : 61f43f57:7fe803b5:61156f1d:c2e7ea0d
         Events : 606
    Number   Major   Minor   RaidDevice State
       0       8        3        0      active sync   /dev/sda3
       1       8       19        1      active sync   /dev/sdb3


Once the new disk is added the RAID Configuration will automatically get re-build using mdadm  utility.