Help Centre
Home Server & Cloud Infrastructure Dedicated Server

Rebuilding software RAID (Linux Dedicated Servers and Bare Metal Servers with UEFI)

Valid for Dedicated Servers and Bare Metal Servers.

This article walks you through rebuilding a software RAID after replacing a defective hard drive (e.g., a hard disk, SSD, or NVMe drive) on a Dedicated Server or Bare Metal Server. This guide covers both configurations with RAID 1, and systems with an additional software RAID 5 or RAID 6 for data partitions.

Please Note

  • This guide is only valid for Dedicated Server and Bare Metal Server that use UEFI as the interface between the hardware and the operating system. If you are using a Dedicated Server with BIOS, please refer to the following article for BIOS instructions. Rebuilding software RAID (Linux/dedicated server with BIOS)

  • Please follow the sequence of steps described exactly.

Check whether a server uses UEFI or BIOS

To check whether your server uses BIOS or UEFI as the interface between the hardware and the operating system, enter the following command:

[root@localhost ~]# [ -d /sys/firmware/efi ] && echo UEFI || echo BIOS

Further information on UEFI

For further information on UEFI, please see the following article: General information on UEFI

Important information about partitioning your Dedicated Server

On Dedicated Servers managed in the Cloud Panel, since October 20, 2021, only a single partition is created during setup and operating system reinstallation. On Dedicated Servers set up before this date and those purchased as Server Power Deals, the OS images are equipped with the Logical Volume Manager (LVM). The Logical Volume Manager adds a logical layer between the file system and the physical partitions. This allows you to create a file system that spans multiple partitions and/or drives, combining their storage space.

Identifying the information required to rebuild the software RAID

Before you begin, you must check the current RAID configuration and the status of the partitions.

List existing disks and partitions

To list the existing disks and partitions, complete the following:

  • Log in to the server using your root account.

  • To list the existing disks and partitions, enter the command fdisk -l. fdisk is a command-line program for partitioning disks. This program allows you to view, create, or delete partitions.

    [root@localhost ~]# fdisk -l

  • Make a note of the drive designations (e.g. /dev/sda, /dev/sdb, /dev/nvme0n1).

Note

Following a hot-swap replacement, the new storage device is often recognised as ‘sdc’. A reboot is mandatory to ensure that the storage device is correctly addressed as ‘sda’ or ‘sdb’ again.

Identifying mount points

  • To view the mount points of the devices and partitions you are using, enter the following command:

    [root@localhost ~]# lsblk

    The following information, for example, will then be displayed:

    [root@2A2E3A1 ~]# lsblk
    NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
    sda 8:0 0 1.8T 0 disk
    ├─sda1 8:1 0 511M 0 part
    │ └─md1 9:1 0 511M 0 raid1 /boot/efi
    ├─sda2 8:2 0 1.4G 0 part
    │ └─md2 9:2 0 1.4G 0 raid1 /boot
    ├─sda3 8:3 0 3.7G 0 part [SWAP]
    └─sda4 8:4 0 1.8T 0 part
    └─md4 9:4 0 1.8T 0 raid1 /
    sdb 8:16 0 1.8T 0 disk
    ├─sdb1 8:17 0 511M 0 part
    │ └─md1 9:1 0 511M 0 raid1 /boot/efi
    ├─sdb2 8:18 0 1.4G 0 part
    │ └─md2 9:2 0 1.4G 0 raid1 /boot
    ├─sdb3 8:19 0 3.7G 0 part [SWAP]
    └─sdb4 8:20 0 1.8T 0 part
    └─md4 9:4 0 1.8T 0 raid1 /

  • Make a note of the devices and partitions, as well as their mount points. Example:
     
    • /md4 is mounted on / (root)
    • /md2 is mounted on /boot
    • /md1 is mounted on /boot/efi
    • sda3 and sdb3 are swap partitions (not part of a RAID array)
Detailed view of the RAID structure using ‘lsblk --inverse’

For an even clearer view of the RAID structure, you can also use the following command:

[root@localhost ~]# lsblk --inverse

This command reverses the order of the output. The structure then begins not with the individual storage devices, but with the md devices or logical devices. The corresponding partitions and storage devices are displayed below these.

This view is particularly helpful when there are several RAID devices. This is the case, for example, when separate arrays are used for /boot, /, swap, or additional data areas. You can then see the associated partitions and mount points for each md device directly in a single view. Example:

[root@localhost ~]# lsblk --inverse
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
sda1 8:1 0 1M 0 part
└─sda 8:0 0 3.6T 0 disk
sdb1 8:17 0 1M 0 part
└─sdb 8:16 0 3.6T 0 disk
md2 9:2 0 487M 0 raid1 /boot
├─sda2 8:2 0 488M 0 part
│ └─sda 8:0 0 3.6T 0 disk
└─sdb2 8:18 0 488M 0 part
└─sdb 8:16 0 3.6T 0 disk
md3 9:3 0 3.9G 0 raid1 [SWAP]
├─sda3 8:3 0 3.9G 0 part
│ └─sda 8:0 0 3.6T 0 disk
└─sdb3 8:19 0 3.9G 0 part
└─sdb 8:16 0 3.6T 0 disk
md4 9:4 0 19.5G 0 raid1 /
├─sda4 8:4 0 19.5G 0 part
│ └─sda 8:0 0 3.6T 0 disk
└─sdb4 8:20 0 19.5G 0 part
└─sdb 8:16 0 3.6T 0 disk
vg-customers 253:1 0 3.3T 0 lvm /customers
└─md5 9:5 0 3.6T 0 raid1
├─sda5 8:5 0 3.6T 0 part
│ └─sda 8:0 0 3.6T 0 disk
└─sdb5 8:21 0 3.6T 0 part
└─sdb 8:16 0 3.6T 0 disk

Note

If the entire storage device does not fail, only individual partitions may be affected. In this case, not all md devices may be degraded. It is therefore possible that one RAID array is faulty, while other arrays on the same storage device remain intact. Using lsblk --inverse often allows you to identify this more quickly.

View the contents of the fstab file and identify the device files

  • In some server configurations, system directories such as /usr, /var, or /home are located on separate partitions. These must also be mounted for commands to work on the system. To do this, check the fstab file on your mounted system. To view the contents of the fstab file and identify the associated device files, enter the following command:

    [root@localhost ~]# cat /etc/fstab

  • Analyse the output.

    Case A (flat filesystem, e.g., Debian 13): If the output shows entries where /usr, /var, and /proc are not listed separately, you do not need to mount any further partitions. In this case, you can skip the next two steps and place the mounted partitions into a chroot environment.

    Case B (Partitioned system / LVM): If entries for /usr, /var, and /proc are displayed, you must mount these as part of the RAID recovery process. The procedure is described in the section ‘Recovering RAID’.

  • Also note whether the devices are mounted via UUID or directly (e.g. as /dev/md0).

Check RAID status

Check which drive is missing from the RAID. To do this, enter the following command:

[root@localhost ~]# cat /proc/mdstat

Note

In addition to cat /proc/mdstat, you can also use the lsblk --inverse command. This command displays the md devices together with their associated partitions and mount points in a single view. This is particularly useful when only individual RAID arrays are degraded.

RAID 1: A healthy array displays [UU]. The U stands for ‘Up’: the disk is active, synchronised, and part of the array. A faulty array displays [_U] or [U_].

RAID 5 and RAID 6: Missing partitions in the list (e.g., only sdb5 and sdc5 instead of sda5 as well) or entries marked with (F) indicate a failure.

The partitions participating in the RAID are listed after the RAID level (e.g. active raid5). A number in square brackets after the partition name (e.g. sda4[0]) indicates its role in the array. A missing entry or an additional (F) for ‘Failed’ indicates a fault. Examples:

RAID 1 (mirroring) – One disk has failed

[root@2A2E3A1 ~]# cat /proc/mdstat
Personalities : [raid1]
md4 : active raid1 sdb4[2] sda4[0]
1947653952 blocks super 1.0 [2/1] [U_]

md2 : active raid1 sdb2[2] sda2[0]
1428416 blocks super 1.0 [2/2] [UU]

md1 : active raid1 sdb1[1] sda1[0]
523200 blocks [2/2] [UU]

unused devices: <none>

RAID 5 (Data) – One drive is completely missing

RAID 5 is often used for large amounts of data. It can cope with the failure of up to one drive.

md5 : active raid5 sdc1[0] sde1[2] [3/2] [U_U]

sdc1[0] and sde1[2]: Only two partitions are listed. One drive is completely missing from the list. This can occur, for example, following a hardware fault where the hard drive is no longer recognised by the controller at all.
[3/2]: The array expects 3 volumes, but only 2 are present.
[_UU]: The first volume is missing.

RAID 6 (Data) – Two volumes failed

RAID 6 can withstand the simultaneous failure of up to two volumes.

md6 : active raid6 sdc1[0] sdf1[3] 3895307904 blocks super 1.2 level 6, 512k chunk, algorithm 2 [4/2] [U__U]

[4/2]: Only 2 of the 4 data volumes are active.

[U__U]: The two middle storage devices (sdd1 and sde1) have failed or been removed. The RAID is still operational (‘degraded’), but urgent action is required.

Note

For servers using RAID 1 and RAID 5, the following devices and partitions are used, for example:

[root@B2E14658 ~]# cat /proc/mdstat
Personalities : [raid1] [raid6] [raid5] [raid4]
md11 : active raid5 sdb1[1] sde1[0] sdc1[2] sdf1[3]
23442074112 blocks super 1.0 level 5, 512k chunk, algorithm 2 [4/4] [UUUU]
bitmap: 2/59 pages [8KB], 65536KB chunk

md125 : active raid1 sda4[0] sdd4[2]
462991168 blocks super 1.0 [2/2] [UU]
bitmap: 2/4 pages [8KB], 65536KB chunk

md126 : active raid1 sda1[0] sdd1[1]
523200 blocks [2/2] [UU]

md127 : active raid1 sda2[0] sdd2[2]
1,428,416 blocks super 1.0 [2/2] [UU]

Restore RAID (RAID 1)

The next steps vary depending on which drive has been replaced. We distinguish between drive 1 (sda/primary) and drive 2 (sdb/secondary). The procedure is the same for RAID 5/6.

Please Note

In the section ‘Drive 1 (sda) has been replaced’, it is assumed that drive 1, labelled sda, has been swapped. In the section ‘Drive 2 (sdb) has been replaced’, it is assumed that drive 2, labelled sdb, has been swapped. Always adapt the commands in this guide before entering them, replacing the drive designations as required. To do this, use the information you have previously identified.

Drive 1 (sda) has been replaced

If drive 1 (sda) has been replaced, you must check whether it has been recognised correctly. You may need to restart the system. As the primary boot loader is usually located on sda, the system often fails to boot normally. You should therefore boot the server into the rescue system and follow the steps listed below.

Note

The following articles describe how to boot your Dedicated Server or Bare Metal Server into the Rescue System:

Dedicated Servers and Bare Metal Servers, which are managed in the Cloud Panel: Booting Dedicated Servers and Bare Metal Server into the Linux Rescue System

Dedicated Servers which are managed in Server Management: Boot Linux servers into the Rescue System

  • Copy the partition tables from drive 2 (e.g., sdb) to the new (empty) disk (e.g., sda).

    rescue:~# sfdisk -d /dev/sdb | sfdisk --force /dev/sda

Note

For NVMe storage devices, check the name carefully (e.g. sfdisk -d /dev/nvme0n1 or sfdisk --force /dev/nvme1n1).

  • Add the partitions to the relevant RAID array. To do this, adapt the following commands and then enter them:

    RAID 1

    rescue:~# mdadm /dev/md4 -a /dev/sda4 (root partition)

    rescue:~# mdadm /dev/md2 -a /dev/sda2 (boot partition)

    rescue:~# mdadm /dev/md1 -a /dev/sda1 (EFI partition)

  • The rebuild will start in the background. To monitor the rebuild, enter the following command:

    rescue:~# cat /proc/mdstat

  • First, mount the root filesystem. Only then can the other partitions be mounted into the directory structure.

    rescue:~# mount /dev/md4 /mnt

  • Mount the boot and EFI partitions.

    rescue:~# mount /dev/md2 /mnt/boot

    rescue:~# mount /dev/md1 /mnt/boot/efi

Note

If you receive error messages stating that the mount points do not exist, create them using the commands mkdir /mnt/boot and mkdir /mnt/boot/efi.

  • Applies only to Dedicated Servers with a partitioned system using LVM: Create the following mount points:

    rescue:~# mkdir /mnt/var

    rescue:~# mkdir /mnt/usr

    rescue:~# mkdir /mnt/home

  • Applies only to Dedicated Servers with a partitioned system using LVM: If necessary, mount the var, usr, and home partitions.

    rescue:~# mount /dev/mapper/vg00-var /mnt/var

    rescue:~# mount /dev/mapper/vg00-usr /mnt/usr

    rescue:~# mount /dev/mapper/vg00-home /mnt/home

  • Applies to Debian 13, AlmaLinux 9 and 10, and Rocky Linux 9 and 10: To mount the system directories for the chroot environment, enter the following command:

    rescue:~# for i in /dev /dev/pts /proc /sys /run; do mount -B $i /mnt$i; done

  • Switch to the chroot environment:

    rescue:~# chroot /mnt

  • Install the bootloader. Example:

    Debian (11/12/13) and Ubuntu (24.04/26.04)

    rescue:/# grub-install --efi-directory=/boot/efi /dev/sda

    AlmaLinux 9 and 10, and Rocky Linux 9 and 10

    First, try restoring via dnf.

    rescue:/# dnf reinstall shim-x64 grub2-efi-x64

    rescue:/# grub2-mkconfig -o /boot/grub2/grub.cfg

    If this command fails, use grub2-install.

    rescue:/# grub2-install --efi-directory=/boot/efi /dev/sda

    rescue:/# grub2-mkconfig -o /boot/grub2/grub.cfg

  • Applies only to Debian (11/12/13) and Ubuntu (24.04/ 26.04): To update the configuration, enter the following command:

    rescue:/# update-grub

  • Exit the chroot environment by typing exit.

    rescue:/# exit

  • Unmount all volumes:

    rescue:~# umount -R /mnt

    Wait until the rebuild process is complete and reboot the server into the normal system.

  • Log in again after the reboot.

  • To set up the /dev/sda3 partition as a Linux swap area, enter the following command:

    [root@host ~]# mkswap /dev/sda3

Please Note

Using mkswap will change the partition’s UUID. If the swap partition is mounted via its UUID in your /etc/fstab (the default for AlmaLinux/Rocky/Debian), you must edit /etc/fstab after restarting and enter the new UUID, otherwise the swap space will not be activated automatically. If your server’s swap partition is not mounted using its UUID, you simply need to enable the swap partition. In this case, you can skip editing the fstab file.

  • To find the UUID, enter the following command:

    [root@host ~]# lsblk -f | grep sda3

  • This only applies if the swap partition is mounted via UUID: To edit /etc/fstab, enter the following command:

    [root@host ~]# vi /etc/fstab

Note

The vi editor has an insert mode and a command mode. You can enter insert mode by pressing the [i] key. In this mode, the characters you type are inserted into the text immediately. To switch to command mode, press [ESC] afterwards. When you are in command mode, your keystrokes are interpreted as commands.

  • Navigate to the line containing "swap" and replace the old UUID with the new UUID in the configuration file at /etc/fstab.
  • To exit vi and save the file, enter the command :wq and then press Enter.

  • Finally, activate the swap partition with the following command:

    [root@host ~]# swapon -a

  • To check whether the swap space is active, enter the following command:

    [root@host ~]# cat /proc/swaps

Drive 2 (sdb) has been replaced

If data volume 2 (sdb) has been replaced, complete the following:

  • Restart the system so that disk 2 (sdb) is recognised.

  • On the local system, copy the partition tables to the new (empty) disk. You may need to use the --force option

    [root@host ~]# sfdisk -d /dev/sda | sfdisk --force /dev/sdb

  • Add the partitions to the RAID. Example:

    RAID 1

    [root@host ~]# mdadm /dev/md1 -a /dev/sdb1

    [root@host ~]# mdadm /dev/md2 -a /dev/sdb2

    [root@host ~]# mdadm /dev/md4 -a /dev/sdb4

  • To monitor the rebuild, enter the following command:

    [root@host ~]# cat /proc/mdstat

  • To ensure the server boots even if sda fails in future, the bootloader must also be written to sdb. To do this, install the bootloader:

    Debian (11/12/13) and Ubuntu (24.04/ 26.04)

    [root@host ~]# grub-install --efi-directory=/boot/efi /dev/sdb

    AlmaLinux 9 and 10, and Rocky Linux 9 and 10

    First, try restoring it via dnf:

    [root@host ~]# dnf reinstall shim-x64 grub2-efi-x64

    [root@host ~]# grub2-mkconfig -o /boot/grub2/grub.cfg

    If this command fails, use the following command:

    [root@host ~]# grub2-install --efi-directory=/boot/efi /dev/sdb

    [root@host ~]# grub2-mkconfig -o /boot/grub2/grub.cfg

  • To update the configuration, enter the following command:

    Debian (11/12/13) and Ubuntu (24.04/ 26.04)

    [root@host ~]# update-grub

    AlmaLinux 9 and 10, and Rocky Linux 9 and 10

    [root@host ~]# grub2-mkconfig -o /boot/grub2/grub.cfg

  • Ensure that the swap partition is not on a RAID array. To do this, enter the following command:

    [root@host ~]# cat /proc/mdstat

  • To set up the partition as a Linux swap area, adapt the following command and then enter it. In this case, the Linux swap area is /dev/sdb3.

    [root@host ~]# mkswap /dev/sdb3

Please Note

Using mkswap will change the partition’s UUID. If the swap partition is mounted via UUID in your /etc/fstab (the default for Alma/Rocky/Debian), you must edit /etc/fstab after restarting and enter the new UUID, otherwise the swap area will not be activated automatically. If your server’s swap partition is not mounted via UUID, you simply need to enable the swap partition. In this case, you can skip editing the fstab file.

  • To find the UUID, adapt the following command and then enter it:

    [root@host ~]# lsblk -f | grep sdb3

  • This only applies if the swap partition is mounted via UUID: To edit /etc/fstab, enter the following command:

    [root@host ~]# vi /etc/fstab

  • Replace the UUID in the configuration in /etc/fstab.
  • To exit vi and save the file, enter the command :wq and then press Enter.

  • Activate the swap partition. To do this, enter the following command:

    [root@host ~]# swapon -a

  • To check whether the swap space is active, enter the following command:

    [root@host ~]# cat /proc/swaps

Restore RAID (RAID 5 or RAID 6)

If you are using a RAID 5 or RAID 6 array, it consists of at least three (RAID 5) or four (RAID 6) storage devices. If you replace a drive, you must ensure that the partition table is copied exactly from one of the remaining, healthy drives. The drive (sdc, sdd, sde, etc.) can be replaced while the system is running. No boot loader needs to be installed on these drives.

In this example, we assume that sdd has been replaced and that sdc and sdf are healthy.

  • Copy the layout from one of the healthy storage devices to the new storage device. To do this, adapt the following command and then enter it. Example:

    [root@host ~]# sfdisk -d /dev/sdc | sfdisk --force /dev/sdd

  • Add the new partition to the RAID array (e.g. md11). To do this, adapt the following command and then enter it.

    [root@host ~]# mdadm /dev/md11 -a /dev/sdd1

  • Rebuilding a RAID 5/6 array can take several hours if there is a large amount of data. Use this command to monitor the progress:

    [root@host ~]# watch -n 5 cat /proc/mdstat

Content