There are many different reasons to choose a network attached storage over a cloud based solution. Some people are concerned about data security and privacy. Some make decisions after calculating the cost. And for content creators, the most important reason is speed! The file sizes have been growing exponentially – E.g., minutes of 4K raw footage can easily occupy gigabytes of storage. This is the same for scientific researchers. Simulations have been playing more and more important roles and e.g. in my field finite element analysis of a large model can quickly generate huge amounts of data. In a cloud based solution, the file transfer speed is bottlenecked by both ISP (internet speed) and cloud service provider, both you have limited control over and cost a great fortune to upgrade. In contrast, for a home NAS, by choosing the proper network hardware (such as in this Multi-Gigabit Home Network Setup Tutorial), you can easily access much faster local network at a fair cost.

However, the downside of a home NAS solution is that compared with a professional cloud service provider, the data is more vulnerable to hard drive failures due to less reliable consumer grade hardware and lack of frequent routine backups. To protect valuable data, RAID is a very popular technology to offer data redundancy automatically – which means data can be still intact even after some of the drives fail. RAID can be implemented with either hardware (RAID controller) or software. This article introduces the setup of software based RAID setup on Linux.

Purchase Hard Drives

First we need to prepare the hard drives for RAID. Mechanical drives (HDDs) are typically used due to their large capacity over low prices. One thing worth noting is that recently there have been many discussions around performance differences between Shingled Magnetic Recording (SMR) and Perpendicular Magnetic Recording (PMR). In short, most hard drive manufacturers have been adopting SMR to reduce cost over the years. However, many reports show SMR has significant performance issues in heavy read/write scenarios due to its overwriting mechanism. As a result, SMR is not ideal for NAS application, and especially it’s not recommended to mix SMR and PMR drives in a RAID setup [1]. Before purchasing any hard drive, you should investigate whether it’s SMR or not. As the time of writing, WD has published the full list of drives using SMR. Seagate has also declared all its Ironwolf series (NAS) are not using SMR.

Prepare Disks

To partition the disks, here we use gdisk command (using GPT partition table). Type gdisk to enter an interactive environment for partitioning the disk (e.g. /dev/sda).

gdisk /dev/sda

Using gdisk is very straightforward:

  1. Use o to create a new GPT partition table. Note this will erase all data in the disk.
  2. Use n to create a new partition. Select the start/end sector to include the whole disk. Select fd00 (Linux RAID) for file system type.
  3. Use w to write changes.

After the above steps, an empty partition /dev/sda1 will be created. Similarly, prepare the other disks that will be included in RAID.

Create RAID

For most Linux distros, the tools for RAID management should have already been included, and you can always easily install them if it’s not the case. mdadm command is used to create a soft RAID device:

sudo mdadm --create /dev/md0 --level=1 --raid-devices=2 /dev/sda1 /dev/sdb1

The same mdadm command can also be used to check the RAID device information.

sudo mdadm /dev/md0

Adding --detail option allows you to get more detailed information.

sudo mdadm --detail /dev/md0

Or if you want to examine whether an actual device belongs to RAID, --examine option can be used. (Note the difference with --detail)

sudo mdadm --examine /dev/sda1

Now create a file system on the created RAID device. Here the XFS file system (default on CentOS) is used. Of course things like block sizes can be fine tuned based on your specific needs.

sudo mkfs.xfs /dev/md0

After mounting the RAID device to a folder, you can already access the space.

mkdir /mnt/raid-storage
mount /dev/md0 /mnt/raid-storage

To automatically mount the RAID device during startup, you can add the corresponding entry in /etc/fstab file.

UUID=[[RAID UUID]]    /mnt/raid-storage    xfs    defaults     0 0

In the first data column, I used Universally Unique IDentifier (UUID) instead of /dev/md0 to make sure the mounting is correct even if the hard drive device ID is changed. UUID can be found using the following command.

ls -l /dev/disk/by-uuid

File Permissions

Finally for a multi-user environment, I prefer to create a new user group nas-storage and make authorized users a member of the control group. Of course the file permission configuration strongly depend on your own use cases.

sudo groupadd -r nas-storage
sudo usermod -a -G nas-storage yeguang
sudo chgrp -R nas-storage /mnt/raid-storage
sudo chmod g+w /mnt/raid-storage

In the future article, I will further share some of my experiences in file server setup (e.g. Samba, NFS). Stay tuned.

References

  1. What are PMR and SMR hard disk drives?
  2. 直接使用 RAID 磁盘阵列

License

Creative Commons License This work is licensed under a Creative Commons Attribution 4.0 International License.