There are many different reasons to choose a network attached storage over a cloud based solution. Some people are concerned about data security and privacy. Some make decisions after calculating the cost. And for content creators, the most important reason is speed! The file sizes have been growing exponentially – E.g., minutes of 4K raw footage can easily occupy gigabytes of storage. This is the same for scientific researchers. Simulations have been playing more and more important roles and e.g. in my field finite element analysis of a large model can quickly generate huge amounts of data. In a cloud based solution, the file transfer speed is bottlenecked by both ISP (internet speed) and cloud service provider, both you have limited control over and cost a great fortune to upgrade. In contrast, for a home NAS, by choosing the proper network hardware (such as in this Multi-Gigabit Home Network Setup Tutorial), you can easily access much faster local network at a fair cost.
However, the downside of a home NAS solution is that compared with a professional cloud service provider, the data is more vulnerable to hard drive failures due to less reliable consumer grade hardware and lack of frequent routine backups. To protect valuable data, RAID is a very popular technology to offer data redundancy automatically – which means data can be still intact even after some of the drives fail. RAID can be implemented with either hardware (RAID controller) or software. This article introduces the setup of software based RAID setup on Linux.
First we need to prepare the hard drives for RAID. Mechanical drives (HDDs) are typically used due to their large capacity over low prices. One thing worth noting is that recently there have been many discussions around performance differences between Shingled Magnetic Recording (SMR) and Perpendicular Magnetic Recording (PMR). In short, most hard drive manufacturers have been adopting SMR to reduce cost over the years. However, many reports show SMR has significant performance issues in heavy read/write scenarios due to its overwriting mechanism. As a result, SMR is not ideal for NAS application, and especially it’s not recommended to mix SMR and PMR drives in a RAID setup . Before purchasing any hard drive, you should investigate whether it’s SMR or not. As the time of writing, WD has published the full list of drives using SMR. Seagate has also declared all its Ironwolf series (NAS) are not using SMR.
To partition the disks, here we use
gdisk command (using GPT partition table). Type
gdisk to enter an interactive environment for partitioning the disk (e.g.
gdisk is very straightforward:
oto create a new GPT partition table. Note this will erase all data in the disk.
nto create a new partition. Select the start/end sector to include the whole disk. Select
fd00(Linux RAID) for file system type.
wto write changes.
After the above steps, an empty partition
/dev/sda1 will be created. Similarly, prepare the other disks that will be included in RAID.
For most Linux distros, the tools for RAID management should have already been included, and you can always easily install them if it’s not the case.
mdadm command is used to create a soft RAID device:
mdadm command can also be used to check the RAID device information.
--detail option allows you to get more detailed information.
Or if you want to examine whether an actual device belongs to RAID,
--examine option can be used. (Note the difference with
Now create a file system on the created RAID device. Here the XFS file system (default on CentOS) is used. Of course things like block sizes can be fine tuned based on your specific needs.
After mounting the RAID device to a folder, you can already access the space.
To automatically mount the RAID device during startup, you can add the corresponding entry in
In the first data column, I used Universally Unique IDentifier (UUID) instead of
/dev/md0 to make sure the mounting is correct even if the hard drive device ID is changed. UUID can be found using the following command.
Finally for a multi-user environment, I prefer to create a new user group
nas-storage and make authorized users a member of the control group. Of course the file permission configuration strongly depend on your own use cases.
In the future article, I will further share some of my experiences in file server setup (e.g. Samba, NFS). Stay tuned.