I have a WD USB 3.0 500gb hard drive for my work laptop who’s sole purpose in life is to record back-ups of my Ubuntu 12.04 system. I’ve never formatted the device and it’s always worked very well for me despite my cavalier treatment: tossing it around in my backpack during my commute mostly.
I use a program called back-in-time for my backups and I like it for it’s ease-of-use coupled with a healthy sense of fire-and-forget.
Recently, my back-ups started failing with generic write-fail errors so I committed to re-formatting the device. Since I was going to nuke it, I decided to implement RAID-1 during it’s resurrection just to have an “extra” copy of my back-ups. The partition I back-up is only 100gb — which is currently about 55% full — so splitting the device into two logical drives and configuring under RAID shouldn’t be a problem, right?
I mean, I have absolutely NO experience with RAID so how hard can this be?
After a few false starts, I honed down the process and this is what I’m sharing with you today. Oh, and as side note — in the first paragraph, I linked to this particular drive on Amazon as a courtesy — I’m not getting and referrals or anything like that…just thought you might want to see what we’re dealing with.
Phase 1 – Installation of Tools
Ok – so, to get started, I needed to install mdadm which was pretty easy:
What was somewhat confusing was the need for mdadm to install a mailing agent – for the monitoring tool, but during the installation I elected to not configure the mail program and everything still went smoothly.
Next, I installed gparted and all of the available options through the Ubuntu Software Center app.
I already had back-in-time installed and I provided the link above. Use the Ubuntu Software Center to install this app if you don’t already have it.
With all the software in-place, it’s time to configure the device.
Part 2 – Wiping and Configuring the Device
Plug your device into your system and wait for it to automount — which it will do if it still has the factory-equipped partition installed. Once it’s installed, go to your command line and unmount the device:
Either sudo this command, or sudo over to root. I chose the latter as I get tired of forgetting to sudo.
Once the device is unmounted, start-up gparted,, authenticate, and wait for gparted to stop scanning for devices. Change your device over to your external drive which was, for me: /dev/sdb.
(Side note: The WD drive comes with it’s own software which just swell if you’re a Windows or a Mac user. Otherwise, the software is just taking up space. Copy these programs if you want but you can always download replacements from the WD website.)
For me, the drive was formatted into one single, large, partition. Delete this partition and don’t look back.
Next, split your partition into two smaller partitions, equally dividing the available space between the two. RAID-1 is mirroring – you’re creating two logical partitions in this device but you will mount it (the system will see it) as a single device. Data is written to the first partition and RAID copies the data over to the second partition, mirroring the data to the second logical partition.
I chose the ext3 filesystem for my partition simply because:
- it’s better than ext2
- it’s robust
- I don’t believe I’ll get any speed benefits from ext4 (USB, eh?)
Note too that you’re not actually doing anything at this point – you’re just building a task-list for gparted to execute when you’re finished creating tasks.
Next, flag each partition as a RAID partition by right-clicking -> Manage Flags -> Raid.
Once this is done, you’re ready to execute your task list which will partition and flag the devices. When the task list completes successfully, you can quit gparted.
Step 3 – MDADM
Next, you need to create the software RAID volume using the mdadm tool.
This is very easy and is done with a single command:
What this command does is:
- invokes mdadm with the –create option (useful for creating RAID arrays)
- specifies the “verbose” flag so you can get meaningful diagnostics should something head south
- specifies the mount-point device (/dev/md0) for your RAID
- specifies your RAID level (–level-1) (remember, 1 = mirroring)
- specifies the number of devices in your RAID (–raid-devices=2)
- lists the device links for the logical drives to be used in the RAID (/dev/sdb1, /dev/sdb2)
# mdadm --create --verbose <strong>/dev/md0</strong> --level=1 --raid-devices=2 /dev/sdb1 /dev/sdb2
mdadm: /dev/sdb1 appears to contain an ext2fs file system
size=244174848K mtime=Wed Dec 31 16:00:00 1969
mdadm: Note: this array has metadata at the start and
may not be suitable as a boot device. If you plan to
store '/boot' on this device please ensure that
your boot-loader understands md/v1.x metadata, or use
mdadm: /dev/sdb2 appears to contain an ext2fs file system
size=244177920K mtime=Wed Dec 31 16:00:00 1969
mdadm: size set to 244173688K
Continue creating array? yes
mdadm: Defaulting to version 1.2 metadata
mdadm: array /dev/md0 started.
Press enter to execute the command and you should get the prompt back after a few seconds. Note that the program requires no input from you.
Step 4 – Making and Mounting
This is the step that most of the online and available pages on mdadm leave out.
You still need to format your filesystem!
So, let’s do this with mkfs…
mke2fs 1.42 (29-Nov-2011)
OS type: Linux
Block size=4096 (log=2)
Fragment size=4096 (log=2)
Stride=0 blocks, Stripe width=0 blocks
15261696 inodes, 61043422 blocks
3052171 blocks (5.00%) reserved for the super user
First data block=0
Maximum filesystem blocks=4294967296
1863 block groups
32768 blocks per group, 32768 fragments per group
8192 inodes per group
Superblock backups stored on blocks:
32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208,
4096000, 7962624, 11239424, 20480000, 23887872
Allocating group tables: done
Writing inode tables: done
Creating journal (32768 blocks): done
Writing superblocks and filesystem accounting information: done
Note that I use the RAID mount-point address of /dev/md0 to create the filesystem.
This will take a few minutes to run so go take a stretch break and play with the dog or annoy your hot receptionist or something.
Once this step completes, you’re pretty much done — all that’s left is mounting the device and using it.
<do stuff like start a back-up>
# df -h
Filesystem Size Used Avail Use% Mounted on
/dev/mapper/PAMCAKES-root 84G 9.4G 70G 12% /
udev 3.9G 4.0K 3.9G 1% /dev
tmpfs 1.6G 920K 1.6G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 3.9G 49M 3.9G 2% /run/shm
/dev/sda2 242M 159M 71M 70% /boot
/dev/mapper/PAMCAKES-user 93G 48G 42G 54% /home
/dev/md0 230G 22G 196G 11% /media/raid
Note that your available filesystem space is approximate one-half the capacity of the drive.
That’s right – you paid a whopping $89 for a 500gb device out of which you can use about 230gb.
But you have two of them. So if your first partition fails, you’ve got a back-up of your back-up thanks to the wondrous magic of RAID. And, you learned how to set-up a RAID device, so, win.
Unless one of your kids (or grandkids) decides that your sleek, new, external device looks better in the fish tank than it does on your desk, you have an additional and available option for recovering lost data.
Keep in mind that this is a software solution — hardware RAID is still the preferred way to go when dealing with issues of redundant data storage. But this works, too.
Step 5 – Maintenance
To stop your RAID device use this command:
# mdadm –stop /dev/md0
To see the state of your RAID, cat /proc/mdstat to your terminal:
Personalities : [raid0] [raid1]
md0 : active raid1 sdb2 sdb1
244173688 blocks super 1.2 [2/2] [UU]
[>....................] resync = 1.8% (4474560/244173688) finish=4948.7min speed=806K/sec
unused devices: <none>
To get details about your RAID, use mdadm:
Version : 1.2
Creation Time : Tue Jul 24 14:41:55 2012
Raid Level : raid1
Array Size : 244173688 (232.86 GiB 250.03 GB)
Used Dev Size : 244173688 (232.86 GiB 250.03 GB)
Raid Devices : 2
Total Devices : 2
Persistence : Superblock is persistent
Update Time : Tue Jul 24 14:42:51 2012
State : active, resyncing
Active Devices : 2
Working Devices : 2
Failed Devices : 0
Spare Devices : 0
Resync Status : 2% complete
Name : pamcakes:0 (local to host pamcakes)
UUID : b542c27c:1d620c3e:6f07b9e8:53aee08d
Events : 1
Number Major Minor RaidDevice State
0 8 17 0 active sync /dev/sdb1
1 8 18 1 active sync /dev/sdb2
To uncouple the RAID device:
# mdadm –remove /dev/md0
That’s it for today…hope this helps…there’s a lot of good information already available on the web for using mdadm and software RAID – I just wanted to consolidate everything into a contiguous process.
Later, should sufficient motivation present itself, I’ll follow-up with alerts and what not…