Welcome to Chucktips Hardware HELP WITH DEAD OS Installing FreeBSD Miscellaneous
 add article

HOWTO: Create RAID1 on FreeBSD with gmirror
Hardware Posted by Jason on Monday April 02, @03:21PM
from the The Dept of redundancy dept... dept.
With the low cost of hard drives these days, and easy to use programs like gmirror, there's no reason not to protect the data on you're FreeBSD system with RAID. In this post I explain how to setup a simple, software based, RAID1 fault tolerant system with FreeBSD-6.2 and the gmirror utility. It's dead simple by the way.

Here are the steps for setting up RAID-1 with gmirror. I am assuming that you have two identical IDE hard drives setup in a master/slave configuration on ad0 and ad1 and that you have FreeBSD-6.2-RELEASE installed on drive ad0. It's important to note that if you're using SATA or SCSI drives then your device names will differ from my example.

  Step 1: Use sysctl to allow the mounted disk to be modified
  Step 2: Create the mirror on ad0 with gmirror
  Step 3: Modify /boot/loader.conf to load gmirror on startup
  Step 4: Modify /etc/fstab for gmirror device names
  Step 6: Use gmirror insert command to add the 2nd drive for mirroring
  Step 7: Check the mirroring status with gmirror status

[Allow the mounted disk (ad0) to be modified by gmirror]

This step allows the gmirror utility to modify your drive while it is mounted and is a one time operation during gmirror setup. Login in as the root user and issue the sysctl command:

Login: root
Password: *******

# sysctl kern.geom.debugflags=16

[Create the mirror on ad0 with gmirror]

This command modifies the drive label metadata on your hard disk so that it is recognized as a gmirror RAID disk. I'm labeling the raid set gm0 but you can give it any label you like as long as you're consistant. It's important to note that this label name becomes your new device file name used in /etc/fstab.

# gmirror label -v -b round-robin gm0 /dev/ad0
Metadata value stored on /dev/ad0

[Modify /boot/loader.conf to load gmirror on startup]

Using your favorite editor, ee or vi for example, modify your loader.conf file so that the gmirror module automatically loads at startup.

# ee /boot/loader.conf

Add the following text to the /boot/loader.conf file and save it:

[Modify /etc/fstab for gmirror device names]

In this step you are modifying the /etc/fstab file so that it will mount the gmirror device gm0 rather than ad0 that the system was previously using. Here is what your old fstab should look like:

# old /etc/fstab device file
/dev/ad0s1b        none ...
/dev/ad0s1a        / ...
/dev/ad0s1e        /tmp ...
/dev/ad0s1f        /usr ...
/dev/ad0s1d        /var ...

Again, use your favorite editor, ee or vi to modify the /etc/fstab file. Here is what your modified /etc/fstab file should look like. Notice that I have added the subdirectory, "mirror" to the path and replaced ad0 with gm0 for all the device names (including the swap file)

# newly modified /etc/fstab device file
/dev/mirror/gm0s1b        none ...
/dev/mirror/gm0s1a        / ...
/dev/mirror/gm0s1e        /tmp ...
/dev/mirror/gm0s1f        /usr ...
/dev/mirror/gm0s1d        /var ...

That's all there is to the gmirror configuration! Now we just need to reboot, let the system come up, and add our ad1 drive to the system for mirroring using the the gmirror insert command. A word of caution- Double check your work. If you mis-typed ANYTHING in the /boot/loder.conf or /etc/fstab file then the system will NOT come up!

[Use the gmirror insert command to add the 2nd drive for mirroring]

Assuming the system comes up fine, and I'm sure it has (grin), then all we have to do to complete our project is tell gmirror to mirror our drives together. We will use the gmirror insert command to do this. We will insert our second hard disk (ad1) into our gm0 RAID. Here is the command to do it.

# gmirror insert gm0 /dev/ad1
GEOM_MIRROR: Device gm0: provider ad1 detected
GEOM_MIRROR: Device gm0: rebuilding provider ad1

The systm will now be busy for awhile mirroring the two drives. The time it takes to complete the task depends on your system hardware and drive sizes. However, you can check the status of your rebuild with the gmirror status command.

[Check the mirroring status with gmirror status]

# gmirror status
Name           Status    Components
mirror/gm0    DEGRADED    ad0
                                                   ad1 (18%)

Note that the system has detected the array to be "DEGRADED" and is automatically rebuilding the drive array. Also note that it is 18% complete. When it reaches 100% the gmirror status reads "COMPLETE" and you're done! Isn't it crazy how simple this is?

My observations when recovering a RAID array

Here's what I did to test the gmirror RAID system. First, I installed FreeBSD and created a mirror set with gmirror (using two IDE drives). After the RAID system was up and running, I simulated a drive failure by shutting the system down and removing the IDE master drive (ad0) from the system. Then I re-installed the IDE slave drive (ad1) as the primary master and rebooted the system. To my surprise and glee the system came right up with no errors and all data intact!!

After my FreeBSD system came up fine, I used another PC to wipe the old ad0 master hard disk. I then placed it back in the system, as a slave, and rebooted. A gmirror insert command returned GEOM_MIRROR: Cannot add disk ad1 to gm0 (error=17) so I tried the command: gmirror insert gm0 /dev/ad1 in an attempt to get the drive back into the mirror. Unfortunately I received another error so I re-read the gmirror man page for more info. I decided to disconnect the second drive (the slave),reboot, login and issue the gmirror forget command:

gmirror forget gm0 /dev/ad1

The system returned an error about not finding the device /dev/ad1. I shutdown, connected the drive again and rebooted. I issued the same command again:

gmirror forget gm0 /dev/ad1

    and then the

gmirror insert gm0 /dev/ad1

command again. This time it inserted the drive and began rebuilding the array. Hurray!

As always, please refer to the man pages for help. The man pages for commands used in this article are:

man gmirror
man vi
man ee
man loader.conf
man fstab
man sysctl

syslog not logging local | FreeNAS is a cool FreebSD based NAS solution!  >


Related Links
  • Articles on Hardware
  • Also by Jason
  • Contact author

    Lilliput Mini USB Computer Monitor

    If you have ever wanted to learn about Cisco Routers, check out my new book "Cisco Routers for the Small Business" - it's Cisco CLI for the Regular Guy!

  • Sponsors

    The Fine Print: The following comments are owned by whoever posted them.
    ( Add a Reply )

    Re: HOWTO: Create RAID1 on FreeBSD with gmirror
    by Andrew Greenwood on Saturday April 11, @02:23AM
    Good article - I tried gmirror out for myself whilst reading it, inside a VM, with success (excluding the fact my kernel seemed to crash on boot a couple of times - I'm assuming this is down to VirtualBox until I try on real hardware.) I suspect the reason gmirror complained when you tried to re-insert the disk may have been due to data being written to an area that didn't get erased? So it might find some kind of "magic data" to indicate that its a member of a gmirror for example. When I tested this in VirtualBox I shutdown, removed the slave hard disk, booted up, and was in a working system (albeit with one hard drive missing.) I shut down again, deleted the virtual disk completely and created a new one, booted up and was able to "gmirror insert gm0 /dev/ad1" without any trouble. ...Which I guess effectively fakes the real-life scenario of a disk dying and being completely replaced.
    [ Add a Reply to this ]
    The Fine Print: The following comments are owned by whoever posted them.
    ( Article Reply )

    "You never know how many friends you have until you own a Condo on the beach." -- Jason's Postulate

    Powered by Zope  Powered by Apache  Squishdot Powered
    All trademarks and copyrights on this page are owned by their respective companies. Comments are owned by the Poster. The Rest ©2001 Jason Neumann.
    [ main | post article | search ]