Comments on: RAID Capacity Expansion, the Suboptimal Way, or How Not to Have a Fun Weekend http://whats.all.this.brouhaha.com/2007/11/19/raid-capacity-expansion-the-suboptimal-way-or-how-not-to-have-a-fun-weekend/ random rantings Thu, 08 Jan 2009 03:32:14 +0000 http://wordpress.org/?v=2.6 By: Eric http://whats.all.this.brouhaha.com/2007/11/19/raid-capacity-expansion-the-suboptimal-way-or-how-not-to-have-a-fun-weekend/#comment-4009 Eric Sun, 18 May 2008 21:39:04 +0000 http://whats.all.this.brouhaha.com/?p=608#comment-4009 The source code for r5test is now available at http://www.brouhaha.com/~eric/software/r5test/ The source code for r5test is now available at
http://www.brouhaha.com/~eric/software/r5test/

]]> By: netproteus http://whats.all.this.brouhaha.com/2007/11/19/raid-capacity-expansion-the-suboptimal-way-or-how-not-to-have-a-fun-weekend/#comment-4003 netproteus Wed, 14 May 2008 15:40:42 +0000 http://whats.all.this.brouhaha.com/?p=608#comment-4003 I've got a similar problem. A disk has completely failed and I've manged to screw up the DCB on one of my good drives. 3ware are being a bit rubbish and generally very slow. A copy of your r5test program as a starting point to manually recover the data would be a real help. I’ve got a similar problem. A disk has completely failed and I’ve manged to screw up the DCB on one of my good drives.

3ware are being a bit rubbish and generally very slow. A copy of your r5test program as a starting point to manually recover the data would be a real help.

]]> By: Eric http://whats.all.this.brouhaha.com/2007/11/19/raid-capacity-expansion-the-suboptimal-way-or-how-not-to-have-a-fun-weekend/#comment-3931 Eric Thu, 29 Nov 2007 08:25:13 +0000 http://whats.all.this.brouhaha.com/?p=608#comment-3931 As it currently exists, r5test assumes that the drives all contain valid data. It doesn't try to do reconstruction from a failed drive. I expect that it wouldn't be too difficult to add that feature. However, dealing with two failed drives is in general not possible, so you'll have to hope that the second failed drive hasn't actually lost your data. What I'd do first, regardless of whether you want to get help from 3ware or hack on r5test, is try to make an exact block-for-block copy of the second failed drive. With Linux/BSD/Solaris, etc, it is trivial to do this with dd. (I'd probably do that for all the drives.) If you want to try hacking r5test, such that it is, send me email, and I'll send you the code. Note that you'll still have to determine somehow what order the 12 drives had within the array, and that seems pretty challenging without knowing the details of the 3ware DCB format. Fortunately since you didn't do anything dumb like I did, the DCBs on all the drives should be intact. I think 3ware support is much more promising at this point that r5test. As it currently exists, r5test assumes that the drives all contain valid data. It doesn’t try to do reconstruction from a failed drive. I expect that it wouldn’t be too difficult to add that feature. However, dealing with two failed drives is in general not possible, so you’ll have to hope that the second failed drive hasn’t actually lost your data.

What I’d do first, regardless of whether you want to get help from 3ware or hack on r5test, is try to make an exact block-for-block copy of the second failed drive. With Linux/BSD/Solaris, etc, it is trivial to do this with dd. (I’d probably do that for all the drives.)

If you want to try hacking r5test, such that it is, send me email, and I’ll send you the code. Note that you’ll still have to determine somehow what order the 12 drives had within the array, and that seems pretty challenging without knowing the details of the 3ware DCB format. Fortunately since you didn’t do anything dumb like I did, the DCBs on all the drives should be intact. I think 3ware support is much more promising at this point that r5test.

]]> By: Vince http://whats.all.this.brouhaha.com/2007/11/19/raid-capacity-expansion-the-suboptimal-way-or-how-not-to-have-a-fun-weekend/#comment-3930 Vince Thu, 29 Nov 2007 07:57:40 +0000 http://whats.all.this.brouhaha.com/?p=608#comment-3930 Nice scary horror story! And good hack to bring the data back to life :-) Are your plans to release r5test out just theoretical at the moment? I'd actually be happy to put my hands on r5test; i just had bad luck with a 12-drive raid5 array on a 3ware controller and i might have to use your trick to recover the data. (In my case i removed a failing drive from the array, shut down the power, swapped the drive with a new one, powered back up and in the process a 2nd drive decided to disappear... I'm hoping 3ware will come up with scripts to reinject the failing-but-not-quite-dead drive into the array but if not, r5test will be my friend.) Thanks! Nice scary horror story! And good hack to bring the data back to life :-)

Are your plans to release r5test out just theoretical at the moment? I’d actually be happy to put my hands on r5test; i just had bad luck with a 12-drive raid5 array on a 3ware controller and i might have to use your trick to recover the data. (In my case i removed a failing drive from the array, shut down the power, swapped the drive with a new one, powered back up and in the process a 2nd drive decided to disappear… I’m hoping 3ware will come up with scripts to reinject the failing-but-not-quite-dead drive into the array but if not, r5test will be my friend.)

Thanks!

]]>