An eight-disk HP ProLiant array that had been running on borrowed time, unnoticed by anyone. A single failing disk is precisely what RAID 5 is meant to survive. Two is precisely what it can’t.
← All case files · from £500 + VAT
A company ProLiant had become unreachable. Its controller could no longer read sections of the array, HP’s own support had gone as far as flagging degraded drives before stopping, and on the server lived financial records, client databases and everyday operational files.
The catch is that a disk slowly accruing bad sectors shows nothing on the surface. Parity papers over the holes, files carry on being served, and for months all looks perfectly ordinary — while the array runs with its redundancy already gone and no one the wiser.
Then a second disk stumbles and everything stops dead. It looks like a bolt-from-the-blue catastrophe; it is anything but. It’s the second of two failures, and the first came long before.
Spread across eight members, those odds climb quickly. Each additional disk is one more chance for that quiet first failure to have already taken place.
the thing that had at last brought the controller down.
so the controller couldn’t settle on the correct configuration.
the discovery that made the job possible — with parity legible, missing blocks are computed back rather than guessed.
The decisive question here is never simply whether two disks have gone. It’s whether their bad sectors land in the same stripe. Where they don’t, every stripe still holds enough live members to rebuild from; where they do, that stripe is finished and nothing brings it back. Imaging settles which — inspection never can.
We imaged all eight members one by one and read-only, the two failing drives going onto DeepSpar imagers so we could wring the most from disks on their last legs while the sound ones cloned as normal. Not a thing was rebuilt on the original disks — the whole reconstruction ran off the clones. From those images we recovered the array’s true RAID 5 geometry (stripe size, disk order, parity rotation), confirmed the two disks’ bad sectors sat in different stripes so parity could bridge every gap, and rebuilt the array virtually. Parity handled the remainder, supplying what the weak disks couldn’t.
Parity held firm, the array was rebuilt off the clones, and the databases, financial records and operational files were all returned.
What decides these jobs isn’t how many disks are degraded — it’s whether the damaged areas overlap. Two failing disks whose bad sectors land in different stripes give a recoverable array; two whose bad sectors coincide give a partial recovery at best, and no method, tool or budget alters that. Anyone quoting you a percentage before every member is imaged is making it up.
Watch your array, and act the instant it raises a flag. A degraded RAID 5 isn’t a job for next quarter — it’s an array stripped of its redundancy, a single bad sector from becoming a forensic case.
And never rebuild onto a replacement disk while another member is already shaky. A rebuild reads every sector off every surviving disk — the hardest they’ve ever been pushed, and they’re the same age, same batch and same running hours as the one that just went. That’s exactly why a second disk so often dies partway through a rebuild.
Image first; rebuild afterwards, from the images, with the originals kept out of harm’s way.
Bring the drive to our Guildford Business Park reception, or post it over — it costs nothing to learn what went wrong. You’ll have a written figure from the fixed bands before any work begins.