| Author |
Message |
Triffid
Guest
|
 Posted:
Sun Nov 21, 2004 6:33 am Post subject:
Damp RAID array |
|
|
I had rather a rude awakening this morning - washer supply hose burst
overnight and sprayed a few thousand gallons of water around the
basement. Fortunately the floor drain was able to cope, so flooding was
minimal, but most everything in the basement got damp from spray.
When I arrived on the scene, the file server RAID controller's audible
alarm (Adaptec 3400S) was screeching loudly. I pulled the power cord to
shut it up while I dealt with more urgent matters.
Later I opened the file server case, but could find no evidence of water
inside, so I powered up. The system booted normally, but the RAID 5
array status was 'degraded', with one of 5 disks marked 'failed' and the
other 4 'optimal'. I removed the failed disk, connected it to a 2940U2W
in another system, and ran 'Verify Disk Media'. This completed with no
errors reported, so I put the disk back in the file server and told the
RAID controller to rebuild the array. The rebuild completed
successfully, and the array and all 5 disks are now marked 'optimal'.
What happened here? Is this episode over, or should I be concerned about
future problems?
TIA |
|
| Back to top |
|
 |
Dorothy Bradbury
Guest
|
| Posted:
Mon Nov 22, 2004 2:03 am Post subject:
Re: Damp RAID array |
|
|
| Quote: | Later I opened the file server case, but could find no evidence of water inside, so I
powered up.
|
So we have...
o Humidity in the environment substantially elevated
---- airflow is drawn through the enclosure
o No direct water evidence inside the enclosure
---- so low probability of condensing humidity or water ingress
Hard drives have a breather hole...
o It exists for pressure equalisation either side of its air filter
o Moisture laden air is thus not directly drawn into the drive
PCs care about humidity within two guidelines...
o Extremely low - risk of static buildup from airflow
---- not a risk in this case, or most usage environments
o Extremely high - risk of condensing if metal temp below dew-point
---- possible in this case, but no evidence of water inside
Outside of powered up component temperatures, the case as a whole
is at a case-ambient somewhat above room-ambient. Case ambient &
components unless contacted by water will be above the dew-point.
I would not assume the RAID error was due to water directly,
but there is still the possibility of indirect effect - eg, power etc.
So I would consider the drive suspicious until proven otherwise.
o Hard drives do fail - otherwise we wouldn't use RAID :-)
o RAID 5 is not infallible - lose parity - ok, lose parity & data - stuffed
Check your logs carefully for any I/O errors before & after the event.
PCs are relatively tolerant of horrible environments - you can have
standing water on motherboards and whilst it will degrade the epoxy
they will continue to operate ok. Surface mount & ball grid array solder
balls can be corroded & affected by contaminants in the water too.
Check your cabling and water ingress impact on electrical feeds/skts.
SMPS primary-side are >300V DC and definately do not like water :-)
Well worth replacing those hoses every 6-8yrs - they do eventually fail.
o They degrade from temperature & from mechanical vibration / stress
o They also degrade on the *inside*, unseen, often the cold side
Water failure alarms are available - not a bad idea for utility basements :-)
--
Dorothy Bradbury
www.dorothybradbury.co.uk for quiet Panaflo fans |
|
| Back to top |
|
|
Triffid
Guest
|
| Posted:
Mon Nov 22, 2004 5:12 am Post subject:
Re: Damp RAID array |
|
|
Dorothy Bradbury wrote:
| Quote: | Later I opened the file server case, but could find no evidence of water inside, so I
powered up.
So we have...
o Humidity in the environment substantially elevated
---- airflow is drawn through the enclosure
o No direct water evidence inside the enclosure
---- so low probability of condensing humidity or water ingress
Hard drives have a breather hole...
o It exists for pressure equalisation either side of its air filter
o Moisture laden air is thus not directly drawn into the drive
PCs care about humidity within two guidelines...
o Extremely low - risk of static buildup from airflow
---- not a risk in this case, or most usage environments
o Extremely high - risk of condensing if metal temp below dew-point
---- possible in this case, but no evidence of water inside
Outside of powered up component temperatures, the case as a whole
is at a case-ambient somewhat above room-ambient. Case ambient &
components unless contacted by water will be above the dew-point.
I would not assume the RAID error was due to water directly,
|
I agree - that's why I posted the query.
| Quote: | but there is still the possibility of indirect effect - eg, power etc.
So I would consider the drive suspicious until proven otherwise.
o Hard drives do fail - otherwise we wouldn't use RAID :-)
o RAID 5 is not infallible - lose parity - ok, lose parity & data - stuffed
Check your logs carefully for any I/O errors before & after the event.
|
I found no indication of any I/O problems prior to the event, after
going back several weeks in the logs. Just completed a full backup to
tape, and tape verification. This will have exercised the disks, but did
not generate any I/O errors. However, I will be keeping an eye on it.
| Quote: | PCs are relatively tolerant of horrible environments - you can have
standing water on motherboards and whilst it will degrade the epoxy
they will continue to operate ok. Surface mount & ball grid array solder
balls can be corroded & affected by contaminants in the water too.
Check your cabling and water ingress impact on electrical feeds/skts.
|
The washer and dryer outlet boxes had water in them, but there's no
evidence it reached other outlets. The file server is now on a stand to
raise it and associated cabling well off the floor :-)
| Quote: | SMPS primary-side are >300V DC and definately do not like water :-)
Well worth replacing those hoses every 6-8yrs - they do eventually fail.
o They degrade from temperature & from mechanical vibration / stress
o They also degrade on the *inside*, unseen, often the cold side
|
No idea how old those hoses were, probably at least 10 years. The
failure occured on the hot side, but the cold shows evidence of swelling
also. No excuses, I should have replaced them prior to failure - I'm
just glad I cleaned the floor drain grate recently! The new hoses are
reinforced with stainless steel braid and appear to be of much better
quality.
| Quote: | Water failure alarms are available - not a bad idea for utility basements :-)
|
I will see what's available locally - I would prefer to know about the
problem *before* it sets off the RAID controller alarm!
Thanks for your input. |
|
| Back to top |
|
|
|
|
|
|
FAQ
Memberlist
Register
Profile
Log in to check your private messages
Log in