Technical Note: Hard Drive Replacement Procedure for BFN 310 NS500 on Suse Linux

[FortiKoala]

Description

Hard Drive Replacement Procedure for BFN 310 NS500 on Suse Linux

Solution

The following procedure should be followed when the hard drive in slot 0 is corrupted or marked for imminent failure.

Hard Drive failure is indicative by

1.) Seeing a message in console similar to "SCSI 0100 Rejecting IO to Dead Device"

2.) The Hard Drive LED will change from green and then flash amber and then shut off. 

The BFN 310 has 2 hard drives one in Slot 0 and one in slot 1 

The system employs raid 1 and is capable of self repair. 

If the hard drive in Slot 0 becomes unstable, corrupted, or failed use the following instructions to replace the hard drive. 

1.) Leave the server up and running

2.) Remove the failed primary drive (in slot 0)

3.) Replace the hard drive (in slot 0) with a new hard drive of the same type and size. 

4.) The RAID process will begin re-building and re-syncing the primary drive (Slot 0)  from the secondary drive (Slot 1)

The rebuild process can take several hours.

To confirm success cat the messages file and you should see messages similar to the messages below. 

Network Sentry NS500

root@campusmanager:/bsc/logs

> cat messages

Mar 21 12:54:47 campusmanager kernel: mptbase: ioc0: LogInfo(0x31110d00): Originator={PL}, Code={Reset}, SubCode(0x0d00) cb_idx mptscsih_io_done

Mar 21 12:54:55 campusmanager last message repeated 55 times

Mar 21 12:54:55 campusmanager kernel: mptbase: ioc0: RAID STATUS CHANGE for PhysDisk 0 id=9

Mar 21 12:54:55 campusmanager kernel: mptbase: ioc0:   PhysDisk is now missing

Mar 21 12:54:55 campusmanager kernel: mptbase: ioc0: RAID STATUS CHANGE for PhysDisk 0 id=9

Mar 21 12:54:55 campusmanager kernel: mptbase: ioc0:   PhysDisk is now missing, out of sync

Mar 21 12:54:55 campusmanager kernel: mptbase: ioc0: RAID STATUS CHANGE for VolumeID 0

Mar 21 12:54:55 campusmanager kernel: mptbase: ioc0:   volume is now degraded, enabled

Mar 21 12:54:55 campusmanager kernel:  target0:0:0: mptsas: ioc0: delete device: fw_channel 0, fw_id 9, phy 0, sas_addr 0x1221000000000000

Mar 21 12:56:02 campusmanager kernel: mptbase: ioc0: RAID STATUS CHANGE for PhysDisk 0 id=0

Mar 21 12:56:02 campusmanager kernel: mptbase: ioc0:   PhysDisk is now online, out of sync

Mar 21 12:56:02 campusmanager kernel: mptsas: ioc0: WARNING - firmware bug: unable to add hidden disk - target_id matchs volume_id

Mar 21 12:56:02 campusmanager kernel: mptbase: ioc0: RAID STATUS CHANGE for VolumeID 0

Mar 21 12:56:02 campusmanager kernel: mptbase: ioc0:   volume is now degraded, enabled, resync in progress

Mar 21 15:57:57 campusmanager kernel: mptbase: ioc0: RAID STATUS CHANGE for PhysDisk 0 id=9

Mar 21 15:57:57 campusmanager kernel: mptbase: ioc0:   PhysDisk is now online

Mar 21 15:57:57 campusmanager kernel: mptbase: ioc0: RAID STATUS CHANGE for VolumeID 0

Mar 21 15:57:57 campusmanager kernel: mptbase: ioc0:   volume is now optimal, enabled, resync in progress

Mar 21 15:57:57 campusmanager kernel: mptbase: ioc0: RAID STATUS CHANGE for VolumeID 0

Mar 21 15:57:57 campusmanager kernel: mptbase: ioc0:   volume is now optimal, enabled

Mar 21 15:57:57 campusmanager kernel: mptsas: ioc0: attaching sata device: fw_channel 0, fw_id 9, phy 0, sas_addr 0x1221000000000000

Mar 21 15:57:57 campusmanager kernel:   Vendor: ATA       Model: WDC WD5003ABYX-1  Rev: 1S05

Mar 21 15:57:57 campusmanager kernel:   Type:   Direct-Access                      ANSI SCSI revision: 05

Mar 21 15:57:58 campusmanager kernel: scsi 0:0:2:0: Attached scsi generic sg0 type 0