It came to mind how to check the raid. Below the key tests. I am not sure if the disk removed by the system (sda) has to be replaced with a new one or it could be forced into place, tried, reformatted, and what else should be better tried. If to be replaced, it would be the second time in a less than 2000 hours service, while there is a good UPS and the disks are mounted on Zahlman coolers with a strong fan directly on them.
Unfortunately I have a badly planned huge raid. If trying to reuse sda, is it correct to command directly: # sfdisk -d /dev/sdb > /tmp/partitions # sfdisk --force /dev/sda < /tmp/partitions # mdadm -add /dev/md0 /dev/sdb1 # mdadm -add /dev/md1 /dev/sdb2 # mdadm -add /dev/md2 /dev/sdb3 # mdadm -add /dev/md3 /dev/sdb5 # mdadm -add /dev/md4 /dev/sdb6 # mdadm -add /dev/md5 /dev/sdb7 # mdadm -add /dev/md6 /dev/sdb8 or should sda be first ext3 formatted? ========================================================================= cat /proc/mdstat Personalities : [raid1] md6 : active raid1 sdb8[1] 102341952 blocks [2/1] [_U] md5 : active raid1 sdb7[1] 1951744 blocks [2/1] [_U] md4 : active raid1 sdb6[1] 2931712 blocks [2/1] [_U] md3 : active raid1 sdb5[1] 14651136 blocks [2/1] [_U] md1 : active (auto-read-only) raid1 sdb2[1] 6835584 blocks [2/1] [_U] md0 : active raid1 sdb1[1] 2931712 blocks [2/1] [_U] md2 : active raid1 sdb3[1] 14651200 blocks [2/1] [_U] unused devices: <none> ======================================================================== mdadm --detail /dev/md0 /dev/md0: Version : 00.90 Creation Time : Wed Sep 13 16:50:11 2006 Raid Level : raid1 Array Size : 2931712 (2.80 GiB 3.00 GB) Used Dev Size : 2931712 (2.80 GiB 3.00 GB) Raid Devices : 2 Total Devices : 1 Preferred Minor : 0 Persistence : Superblock is persistent Update Time : Tue Mar 3 19:05:21 2009 State : clean, degraded Active Devices : 1 Working Devices : 1 Failed Devices : 0 Spare Devices : 0 UUID : 5afe70f3:fa1dcb8c:5cc73e84:1c5db3e0 Events : 0.912 Number Major Minor RaidDevice State 0 0 0 0 removed 1 8 17 1 active sync /dev/sdb1 ================================================================== Similar situation with all other raids ================================================================== smartctl --all /dev/sda (the disk removed): smartctl version 5.38 [x86_64-unknown-linux-gnu] Copyright (C) 2002-8 Bruce Allen Home page is http://smartmontools.sourceforge.net/ === START OF INFORMATION SECTION === Model Family: Western Digital Raptor family Device Model: WDC WD1500ADFD-00NLR1 Serial Number: WD-WMAP41185084 Firmware Version: 20.07P20 User Capacity: 150,039,945,216 bytes Device is: In smartctl database [for details use: -P show] ATA Version is: 7 ATA Standard is: ATA/ATAPI-7 published, ANSI INCITS 397-2005 Local Time is: Tue Mar 3 19:41:15 2009 CET SMART support is: Available - device has SMART capability. SMART support is: Enabled === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED General SMART Values: Offline data collection status: (0x84) Offline data collection activity was suspended by an interrupting command from host. Auto Offline Data Collection: Enabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: (4783) seconds. Offline data collection capabilities: (0x7b) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 2) minutes. Extended self-test routine recommended polling time: ( 72) minutes. Conveyance self-test routine recommended polling time: ( 5) minutes. SCT capabilities: (0x103f) SCT Status supported. SCT Feature Control supported. SCT Data Table supported. SMART Attributes Data Structure revision number: 16 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAG VALUE WORST THRESH TYPE UPDATED WHEN_FAILED RAW_VALUE 1 Raw_Read_Error_Rate 0x000b 200 200 051 Pre-fail Always - 0 3 Spin_Up_Time 0x0007 168 166 021 Pre-fail Always - 4625 4 Start_Stop_Count 0x0032 099 099 040 Old_age Always - 1031 5 Reallocated_Sector_Ct 0x0033 189 189 140 Pre-fail Always - 168 7 Seek_Error_Rate 0x000a 200 200 051 Old_age Always - 0 9 Power_On_Hours 0x0032 091 091 000 Old_age Always - 7259 10 Spin_Retry_Count 0x0012 100 100 051 Old_age Always - 0 11 Calibration_Retry_Count 0x0012 100 100 051 Old_age Always - 0 12 Power_Cycle_Count 0x0032 099 099 000 Old_age Always - 1031 194 Temperature_Celsius 0x0022 123 106 000 Old_age Always - 24 196 Reallocated_Event_Count 0x0032 180 180 000 Old_age Always - 20 197 Current_Pending_Sector 0x0012 200 200 000 Old_age Always - 0 198 Offline_Uncorrectable 0x0012 200 200 000 Old_age Always - 0 199 UDMA_CRC_Error_Count 0x000a 200 253 000 Old_age Always - 0 200 Multi_Zone_Error_Rate 0x0008 200 200 051 Old_age Offline - 0 SMART Error Log Version: 1 ATA Error Count: 12 (device log contains only the most recent five errors) CR = Command Register [HEX] FR = Features Register [HEX] SC = Sector Count Register [HEX] SN = Sector Number Register [HEX] CL = Cylinder Low Register [HEX] CH = Cylinder High Register [HEX] DH = Device/Head Register [HEX] DC = Device Command Register [HEX] ER = Error register [HEX] ST = Status register [HEX] Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, SS=sec, and sss=millisec. It "wraps" after 49.710 days. Error 12 occurred at disk power-on lifetime: 1940 hours (80 days + 20 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 08 4a 9d 9b ec Error: UNC 8 sectors at LBA = 0x0c9b9d4a = 211524938 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 47 9d 9b 4c 00 00:52:49.600 READ DMA ec 00 08 4a 9d 9b 00 00 00:52:49.600 IDENTIFY DEVICE c8 00 08 47 9d 9b 4c 00 00:52:49.600 READ DMA ec 00 08 4a 9d 9b 00 00 00:52:49.600 IDENTIFY DEVICE c8 00 08 47 9d 9b 4c 00 00:52:49.600 READ DMA Error 11 occurred at disk power-on lifetime: 1940 hours (80 days + 20 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 08 4a 9d 9b ec Error: UNC 8 sectors at LBA = 0x0c9b9d4a = 211524938 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 47 9d 9b 4c 00 00:52:46.400 READ DMA ec 00 08 4a 9d 9b 00 00 00:52:46.400 IDENTIFY DEVICE c8 00 08 47 9d 9b 4c 00 00:52:46.400 READ DMA ec 00 08 4a 9d 9b 00 00 00:52:46.400 IDENTIFY DEVICE c8 00 08 47 9d 9b 4c 00 00:52:46.400 READ DMA Error 10 occurred at disk power-on lifetime: 1940 hours (80 days + 20 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 08 4a 9d 9b ec Error: UNC 8 sectors at LBA = 0x0c9b9d4a = 211524938 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 47 9d 9b 4c 00 00:52:42.950 READ DMA ec 00 08 4a 9d 9b 00 00 00:52:42.950 IDENTIFY DEVICE c8 00 08 47 9d 9b 4c 00 00:52:42.950 READ DMA ec 00 08 4a 9d 9b 00 00 00:52:42.950 IDENTIFY DEVICE c8 00 08 47 9d 9b 4c 00 00:52:42.950 READ DMA Error 9 occurred at disk power-on lifetime: 1940 hours (80 days + 20 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 08 4a 9d 9b ec Error: UNC 8 sectors at LBA = 0x0c9b9d4a = 211524938 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 47 9d 9b 4c 00 00:52:39.800 READ DMA ec 00 08 4a 9d 9b 00 00 00:52:39.800 IDENTIFY DEVICE c8 00 08 47 9d 9b 4c 00 00:52:39.800 READ DMA ec 00 08 4a 9d 9b 00 00 00:52:39.800 IDENTIFY DEVICE c8 00 08 47 9d 9b 4c 00 00:52:39.800 READ DMA Error 8 occurred at disk power-on lifetime: 1940 hours (80 days + 20 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER ST SC SN CL CH DH -- -- -- -- -- -- -- 40 51 08 4a 9d 9b ec Error: UNC 8 sectors at LBA = 0x0c9b9d4a = 211524938 Commands leading to the command that caused the error were: CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name -- -- -- -- -- -- -- -- ---------------- -------------------- c8 00 08 47 9d 9b 4c 00 00:52:36.600 READ DMA ec 00 08 4a 9d 9b 00 00 00:52:36.600 IDENTIFY DEVICE c8 00 08 47 9d 9b 4c 00 00:52:36.600 READ DMA c8 00 08 3f 9d 9b 4c 00 00:52:36.600 READ DMA c8 00 08 37 9d 9b 4c 00 00:52:36.600 READ DMA SMART Self-test log structure revision number 1 No self-tests have been logged. [To run self-tests, use: smartctl -t] SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. ============================================================ sdb reported no errors ============================================================ thanks francesco pietra On Tue, Mar 3, 2009 at 5:02 PM, Francesco Pietra <chiendar...@gmail.com> wrote: > How to run selftest with raid1? > > smartctl -t /dev/sda (or sda1, sda2, ..) > > is incorrect. I don't remember what should first be done with mdadm. > > I would like to run the test because > > smartctl --all /dev/sd# > > report errors for only sda, as follows: > > SMART Error Log Version: 1 > ATA Error Count: 12 (device log contains only the most recent five errors) > CR = Command Register [HEX] > FR = Features Register [HEX] > SC = Sector Count Register [HEX] > SN = Sector Number Register [HEX] > CL = Cylinder Low Register [HEX] > CH = Cylinder High Register [HEX] > DH = Device/Head Register [HEX] > DC = Device Command Register [HEX] > ER = Error register [HEX] > ST = Status register [HEX] > Powered_Up_Time is measured from power on, and printed as > DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, > SS=sec, and sss=millisec. It "wraps" after 49.710 days. > > Error 12 occurred at disk power-on lifetime: 1940 hours (80 days + 20 hours) > When the command that caused the error occurred, the device was > active or idle. > > After command completion occurred, registers were: > ER ST SC SN CL CH DH > -- -- -- -- -- -- -- > 40 51 08 4a 9d 9b ec Error: UNC 8 sectors at LBA = 0x0c9b9d4a = 211524938 > > Commands leading to the command that caused the error were: > CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name > -- -- -- -- -- -- -- -- ---------------- -------------------- > c8 00 08 47 9d 9b 4c 00 00:52:49.600 READ DMA > ec 00 08 4a 9d 9b 00 00 00:52:49.600 IDENTIFY DEVICE > c8 00 08 47 9d 9b 4c 00 00:52:49.600 READ DMA > ec 00 08 4a 9d 9b 00 00 00:52:49.600 IDENTIFY DEVICE > c8 00 08 47 9d 9b 4c 00 00:52:49.600 READ DMA > > Error 11 occurred at disk power-on lifetime: 1940 hours (80 days + 20 hours) > When the command that caused the error occurred, the device was > active or idle. > > After command completion occurred, registers were: > ER ST SC SN CL CH DH > -- -- -- -- -- -- -- > 40 51 08 4a 9d 9b ec Error: UNC 8 sectors at LBA = 0x0c9b9d4a = 211524938 > > Commands leading to the command that caused the error were: > CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name > -- -- -- -- -- -- -- -- ---------------- -------------------- > c8 00 08 47 9d 9b 4c 00 00:52:46.400 READ DMA > ec 00 08 4a 9d 9b 00 00 00:52:46.400 IDENTIFY DEVICE > c8 00 08 47 9d 9b 4c 00 00:52:46.400 READ DMA > ec 00 08 4a 9d 9b 00 00 00:52:46.400 IDENTIFY DEVICE > c8 00 08 47 9d 9b 4c 00 00:52:46.400 READ DMA > > Error 10 occurred at disk power-on lifetime: 1940 hours (80 days + 20 hours) > When the command that caused the error occurred, the device was > active or idle. > > After command completion occurred, registers were: > ER ST SC SN CL CH DH > -- -- -- -- -- -- -- > 40 51 08 4a 9d 9b ec Error: UNC 8 sectors at LBA = 0x0c9b9d4a = 211524938 > > Commands leading to the command that caused the error were: > CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name > -- -- -- -- -- -- -- -- ---------------- -------------------- > c8 00 08 47 9d 9b 4c 00 00:52:42.950 READ DMA > ec 00 08 4a 9d 9b 00 00 00:52:42.950 IDENTIFY DEVICE > c8 00 08 47 9d 9b 4c 00 00:52:42.950 READ DMA > ec 00 08 4a 9d 9b 00 00 00:52:42.950 IDENTIFY DEVICE > c8 00 08 47 9d 9b 4c 00 00:52:42.950 READ DMA > > Error 9 occurred at disk power-on lifetime: 1940 hours (80 days + 20 hours) > When the command that caused the error occurred, the device was > active or idle. > > After command completion occurred, registers were: > ER ST SC SN CL CH DH > -- -- -- -- -- -- -- > 40 51 08 4a 9d 9b ec Error: UNC 8 sectors at LBA = 0x0c9b9d4a = 211524938 > > Commands leading to the command that caused the error were: > CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name > -- -- -- -- -- -- -- -- ---------------- -------------------- > c8 00 08 47 9d 9b 4c 00 00:52:39.800 READ DMA > ec 00 08 4a 9d 9b 00 00 00:52:39.800 IDENTIFY DEVICE > c8 00 08 47 9d 9b 4c 00 00:52:39.800 READ DMA > ec 00 08 4a 9d 9b 00 00 00:52:39.800 IDENTIFY DEVICE > c8 00 08 47 9d 9b 4c 00 00:52:39.800 READ DMA > > Error 8 occurred at disk power-on lifetime: 1940 hours (80 days + 20 hours) > When the command that caused the error occurred, the device was > active or idle. > > After command completion occurred, registers were: > ER ST SC SN CL CH DH > -- -- -- -- -- -- -- > 40 51 08 4a 9d 9b ec Error: UNC 8 sectors at LBA = 0x0c9b9d4a = 211524938 > > Commands leading to the command that caused the error were: > CR FR SC SN CL CH DH DC Powered_Up_Time Command/Feature_Name > -- -- -- -- -- -- -- -- ---------------- -------------------- > c8 00 08 47 9d 9b 4c 00 00:52:36.600 READ DMA > ec 00 08 4a 9d 9b 00 00 00:52:36.600 IDENTIFY DEVICE > c8 00 08 47 9d 9b 4c 00 00:52:36.600 READ DMA > c8 00 08 3f 9d 9b 4c 00 00:52:36.600 READ DMA > c8 00 08 37 9d 9b 4c 00 00:52:36.600 READ DMA > > SMART Self-test log structure revision number 1 > No self-tests have been logged. [To run self-tests, use: smartctl -t] > ======= > > I.e., I am still uncertain if sda has to be replaced with a new disk, > or the errors reported were temporary and have been removed by raid1. > > thanks > > francesco ============================================================================================== ===============================================================================================> > > ---------- Forwarded message ---------- > From: Francesco Pietra <chiendar...@gmail.com> > Date: Tue, Mar 3, 2009 at 11:21 AM > Subject: Re: raid1 issue, somewhat related to recent "debian on big machines" > To: Ron Johnson <ron.l.john...@cox.net> > > > On Tue, Mar 3, 2009 at 10:08 AM, Ron Johnson <ron.l.john...@cox.net> wrote: >> On 03/03/2009 02:53 AM, Francesco Pietra wrote: >>> >>> lupus in fabula as a follow up of my short intervention on raid1 with >>> my machine to the thread "Debian on big systems". >>> >>> System: supermicro H8QC8 m.board, two WD Raptor SATA 150GB, Debian >>> amd64 lenny, raid1 >>> >>> While running an electronic molecular calculation - estimated to four >>> days time - I noticed by chance on the screen (what is not in the out >>> file of the calculation) that there was a disk problem. I took some >>> scattered notes from the scree: >>> >>> RAID1 conf printout >>> >>> wd: 1 rd:2 >> >> [snip] >> >> What you are looking for should be in syslog, not your application's log. > > > OK, but /var/log/syslog > > tels nothing more that I took notice about from the screen: sda sector > 0 problematic, disk failure, continuing on one disk. My question is, > what does the lshw -disk output mean (SCSI vs SATA, as I have shown), > and if one disk has to be replaced with a new one. If so, to identify > which is which can I detach the SATA connection to the disks and see > which one works? > > thanks > francesco > > >> >> -- >> Ron Johnson, Jr. >> Jefferson LA USA >> >> The feeling of disgust at seeing a human female in a Relationship >> with a chimp male is Homininphobia, and you should be ashamed of >> yourself. >> >> >> -- >> To UNSUBSCRIBE, email to debian-amd64-requ...@lists.debian.org >> with a subject of "unsubscribe". Trouble? Contact >> listmas...@lists.debian.org >> >> > -- To UNSUBSCRIBE, email to debian-amd64-requ...@lists.debian.org with a subject of "unsubscribe". Trouble? Contact listmas...@lists.debian.org