Re: Can't get md array to shut down cleanly
> My md raid5 still does not always shut down cleanly. The last few > lines of the shutdown sequence are always as follows: > > [...] > Will now halt. > md: stopping all md devices. > md: md0 still in use. > Synchronizing SCSI cache for disk /dev/sdd: > Synchronizing SCSI cache for disk /dev/sdc: > Synchronizing SCSI cache for disk /dev/sdb: > Synchronizing SCSI cache for disk /dev/sda: > Shutdown: hde > System halted. May be your shutdown script is doing "halt -h"? Halting the disk immediately without letting the RAID to settle to a clean state can be the cause? I see that my Debian avoids the -h option if running RAID, from /etc/init.d/halt: # Don't shut down drives if we're using RAID. hddown="-h" if grep -qs '^md.*active' /proc/mdstat then hddown="" fi # If INIT_HALT=HALT don't poweroff. poweroff="-p" if [ "$INIT_HALT" = "HALT" ] then poweroff="" fi log_action_msg "Will now halt" sleep 1 halt -d -f -i $poweroff $hddown -- Niccolo Rigacci Firenze - Italy Iraq, missione di pace: 38839 morti - www.iraqbodycount.net - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Cutting power without breaking RAID
On Thu, Jun 29, 2006 at 02:00:09PM +1000, Neil Brown wrote: > With 2.6, >killall -9 md0_raid1 > > should do the trick (assuming root is on /dev/md0. If it is elsewhere, > choose a different process name). Thanks, this is what I was looking for! I will try remounting read-only and killing the md0_raid1. I will keep you informed. -- Niccolo Rigacci Firenze - Italy Iraq, missione di pace: 38725 morti - www.iraqbodycount.net - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Cutting power without breaking RAID
On Wed, Jun 28, 2006 at 06:03:01PM -0700, Tim wrote: > Your UPS won't accept a timer value to wait before actually cutting > power? Unfortunately my UPS des not! -- Niccolo Rigacci Firenze - Italy Iraq, missione di pace: 38725 morti - www.iraqbodycount.net - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Cutting power without breaking RAID
Hello, I'm facing this problem: when my Linux box detects a POWER FAIL event from the UPS, it starts a normal shutdown. Just before the normal kernel poweroff, it sends to the UPS a signal on the serial line which says "cut-off the power to the server and switch-off the UPS". This is required to reboot the server as soon as the power is restored. The problem is that the root partition is on top of a RAID-1 filesystem which is still mounted when the program that kills the power is run, so the system goes down with a non clean RAID volume. What can be the proper action to do before killing the power to ensure that RAID will remain clean? It seems that remounting the partition read-only is not sufficient. -- Niccolo Rigacci Firenze - Italy Iraq, missione di pace: 38475 morti - www.iraqbodycount.net - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: New FAQ entry? (was IBM xSeries stop responding during RAID1 reconstruction)
> > personally, I don't see any point to worrying about the default, > > compile-time or boot time: > > > > for f in `find /sys/block/* -name scheduler`; do echo cfq > $f; done I tested this case: - reboot as per power failure (RAID goes dirty) - RAID start resyncing as soon as the kernel assemble it - every disk activity is blocked, even DHCP failed! - host services are unavailable This is why I changed the kernel default. -- Niccolo Rigacci Firenze - Italy Iraq, missione di pace: 38475 morti - www.iraqbodycount.net - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
New FAQ entry? (was IBM xSeries stop responding during RAID1 reconstruction)
Thanks to the several guys in this list, I have solved my problem and elaborated this, can be a new FAQ entry? Q: Sometimes when a RAID volume is resyncing, the system seems to locks-up: every disk activity is blocked until resync is done. A: This is not strictly related to Linux RAID, this is a problem related to the Linux kernel and the disk subsytem: in no circumstances a process should get all the disk resources preventing others to access them. You can control the max speed at which RAID reconstruction is done by setting it, say at 5 Mb/s: echo 5000 > /proc/sys/dev/raid/speed_limit_max This is just a workaround, you have to determine the max speed that does not lock your system by trial and error and you cannot predict what will be the disk load in the future when the RAID will be resyncing for some reason. Starting from version 2.6, Linux kernel has several choices about the I/O scheduler to be used. The default is the anticipatory scheduler, which seems to be sub-optimal on resync high load. If your kernel has the CFQ scheduler compiled in, use it during resync. >From the command line you can see which schedulers are supported and change it on the fly (remember to do it for each RAID disk): # cat /sys/block/hda/queue/scheduler noop [anticipatory] deadline cfq # echo cfq > /sys/block/hda/queue/scheduler Otherwise you can recompile your kernel and set CFQ as the default I/O scheduler (CONFIG_DEFAULT_CFQ=y in Block layer, IO Schedulers, Default I/O scheduler). -- Niccolo Rigacci Firenze - Italy Iraq, missione di pace: 38475 morti - www.iraqbodycount.net - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: IBM xSeries stop responding during RAID1 reconstruction
> At least one can do a ls of the /sys/block area & then do an > automated > echo cfq down the tree . Does anyone know of a method to set a > default > scheduler ? My be I didn't understand the question... You decide what schedulers are available at kernel compile time, also at kernel compile time you decide which is the default i/o scheduler. -- Niccolo Rigacci Firenze - Italy Iraq, missione di pace: 38475 morti - www.iraqbodycount.net - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: IBM xSeries stop responding during RAID1 reconstruction
On Mon, Jun 19, 2006 at 05:05:56PM +0200, Gabor Gombas wrote: > > IMHO a much better fix is to use the cfq I/O scheduler during the > rebuild. Yes, changing the default I/O to DEFAULT_CFQ solve the problem very well, I get over 40 Mb/s resync speed with no lock-up at all! Thank you very much, I think we can elaborate a new FAQ entry. Do you know if it is possible to switch the scheduler at runtime? -- Niccolo Rigacci Firenze - Italy Iraq, missione di pace: 38475 morti - www.iraqbodycount.net - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: IBM xSeries stop responding during RAID1 reconstruction
On Thursday 15 June 2006 12:13, you wrote: > If this is causing a lockup, then there is something else wrong, just > as any single process should not - by writing constantly to disks - be > able to clog up the whole system. > > Maybe if you could get the result of > alt-sysrq-P I tried some kernel changes enabling the HyperThreading on the (single) P4 processor and enabling CONFIG_PREEMPT_VOLUNTARY=y, but with no success. During the lookup Alt-SysRq-P constantly says that: EIP is at mwait_idle+0x1a/0x2e While Alt-SysRq-T shows - among other processes - the MD syncing and the bash looked-up; this is the hand-copied call traces: md3_resync device_barrier default_wake_function sync_request __generic_unplug_device md_do_sync schedule md_thread md_thread kthread kthread kernel_thread_helper bash io_schedule sync_buffer sync_buffer __wait_on_bit_lock sync_buffer out_of_line_wait_on_bit_lock wake_bit_function __lock_buffer do_get_write_access __ext3_get_inode_loc jurnal_get_write_access ext3_reserve_inode_write ext3_mark_inode_dirty ext3_dirty_inode __mark_inode_dirty update_atime vfs_readdir sys_getdents64 filldir64 syscall_call This is also the top output, which runs regularly during the lookup: top - 11:40:41 up 7 min, 2 users, load average: 8.70, 4.92, 2.04 Tasks: 70 total, 1 running, 69 sleeping, 0 stopped, 0 zombie Cpu(s): 0.2% us, 0.7% sy, 0.0% ni, 98.7% id, 0.0% wa, 0.0% hi, 0.5% si Mem:906212k total,58620k used, 847592k free, 3420k buffers Swap: 1951736k total,0k used, 1951736k free,23848k cached PID USER PR NI VIRT RES SHR S %CPU %MEMTIME+ COMMAND 829 root 10 -5 000 S1 0.0 0:01.70 md3_raid1 2823 root 10 -5 000 D1 0.0 0:01.62 md3_resync 1 root 16 0 1956 656 560 S0 0.1 0:00.52 init 2 root RT 0 000 S0 0.0 0:00.00 migration/0 3 root 34 19 000 S0 0.0 0:00.00 ksoftirqd/0 4 root RT 0 000 S0 0.0 0:00.00 watchdog/0 5 root RT 0 000 S0 0.0 0:00.00 migration/1 6 root 34 19 000 S0 0.0 0:00.00 ksoftirqd/1 7 root RT 0 000 S0 0.0 0:00.00 watchdog/1 8 root 10 -5 000 S0 0.0 0:00.01 events/0 9 root 10 -5 000 S0 0.0 0:00.01 events/1 10 root 10 -5 000 S0 0.0 0:00.00 khelper 11 root 10 -5 000 S0 0.0 0:00.00 kthread 14 root 10 -5 000 S0 0.0 0:00.00 kblockd/0 15 root 10 -5 000 S0 0.0 0:00.00 kblockd/1 16 root 11 -5 000 S0 0.0 0:00.00 kacpid 152 root 20 0 000 S0 0.0 0:00.00 pdflush 153 root 15 0 000 D0 0.0 0:00.00 pdflush 154 root 17 0 000 S0 0.0 0:00.00 kswapd0 155 root 11 -5 000 S0 0.0 0:00.00 aio/0 156 root 11 -5 000 S0 0.0 0:00.00 aio/1 755 root 10 -5 000 S0 0.0 0:00.00 kseriod 796 root 10 -5 000 S0 0.0 0:00.00 ata/0 797 root 11 -5 000 S0 0.0 0:00.00 ata/1 799 root 11 -5 000 S0 0.0 0:00.00 scsi_eh_0 800 root 11 -5 000 S0 0.0 0:00.00 scsi_eh_1 825 root 15 0 000 S0 0.0 0:00.00 kirqd 831 root 10 -5 000 D0 0.0 0:00.00 md2_raid1 833 root 10 -5 000 S0 0.0 0:00.00 md1_raid1 834 root 10 -5 000 D0 0.0 0:00.00 md0_raid1 835 root 15 0 000 D0 0.0 0:00.00 kjournald 932 root 18 -4 2192 584 368 S0 0.1 0:00.19 udevd 1698 root 10 -5 000 S0 0.0 0:00.00 khubd 2031 root 22 0 000 S0 0.0 0:00.00 kjournald 2032 root 15 0 000 D0 0.0 0:00.00 kjournald 2142 daemon16 0 1708 364 272 S0 0.0 0:00.00 portmap 2464 root 16 0 2588 932 796 S0 0.1 0:00.01 syslogd -- Niccolo Rigacci Firenze - Italy War against Iraq? Not in my name! - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: IBM xSeries stop responding during RAID1 reconstruction
On Wed, Jun 14, 2006 at 10:46:09AM -0500, Bill Cizek wrote: > Niccolo Rigacci wrote: > > >When the sync is complete, the machine start to respond again > >perfectly. > > > I was able to work around this by lowering > /proc/sys/dev/raid/speed_limit_max to a value > below my disk thruput value (~ 50 MB/s) as follows: > > $ echo 45000 > /proc/sys/dev/raid/speed_limit_max Thanks! This hack seems to solve my problem too. So it seems that the RAID subsystem does not detect a proper speed to throttle the sync. Can you please send me some details of your system? - SATA chipset (or motherboard model)? - Disks make/model? - Do you have the config file of the kernel that you was running (look at /boot/config- file)? I wonder if kernel preemption can be blamed for that, or burst speed of disks can fool the throttle calculation. -- Niccolo Rigacci Firenze - Italy Iraq, missione di pace: 38355 morti - www.iraqbodycount.net - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
IBM xSeries stop responding during RAID1 reconstruction
Hi to all, I have a new IBM xSeries 206m with two SATA drives, I installed a Debian Testing (Etch) and configured a software RAID as shown: Personalities : [raid1] md1 : active raid1 sdb5[1] sda5[0] 1951744 blocks [2/2] [UU] md2 : active raid1 sdb6[1] sda6[0] 2931712 blocks [2/2] [UU] md3 : active raid1 sdb7[1] sda7[0] 39061952 blocks [2/2] [UU] md0 : active raid1 sdb1[1] sda1[0] 582 blocks [2/2] [UU] I experience this problem: whenever a volume is reconstructing (syncing), the system stops responding. The machine is alive, because it responds to the ping, the console is responsive but I cannot pass the login prompt. It seems that every disk activity is delayed and blocking. When the sync is complete, the machine start to respond again perfectly. Any hints on how to start debugging? In the log file nothing strange appears: Jun 13 18:48:04 paros kernel: md: bind Jun 13 18:48:04 paros kernel: RAID1 conf printout: Jun 13 18:48:04 paros kernel: --- wd:1 rd:2 Jun 13 18:48:04 paros kernel: disk 0, wo:0, o:1, dev:sda1 Jun 13 18:48:04 paros kernel: disk 1, wo:1, o:1, dev:sdb1 Jun 13 18:48:04 paros kernel: md: syncing RAID array md0 Jun 13 18:48:04 paros kernel: md: minimum _guaranteed_ reconstruction speed: 1000 KB/sec/disc. Jun 13 18:48:04 paros kernel: md: using maximum available idle IO bandwidth (but not more than 20 KB/sec) for reconstruction. Jun 13 18:48:04 paros kernel: md: using 128k window, over a total of 582 blocks. Jun 13 18:57:30 paros kernel: md: md0: sync done. Jun 13 18:57:30 paros kernel: RAID1 conf printout: Jun 13 18:57:30 paros kernel: --- wd:2 rd:2 Jun 13 18:57:30 paros kernel: disk 0, wo:0, o:1, dev:sda1 Jun 13 18:57:30 paros kernel: disk 1, wo:0, o:1, dev:sdb1 Those are the IDE controllers as identified by lspci: :00:1f.1 IDE interface: Intel Corporation 82801G (ICH7 Family) IDE Controller (rev 01) :00:1f.2 IDE interface: Intel Corporation 82801GB/GR/GH (ICH7 Family) Serial ATA Storage Controllers cc=IDE (rev 01) And those are the SATA drivers detected by the kernel: ata1: SATA max UDMA/133 cmd 0x30C8 ctl 0x30BE bmdma 0x3090 irq 233 ata2: SATA max UDMA/133 cmd 0x30C0 ctl 0x30BA bmdma 0x3098 irq 233 ata1: dev 1 cfg 49:2f00 82:346b 83:7fe9 84:4773 85:3469 86:3c01 87:4763 88:207f ata1: dev 1 ATA-7, max UDMA/133, 156312576 sectors: LBA48 ata1: dev 1 configured for UDMA/133 scsi0 : ata_piix ata2: dev 1 cfg 49:2f00 82:346b 83:7fe9 84:4773 85:3469 86:3c01 87:4763 88:207f ata2: dev 1 ATA-7, max UDMA/133, 156312576 sectors: LBA48 ata2: dev 1 configured for UDMA/133 scsi1 : ata_piix Vendor: ATA Model: HDS728080PLA380 Rev: PF2O Type: Direct-AccessANSI SCSI revision: 05 Vendor: ATA Model: HDS728080PLA380 Rev: PF2O Type: Direct-AccessANSI SCSI revision: 05 SCSI device sda: 156312576 512-byte hdwr sectors (80032 MB) SCSI device sda: drive cache: write back SCSI device sda: 156312576 512-byte hdwr sectors (80032 MB) SCSI device sda: drive cache: write back sda: sda1 sda2 < sda5 sda6 sda7 > sd 0:0:1:0: Attached scsi disk sda SCSI device sdb: 156312576 512-byte hdwr sectors (80032 MB) SCSI device sdb: drive cache: write back SCSI device sdb: 156312576 512-byte hdwr sectors (80032 MB) SCSI device sdb: drive cache: write back sdb: sdb1 sdb2 < sdb5 sdb6 sdb7 > sd 1:0:1:0: Attached scsi disk sdb -- Niccolo Rigacci Firenze - Italy Iraq, missione di pace: 38355 morti - www.iraqbodycount.net - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Forcing a more random uuid (random seed bug)
> I get /dev/md5, /dev/md6, /dev/md7
> and /dev/md8 all with the same UUID!
It seems that there is a bug in mdadm: when generating the UUID for a
volume, the random() function is called, but the random sequence is never
initialized.
The result is that every volume created with mdadm has an uuid of:
6b8b4567:327b23c6:643c9869:66334873
See also Debian bug 292784 at
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=292784
I fixed the problem adding the following patch to mdadm.c, but please bear
in mind that I'm totally unaware of mdadm code and quite naive in C
programming:
$ diff -u mdadm.c.orig mdadm.c
--- mdadm.c.orig2004-11-02 06:11:06.0 +0100
+++ mdadm.c 2005-02-02 14:27:55.0 +0100
@@ -86,6 +86,15 @@
ident.super_minor= UnSet;
ident.devices=0;
+int my_fd;
+unsigned int my_seed;
+if ((my_fd = open("/dev/random", O_RDONLY)) != -1) {
+if (read(my_fd, &my_seed, sizeof(my_seed)) == sizeof(my_seed))
{
+srandom(my_seed);
+}
+close(my_fd);
+}
+
while ((option_index = -1) ,
(opt=getopt_long(argc, argv,
short_options, long_options,
--
Niccolo Rigacci
http://www.texnet.it/
-
To unsubscribe from this list: send the line "unsubscribe linux-raid" in
the body of a message to [EMAIL PROTECTED]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Forcing a more random uuid
My previous post was answered very quickly and with an optimal solution, but unfortunately I have a new question... I'm writing a script to clone a disk with several md devices. The raid1 volumes are created on the blank disk within a very quick loop, I think this is the reason why I get /dev/md5, /dev/md6, /dev/md7 and /dev/md8 all with the same UUID! Is there a way to force a more random UUID, or do I have to provide an --uid option on the command line? -- Niccolo Rigacci http://www.texnet.it/ - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Change preferred minor number of an md device?
Hi to all, md gurus! Is there a way to edit the preferred minor of a stopped device? Alternatively, is there a way to create a raid1 device specifying the preferred minor number md0, but activating it provisionally as a different minor, say md5? An md0 is already running, so "mdadm --create /dev/md0" fails... I have to dump my /dev/md0 to a different disk (/dev/md5), but when I boot from the new disk, I want the kernel to autmatically detect the device as /dev/md0. Many thanks -- Niccolo Rigacci http://www.texnet.it/ - To unsubscribe from this list: send the line "unsubscribe linux-raid" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
