Re: geom mirror now rebuilding on every reboot?
On Mon, 2012-08-06 at 08:34 -0700, Maksim Yevmenkin wrote: Michael, Something in -current and recently MFC'd to -stable is causing all of my gmirror drives to rebuild on reboot :-( Being remote and these being production machines, I suspect SVN r237929 and r237930 in -current and SVN r238500 to -stable but haven't yet been able to prove it. Is anyone else seeing this? yes, i've seem something similar only much, much worse. one of our production systems completely kept loosing its gmirror volumes on every reboot. it looked like gmirror metadata were completely corrupted. rebuilding mirrors and reverting back to previous kernel seemed to work. someone else is tracking it down. Have they managed to track this down to a commit or range of commits yet? Thanks, Gavin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 08/06/12 14:06, Gleb Smirnoff wrote: Michael, On Sat, Aug 04, 2012 at 12:49:49PM -0400, Michael Butler wrote: M Something in -current and recently MFC'd to -stable is causing all of my M gmirror drives to rebuild on reboot :-( M M Being remote and these being production machines, I suspect SVN r237929 M and r237930 in -current and SVN r238500 to -stable but haven't yet been M able to prove it. I'd appreciate if you test that and either confirm or disclaim that r238500 introduces such regression. Thanks! I'm only home for a few hours this weekend (enough to do the laundry before hitting the road again :-() but I did prove that disabling SUJ and moving back to SU mitigates the problem, imb -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.12 (FreeBSD) iEYEARECAAYFAlAmhXIACgkQQv9rrgRC1JLqCQCghWIHeG9m3q6qY2o4vuTqXAx1 PSkAoIKgsEfiDxJJAqni/bL2CqRBG4+D =3fLJ -END PGP SIGNATURE- ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
Alexander Motin wrote: I think the right answer would be to remove stale Promise format RAID metadata from the disk. You should be able to do it by booting from any source and doing `graid list -a` to see all RAID GEOMs (even broken) and then remove them with `graid remove Promise ad4`. thanks, that fixed it. tom ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
On 07.08.2012 04:03, Tom Uffner wrote: GEOM_RAID: Promise: Array Promise Created GEOM_RAID: Promise: Disk ad4 state changed from NONE to SPARE GEOM_MIRROR: Cannot open consumer ad4 (error=1) GEOM_MIRROR: Cannot add disk ad4 to gm0 (error=1) wondering if anyone knew offhand what is going on how to fix it. You can remove options GEOM_RAID from your kernel config. -- WBR, Andrey V. Elsukov ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
On 07.08.2012 17:09, Andrey V. Elsukov wrote: On 07.08.2012 04:03, Tom Uffner wrote: GEOM_RAID: Promise: Array Promise Created GEOM_RAID: Promise: Disk ad4 state changed from NONE to SPARE GEOM_MIRROR: Cannot open consumer ad4 (error=1) GEOM_MIRROR: Cannot add disk ad4 to gm0 (error=1) wondering if anyone knew offhand what is going on how to fix it. You can remove options GEOM_RAID from your kernel config. I think the right answer would be to remove stale Promise format RAID metadata from the disk. You should be able to do it by booting from any source and doing `graid list -a` to see all RAID GEOMs (even broken) and then remove them with `graid remove Promise ad4`. -- Alexander Motin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
Michael, Something in -current and recently MFC'd to -stable is causing all of my gmirror drives to rebuild on reboot :-( Being remote and these being production machines, I suspect SVN r237929 and r237930 in -current and SVN r238500 to -stable but haven't yet been able to prove it. Is anyone else seeing this? yes, i've seem something similar only much, much worse. one of our production systems completely kept loosing its gmirror volumes on every reboot. it looked like gmirror metadata were completely corrupted. rebuilding mirrors and reverting back to previous kernel seemed to work. someone else is tracking it down. thanks, max ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
On Mon, Aug 6, 2012 at 8:34 AM, Maksim Yevmenkin maksim.yevmen...@gmail.com wrote: Michael, Something in -current and recently MFC'd to -stable is causing all of my gmirror drives to rebuild on reboot :-( Being remote and these being production machines, I suspect SVN r237929 and r237930 in -current and SVN r238500 to -stable but haven't yet been able to prove it. Is anyone else seeing this? yes, i've seem something similar only much, much worse. one of our production systems completely kept loosing its gmirror volumes on every reboot. it looked like gmirror metadata were completely corrupted. rebuilding mirrors and reverting back to previous kernel seemed to work. someone else is tracking it down. Is this is in 9.1 -PRERELEASE, -RELEASE (or whatever the official label is...)? If so, it seems like this would be a ship blocker. Thanks! -Garrett ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
On Mon, Aug 6, 2012 at 9:23 AM, Garrett Cooper yaneg...@gmail.com wrote: On Mon, Aug 6, 2012 at 8:34 AM, Maksim Yevmenkin maksim.yevmen...@gmail.com wrote: Michael, Something in -current and recently MFC'd to -stable is causing all of my gmirror drives to rebuild on reboot :-( Being remote and these being production machines, I suspect SVN r237929 and r237930 in -current and SVN r238500 to -stable but haven't yet been able to prove it. Is anyone else seeing this? yes, i've seem something similar only much, much worse. one of our production systems completely kept loosing its gmirror volumes on every reboot. it looked like gmirror metadata were completely corrupted. rebuilding mirrors and reverting back to previous kernel seemed to work. someone else is tracking it down. Is this is in 9.1 -PRERELEASE, -RELEASE (or whatever the official label is...)? If so, it seems like this would be a ship blocker. sorry. its releng_9/9-stable. gmirrors are on two ssds. we use gpt and gmirror individual partitions, not entire disks. thanks, max ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
On Mon, Aug 6, 2012 at 9:26 AM, Maksim Yevmenkin maksim.yevmen...@gmail.com wrote: On Mon, Aug 6, 2012 at 9:23 AM, Garrett Cooper yaneg...@gmail.com wrote: On Mon, Aug 6, 2012 at 8:34 AM, Maksim Yevmenkin maksim.yevmen...@gmail.com wrote: Michael, Something in -current and recently MFC'd to -stable is causing all of my gmirror drives to rebuild on reboot :-( Being remote and these being production machines, I suspect SVN r237929 and r237930 in -current and SVN r238500 to -stable but haven't yet been able to prove it. Is anyone else seeing this? yes, i've seem something similar only much, much worse. one of our production systems completely kept loosing its gmirror volumes on every reboot. it looked like gmirror metadata were completely corrupted. rebuilding mirrors and reverting back to previous kernel seemed to work. someone else is tracking it down. Is this is in 9.1 -PRERELEASE, -RELEASE (or whatever the official label is...)? If so, it seems like this would be a ship blocker. sorry. its releng_9/9-stable. gmirrors are on two ssds. we use gpt and gmirror individual partitions, not entire disks. I may well be confused, but I don't understand how you can use GPT for a single partition. Looks to me like a disk is GPT or legacy. Declaring which is the first command when setting up a new disk. -- R. Kevin Oberman, Network Engineer E-mail: kob6...@gmail.com ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
On Mon, Aug 6, 2012 at 9:33 AM, Kevin Oberman kob6...@gmail.com wrote: On Mon, Aug 6, 2012 at 9:26 AM, Maksim Yevmenkin maksim.yevmen...@gmail.com wrote: On Mon, Aug 6, 2012 at 9:23 AM, Garrett Cooper yaneg...@gmail.com wrote: On Mon, Aug 6, 2012 at 8:34 AM, Maksim Yevmenkin maksim.yevmen...@gmail.com wrote: Michael, Something in -current and recently MFC'd to -stable is causing all of my gmirror drives to rebuild on reboot :-( Being remote and these being production machines, I suspect SVN r237929 and r237930 in -current and SVN r238500 to -stable but haven't yet been able to prove it. Is anyone else seeing this? yes, i've seem something similar only much, much worse. one of our production systems completely kept loosing its gmirror volumes on every reboot. it looked like gmirror metadata were completely corrupted. rebuilding mirrors and reverting back to previous kernel seemed to work. someone else is tracking it down. Is this is in 9.1 -PRERELEASE, -RELEASE (or whatever the official label is...)? If so, it seems like this would be a ship blocker. sorry. its releng_9/9-stable. gmirrors are on two ssds. we use gpt and gmirror individual partitions, not entire disks. I may well be confused, but I don't understand how you can use GPT for a single partition. Looks to me like a disk is GPT or legacy. Declaring which is the first command when setting up a new disk. i'm sorry, to make it (hopefully) clear, we are using gpt partitioning scheme (as opposed to mbr partitioning scheme), and then use gmirror on individual partitions, i.e. we gimirror /dev/ada0p1 and /dev/ada1p1 and not /dev/ada0 and /dev/ada1. thanks max ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
On Aug 6, 2012, at 9:40 AM, Maksim Yevmenkin maksim.yevmen...@gmail.com wrote: On Mon, Aug 6, 2012 at 9:33 AM, Kevin Oberman kob6...@gmail.com wrote: On Mon, Aug 6, 2012 at 9:26 AM, Maksim Yevmenkin maksim.yevmen...@gmail.com wrote: On Mon, Aug 6, 2012 at 9:23 AM, Garrett Cooper yaneg...@gmail.com wrote: On Mon, Aug 6, 2012 at 8:34 AM, Maksim Yevmenkin maksim.yevmen...@gmail.com wrote: Michael, Something in -current and recently MFC'd to -stable is causing all of my gmirror drives to rebuild on reboot :-( Being remote and these being production machines, I suspect SVN r237929 and r237930 in -current and SVN r238500 to -stable but haven't yet been able to prove it. Is anyone else seeing this? yes, i've seem something similar only much, much worse. one of our production systems completely kept loosing its gmirror volumes on every reboot. it looked like gmirror metadata were completely corrupted. rebuilding mirrors and reverting back to previous kernel seemed to work. someone else is tracking it down. Is this is in 9.1 -PRERELEASE, -RELEASE (or whatever the official label is...)? If so, it seems like this would be a ship blocker. sorry. its releng_9/9-stable. gmirrors are on two ssds. we use gpt and gmirror individual partitions, not entire disks. I may well be confused, but I don't understand how you can use GPT for a single partition. Looks to me like a disk is GPT or legacy. Declaring which is the first command when setting up a new disk. i'm sorry, to make it (hopefully) clear, we are using gpt partitioning scheme (as opposed to mbr partitioning scheme), and then use gmirror on individual partitions, i.e. we gimirror /dev/ada0p1 and /dev/ada1p1 and not /dev/ada0 and /dev/ada1. Can you provide more helpful info like svn revisions?___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
Michael, On Sat, Aug 04, 2012 at 12:49:49PM -0400, Michael Butler wrote: M Something in -current and recently MFC'd to -stable is causing all of my M gmirror drives to rebuild on reboot :-( M M Being remote and these being production machines, I suspect SVN r237929 M and r237930 in -current and SVN r238500 to -stable but haven't yet been M able to prove it. I'd appreciate if you test that and either confirm or disclaim that r238500 introduces such regression. Thanks! -- Totus tuus, Glebius. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: geom mirror now rebuilding on every reboot?
Ding, ding - I use the old partitioning scheme but also mirror individual partitions rather than whole disks, imb -Original Message- From: Maksim Yevmenkin maksim.yevmen...@gmail.com Sent: Monday, 6 August 2012 12:26 To: Garrett Cooper yaneg...@gmail.com Cc: sta...@freebsd.org; Michael Butler i...@protected-networks.net; curr...@freebsd.org Subject: Re: geom mirror now rebuilding on every reboot? On Mon, Aug 6, 2012 at 9:23 AM, Garrett Cooper yaneg...@gmail.com wrote: On Mon, Aug 6, 2012 at 8:34 AM, Maksim Yevmenkin maksim.yevmen...@gmail.com wrote: Michael, Something in -current and recently MFC'd to -stable is causing all of my gmirror drives to rebuild on reboot :-( Being remote and these being production machines, I suspect SVN r237929 and r237930 in -current and SVN r238500 to -stable but haven't yet been able to prove it. Is anyone else seeing this? yes, i've seem something similar only much, much worse. one of our production systems completely kept loosing its gmirror volumes on every reboot. it looked like gmirror metadata were completely corrupted. rebuilding mirrors and reverting back to previous kernel seemed to work. someone else is tracking it down. Is this is in 9.1 -PRERELEASE, -RELEASE (or whatever the official label is...)? If so, it seems like this would be a ship blocker. sorry. its releng_9/9-stable. gmirrors are on two ssds. we use gpt and gmirror individual partitions, not entire disks. thanks, max ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: geom mirror now rebuilding on every reboot?
Sorry but I'm not going to be able to get to this until next w/e as I'm ~1500 miles away from them at present :-( -Original Message- From: Gleb Smirnoff gleb...@freebsd.org Sent: Monday, 6 August 2012 14:06 To: Michael Butler i...@protected-networks.net Cc: curr...@freebsd.org; sta...@freebsd.org Subject: Re: geom mirror now rebuilding on every reboot? Michael, On Sat, Aug 04, 2012 at 12:49:49PM -0400, Michael Butler wrote: M Something in -current and recently MFC'd to -stable is causing all of my M gmirror drives to rebuild on reboot :-( M M Being remote and these being production machines, I suspect SVN r237929 M and r237930 in -current and SVN r238500 to -stable but haven't yet been M able to prove it. I'd appreciate if you test that and either confirm or disclaim that r238500 introduces such regression. Thanks! -- Totus tuus, Glebius. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
On Aug 6, 2012, at 12:06 PM, Gleb Smirnoff gleb...@freebsd.org wrote: Michael, On Sat, Aug 04, 2012 at 12:49:49PM -0400, Michael Butler wrote: M Something in -current and recently MFC'd to -stable is causing all of my M gmirror drives to rebuild on reboot :-( M M Being remote and these being production machines, I suspect SVN r237929 M and r237930 in -current and SVN r238500 to -stable but haven't yet been M able to prove it. I'd appreciate if you test that and either confirm or disclaim that r238500 introduces such regression. Thanks! I'm not sure how r238500 could affect what Max, myself, and presumably the original poster are seeing. There's one other change in 9-stable, r235599, but it looks like a benign change as well. Scott ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
not quite the same issue, but i tried to update a system with gmirror from FreeBSD eris.uffner.com 10.0-CURRENT FreeBSD 10.0-CURRENT #210: Thu Mar 15 16:41:18 EDT 2012 t...@eris.uffner.com:/usr/obj/usr/src/sys/ERIS i386 to -current from Aug 4th, and booting now fails into mountroot with GEOM_RAID: Promise: Array Promise Created GEOM_RAID: Promise: Disk ad4 state changed from NONE to SPARE GEOM_MIRROR: Cannot open consumer ad4 (error=1) GEOM_MIRROR: Cannot add disk ad4 to gm0 (error=1) GEOM_MIRROR: Device gm0 destroyed GEOM_RAID: Promise: Array Promise Created GEOM_RAID: Promise: Disk ad6 state changed from NONE to SPARE GEOM_MIRROR: Cannot open consumer ad6 (error=1) GEOM_MIRROR: Cannot add disk ad6 to gm0 (error=1) GEOM_MIRROR: Device gm0 destroyed nor will it boot from the underlying partitions (ad4s1a, ad6s1a), even if i unplug one of the drives. the partitions metadata are not corrupt and work just fine if i revert to the previous kernel. i haven't had time yet to track down the commit where this breaks. i was just wondering if anyone knew offhand what is going on how to fix it. tom ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: geom mirror now rebuilding on every reboot?
by any chance... ... is this maybe related to my PR: http://www.freebsd.org/cgi/query-pr.cgi?pr=170038 cheers, Frank Reppin -- 43rd Law of Computing: Anything that can go wr fortune: Segmentation violation -- Core dumped ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
