Re: How to remove missing device on RAID1?
On Wed, Oct 21, 2015 at 11:54 AM, Dmitry Katsubowrote: > On 2015-10-21 00:40, Henk Slager wrote: >> I had a similar issue some time ago, around the time kernel 4.1.6 was >> just there. >> In case you don't want to wait for new disk or decide to just run the >> filesystem with 1 disk less or maybe later on replace 1 of the still >> healthy disks with a double/bigger sized one and use current/older >> kernel+tools, you could do this (assuming the filesystem is not too >> full of course): >> - mount degraded >> - btrfs balance start -f -v -sdevid=1 -mdevid=1 -ddevid=1 >> (where missing disk has devid 1) > > Am I right that one can "btrfs dev delete 1" after balance succeeded? If the patch that Ducan decribes is applied, then yes, this is then the way to remove (delete) the missing device from the filesystem. >> After completion the (virtual/missing) device shall be fully unallocated >> - create /dev/loopX with sparse file of same size as missing disk on >> some other filesystem >> - btrfs replace start 1 /dev/loopX >> - remove /dev/loopX from the filesystem >> - remount filesystyem without degraded >> And remove /dev/loopX > > If would be nice if btrfs allows to delete device and perform rebalance > automatically (provided that left devices still have enough space to > sustain raidX prerequisite). This would indeed be nice, but I guess it should be working already, looking at older (3.x) btrfs-tools documentation. It's just that it could not be used for some time. Anyhow, once this patch is more commonly available, people can test/use it and see if still a manual extra specific balance is needed in order to get rid of or work-around issues. It is actually the functionality of the '-r' option of 'btrfs replace' that is then applied. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to remove missing device on RAID1?
Hi Henk, This trick/hack worked great for me. After the rebalance was complete, a sparse file + loop device, `btrfs replace` and `btrfs delete` worked as expected. Thanks. In other news, I did hit a btrfs bug 3 times while attempting to balance. I've added my comments @ https://bugzilla.kernel.org/show_bug.cgi?id=105681#c14 ᐧ On Tue, Oct 20, 2015 at 3:46 PM, Henk Slagerwrote: > copy-paste error corrected > On Wed, Oct 21, 2015 at 12:40 AM, Henk Slager wrote: >> I had a similar issue some time ago, around the time kernel 4.1.6 was >> just there. >> In case you don't want to wait for new disk or decide to just run the >> filesystem with 1 disk less or maybe later on replace 1 of the still >> healthy disks with a double/bigger sized one and use current/older >> kernel+tools, you could do this (assuming the filesystem is not too >> full of course): >> - mount degraded > - btrfs balance start -f -v -sdevid=1 -mdevid=1 -mdevid=1 >> (where missing disk has devid 1) >> After completion the (virtual/missing) device shall be fully unallocated >> - create /dev/loopX with sparse file of same size as missing disk on >> some other filesystem >> - btrfs replace start 1 /dev/loopX >> - remove /dev/loopX from the filesystem >> - remount filesystyem without degraded >> And remove /dev/loopX >> >> >> On Tue, Oct 20, 2015 at 11:48 PM, Kyle Manna <2blu...@gmail.com> wrote: >>> Thanks for the follow-up Duncan, that makes sense. I assumed I was >>> doing something wrong. >>> >>> I downloaded the devel branch of of btrfs-progs and got it running >>> before I saw the need for a kernel patch and decided to wait. >>> >>> For anyone following this later, I needed to use the following to get >>> the missing device ID: >>> >>> btrfs device usage >>> ᐧ >>> >>> On Tue, Oct 20, 2015 at 1:58 PM, Duncan <1i5t5.dun...@cox.net> wrote: Kyle Manna posted on Tue, 20 Oct 2015 10:24:48 -0700 as excerpted: > Hi all, > > I have a collection of three (was 4) 1-2TB devices with data and > metadata in a RAID1 mirror. Last night I was struck by the Click of > Death on an old Samsung drive. > > I removed the device from the system, rebooted and mounted the volume > with `-o degraded` and the file system seems fine and usable. I'm > waiting on a replacement, drive but want to remove the old drive and > re-balance in the meantime. > > How do I remove the missing device? I tried the `btrfs device delete > missing /mnt` but was greeted with "ERROR: missing is not a block > device". A quick look at that btrfs-progs git repo shows that > `stat("missing")` is called, which of course fails since missing isn't a > block device. Nothing other then `btrfs replace` seemed intuitive and > all the docs mention the older command. What's the move? > > Thanks! > - Kyle > > Versions: > Kernel: 4.2.3-1-ARCH > btrfs-progs: 4.2.2-1 ᐧ I believe the current advice given here (that you were likely trying to follow, wrapped link)... https://btrfs.wiki.kernel.org/index.php/ Using_Btrfs_with_Multiple_Devices#Replacing_failed_devices ... is dated and no longer works due to code change some time in the past. There's a set of (very) recent patches, to the kernel and userspace both (I just updated userspace and it's in the git devel-branch v4.2.3-49- g4db87a1 I just built, kernelspace, I don't see it in linus-mainline yet, so I'd guess it's in the btrfs-integration patches, to land in the v4.4 commit window if not in 4.3 as it's getting late in the cycle for that. btrfs fi show That will list the btrfs component devices together with their devids. Then use the appropriate devid like so: btrfs dev del The -progs commit is d462081f, by Anand Jain, titled: btrfs-progs: Introduce device delete by devid According to it, the required kernel commit (title only listed) is similar: Btrfs: Introduce device delete by devid You can probably find them on-list if you wish to cherry-pick them into a current version. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >>> the body of a message to majord...@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majord...@vger.kernel.org >>
Re: How to remove missing device on RAID1?
On 2015-10-21 00:40, Henk Slager wrote: > I had a similar issue some time ago, around the time kernel 4.1.6 was > just there. > In case you don't want to wait for new disk or decide to just run the > filesystem with 1 disk less or maybe later on replace 1 of the still > healthy disks with a double/bigger sized one and use current/older > kernel+tools, you could do this (assuming the filesystem is not too > full of course): > - mount degraded > - btrfs balance start -f -v -sdevid=1 -sdevid=1 -sdevid=1 > (where missing disk has devid 1) Am I right that one can "btrfs dev delete 1" after balance succeeded? > After completion the (virtual/missing) device shall be fully unallocated > - create /dev/loopX with sparse file of same size as missing disk on > some other filesystem > - btrfs replace start 1 /dev/loopX > - remove /dev/loopX from the filesystem > - remount filesystyem without degraded > And remove /dev/loopX If would be nice if btrfs allows to delete device and perform rebalance automatically (provided that left devices still have enough space to sustain raidX prerequisite). -- With best regards, Dmitry -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
How to remove missing device on RAID1?
Hi all, I have a collection of three (was 4) 1-2TB devices with data and metadata in a RAID1 mirror. Last night I was struck by the Click of Death on an old Samsung drive. I removed the device from the system, rebooted and mounted the volume with `-o degraded` and the file system seems fine and usable. I'm waiting on a replacement, drive but want to remove the old drive and re-balance in the meantime. How do I remove the missing device? I tried the `btrfs device delete missing /mnt` but was greeted with "ERROR: missing is not a block device". A quick look at that btrfs-progs git repo shows that `stat("missing")` is called, which of course fails since missing isn't a block device. Nothing other then `btrfs replace` seemed intuitive and all the docs mention the older command. What's the move? Thanks! - Kyle Versions: Kernel: 4.2.3-1-ARCH btrfs-progs: 4.2.2-1 ᐧ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to remove missing device on RAID1?
Hi Kyle, On 10/20/2015 07:24 PM, Kyle Manna wrote: I removed the device from the system, rebooted and mounted the volume with `-o degraded` and the file system seems fine and usable. I'm waiting on a replacement, drive but want to remove the old drive and re-balance in the meantime. This won't answer your question (not sure why missing doesn't work anymore), but you could wait for the replacement and then run "btrfs replace start 1 /dev/sdd /mnt", where 1 is the devid of the missing drive (dmesg will tell you) and /dev/sdd is the new drive. This way, the drive will keep its devid, which may be useful if the ids are used to identify the drives (mirror will still have id 1 as first drive and id 2 as second drive as opposed to 2 = first and 3 = second). Philip -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to remove missing device on RAID1?
Kyle Manna posted on Tue, 20 Oct 2015 10:24:48 -0700 as excerpted: > Hi all, > > I have a collection of three (was 4) 1-2TB devices with data and > metadata in a RAID1 mirror. Last night I was struck by the Click of > Death on an old Samsung drive. > > I removed the device from the system, rebooted and mounted the volume > with `-o degraded` and the file system seems fine and usable. I'm > waiting on a replacement, drive but want to remove the old drive and > re-balance in the meantime. > > How do I remove the missing device? I tried the `btrfs device delete > missing /mnt` but was greeted with "ERROR: missing is not a block > device". A quick look at that btrfs-progs git repo shows that > `stat("missing")` is called, which of course fails since missing isn't a > block device. Nothing other then `btrfs replace` seemed intuitive and > all the docs mention the older command. What's the move? > > Thanks! > - Kyle > > Versions: > Kernel: 4.2.3-1-ARCH > btrfs-progs: 4.2.2-1 ᐧ I believe the current advice given here (that you were likely trying to follow, wrapped link)... https://btrfs.wiki.kernel.org/index.php/ Using_Btrfs_with_Multiple_Devices#Replacing_failed_devices ... is dated and no longer works due to code change some time in the past. There's a set of (very) recent patches, to the kernel and userspace both (I just updated userspace and it's in the git devel-branch v4.2.3-49- g4db87a1 I just built, kernelspace, I don't see it in linus-mainline yet, so I'd guess it's in the btrfs-integration patches, to land in the v4.4 commit window if not in 4.3 as it's getting late in the cycle for that. btrfs fi show That will list the btrfs component devices together with their devids. Then use the appropriate devid like so: btrfs dev del The -progs commit is d462081f, by Anand Jain, titled: btrfs-progs: Introduce device delete by devid According to it, the required kernel commit (title only listed) is similar: Btrfs: Introduce device delete by devid You can probably find them on-list if you wish to cherry-pick them into a current version. -- Duncan - List replies preferred. No HTML msgs. "Every nonfree program has a lord, a master -- and if you use the program, he is your master." Richard Stallman -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to remove missing device on RAID1?
On 2015-10-20 19:24, Kyle Manna wrote: > Hi all, [...] > How do I remove the missing device? I tried the `btrfs device delete > missing /mnt` but was greeted with "ERROR: missing is not a block > device". A quick look at that btrfs-progs git repo shows that > `stat("missing")` is called, which of course fails since missing isn't > a block device. Nothing other then `btrfs replace` seemed intuitive > and all the docs mention the older command. What's the move? In the past the device name was passed directly to the kernel, which operated in an appropriate manner if it was "missing". The commit e3c9596d3fc8b0df647a6cbae55c2d77570de759 introduced a check so if the argument is not a real device, an error is raised. So, now, if "missing" is passed, btrfs(-progs) stops because it is not a real device. This change is quite old (dec 2013); however it seems that this is the first time that a complaint is raised. > > Thanks! > - Kyle > > Versions: > Kernel: 4.2.3-1-ARCH > btrfs-progs: 4.2.2-1 > ᐧ > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- gpg @keyserver.linux.it: Goffredo Baroncelli Key fingerprint BBF5 1610 0B64 DAC6 5F7D 17B2 0EDA 9B37 8B82 E0B5 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to remove missing device on RAID1?
Thanks for the follow-up Duncan, that makes sense. I assumed I was doing something wrong. I downloaded the devel branch of of btrfs-progs and got it running before I saw the need for a kernel patch and decided to wait. For anyone following this later, I needed to use the following to get the missing device ID: btrfs device usage ᐧ On Tue, Oct 20, 2015 at 1:58 PM, Duncan <1i5t5.dun...@cox.net> wrote: > Kyle Manna posted on Tue, 20 Oct 2015 10:24:48 -0700 as excerpted: > >> Hi all, >> >> I have a collection of three (was 4) 1-2TB devices with data and >> metadata in a RAID1 mirror. Last night I was struck by the Click of >> Death on an old Samsung drive. >> >> I removed the device from the system, rebooted and mounted the volume >> with `-o degraded` and the file system seems fine and usable. I'm >> waiting on a replacement, drive but want to remove the old drive and >> re-balance in the meantime. >> >> How do I remove the missing device? I tried the `btrfs device delete >> missing /mnt` but was greeted with "ERROR: missing is not a block >> device". A quick look at that btrfs-progs git repo shows that >> `stat("missing")` is called, which of course fails since missing isn't a >> block device. Nothing other then `btrfs replace` seemed intuitive and >> all the docs mention the older command. What's the move? >> >> Thanks! >> - Kyle >> >> Versions: >> Kernel: 4.2.3-1-ARCH >> btrfs-progs: 4.2.2-1 ᐧ > > I believe the current advice given here (that you were likely trying to > follow, wrapped link)... > > https://btrfs.wiki.kernel.org/index.php/ > Using_Btrfs_with_Multiple_Devices#Replacing_failed_devices > > ... is dated and no longer works due to code change some time in the past. > > There's a set of (very) recent patches, to the kernel and userspace both > (I just updated userspace and it's in the git devel-branch v4.2.3-49- > g4db87a1 I just built, kernelspace, I don't see it in linus-mainline yet, > so I'd guess it's in the btrfs-integration patches, to land in the v4.4 > commit window if not in 4.3 as it's getting late in the cycle for that. > > btrfs fi show > > That will list the btrfs component devices together with their devids. > > Then use the appropriate devid like so: > > btrfs dev del > > The -progs commit is d462081f, by Anand Jain, titled: > > btrfs-progs: Introduce device delete by devid > > According to it, the required kernel commit (title only listed) is > similar: > > Btrfs: Introduce device delete by devid > > You can probably find them on-list if you wish to cherry-pick them into a > current version. > > -- > Duncan - List replies preferred. No HTML msgs. > "Every nonfree program has a lord, a master -- > and if you use the program, he is your master." Richard Stallman > > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to remove missing device on RAID1?
copy-paste error corrected On Wed, Oct 21, 2015 at 12:40 AM, Henk Slagerwrote: > I had a similar issue some time ago, around the time kernel 4.1.6 was > just there. > In case you don't want to wait for new disk or decide to just run the > filesystem with 1 disk less or maybe later on replace 1 of the still > healthy disks with a double/bigger sized one and use current/older > kernel+tools, you could do this (assuming the filesystem is not too > full of course): > - mount degraded - btrfs balance start -f -v -sdevid=1 -mdevid=1 -mdevid=1 > (where missing disk has devid 1) > After completion the (virtual/missing) device shall be fully unallocated > - create /dev/loopX with sparse file of same size as missing disk on > some other filesystem > - btrfs replace start 1 /dev/loopX > - remove /dev/loopX from the filesystem > - remount filesystyem without degraded > And remove /dev/loopX > > > On Tue, Oct 20, 2015 at 11:48 PM, Kyle Manna <2blu...@gmail.com> wrote: >> Thanks for the follow-up Duncan, that makes sense. I assumed I was >> doing something wrong. >> >> I downloaded the devel branch of of btrfs-progs and got it running >> before I saw the need for a kernel patch and decided to wait. >> >> For anyone following this later, I needed to use the following to get >> the missing device ID: >> >> btrfs device usage >> ᐧ >> >> On Tue, Oct 20, 2015 at 1:58 PM, Duncan <1i5t5.dun...@cox.net> wrote: >>> Kyle Manna posted on Tue, 20 Oct 2015 10:24:48 -0700 as excerpted: >>> Hi all, I have a collection of three (was 4) 1-2TB devices with data and metadata in a RAID1 mirror. Last night I was struck by the Click of Death on an old Samsung drive. I removed the device from the system, rebooted and mounted the volume with `-o degraded` and the file system seems fine and usable. I'm waiting on a replacement, drive but want to remove the old drive and re-balance in the meantime. How do I remove the missing device? I tried the `btrfs device delete missing /mnt` but was greeted with "ERROR: missing is not a block device". A quick look at that btrfs-progs git repo shows that `stat("missing")` is called, which of course fails since missing isn't a block device. Nothing other then `btrfs replace` seemed intuitive and all the docs mention the older command. What's the move? Thanks! - Kyle Versions: Kernel: 4.2.3-1-ARCH btrfs-progs: 4.2.2-1 ᐧ >>> >>> I believe the current advice given here (that you were likely trying to >>> follow, wrapped link)... >>> >>> https://btrfs.wiki.kernel.org/index.php/ >>> Using_Btrfs_with_Multiple_Devices#Replacing_failed_devices >>> >>> ... is dated and no longer works due to code change some time in the past. >>> >>> There's a set of (very) recent patches, to the kernel and userspace both >>> (I just updated userspace and it's in the git devel-branch v4.2.3-49- >>> g4db87a1 I just built, kernelspace, I don't see it in linus-mainline yet, >>> so I'd guess it's in the btrfs-integration patches, to land in the v4.4 >>> commit window if not in 4.3 as it's getting late in the cycle for that. >>> >>> btrfs fi show >>> >>> That will list the btrfs component devices together with their devids. >>> >>> Then use the appropriate devid like so: >>> >>> btrfs dev del >>> >>> The -progs commit is d462081f, by Anand Jain, titled: >>> >>> btrfs-progs: Introduce device delete by devid >>> >>> According to it, the required kernel commit (title only listed) is >>> similar: >>> >>> Btrfs: Introduce device delete by devid >>> >>> You can probably find them on-list if you wish to cherry-pick them into a >>> current version. >>> >>> -- >>> Duncan - List replies preferred. No HTML msgs. >>> "Every nonfree program has a lord, a master -- >>> and if you use the program, he is your master." Richard Stallman >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >>> the body of a message to majord...@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: How to remove missing device on RAID1?
I had a similar issue some time ago, around the time kernel 4.1.6 was just there. In case you don't want to wait for new disk or decide to just run the filesystem with 1 disk less or maybe later on replace 1 of the still healthy disks with a double/bigger sized one and use current/older kernel+tools, you could do this (assuming the filesystem is not too full of course): - mount degraded - btrfs balance start -f -v -sdevid=1 -sdevid=1 -sdevid=1 (where missing disk has devid 1) After completion the (virtual/missing) device shall be fully unallocated - create /dev/loopX with sparse file of same size as missing disk on some other filesystem - btrfs replace start 1 /dev/loopX - remove /dev/loopX from the filesystem - remount filesystyem without degraded And remove /dev/loopX On Tue, Oct 20, 2015 at 11:48 PM, Kyle Manna <2blu...@gmail.com> wrote: > Thanks for the follow-up Duncan, that makes sense. I assumed I was > doing something wrong. > > I downloaded the devel branch of of btrfs-progs and got it running > before I saw the need for a kernel patch and decided to wait. > > For anyone following this later, I needed to use the following to get > the missing device ID: > > btrfs device usage > ᐧ > > On Tue, Oct 20, 2015 at 1:58 PM, Duncan <1i5t5.dun...@cox.net> wrote: >> Kyle Manna posted on Tue, 20 Oct 2015 10:24:48 -0700 as excerpted: >> >>> Hi all, >>> >>> I have a collection of three (was 4) 1-2TB devices with data and >>> metadata in a RAID1 mirror. Last night I was struck by the Click of >>> Death on an old Samsung drive. >>> >>> I removed the device from the system, rebooted and mounted the volume >>> with `-o degraded` and the file system seems fine and usable. I'm >>> waiting on a replacement, drive but want to remove the old drive and >>> re-balance in the meantime. >>> >>> How do I remove the missing device? I tried the `btrfs device delete >>> missing /mnt` but was greeted with "ERROR: missing is not a block >>> device". A quick look at that btrfs-progs git repo shows that >>> `stat("missing")` is called, which of course fails since missing isn't a >>> block device. Nothing other then `btrfs replace` seemed intuitive and >>> all the docs mention the older command. What's the move? >>> >>> Thanks! >>> - Kyle >>> >>> Versions: >>> Kernel: 4.2.3-1-ARCH >>> btrfs-progs: 4.2.2-1 ᐧ >> >> I believe the current advice given here (that you were likely trying to >> follow, wrapped link)... >> >> https://btrfs.wiki.kernel.org/index.php/ >> Using_Btrfs_with_Multiple_Devices#Replacing_failed_devices >> >> ... is dated and no longer works due to code change some time in the past. >> >> There's a set of (very) recent patches, to the kernel and userspace both >> (I just updated userspace and it's in the git devel-branch v4.2.3-49- >> g4db87a1 I just built, kernelspace, I don't see it in linus-mainline yet, >> so I'd guess it's in the btrfs-integration patches, to land in the v4.4 >> commit window if not in 4.3 as it's getting late in the cycle for that. >> >> btrfs fi show >> >> That will list the btrfs component devices together with their devids. >> >> Then use the appropriate devid like so: >> >> btrfs dev del >> >> The -progs commit is d462081f, by Anand Jain, titled: >> >> btrfs-progs: Introduce device delete by devid >> >> According to it, the required kernel commit (title only listed) is >> similar: >> >> Btrfs: Introduce device delete by devid >> >> You can probably find them on-list if you wish to cherry-pick them into a >> current version. >> >> -- >> Duncan - List replies preferred. No HTML msgs. >> "Every nonfree program has a lord, a master -- >> and if you use the program, he is your master." Richard Stallman >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html > -- > To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html