Re: kernel debugging and ULE
on 06/02/2012 07:52 Julian Elischer said the following: so if I'm sitting still in the debugger for too long, a hardclock event happens that goes into ULE, which then hits the following KASSERT. KASSERT(pri = PRI_MIN_BATCH pri = PRI_MAX_BATCH, (sched_priority: invalid priority %d: nice %d, ticks %d ftick %d ltick %d tick pri %d, pri, td-td_proc-p_nice, td-td_sched-ts_ticks, td-td_sched-ts_ftick, td-td_sched-ts_ltick, SCHED_PRI_TICKS(td-td_sched))); The reason seems to be that I've been sitting still for too long and things have become pear shaped. how is it that being in the debugger doesn't stop hardclock events? is there something I can do to make them not happen.. It means I have to ge tmy debugging done in less than about 60 seconds. suggesions welcome. Does this really happen when you just sit in the debugger? Or does it happen when you let the kernel run? Like stepping through the code, etc -- Andriy Gapon ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [ptrace] please review follow fork/exec changes
On Mon, Feb 06, 2012 at 01:19:30PM -0800, Dmitry Mikulin wrote: I see what is going on. The wait loop for P_PPWAIT in do_fork() simply do not allow the ptracestop() in the syscall return path to be reached. There seems to be more problems. In particular, I do not see anything which would prevent the child from being reapped while the loop is executing (assume that the parent is multithreaded and other thread processed SIGCHLD and called wait). Lets deal with these bugs after your proposal for interface changes is dealt with. OK. Yes, I agree with the proposal to add flag to the child lwp info. I think it will be easier if the flag is different from PL_FLAG_FORKED. I named it PL_FLAG_CHILD. PT_FOLLOW_EXEC is easy to implement, but my question is, how can debugger operate (correctly) if it ignores exec events ? After exec, the whole cached state of the debuggee must be invalidated, and since debugger ignores the notification when the invalidation shall be done, it probably gets very confused. You're right, the debugger needs to handle exec() events implicitly when it starts up executables. The problem is that there is OS-independent machinery in gdb which handles statup fork-exec sequence differently from when the debuggee itself does an exec(). Basically in the event handling code I need to be able to distinguish app startup by gdb from an exec done by the app. Other OS-es have flags like PL_FLAG_EXEC set on demand: they have an equivalent of PT_FOLLOW_EXEC. I attached a modified patch that solves the problem. It tries to separate the always-on TDB_EXEC from the on-demand TDB_FOLLOWEXEC without changing existing functionality. Let me know if it's acceptable. So, do you in fact need to distinguish exec stops from syscall exit against exec stops from PT_FOLLOW_EXEC, or do you need to only get stops at exec returns from PT_CONTINUE when explicitely requested them ? I would prefer to not introduce another PL_FLAG_somethingEXEC with the same semantic as PL_FLAG_EXEC. Instead, would the following patch be fine for your purposes ? With it, stop on exec should only occur if PT_SCX is requested, or PT_CONTINUE and PT_FOLLOW_EXEC. [I am unable to fully test this until tomorrow]. diff --git a/sys/kern/kern_exec.c b/sys/kern/kern_exec.c index 135f798..67cb1b2 100644 --- a/sys/kern/kern_exec.c +++ b/sys/kern/kern_exec.c @@ -56,6 +56,7 @@ __FBSDID($FreeBSD$); #include sys/proc.h #include sys/pioctl.h #include sys/namei.h +#include sys/ptrace.h #include sys/resourcevar.h #include sys/sdt.h #include sys/sf_buf.h @@ -889,7 +890,9 @@ exec_fail_dealloc: if (error == 0) { PROC_LOCK(p); - td-td_dbgflags |= TDB_EXEC; + if ((p-p_flag P_TRACED) != 0 + ((P_FOLLOWEXEC) != 0 || (p-p_stops S_PT_SCX) != 0)) + td-td_dbgflags |= TDB_EXEC; PROC_UNLOCK(p); /* diff --git a/sys/kern/kern_fork.c b/sys/kern/kern_fork.c index 60639c9..e447c93 100644 --- a/sys/kern/kern_fork.c +++ b/sys/kern/kern_fork.c @@ -1035,7 +1035,9 @@ fork_return(struct thread *td, struct trapframe *frame) p-p_oppid = p-p_pptr-p_pid; proc_reparent(p, dbg); sx_xunlock(proctree_lock); + td-td_dbgflags |= TDB_CHILD; ptracestop(td, SIGSTOP); + td-td_dbgflags = ~TDB_CHILD; } else { /* * ... otherwise clear the request. diff --git a/sys/kern/sys_process.c b/sys/kern/sys_process.c index 4510380..79bbaed 100644 --- a/sys/kern/sys_process.c +++ b/sys/kern/sys_process.c @@ -660,6 +660,7 @@ kern_ptrace(struct thread *td, int req, pid_t pid, void *addr, int data) case PT_TO_SCX: case PT_SYSCALL: case PT_FOLLOW_FORK: + case PT_FOLLOW_EXEC: case PT_DETACH: sx_xlock(proctree_lock); proctree_locked = 1; @@ -873,6 +874,12 @@ kern_ptrace(struct thread *td, int req, pid_t pid, void *addr, int data) else p-p_flag = ~P_FOLLOWFORK; break; + case PT_FOLLOW_EXEC: + if (data) + p-p_flag |= P_FOLLOWEXEC; + else + p-p_flag = ~P_FOLLOWEXEC; + break; case PT_STEP: case PT_CONTINUE: @@ -936,7 +943,8 @@ kern_ptrace(struct thread *td, int req, pid_t pid, void *addr, int data) p-p_sigparent = SIGCHLD; } p-p_oppid = 0; - p-p_flag = ~(P_TRACED | P_WAITED | P_FOLLOWFORK); + p-p_flag = ~(P_TRACED | P_WAITED | P_FOLLOWFORK | + P_FOLLOWEXEC); /* should we send SIGCHLD? */ /*
Re: Freebsd 9.0 release and dmesg
On 06/02/2012 18:24, JD wrote: dmesg no longer outputs the kernel messages. $ dmesg $ $ which dmesg /sbin/dmesg $what /sbin/dmesg /sbin/dmesg: So, I have no idea what version of dmesg got installed. Anyone on 9.0 Release have this problem? How to fix it? I thought this was by design, I've noticed it very early (7.x, probably earlier) and just thought that when lot of data is written to the console, the actual kernel message buffer gets invalidated or freed. This made me stop using dmesg altogether and just look at either /var/run/dmesg.boot for the boot messages or /var/log/messages. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
RE: LSI supported mps(4) driver available
-Original Message- From: Stas Orlov [mailto:senn...@gmail.com] Sent: Monday, February 06, 2012 11:54 PM To: Desai, Kashyap Cc: Kenneth D. Merry; freebsd-s...@freebsd.org; freebsd- curr...@freebsd.org; Dennis Glatting Subject: Re: LSI supported mps(4) driver available Sorry for the swarm of screenshots :) Spare drive should be a iDRAC Virtual usb. On that particular machine, as I said, two RAID arrays (1 and 10) Is this all SATA drives part of your raid1 and raid10 ? http://oi39.tinypic.com/10hlfmg.jpg -- they initialized as they should. http://oi43.tinypic.com/2db83g8.jpg -- loader recognizes 3 disks (3rd one is the iDRAC usb as I suppose) So, I flashed the FW and installed the latest one from Dell (7.15.08.00 \ 7.03.05.00), and things have gone slightly better. http://oi39.tinypic.com/69dtuq.jpg -- controller . http://oi44.tinypic.com/10s6jxi.jpg -- still. I am debugging this issue. ! I can understand the issue, but not sure how to progress. Do you willing to make more test with my debug builds ? Here In my setup I am not able to reproduce the issue. ~ Kashyap http://oi42.tinypic.com/ipmh37.jpg -- It boots in single user, in multiuser it prints probe errors, skips them and hang on the daemon startup. http://oi43.tinypic.com/13yhz02.jpg -- in single user they initialized\UFSed\mounted just fine. To sum everything up, FW upgrade helped, still some device handle errors and SCSI errors, hangs in multiuser-mode(may or may not be related). Tested with the same FreeBSD10-CURRENT r230857, I saw ken@'s commit to -STABLE and will try it somewhere tomorrow. Thanks for your time:) On Fri, Feb 3, 2012 at 12:10 PM, Desai, Kashyap kashyap.de...@lsi.com wrote: From: Stas Orlov [mailto:senn...@gmail.com] Sent: Thursday, February 02, 2012 8:29 PM To: Desai, Kashyap Cc: Kenneth D. Merry; freebsd-s...@freebsd.org; freebsd- curr...@freebsd.org; Dennis Glatting Subject: Re: LSI supported mps(4) driver available (Again clarify which version of FreeBSD you are using) As I've stated earlier it's current snapshot from the other day, to be more specific FreeBSD10-CURRENT r230857 Ok, I'll try to upgrade firmware. On Thu, Feb 2, 2012 at 6:43 PM, Desai, Kashyap kashyap.de...@lsi.com wrote: Can you switch your mail client to default text mode reply. It is always turning into html format and difficult for inline reply. I have done some analysis on of your logs provided at http://oi40.tinypic.com/25gdw8o.jpg; 1. it seems Driver is somehow not handling error condition which should be better handled at driver. e.a driver does not reinit HBA if any config request time out. I will add this feature sometime later, since I have some more item queued up as well. 2. Your logs mentioned there are three different handled got from FW to add as Bare Drive. (it is not a volume entry) So just curious to know why those entries are coming as bare drive. ? (do you have any other bare drives in your topology ? ) ` Kashyap From: Stas Orlov [mailto:senn...@gmail.com] Sent: Thursday, February 02, 2012 7:45 PM To: Desai, Kashyap Cc: Kenneth D. Merry; freebsd-s...@freebsd.org; freebsd- curr...@freebsd.org; Dennis Glatting Subject: Re: LSI supported mps(4) driver available Sure, I've made two RAID (1 and 10 ) arrays within LSI Config Utility and tried to install fresh current, every time I get error linked above, so yes, it's reproducible. - What I understood here is, you have two raid volumes RAID1 and RAID10, and trying to install FreeBSD (Again clarify which version of FreeBSD you are using) Try erasing a controller FW completely and re-install everything from fresh. (Here make sure you flash completely. Hope you are aware of controller firmware upgrade process) Our board has DPM tables and for Raid volume it is maximum 2 entry. When you have more than two inactive volumes, we cannot add another raid volume. There is some implementation recently done by BIOS team related to this area. Where BIOS itself will erase inactive Raid volume entry from DPM pages. ~ Kashyap MPT Firmware 2.15.63.00-IR Package Version 7.01.33.00 I do have another spare R610 with that card, I'll test it with current later this evening. On Thu, Feb 2, 2012 at 5:53 PM, Desai, Kashyap kashyap.de...@lsi.com wrote: -Original Message- From: owner-freebsd-s...@freebsd.org [mailto:owner-freebsd- s...@freebsd.org] On Behalf Of Stas Orlov Sent: Thursday, February 02, 2012 6:48 PM To: Kenneth D. Merry Cc: freebsd-s...@freebsd.org; freebsd-current@freebsd.org; Dennis Glatting Subject: Re: LSI supported mps(4) driver available Hi, We have a pack of identical Dell R610 mahcines with H200 cards. pciconf from R610 with FreeBSD9 on ZFS, disks in JBOD mode. mps0@pci0:3:0:0: class=0x010700 card=0x1f1e1028 chip=0x00721000 rev=0x02 hdr=0x00
RE: LSI supported mps(4) driver available
Can you to reproduce issue with below mentioned changes.. In mps.c mps_get_tunables(struct mps_softc *sc) { char tmpstr[80]; /* XXX default to some debugging for now */ sc-mps_debug = MPS_FAULT; Instead of above line make sc-mps_debug = 0xd; This will dump debug prints on screen at the time of boot itself. ~ Kashyap -Original Message- From: owner-freebsd-curr...@freebsd.org [mailto:owner-freebsd- curr...@freebsd.org] On Behalf Of Desai, Kashyap Sent: Tuesday, February 07, 2012 8:37 PM To: Stas Orlov Cc: freebsd-s...@freebsd.org; freebsd-current@freebsd.org; Kenneth D. Merry; Dennis Glatting Subject: RE: LSI supported mps(4) driver available -Original Message- From: Stas Orlov [mailto:senn...@gmail.com] Sent: Monday, February 06, 2012 11:54 PM To: Desai, Kashyap Cc: Kenneth D. Merry; freebsd-s...@freebsd.org; freebsd- curr...@freebsd.org; Dennis Glatting Subject: Re: LSI supported mps(4) driver available Sorry for the swarm of screenshots :) Spare drive should be a iDRAC Virtual usb. On that particular machine, as I said, two RAID arrays (1 and 10) Is this all SATA drives part of your raid1 and raid10 ? http://oi39.tinypic.com/10hlfmg.jpg -- they initialized as they should. http://oi43.tinypic.com/2db83g8.jpg -- loader recognizes 3 disks (3rd one is the iDRAC usb as I suppose) So, I flashed the FW and installed the latest one from Dell (7.15.08.00 \ 7.03.05.00), and things have gone slightly better. http://oi39.tinypic.com/69dtuq.jpg -- controller . http://oi44.tinypic.com/10s6jxi.jpg -- still. I am debugging this issue. ! I can understand the issue, but not sure how to progress. Do you willing to make more test with my debug builds ? Here In my setup I am not able to reproduce the issue. ~ Kashyap http://oi42.tinypic.com/ipmh37.jpg -- It boots in single user, in multiuser it prints probe errors, skips them and hang on the daemon startup. http://oi43.tinypic.com/13yhz02.jpg -- in single user they initialized\UFSed\mounted just fine. To sum everything up, FW upgrade helped, still some device handle errors and SCSI errors, hangs in multiuser-mode(may or may not be related). Tested with the same FreeBSD10-CURRENT r230857, I saw ken@'s commit to -STABLE and will try it somewhere tomorrow. Thanks for your time:) On Fri, Feb 3, 2012 at 12:10 PM, Desai, Kashyap kashyap.de...@lsi.com wrote: From: Stas Orlov [mailto:senn...@gmail.com] Sent: Thursday, February 02, 2012 8:29 PM To: Desai, Kashyap Cc: Kenneth D. Merry; freebsd-s...@freebsd.org; freebsd- curr...@freebsd.org; Dennis Glatting Subject: Re: LSI supported mps(4) driver available (Again clarify which version of FreeBSD you are using) As I've stated earlier it's current snapshot from the other day, to be more specific FreeBSD10-CURRENT r230857 Ok, I'll try to upgrade firmware. On Thu, Feb 2, 2012 at 6:43 PM, Desai, Kashyap kashyap.de...@lsi.com wrote: Can you switch your mail client to default text mode reply. It is always turning into html format and difficult for inline reply. I have done some analysis on of your logs provided at http://oi40.tinypic.com/25gdw8o.jpg; 1. it seems Driver is somehow not handling error condition which should be better handled at driver. e.a driver does not reinit HBA if any config request time out. I will add this feature sometime later, since I have some more item queued up as well. 2. Your logs mentioned there are three different handled got from FW to add as Bare Drive. (it is not a volume entry) So just curious to know why those entries are coming as bare drive. ? (do you have any other bare drives in your topology ? ) ` Kashyap From: Stas Orlov [mailto:senn...@gmail.com] Sent: Thursday, February 02, 2012 7:45 PM To: Desai, Kashyap Cc: Kenneth D. Merry; freebsd-s...@freebsd.org; freebsd- curr...@freebsd.org; Dennis Glatting Subject: Re: LSI supported mps(4) driver available Sure, I've made two RAID (1 and 10 ) arrays within LSI Config Utility and tried to install fresh current, every time I get error linked above, so yes, it's reproducible. - What I understood here is, you have two raid volumes RAID1 and RAID10, and trying to install FreeBSD (Again clarify which version of FreeBSD you are using) Try erasing a controller FW completely and re-install everything from fresh. (Here make sure you flash completely. Hope you are aware of controller firmware upgrade process) Our board has DPM tables and for Raid volume it is maximum 2 entry. When you have more than two inactive volumes, we cannot add another raid volume. There is some implementation recently done by BIOS team related to this area. Where BIOS itself will erase inactive Raid volume entry from DPM pages. ~
Re: LSI supported mps(4) driver available
On Tue, Feb 07, 2012 at 21:00:28 +0530, Desai, Kashyap wrote: Can you to reproduce issue with below mentioned changes.. In mps.c mps_get_tunables(struct mps_softc *sc) { char tmpstr[80]; /* XXX default to some debugging for now */ sc-mps_debug = MPS_FAULT; Instead of above line make sc-mps_debug = 0xd; You can also put the following in /boot/loader.conf: hw.mps.debug_level=0xd Ken -- Kenneth Merry k...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [ptrace] please review follow fork/exec changes
On Monday, February 06, 2012 10:12:11 pm David Xu wrote: On 2012/1/26 7:48, Dmitry Mikulin wrote: snip The debugger needs to intercept fork() in both parent and child so it can detach from the old process and attach to the new one. Maybe it'll make more sense in the context of gdb changes. Should I send them too? Don't think Marcel included that patch... Does the orphan list change intended to not lost the child after fork ? But the child shall be traced, so debugger would get the SIGTRAP on the attach on fork returning to usermode. I remember that I explicitely tested this when adding followfork changes. Yes, the debugger gets SIGTRAPs. The problem arises when the real parent of the forked process has the code to collect termination status. Since attaching to a process changes the parent/child relationships, we need to keep track of the children lost due to re-parenting so we can properly attribute their exit status to the real parent. I recall that someone brought a topic in the list said that this should be fixed, debugging a process should not change parent-child relation, instead a new link list data structure should be added to struct proc to trace debugged process, this will make code clean with a small memory overhead. Yes, I have some old patches to start on this, but I hadn't really finished them, and it makes wait() a bit more complicated. It would be nice if ptrace() had its own pwait() or some such instead of overloading wait(). -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: kernel debugging and ULE
On Monday, February 06, 2012 12:52:58 am Julian Elischer wrote: so if I'm sitting still in the debugger for too long, a hardclock event happens that goes into ULE, which then hits the following KASSERT. KASSERT(pri = PRI_MIN_BATCH pri = PRI_MAX_BATCH, (sched_priority: invalid priority %d: nice %d, ticks %d ftick %d ltick %d tick pri %d, pri, td-td_proc-p_nice, td-td_sched-ts_ticks, td-td_sched-ts_ftick, td-td_sched-ts_ltick, SCHED_PRI_TICKS(td-td_sched))); The reason seems to be that I've been sitting still for too long and things have become pear shaped. how is it that being in the debugger doesn't stop hardclock events? is there something I can do to make them not happen.. It means I have to ge tmy debugging done in less than about 60 seconds. suggesions welcome. I committed a workaround to HEAD for this recently (r228960). Just make sure that is merged into whatever tree you are using. -- John Baldwin ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [ptrace] please review follow fork/exec changes
So, do you in fact need to distinguish exec stops from syscall exit against exec stops from PT_FOLLOW_EXEC, This is pretty much what I need. It's the same stop in syscall return right? I don't want to change when the stop happens, I want to have an lwpinfo flag that tells me when a stop occurred in a process under PT_FOLLOW_EXEC. @@ -889,7 +890,9 @@ exec_fail_dealloc: if (error == 0) { PROC_LOCK(p); - td-td_dbgflags |= TDB_EXEC; + if ((p-p_flag P_TRACED) != 0 + ((P_FOLLOWEXEC) != 0 || (p-p_stops S_PT_SCX) != 0)) + td-td_dbgflags |= TDB_EXEC; PROC_UNLOCK(p); There's a small bug in the patch that makes it not work. The check for P_FOLLOWEXEC should be: + ((p-p_flag P_FOLLOWEXEC) != 0 || (p-p_stops S_PT_SCX) != 0)) Looks like the patch should work for me but I need to verify. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Packages for Current ( 10.0 )
Dear All , At present , ftp://ftp.freebsd.org/pub/FreeBSD/ports/amd64/packages-10-current/ is empty , and , http://pub.allbsd.org/FreeBSD-snapshots/ amd64 , head is prepared without ports.txz . To download a snapshot and test Current ( 10.0 ) , without ports seems to be not possible . Which ports can be used for Current ( 10.0 ) for X , Gnome , KDE , etc . ? Thank you very much . Mehmet Erol Sanliturk ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: [ptrace] please review follow fork/exec changes
Well, that didn't work... Not sure why since it broke existing gdb. My guess is we're not getting the exec stops we used to get. Might have to wait till tomorrow to get more details. On 02/07/2012 12:45 PM, Dmitry Mikulin wrote: So, do you in fact need to distinguish exec stops from syscall exit against exec stops from PT_FOLLOW_EXEC, This is pretty much what I need. It's the same stop in syscall return right? I don't want to change when the stop happens, I want to have an lwpinfo flag that tells me when a stop occurred in a process under PT_FOLLOW_EXEC. @@ -889,7 +890,9 @@ exec_fail_dealloc: if (error == 0) { PROC_LOCK(p); -td-td_dbgflags |= TDB_EXEC; +if ((p-p_flag P_TRACED) != 0 +((P_FOLLOWEXEC) != 0 || (p-p_stops S_PT_SCX) != 0)) +td-td_dbgflags |= TDB_EXEC; PROC_UNLOCK(p); There's a small bug in the patch that makes it not work. The check for P_FOLLOWEXEC should be: +((p-p_flag P_FOLLOWEXEC) != 0 || (p-p_stops S_PT_SCX) != 0)) Looks like the patch should work for me but I need to verify. ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Announcing Emulex 10G Ethernet NIC driver availability
Hi All, Please find the 10Gb Ethernet NIC driver for Emulex OneConnect (BladeEngine) and Lancer family of network adapters at http://info.iet.unipi.it/~luigi/20120207-emulex-nic.tgzhttps://iwebmail.emulex.com/OWA/redir.aspx?C=fc4deb7ba3ea44aa8034c6a14f2f59f6URL=http%3a%2f%2finfo.iet.unipi.it%2f%7eluigi%2f20120207-emulex-nic.tgz . Luigi Rizzo has already reviewed the driver and has agreed to do commit-to-the-tree (thanks a lot Luigi). Please review and send us any comments you may have. Thanks, ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org
Re: Announcing Emulex 10G Ethernet NIC driver availability
Any plans for iscsi, fcoe? Hi All, Please find the 10Gb Ethernet NIC driver for Emulex OneConnect (BladeEngine) and Lancer family of network adapters at http://info.iet.unipi.it/~luigi/20120207-emulex-nic.tgzhttps://iwebmail.emulex.com/OWA/redir.aspx?C=fc4deb7ba3ea44aa8034c6a14f2f59f6URL=http%3a%2f%2finfo.iet.unipi.it%2f%7eluigi%2f20120207-emulex-nic.tgz . Luigi Rizzo has already reviewed the driver and has agreed to do commit-to-the-tree (thanks a lot Luigi). Please review and send us any comments you may have. Thanks, ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org ___ freebsd-current@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-current To unsubscribe, send any mail to freebsd-current-unsubscr...@freebsd.org