Re: x86intrin.h not found after 10.4->11.2 upgrade
Ah, that's the problem, /usr/include/x86intrin.h was a symlink to an old clang 3.4.1 version of that header which doesn't exist on the filesystem any more, I've deleted it and now things are much happier. Really ought to do a clean rebuild on this box one of these days... Thanks, Morgan On Wed, Jan 9, 2019 at 5:47 AM Dimitry Andric wrote: > On 8 Jan 2019, at 12:50, Morgan Reed wrote: > > > > Just did a find across /usr for the file and it's definitely there so I'm > > not sure why the compiler can't find it :/ > > Please post the output of: > > cc -v -x c -c /dev/null -o /dev/null > > > > > > # find /usr -name x86intrin.h > > /usr/include/x86intrin.h > > This copy should not exist. Any idea where it came from, and what is > its contents? > > > > /usr/src/contrib/llvm/tools/clang/lib/Headers/x86intrin.h > > > /usr/local/lib/gcc7/gcc/x86_64-portbld-freebsd11.2/7.4.0/include/x86intrin.h > > /usr/local/llvm60/lib/clang/6.0.1/include/x86intrin.h > > /usr/lib/clang/6.0.0/include/x86intrin.h > > If your /usr/bin/cc reports being clang 6.0.0, then the latter one is > correct. > > My guess would be that something (/etc/make.conf, for instance) is > messing with your CFLAGS, causing the internal headers to not be found. > > -Dimitry > > -- "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759 ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: x86intrin.h not found after 10.4->11.2 upgrade
Just did a find across /usr for the file and it's definitely there so I'm not sure why the compiler can't find it :/ # find /usr -name x86intrin.h /usr/include/x86intrin.h /usr/src/contrib/llvm/tools/clang/lib/Headers/x86intrin.h /usr/local/lib/gcc7/gcc/x86_64-portbld-freebsd11.2/7.4.0/include/x86intrin.h /usr/local/llvm60/lib/clang/6.0.1/include/x86intrin.h /usr/lib/clang/6.0.0/include/x86intrin.h This system's been around the block (started out as 9.3 I think, had a few upgrades over the years) which would explain the gcc7 stuff. On Tue, Jan 8, 2019 at 6:52 PM Morgan Reed wrote: > Hi All, > > Finally got around to upgrading my NAS which was running FreeBSD > 10.4 to a supported version (11.2). > > Ran into an issue when I came to do a portupgrade -a to update all my > installed ports, a number of the ports are failing with "x86intrin.h No > such file or directory", not sure what's going on there. > > I've updated my src tree just in case it was something that only got added > in 11, no change. > > All the similar reports I found online were related to OLD versions of gcc > (e.g. 4.3), which is not relevant since we're using clang (I assume > anyway), but I reinstalled llvm60 from package just in case, no change > again. > > An pointers would be greatly appreciated, thanks. > > Morgan > > > > -- "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759 ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
x86intrin.h not found after 10.4->11.2 upgrade
Hi All, Finally got around to upgrading my NAS which was running FreeBSD 10.4 to a supported version (11.2). Ran into an issue when I came to do a portupgrade -a to update all my installed ports, a number of the ports are failing with "x86intrin.h No such file or directory", not sure what's going on there. I've updated my src tree just in case it was something that only got added in 11, no change. All the similar reports I found online were related to OLD versions of gcc (e.g. 4.3), which is not relevant since we're using clang (I assume anyway), but I reinstalled llvm60 from package just in case, no change again. An pointers would be greatly appreciated, thanks. Morgan ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: entropy lockup
In all likelihood the process wasn't "hung" per-se, more likely random hadn't been seeded yet and as such you *can't* get 4096b of entropy out of it (so the process was attempting to do its job, just nothing on the device). The issue is basically that there are very few "good" entropy sources in a VM environment, and as such (particularly on a machine which is only running SSH) there's not enough data available to seed random. Check 'man 4 random' for some discussion. On Tue, Sep 12, 2017 at 12:02 AM, Vick Khera wrote: > I just had a VM running in Google's cloud become totally useless, and I > tracked it down to the save-entropy operation. > > Basically this process was sucking up all CPU, even when nothing else > running other than my ssh shell: > > % ps axuw803 > USER PID %CPU %MEM VSZ RSS TT STAT STARTED TIME COMMAND > operator 803 100.0 0.1 8336 2096 - RL 08:55 48:20.14 dd > if=/dev/random of=saved-entropy.1 bs=4096 count=1 > > > The process is unkillable, and I cannot even get the system to shut down. > That has been hanging for about 10 minutes so far, with the last output > being > > System shutdown time has arrive90 second watchdSep 11 09:50:02 yertle init: > /bin/sh on /etc/rc.shutdown terminated abnormally, going to single user > mode > Sep 11 09:50:47 init: some processes would not die; ps axl advised > Waiting (max 60 seconds) for system process `vnlru' to stop... done > Waiting (max 60 seconds) for system process `bufdaemon' to stop... done > Waiting (max 60 seconds) for system process `syncer' to stop... > Syncing disks, vnodes remaining... 4 4 4 4 4 4 4 timed out > 2 2 2 2 2 2 2 > > > Running FreeBSD 11.1-p1 on a 1CPU standard machine in GCE. > > What's the proper recovery from this kind of lockup, or how to prevent it? > I've never encountered this on a bare metal system, or other KVM based > machines. > ___ > freebsd-stable@freebsd.org mailing list > https://lists.freebsd.org/mailman/listinfo/freebsd-stable > To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org" > -- "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759 ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Not all ZFS pools mounting at boot
On Wed, Mar 9, 2016 at 10:04 PM, Morgan Reed wrote: > On Wed, Mar 9, 2016 at 7:23 PM, Andriy Gapon wrote: > >> On 09/03/2016 06:54, Morgan Reed wrote: >> > I am mounting the third pool under an alternate root, if that has any >> > bearing on things. >> >> It does. A pool imported under an alternative root (-R option) is not >> added to >> zpool.cache. >> > > Ah, it would seem that this is the root of my issue then, I'll try > importing it without the altroot switch and we'll see what happens. > We have a winner, pool now mounts, a quick zfs set mountpoint later and it even mounts in the right place. Thanks for the assist. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: Not all ZFS pools mounting at boot
On Wed, Mar 9, 2016 at 7:23 PM, Andriy Gapon wrote: > On 09/03/2016 06:54, Morgan Reed wrote: > > I am mounting the third pool under an alternate root, if that has any > > bearing on things. > > It does. A pool imported under an alternative root (-R option) is not > added to > zpool.cache. > Ah, it would seem that this is the root of my issue then, I'll try importing it without the altroot switch and we'll see what happens. ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Not all ZFS pools mounting at boot
Hi All, Been fighting this one for a while and I figure it's time to take it to the brainstrust. I've recently added a third storage array to my NAS, it's a RAID-Z pool the same as the other two arrays in the box, the new array isn't mounting at boot, I have to manually import it every time. Done some digging around online but everything I've found has come down to the person not having zfs_enable=YES in rc.conf, obviously as two of my three pools are mounting at boot this is not my issue. Only oddity I've found is that, the output of zdb does NOT show the third storage pool, although it is presently imported and mounted. I am mounting the third pool under an alternate root, if that has any bearing on things. Any thoughts? Thanks, Morgan Reed ___ freebsd-stable@freebsd.org mailing list https://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: natd in a jail
SOLVED: Thanks all for your assistance.
SUMMARY:
- Kernel rebuilt with option IPFIREWALL and friends turned on (not
necessary if your ipfw modules work you should just be able to load
them, mine didn't for reasons I don't really have the time or
inclination to track down)
- OpenVPN configurations modified to use a specific tun device
('device tun' directive replaced with 'device tun0')
- OpenVPN configurations modified to run the following script prior
to dropping privs (via the 'up' directive);
/usr/local/etc/openvpn/up.sh
---
ipfw -q flush
pfw nat 1 config if tun0 reset same_ports deny_in
ipfw add 500 nat 1 ip from any to any via tun0
---
This script assumes that option IPFILTER_DEFAULT_TO_ACCEPT or the
equivalent sysctl frob is set, this is most probably *not* what you
want to do in the "real world". Modify as needed, and be sure to set
the permissions on the file appropriately as the script will be
executed by root.
A warning though; this is a total hack, the ipfw stuff should be moved
to /etc/ipfw.rules or similar and processed by ipfw at boot but I'm
not sure how it'll react if you try to do this config before the tun
device is created, I expect it'll be unhappy so you'll need to create
a static tun device for openvpn, this is the "right" way to do things
but I'm being exceedingly lazy. The script above is a filthy hack, and
potentially dangerous.
- Normal requirements for gateway operation also apply here (which
essentially means set gateway_enable=YES in rc.conf on the host and
all router jails).
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: natd in a jail
On Sat, Nov 24, 2012 at 5:44 PM, Morgan Reed wrote: > Works like a charm, just one last thing I'd like to get squared away > here though, currently OpenVPN is using a dynamically created tun > device, I'd like to have a static /dev/tun0 exist prior to the > /etc/rc.d/natd start launching (because as it is I have to restart > natd after the openvpn tunnel comes up), not sure what the best way to > achieve this is in a jailed environment though. Scratch that, I definitely need a holiday... natd_enable removed from rc.conf, appropriate ipfw script being run by openvpn prior to dropping its privs (by way of the up directive) and it "just works" (tm) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: natd in a jail
On Sat, Nov 24, 2012 at 9:16 PM, Morgan Reed wrote: >> And with ipfw nat you won't be needing ipdivert. Again, no harm. > > Yeah, I didn't think it should be necessary but something was trying > to load it from within the jails and throwing an error, probably the > natd startup script, not sure why, I might do some digging if I get > bored at some point. *facepalm* or I could just remove natd_enable from rc.conf since I don't need it anymore... ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: natd in a jail
On Sat, Nov 24, 2012 at 7:26 PM, Ian Smith wrote: > Unless you needed to include FIREWALL_FORWARD, you really didn't need to > build ipfw into the kernel, it's all loadable by module. No harm, but. The ipfw_nat module was causing an instant panic at load and I was going to have to rebuild my kernel to debug that anyway, went with the sledgehammer approach and built it in, this box won't be doing anything else so it's no problem. > And with ipfw nat you won't be needing ipdivert. Again, no harm. Yeah, I didn't think it should be necessary but something was trying to load it from within the jails and throwing an error, probably the natd startup script, not sure why, I might do some digging if I get bored at some point. > If the address of the tunX interface is fixed in the jail, you can > specify it by IP instead of the interface in the nat setup, like: > > ipfw nat 1 config ip $address same_ports deny_in > ipfw add 500 nat 1 ip from any to any via $address > > Your use of 'reset' in nat config makes me wonder if it's a variable > address though? If IP varies you will need to specify the interface. Dynamically assigned IP address, I don't control the remote end of the tunnel, IP changes each time the tunnel connects. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: natd in a jail
On Fri, Nov 23, 2012 at 5:16 PM, Morgan Reed wrote: > So it turns out I'd not bought bpf into the jails, however even with > that and raw_sockets enabled I'm still having no joy with natd. > > I've been looking at ipfw a bit today but I've run into an issue, > loading ipfw_nat causes my kernel to instantly panic, I need to > recompile with KDB and DDB turned on so I can actually catch the trace > though... Might look at netgraph before going too far down that path. Rebuilt the kernel with option IPFIREWALL and friends turned on (including IPFILTER_DEFAULT_TO_ACCEPT or whatever it is). Throw ipfw_nat_load="YES" and ipdivert_load="YES" into /boot/loader.conf so the modules are available for the jails. Run a quick and dirty ipfw script (running out of an 'up' script I wrote into the OpenVPN config); ipfw nat 1 config if tun0 reset same_ports deny_in ipfw add 500 nat 1 ip from any to any via tun0 Works like a charm, just one last thing I'd like to get squared away here though, currently OpenVPN is using a dynamically created tun device, I'd like to have a static /dev/tun0 exist prior to the /etc/rc.d/natd start launching (because as it is I have to restart natd after the openvpn tunnel comes up), not sure what the best way to achieve this is in a jailed environment though. The next trick will be migrating from my spaghetti script into rc launched jails... ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: natd in a jail
On Fri, Nov 23, 2012 at 7:48 PM, Andreas Nilsson wrote: > Why not just load the module? Yeah, you got beaten to the punch on that one offlist, it's late in the day here ;) ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: natd in a jail
On Fri, Nov 23, 2012 at 5:16 PM, Morgan Reed wrote: > So it turns out I'd not bought bpf into the jails, however even with > that and raw_sockets enabled I'm still having no joy with natd. > > I've been looking at ipfw a bit today but I've run into an issue, > loading ipfw_nat causes my kernel to instantly panic, I need to > recompile with KDB and DDB turned on so I can actually catch the trace > though... Might look at netgraph before going too far down that path. Scratch that, netgtaph isn't in the GENERIC kernel, so I'll have to rebuild anyway. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: natd in a jail
On Thu, Nov 22, 2012 at 10:36 PM, Morgan Reed wrote: > BPF is enabled for the jails, and the traffic is getting to where it > needs to (but not via natd). I'll try enabling raw_sockets in the > jails, it is entirely conceivable that natd requires that > functionality. So it turns out I'd not bought bpf into the jails, however even with that and raw_sockets enabled I'm still having no joy with natd. I've been looking at ipfw a bit today but I've run into an issue, loading ipfw_nat causes my kernel to instantly panic, I need to recompile with KDB and DDB turned on so I can actually catch the trace though... Might look at netgraph before going too far down that path. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: natd in a jail
On Thu, Nov 22, 2012 at 10:32 PM, Teske, Devin wrote: > I have created a boot script for managing vimages (downloadable as a FreeBSD > package) and made a little write-up on how to use it... > http://druidbsd.sf.net/vimage.shtml As noted elsewhere, these are VIMAGE jails, but I'm managing them manually with a spaghetti script at the moment (just proof-of-concept at this point), I'll have a look at the script, might make my life easier. > Note that I use netgraph for bridging (not if_bridge+epair method which seems > to be popular in some other setups -- we've benchmarked netgraph and it > scales well). Not to mention that "ngctl dot | dot -Tsvg -o network.svg" can > produce nice pretty graphs of your vimage structure when using my setup. Hmmm, I've not done anything with netgraph before, I'll have a look into it, if it is an issue of the appropriate interfaces not being exposed to natd from the epair/bridge setup that might be an alternate solution, not hugely concerned about scale, it'll pretty much only be my traffic that gets routed this way, but I am interested in making it as efficient as possible (no sense adding additional latency unnecessarily when one already has the tunnel latency to deal with). Thanks, Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Fwd: natd in a jail
Hmm, list was missing from reply-to on this one. -- Forwarded message -- From: Morgan Reed Date: Thu, Nov 22, 2012 at 10:36 PM Subject: Re: natd in a jail To: Dewayne Geraghty On Thu, Nov 22, 2012 at 9:33 PM, Dewayne Geraghty wrote: > We run a lot of jails with kernel nat and ipfw (& ipsec but that's not what > you need here). Some of the hosts haven't migrated from natd to kernel nat, > so we're probably similar to your setup. Sounds very similar, just substituting OpenVPN for IPSec. > 90% of our jails have an 192.168/16 that nat via an external interface with > a routable address, and an internal non-routeable address (ie non-RFC1918); > which is probably what you're doing for your VPN stuff. > > Our openvpn's all use tun, I would suggest that your natd isn't doing > exactly like you'd wish - on a good day it can be tricky to get right and > tcpdump is your friend, which should be monitored in both your host > environment and within the jail. You'll need to enable allow.raw_sockets > and you'll probably want to enable bpf to be available in your jail, if you > haven't already done so. BPF is enabled for the jails, and the traffic is getting to where it needs to (but not via natd). I'll try enabling raw_sockets in the jails, it is entirely conceivable that natd requires that functionality. Thanks for your assistance, I'll see how I go and report back. Best Regards, Morgan Reed -- "They that can give up essential liberty to obtain a little temporary safety deserve neither liberty nor safety." -- Benjamin Franklin, 1759 ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: natd in a jail
On Thu, Nov 22, 2012 at 9:38 PM, Simon Dick wrote: > I've not used it myself, but this sound like something VIMAGE may be good > for, basically it's a virtual tcp stack per jail, there's some docs at > http://wiki.freebsd.org/Image but I seem to remember a more up to date one > elsewhere but can't find it at the moment! These are all VIMAGE jails :) I originally tried to do this without VIMAGE but OpenVPN won't work properly in that environment as if it updated the kernel routing table (which ISTR it couldn't, makes sense given the nature of jail) it would have changed it on the host and all jail images. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
natd in a jail
Hi All, I've a bit of an odd query which I hope somebody may be able to assist with. I'm looking to set up several OpenVPN tunnels on a single machine (each residing in its own jail) and route data to different destinations over different tunnels by selectively routing the traffic via a particular jail. I have three jails set up with OpenVPN tunnels terminated in each, they all work as expected from the "local" machine. I can't do a straight forward route over the VPN tunnel as I don't control the other end of the tunnel, I need to treat it as a point-to-point connection as a result, hence I need to use NAT. I've tested this setup with a single tunnel running off a "real" machine with natd providing NAT, it works like a charm, however, when I move the config into a jail I run into issues, natd doesn't seem to be able to see the incoming traffic, nothing shows up in the logs at all. I'm not even sure if this is actually possible, I'm starting to suspect that natd can't hook in low enough from the jails to access the incoming traffic. Traffic gets into the jail by way of an epair interface between the host and the jail, bridged to the ethernet adapter by way of a bridge device, I can see the traffic attempting to route over the tun interface in the jail (but obviously it's not being NATted so nothing comes back) so the traffic is making it in and through the routing engine, just not via natd. Any suggestions here? The host is FreeBSD-8.3. Thanks, Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: ZFS panics on pool moved from OpenSolaris
Hi Andriy, Scratch that, it would appear to be a PEBKAC issue, looks like I omitted to actually save sid.h when I made the mods. Patch is good, I now have access to my whole pool again. Thanks, Morgan On Mon, Feb 6, 2012 at 10:23, Morgan Reed wrote: > Hi Andriy, > > Thanks for that, the patch has significantly improved > matters, I'm now able to run a find across part of the drive without > issue, however I'm still seeing the panics on some directories, stack > trace below; > > panic: avl_find() succeeded inside avl_add() > cpuid = 0 > KDB: stack backtrace: > #0 0xc0a4b157 at kdb_backtrace+0x47 > #1 0xc0a186b7 at panic+0x117 > #2 0xc5a2d7b2 at avl_add+0x52 > #3 0xc5ac44e6 at zfs_fuid_table_load+0x1f6 > #4 0xc5ac479e at zfs_fuid_init+0x14e > #5 0xc5ac4893 at zfs_fuid_find_by_idx+0xc3 > #6 0xc5ac48ed at zfs_fuid_map_id+0x2d > #7 0xc5ac492f at zfs_groupmember+0x2f > #8 0xc5adbdcb at zfs_zaccess_aces_check+0x1db > #9 0xc5adc257 at zfs_zaccess+0xb7 > #10 0xc5afa7d4 at zfs_freebsd_getattr+0x1f4 > #11 0xc0d69322 at VOP_GETATTR_APV+0x42 > #12 0xc0ab81c9 at vn_stat+0x79 > #13 0xc0aaefdd at kern_statat_vnhook+0xfd > #14 0xc0aaf1cc at kern_statat+0x3c > #15 0xc0aaf156 at kern_lstat+0x36 > #16 0xc0aaf1ff at sys_lstat+0x2f > #17 0xc0d49315 at syscall+0x355 > > Same trace as previously on 9.0. > > I'm following your advice to Karli and throwing some printfs into > zfs_fuid_table_load, I'll advise if I find anything enlightening. > > Thanks, > > Morgan > > On Sun, Feb 5, 2012 at 22:20, Andriy Gapon wrote: >> >> Please see this thread: >> http://lists.freebsd.org/pipermail/freebsd-fs/2011-December/013215.html >> It looks like the same issue. >> The patch has been committed in head, not sure if it's MFCed. >> >> on 05/02/2012 06:57 Morgan Reed said the following: >>> Hi all, >>> >>> I'm experiencing an issue in migrating my NAS from OpenSolaris >>> over to FreeBSD, I've tried both releng_8_2 and releng_9 I have >>> similar issues in both cases. >>> >>> The pool is a RAID-Z pool comprising 4 1TB drives, it was originally >>> created on OpenSolaris (not sure what version, 2010.09 maybe, it was >>> one of the last ones prior to the Oracle acquisition), pool was a V14 >>> pool, initially I built a FreeBSD-8.2 system to migrate the pool to, >>> migrated it over OK, upgraded it from V14 to V15, but later testing >>> revealed something wasn't happy, when listing certain directories (and >>> even doing an ls -la at the root of the pool) resulted in a kernel >>> panic (Mostly GENERIC kernel, rebuilt with KVA_PAGES 512 but other >>> than that stock); >>> >>> panic: avl_find() succeeded inside avl_add() >>> cpuid = 0 >>> KDB: stack backtrace: >>> #0 0x808e0d07 at kdb_backtrace+0x47 >>> #1 0x808b1dc7 at panic+0x117 >>> #2 0x862e6602 at avl_add+0x52 >>> #3 0x8635c136 at zfs_fuid_table_load+0x1f6 >>> #4 0x8635c3ee at zfs_fuid_init+0x14e >>> #5 0x8635c4d7 at zfs_fuid_find_by_idx+0xb7 >>> #6 0x8635c52d at zfs_fuid_map_id+0x2d >>> #7 0x8635d56f at zfs_groupmember+0x2f >>> #8 0x8636df0b at zfs_zaccess_aces_check+0x1db >>> #9 0x8636377 at zfs_zaccess+0x57 >>> #10 0x8636d6fb at zfs_zaccess_rwx+0x3b >>> #11 0x86385f61 at zfs_freebsd_access+0xf1 >>> #12 0x80c02ea2 at VOP_ACCESS_APV+0x42 >>> #13 0x809457cf at change_dir+0x5f >>> #14 0x809467b1 at kern_chdir+0x81 >>> #15 0x80946a22 at chdir+0x22 >>> #16 0x808eca39 at syscallenter+0x329 >>> #17 0x80be4e14 at syscall+0x34 >>> >>> Looks like something in the permissions structure was causing grief, >>> tried running a scrub across the pool, didn't resolve the issue. >>> >>> After spending some time fighting with it I decided that it wasn't >>> worth the effort, and I upgraded to FreeBSD-9.0 to see if that would >>> assist (I normally avoid x.0 releases), once again pool imported fine, >>> however I was still seeing similar panics, ran a scrub across the >>> pool, still not happy, also upgraded the pool to v28 tried again, when >>> that failed I scrubbed again but still no joy. >>> >>> As a matter of interest I booted an OpenIndiana live CD and tried >>> copying the directories contents to another location, I am now able to >>> list the directories. However there are still issues. >>> >>> The issue seems to have shifted slightly, stack trace from a recent >>> panic is below (GENERIC kernel o
Re: ZFS panics on pool moved from OpenSolaris
Hi Andriy, Thanks for that, the patch has significantly improved matters, I'm now able to run a find across part of the drive without issue, however I'm still seeing the panics on some directories, stack trace below; panic: avl_find() succeeded inside avl_add() cpuid = 0 KDB: stack backtrace: #0 0xc0a4b157 at kdb_backtrace+0x47 #1 0xc0a186b7 at panic+0x117 #2 0xc5a2d7b2 at avl_add+0x52 #3 0xc5ac44e6 at zfs_fuid_table_load+0x1f6 #4 0xc5ac479e at zfs_fuid_init+0x14e #5 0xc5ac4893 at zfs_fuid_find_by_idx+0xc3 #6 0xc5ac48ed at zfs_fuid_map_id+0x2d #7 0xc5ac492f at zfs_groupmember+0x2f #8 0xc5adbdcb at zfs_zaccess_aces_check+0x1db #9 0xc5adc257 at zfs_zaccess+0xb7 #10 0xc5afa7d4 at zfs_freebsd_getattr+0x1f4 #11 0xc0d69322 at VOP_GETATTR_APV+0x42 #12 0xc0ab81c9 at vn_stat+0x79 #13 0xc0aaefdd at kern_statat_vnhook+0xfd #14 0xc0aaf1cc at kern_statat+0x3c #15 0xc0aaf156 at kern_lstat+0x36 #16 0xc0aaf1ff at sys_lstat+0x2f #17 0xc0d49315 at syscall+0x355 Same trace as previously on 9.0. I'm following your advice to Karli and throwing some printfs into zfs_fuid_table_load, I'll advise if I find anything enlightening. Thanks, Morgan On Sun, Feb 5, 2012 at 22:20, Andriy Gapon wrote: > > Please see this thread: > http://lists.freebsd.org/pipermail/freebsd-fs/2011-December/013215.html > It looks like the same issue. > The patch has been committed in head, not sure if it's MFCed. > > on 05/02/2012 06:57 Morgan Reed said the following: >> Hi all, >> >> I'm experiencing an issue in migrating my NAS from OpenSolaris >> over to FreeBSD, I've tried both releng_8_2 and releng_9 I have >> similar issues in both cases. >> >> The pool is a RAID-Z pool comprising 4 1TB drives, it was originally >> created on OpenSolaris (not sure what version, 2010.09 maybe, it was >> one of the last ones prior to the Oracle acquisition), pool was a V14 >> pool, initially I built a FreeBSD-8.2 system to migrate the pool to, >> migrated it over OK, upgraded it from V14 to V15, but later testing >> revealed something wasn't happy, when listing certain directories (and >> even doing an ls -la at the root of the pool) resulted in a kernel >> panic (Mostly GENERIC kernel, rebuilt with KVA_PAGES 512 but other >> than that stock); >> >> panic: avl_find() succeeded inside avl_add() >> cpuid = 0 >> KDB: stack backtrace: >> #0 0x808e0d07 at kdb_backtrace+0x47 >> #1 0x808b1dc7 at panic+0x117 >> #2 0x862e6602 at avl_add+0x52 >> #3 0x8635c136 at zfs_fuid_table_load+0x1f6 >> #4 0x8635c3ee at zfs_fuid_init+0x14e >> #5 0x8635c4d7 at zfs_fuid_find_by_idx+0xb7 >> #6 0x8635c52d at zfs_fuid_map_id+0x2d >> #7 0x8635d56f at zfs_groupmember+0x2f >> #8 0x8636df0b at zfs_zaccess_aces_check+0x1db >> #9 0x8636377 at zfs_zaccess+0x57 >> #10 0x8636d6fb at zfs_zaccess_rwx+0x3b >> #11 0x86385f61 at zfs_freebsd_access+0xf1 >> #12 0x80c02ea2 at VOP_ACCESS_APV+0x42 >> #13 0x809457cf at change_dir+0x5f >> #14 0x809467b1 at kern_chdir+0x81 >> #15 0x80946a22 at chdir+0x22 >> #16 0x808eca39 at syscallenter+0x329 >> #17 0x80be4e14 at syscall+0x34 >> >> Looks like something in the permissions structure was causing grief, >> tried running a scrub across the pool, didn't resolve the issue. >> >> After spending some time fighting with it I decided that it wasn't >> worth the effort, and I upgraded to FreeBSD-9.0 to see if that would >> assist (I normally avoid x.0 releases), once again pool imported fine, >> however I was still seeing similar panics, ran a scrub across the >> pool, still not happy, also upgraded the pool to v28 tried again, when >> that failed I scrubbed again but still no joy. >> >> As a matter of interest I booted an OpenIndiana live CD and tried >> copying the directories contents to another location, I am now able to >> list the directories. However there are still issues. >> >> The issue seems to have shifted slightly, stack trace from a recent >> panic is below (GENERIC kernel on 9.0-RELEASE); >> >> panic: avl_find() succeeded inside avl_add() >> cpuid = 0 >> KDB: stack backtrace: >> #0 0xc0a4b157 at kdb_backtrace+0x47 >> #1 0xc0a186b7 at panic+0x117 >> #2 0xc5a2d7b2 at avl_add+0x52 >> #3 0xc5ac44e6 at zfs_fuid_table_load+0x1f6 >> #4 0xc5ac479e at zfs_fuid_init+0x14e >> #5 0xc5ac4893 at zfs_fuid_find_by_idx+0xc3 >> #6 0xc5ac48ed at zfs_fuid_map_id+0x2d >> #7 0xc5ac492f at zfs_groupmember+0x2f >> #8 0xc5adbdcb at zfs_zaccess_aces_check+0x1db >> #9 0xc5adc257 at zfs_zaccess+0xb7 >> #10 0xc5afa7d4 at zfs_freebsd_getattr+0x1f4 >> #11 0xc0d69322 at VOP_GETATTR_AP
ZFS panics on pool moved from OpenSolaris
Hi all, I'm experiencing an issue in migrating my NAS from OpenSolaris over to FreeBSD, I've tried both releng_8_2 and releng_9 I have similar issues in both cases. The pool is a RAID-Z pool comprising 4 1TB drives, it was originally created on OpenSolaris (not sure what version, 2010.09 maybe, it was one of the last ones prior to the Oracle acquisition), pool was a V14 pool, initially I built a FreeBSD-8.2 system to migrate the pool to, migrated it over OK, upgraded it from V14 to V15, but later testing revealed something wasn't happy, when listing certain directories (and even doing an ls -la at the root of the pool) resulted in a kernel panic (Mostly GENERIC kernel, rebuilt with KVA_PAGES 512 but other than that stock); panic: avl_find() succeeded inside avl_add() cpuid = 0 KDB: stack backtrace: #0 0x808e0d07 at kdb_backtrace+0x47 #1 0x808b1dc7 at panic+0x117 #2 0x862e6602 at avl_add+0x52 #3 0x8635c136 at zfs_fuid_table_load+0x1f6 #4 0x8635c3ee at zfs_fuid_init+0x14e #5 0x8635c4d7 at zfs_fuid_find_by_idx+0xb7 #6 0x8635c52d at zfs_fuid_map_id+0x2d #7 0x8635d56f at zfs_groupmember+0x2f #8 0x8636df0b at zfs_zaccess_aces_check+0x1db #9 0x8636377 at zfs_zaccess+0x57 #10 0x8636d6fb at zfs_zaccess_rwx+0x3b #11 0x86385f61 at zfs_freebsd_access+0xf1 #12 0x80c02ea2 at VOP_ACCESS_APV+0x42 #13 0x809457cf at change_dir+0x5f #14 0x809467b1 at kern_chdir+0x81 #15 0x80946a22 at chdir+0x22 #16 0x808eca39 at syscallenter+0x329 #17 0x80be4e14 at syscall+0x34 Looks like something in the permissions structure was causing grief, tried running a scrub across the pool, didn't resolve the issue. After spending some time fighting with it I decided that it wasn't worth the effort, and I upgraded to FreeBSD-9.0 to see if that would assist (I normally avoid x.0 releases), once again pool imported fine, however I was still seeing similar panics, ran a scrub across the pool, still not happy, also upgraded the pool to v28 tried again, when that failed I scrubbed again but still no joy. As a matter of interest I booted an OpenIndiana live CD and tried copying the directories contents to another location, I am now able to list the directories. However there are still issues. The issue seems to have shifted slightly, stack trace from a recent panic is below (GENERIC kernel on 9.0-RELEASE); panic: avl_find() succeeded inside avl_add() cpuid = 0 KDB: stack backtrace: #0 0xc0a4b157 at kdb_backtrace+0x47 #1 0xc0a186b7 at panic+0x117 #2 0xc5a2d7b2 at avl_add+0x52 #3 0xc5ac44e6 at zfs_fuid_table_load+0x1f6 #4 0xc5ac479e at zfs_fuid_init+0x14e #5 0xc5ac4893 at zfs_fuid_find_by_idx+0xc3 #6 0xc5ac48ed at zfs_fuid_map_id+0x2d #7 0xc5ac492f at zfs_groupmember+0x2f #8 0xc5adbdcb at zfs_zaccess_aces_check+0x1db #9 0xc5adc257 at zfs_zaccess+0xb7 #10 0xc5afa7d4 at zfs_freebsd_getattr+0x1f4 #11 0xc0d69322 at VOP_GETATTR_APV+0x42 #12 0xc0ab81c9 at vn_stat+0x79 #13 0xc0aaefdd at kern_statat_vnhook+0xfd #14 0xc0aaf1cc at kern_statat+0x3c #15 0xc0aaf156 at kern_lstat+0x36 #16 0xc0aaf1ff at sys_lstat+0x2f #17 0xc0d49315 at syscall+0x355 This time it appears to be related to some extended attribute(s), I can do an ls on one of the directories in question but an ls -la causes a panic, so it would seem that it's some attribute which is only shown in the long form of the ls output that is causing the issue. I've done some digging around via the magic of google and this seems to be a fairly common issue, but I've not found a solution for it (barring copying the data off, recreating the pool and restoring the data, I'd like to avoid this if at all possible. If I could determine what the problematic attribute was and a means to strip it (be that from FreeBSD or from an OpenIndiana liveCD) I think that will get me back up and running. If anybody can provide some suggestions as to what I may be able to do to resolve this issue in situ I would be very grateful. Thanks, Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Accessing tun devices from inside a Jail
Hi all, I'm currently attempting to setup, I suppose you'd call it a multi-VPN-tunnel gateway. Basically I have several OpenVPN Servers in different locations, I want to have various tunnels up to them and be able to choose an exit by way of pointing my browser at a particular instance of Squid running in a particular jail which routes via a particular tunnel (HTTP/S traffic is the primary concern at this point, though I might want to extend the concept to all traffic in future). First issue I ran into was routing tables, that was resolved by recompiling my kernel with option ROUTETABLES=10 and pointing each of my jails to their own FIB, however as it's not possible to configure route tables from inside the jail (as far as I'm aware anyway) I need to bring the OpenVPN tunnel up from the host and utilise a route-up script to configure the routing table for the jail (utilising setfib), I run into problems though, as even though the tun device is visible in the jail it does not appear to be configured (no IP addersses, etc) so the jail is unable to route traffic. All the stuff I've been able to find online has been geared to static addresses on each end of the tunnel, this is not the case with my VPN provider, tunnel addresses are dynamically assigned. I think that worst case I can probably use pf on the host to route traffic from a given jail via a particular interface or possibly cobble something up around VIMAGE, but I think I'd rather not have to go down those paths. I'm not sure if what I'm looking for is actually possible, any suggestions would be much appreciated. Thanks, Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Diskless/readonly root booting issues
Hi all, I've been working on updating my semi-embedded images to 7.3-stable of late (I generally wait for .3+ releases), it's been a few years since the last time I did one of these and I'm having some issues getting my netboot test environment to behave itself. I'm sure it's something simple but I've spent quite a bit of time looking for answers and poking the system but no joy yet. Basically I use a PXE booted NFS root to test my reduced footprint image builds, the boot is working but init is attempting to remount / rw (in spite of it being marked ro in fstab) which of course fails because the directory is exported ro from the NFS server at which point the system dumps me to single user mode; === OUTPUT === Starting file system checks: udp: Netconfig database not found Mounting root filesystem rw failed, startup aborted ERROR: ABORTING BOOT (sending SIGTERM to parent)! Sep 30 09:60:02 init: /bin/sh on /etc/rc terminated abnormally, going to single user mode Enter full pathname of shell or RETURN for /bin/sh: Relevant configs from the diskless root == rc.conf == ifconfig_le0="DHCP" diskless_mount=/etc/rc.initdiskless varsize=8192 varmfs="YES" tmpsize=8192 tmpmfs="YES" nfs_client_enable="YES" dumpdev="NO" = rc.initdiskless is the version from /usr/share/examples/rc.initdiskless == fstab == 192.168.2.2:/usr/fbtest / nfs ro 0 0 proc /proc procfs rw 0 0 == loader.conf == verbose_loading="YES" autoboot_delay="2" Kernel is (obviously) built with NFS_ROOT and NFSCLIENT, relatively minimalist otherwise, have also tested with GENERIC, same result. I must be forgetting something simple in all of this, I don't recall it being terribly difficult to get this stuff working when I was doing my original work with 6.3, though I don't recall the use of the initdiskless script, IIRC I was using rc.diskless2 which (again IIRC) was later replaced by /etc/rc.d/diskless but I've not been able to find this script anywhere. Any suggestions would be greatly appreciated at this point. Thanks, Morgan Reed ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "freebsd-stable-unsubscr...@freebsd.org"
Re: 5.x to 6.x or 7.x with 64MB /
On Tue, Aug 5, 2008 at 2:48 AM, Nick Barnes <[EMAIL PROTECTED]> wrote: > It occurs to me that if ad0s1a is insufficient then I could use ad0s1g > as swap, and repurpose ad0s1b as a new /. Is it straightforward to > installworld/mergemaster to somewhere other than / ? The DESTDIR directive will allow you to redirect your installworld to a different location, as for mergemaster IIRC this can be done (been a while since I was working on my embedded stuff that needed all this FreeBSD 6.something in about 8MB) but I can't remember what switches. It might be worth considering building /bin and /sbin dynamically (20oddMB to about 500kB), mind I can't remember where the required libs would be, if they're in /lib it'd be all good, if they're in /usr/lib you're stuck. All things considered I think your best option is to move / to a different partition. Should be relatively painless to do from a LiveCD, mount everything, duplicate / to somewhere else, modify fstab and the bootloader config, reboot. ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: rc.local equivalent
On 7/12/07, Brian <[EMAIL PROTECTED]> wrote: man rc.local on a freebsd 7 box says Same for 6.2-STABLE. So, rc.local, though not current is still supported. Yes, effectively deprecated (although the mechanism may not be removed for a significant time, if at all). I'm going to write an rc.d script to do the loading and saving of configs for my system. Thanks for your assistance. Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
rc.local equivalent
I'm working on a small footprint semi-embedded system, I need a means to load and save parts of /var (which is a memory-backed filesystem) at boot and shutdown. Given that rc.local is now deprecated, what is the "correct" way to perform extra startup/shutdown processes, should I write an rc.d script for it and insert it into the rcorder appropriately, or is there another mechanism? Thanks in advance Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
PF Question
Not sure if this is the most appropriate place to ask, feel free to redirect me if it isn't. I've got an issue with a simple NAT with pf. I've got two machines; the first (I will call m1) has 2 ethernet interfaces (I will call them m1.0 and m1.1) the second (I will call m2) has 1 ethernet interface (I will call it m2.0) m1.0 faces my LAN, m1.1 and m2.0 are on a separate, isolated segment. what I need to be able to do is to access the "outside world" from m2 and be able to get to Ports 80, 443 and 3128 on m2 from my LAN by connecting to ports 80, 443 and 3128 on m1 and having traffic forwarded appropriately. m1.0 - 192.168.0.X/24 (DHCP assigned) m1.1 - 192.168.1.2/24 m2.0 - 192.168.1.30/24 /etc/pf.conf = ext_if="m1.0" int_if="m1.1" nat on $ext_if from !($ext_if) -> ($ext_if:0) rdr pass on $ext_if proto tcp to port 80 -> 192.168.1.30 port 80 rdr pass on $ext_if proto tcp to port 443 -> 192.168.1.30 port 443 rdr pass on $ext_if proto tcp to port 3128 -> 192.168.1.30 port 3128 pass in keep state pass out keep state = The current status is as follows; * I can ping m1.0 from m2 * I can't ping any of the other address on the 192.168.0.0/24 network from m2 - tcpdump'ing m1.1 * Connecting to one of the forwarded ports on m1.0 I see nothing * Connecting from m2 to a host on the LAN I see the connections going out but, not coming back Your assistance is greatly appreciated. Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
miniBSD vs 6.2-STABLE
I now have my target PXE booting from my FreeBSD host (chalk one up to not writing config files at ungodly hours of the night, thanks to Danny for spotting the error.) The boot is proceeding to the boot menu but after this the kernel freezes; With acpi.ko in the image the acpi module loads, the next spinner appears and the system freezes. Without acpi.ko I get the message "ACPI autoload failed - no such file or directory", the next spinner appears and the system freezes. My kernel is a stripped GENERIC, essentially minus a bunch of network drivers (the driver for the adapter in the machine is still there ;o) ), and all the SCSI controllers. Any suggestions would be greatly appreciated. Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: PXE booting issues
On 6/25/07, Jeremy Chadwick <[EMAIL PROTECTED]> wrote: I hope this helps clear up any concern over that part. Thanks for the explanation, that makes sense. Unfortunately it still doesn't explain why the file is not being created, the fact that the dd command is outputting a count for records out suggests to me that it *SHOULD* have been created but even doing a find across my /usr/src/sys/boot doesn't turn it up. Thanks Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: PXE booting issues
On 6/25/07, Robert Joosten <[EMAIL PROTECTED]> wrote: And next-server My dhcpd.conf has an appropriate next-server directive, all the configuration (for the network boot) has been duplicated from the functional DragonFly boot server. I have a couple of version 6.2 pxe-clients booting of a 4.11 box without a problem. My FreeBSD boot host is running 6.2-STABLE and the kernel config: #grep -i bootp /usr/src/sys/i386/conf/GISTER options BOOTP options BOOTP_NFSROOT options BOOTP_NFSV3 options BOOTP_COMPAT options BOOTP_WIRED_TO=fxp0 I'm not using the BOOTP options in my kernel, the same image is able to get its kernel on another system so I'm happy it's not the image causing this issue. At Danny's suggestion I've done some sniffing of the virtual interface and discovered that none of the NFS calls from the client are being replied to, yet another host on the same subnet can mount the same filesystem without issue, that said I've just run another trace, and I do notice that the calls when I mount the filesystem on the other host are a mixture of V2 and V3 calls whereas the call from the PXE host is a V2 call (although this is the GETPORT call which is V2 in both cases). Thanks for your time. Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
PXE booting issues
I'm currently working on an embedded project which will be built around a BSD (I'm not sure which yet), currently I have an image up and running DragonFly and I'm currently attempting to do the same with FreeBSD for comparison. I'm more or less following the miniBSD tutorials (updating the various file lists and such as I go). The system is being built around 6.2-STABLE but I'm having some issues getting it to PXE boot. On my FreeBSD host I've enabled TFTP, exported the rootfs for the system via NFS, built and installed isc-dhcpd and configured it with the extra options for PXE booting. The client currently gets its IP address from the server successfully, retrieves and loads pxeboot but when it comes to launch the kernel it eventually throws an NFS timeout. I know the export is working because I used NFS to pull the rootfs over to my DragonFly boot host to see what the result was when I booted the same image from a known working boot host (it worked correctly until it hit a problem in the image I will detail in a separate message). I did attempt to rebuild the pxeboot loader, following the standard instructions; set LOADER_TFTP_SUPPORT=YES in /etc/make.conf cd /usr/src/sys/boot make clean && make depend && make It appears to be successful (and the output would support this) but the i386/pxeldr/pxeboot and i386/loader/loader files do not exist, my only guess is that I've not set a make variable I should have, the most confusing part is that the dd command which is the final step in generating pxeboot appears in the output and appears to be successful; == dd if=pxeboot.tmp of=pxeboot obs=2k conv=osync 425+0 records in 107+0 records out == The discrepancy in the records in and the records out is concerning but I would expect the file to exist regardless, I'm currently using the default /boot/pxeboot. Any suggestions as to what might be causing this would be greatly appreciated. Thanks in advance, Morgan ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
