Re: Sed error message on latest ramdisk_CD #164
- Original Message From: Kevin Chadwick ma1l1i...@yahoo.co.uk To: misc@openbsd.org Sent: Tue, October 19, 2010 9:32:39 AM Subject: Sed error message on latest ramdisk_CD #164 At the last part of the install, just after timezone entry using install48.iso. (Ramdisk_CD) #164 Oct 18 17:42:33 An error message is given saying. Uid0 on /: file system full /: write failed , file system is full sed: stdout: No space left on device /bin/df gives blocksused avail /dev/rd0a3487 34404799% Install seems fine and even the mail to root is there. Perhaps that is what this commit is for? CVSROOT:/cvs Module name: src Changes by: dera...@cvs.openbsd.org 2010/10/19 14:23:55 Modified files: distrib/i386/common: Makefile.inc etc/etc.i386 : disktab sys/arch/i386/conf: RAMDISK RAMDISKB RAMDISKC RAMDISK_CD Log message: grow i386 inside media a teeny bit --- James A. Peltier james_a_pelt...@yahoo.ca
Re: em(4) ierrs [solved]
- Original Message From: Stuart Henderson s...@spacehopper.org To: Andre Keller a...@list.ak.cx Cc: misc@openbsd.org Sent: Wed, September 22, 2010 8:44:26 AM Subject: Re: em(4) ierrs [solved] On 2010/09/22 17:38, Andre Keller wrote: Hi Stuart On 21.09.2010 01:28, schrieb Stuart Henderson: I would try wbng first. Failing that, lm. I doubt you would need to disable ichiic but that would be the next step if there's no improvement. well disabling wbng seems to be the solution. After one day of normal traffic levels we do not see any Ierrs anymore... Thank you Stuart for the helpful advise. Can somebody explain how this driver (which is for getting voltage levels, fan speeds etc, if i did not misinterpret the manpage) is causing this strange behavior? I'm just curious... Great, thanks for the feedback. If any code ties up the kernel for too long, it can't handle other tasks in a timely fashion. I, unfortunately, am still experiencing livelocks on my em interfaces on my Dell R200 server in bridging mode. I'm going to have to schedule an upgrade to the latest snapshot first to see if that clears up any issues, but barring that I'm not sure where to look. Perhaps I'll also try the UP kernel. --- James A. Peltier james_a_pelt...@yahoo.ca
Re: em(4) ierrs [solved]
- Original Message From: Stuart Henderson s...@spacehopper.org To: James Peltier james_a_pelt...@yahoo.ca Cc: Andre Keller a...@list.ak.cx; misc@openbsd.org Sent: Wed, September 22, 2010 12:31:43 PM Subject: Re: em(4) ierrs [solved] snip I, unfortunately, am still experiencing livelocks on my em interfaces on my Dell R200 server in bridging mode. I'm going to have to schedule an upgrade to the latest snapshot first to see if that clears up any issues, but barring that I'm not sure where to look. Perhaps I'll also try the UP kernel. the livelock counter means a timeout wasn't reached in time, indicating the system being too busy to run userland. (see m_cltick(), m_cldrop() etc in sys/kern/uipc_mbuf.c, and the video from asiabsdcon starting about 15 minutes into http://www.youtube.com/watch?v=fv-AQJqUzRI). when this happens, nics with drivers using the MCLGETI mechanism halve the size of their receive rings, so that packets drop earlier, more effectively limiting system load than if they were allowed to proceed up the network stack. so for some reason or other the timeout wasn't processed quickly enough and the system responds in this way to limit the overload. so the challenge is to identify what causes the system to become non-responsive (could be in the network stack or could be for other reasons) and work out ways to alleviate that.. Watching now. :)
Re: em(4) ierrs [solved]
- Original Message From: Stuart Henderson s...@spacehopper.org To: James Peltier james_a_pelt...@yahoo.ca Cc: Andre Keller a...@list.ak.cx; misc@openbsd.org Sent: Wed, September 22, 2010 12:31:43 PM Subject: Re: em(4) ierrs [solved] the livelock counter means a timeout wasn't reached in time, indicating the system being too busy to run userland. (see m_cltick(), m_cldrop() etc in sys/kern/uipc_mbuf.c, and the video from asiabsdcon starting about 15 minutes into http://www.youtube.com/watch?v=fv-AQJqUzRI). when this happens, nics with drivers using the MCLGETI mechanism halve the size of their receive rings, so that packets drop earlier, more effectively limiting system load than if they were allowed to proceed up the network stack. so for some reason or other the timeout wasn't processed quickly enough and the system responds in this way to limit the overload. so the challenge is to identify what causes the system to become non-responsive (could be in the network stack or could be for other reasons) and work out ways to alleviate that.. Thanks for the notes. Below are snapshots of vmstat -i and systat vmstat which do show high interrupt levels (6-12k). I put quotes around high because I'm not really sure if that is high. That said, is there any benefit to the use of blocknonip clause being added to the bridge devices? I also note, that according to the m_cldrop() that the halving is done on all interfaces. This seems odd, in that, if you had a device with multiple cards that all traffic would be affected at the expense of one. Am I correct in this? # vmstat -i interrupt total rate irq0/clock 819075628 199 irq0/ipi 208550295 irq112/em012478765512 3047 irq113/em113607027530 3322 irq113/bge1 126355323 irq97/uhci1 19490 irq96/ehci0220 irq98/pciide0 52040391 irq145/com0 3390 Total 26943565580 6578 and #systat vmstat 1 usersLoad 0.64 0.67 0.66 Wed Sep 22 16:56:35 2010 memory totals (in KB)PAGING SWAPPING Interrupts real virtual free in out in out11067 total Active15388 15388 2918228 ops200 clock All 383480383480 6585880 pages 48 ipi 5586 em0 Proc:r d s wCsw Trp Sys Int Sof Flt 1 forks5212 em1 7 101 561 1525 9438 105 595 fkppw 21 bge1 fksvm uhci1 18.8%Int 1.3%Sys 1.9%Usr 0.0%Nic 77.9%Idle pwait ehci0 ||||||||||| relck pciide0 |= rlkok com0 noram Namei Sys-cacheProc-cacheNo-cache 96 ndcpy Calls hits%hits %miss % 18 fltcp 55 55 100 106 zfod 31 cow Disks wd0 cd0 27514 fmin seeks 36685 ftarg xfers itarg speed 17 wired sec pdfre pdscn pzidle 13 kmapent --- James A. Peltier james_a_pelt...@yahoo.ca8
Re: CARP-ed dns server ?
- Original Message From: PP;Q Q P(P8P?P8QP8P= chipits...@gmail.com To: James Peltier james_a_pelt...@yahoo.ca Sent: Mon, September 20, 2010 1:40:16 PM Subject: Re: CARP-ed dns server ? if you have nothing to say except RTFM, can you do everybody a favour and be silent, please ? 2010/9/20 James Peltier james_a_pelt...@yahoo.ca: - Original Message From: P P;Q Q P(P8P?P8Q P8P= chipits...@gmail.com To: misc@openbsd.org Sent: Mon, September 20, 2010 2:04:18 AM Subject: Re: CARP-ed dns server ? hello! can you provide more details ? 1. what is dns software ? 2. how two copies of dns server (on master and backup) are replicated ? 3. any carp hooks on switching ? cheers, Ilia Chipitsine If BIND: read the documentation Get the book Pro DNS and BIND or the O'Rielly BIND book. If Unbound: Read the documentation If djbdns: Read the documentation There is nothing really special about running any of these on a CARP interface other than it is highly available. --- James A. Peltier james_a_pelt...@yahoo.ca Your questions are basic! What is a DNS software? I mean come on! Don't tell me to be silent, when clearly it is you who needs to do the research. You asked about running DNS on CARP. I told you there was nothing special about a CARP interface and pointed you to answers to your other questions. Don't like the answer... then piss off. ;) --- James A. Peltier james_a_pelt...@yahoo.ca
Re: em(4) ierrs
- Original Message From: Andre Keller a...@list.ak.cx To: misc@openbsd.org Cc: James Peltier james_a_pelt...@yahoo.ca Sent: Mon, September 20, 2010 3:51:16 PM Subject: Re: em(4) ierrs Am 20.09.2010 19:54, schrieb James Peltier: I see you are using LACP as your trunk protocol. You might want to check that all the LACP settings are correct or that there aren't any links being dropped for some reason that might cause the errors to occur. Additionally, have you tried with only one link in the LACP pairs being active? Does it stop then? Just tried that. There is not much I can configure for LACP. On the switch I see no errors. I've now pulled one cable so that only on interface in the trunk is active. The problem is still existing. Ierrs on the interfaces (mostly em2) (btw. there are no ifq.drops) It seems to me that some buffers are running full. As now when there is low traffic there is only a small amount of errors (about 150 in 5minutes) Are there any other knobs I could try to tune? Regards Andri I would be tempted to say, back out all your changes and return to a stock configuration, except for the net.inet.ip.ifq.maxlen parameter. I posted in early august that I was able to push nearly full gigabit speeds with a Dell R200 w/4GB of RAM with a pretty stock configuration. Eventually I had to bump maxlen and the state table but that's about it. I don't see these problems on an mid August snapshot. I haven't had a chance to try the latest ones yet though. --- James A. Peltier james_a_pelt...@yahoo.ca
Re: em(4) ierrs
- Original Message From: Joerg Goltermann go...@openbsd.org To: Andre Keller a...@list.ak.cx Cc: misc@openbsd.org Sent: Tue, September 21, 2010 12:21:28 AM Subject: Re: em(4) ierrs On 20.09.2010 19:15, Andre Keller wrote: Hi I have some odd packet loss on a openbsd based router (running -current as of the beginning of september) . The router has 6 physical interfaces (all em, Intel 82575EB), 4 of them have traffic (about 10-20 Mbps). which packet rate do you expect on the interfaces? Do you see livelocks (systat -b mbuf)? - Joerg livelocks are seen on my em interfaces as well. I also have livelocks on my far less busy bge1 management interface. See below IFACE LIVELOCKS SIZE ALIVE LWM HWM CWM System256 116 84 2k92 504 lo0 em0 293632k37 4 25637 em1 101742k37 4 25637 bge0 bge1 42k1717 51217 enc0 vlan300 bridge0 pflog0 pflow0 --- James A. Peltier james_a_pelt...@yahoo.ca
Re: em(4) ierrs
- Original Message From: James Peltier james_a_pelt...@yahoo.ca To: misc@openbsd.org Cc: misc@openbsd.org Sent: Tue, September 21, 2010 9:46:40 AM Subject: Re: em(4) ierrs - Original Message From: Joerg Goltermann go...@openbsd.org To: Andre Keller a...@list.ak.cx Cc: misc@openbsd.org Sent: Tue, September 21, 2010 12:21:28 AM Subject: Re: em(4) ierrs On 20.09.2010 19:15, Andre Keller wrote: Hi I have some odd packet loss on a openbsd based router (running -current as of the beginning of september) . The router has 6 physical interfaces (all em, Intel 82575EB), 4 of them have traffic (about 10-20 Mbps). which packet rate do you expect on the interfaces? Do you see livelocks (systat -b mbuf)? - Joerg livelocks are seen on my em interfaces as well. I also have livelocks on my far less busy bge1 management interface. See below IFACE LIVELOCKS SIZE ALIVE LWM HWM CWM System 256 116 84 2k 92 504 lo0 em0293632k37 4 25637 em1101742k37 4 25637 bge0 bge1 42k 1717 512 17 enc0 vlan300 bridge0 pflog0 pflow0 I should mention that these might have been made prior to some recent tuning. However, for the purpose of following this thread I will keep an eye on it to be sure.
Re: em(4) ierrs
- Original Message From: James Peltier james_a_pelt...@yahoo.ca To: misc@openbsd.org Sent: Tue, September 21, 2010 9:51:05 AM Subject: Re: em(4) ierrs - Original Message From: James Peltier james_a_pelt...@yahoo.ca To: misc@openbsd.org Cc: misc@openbsd.org Sent: Tue, September 21, 2010 9:46:40 AM Subject: Re: em(4) ierrs - Original Message From: Joerg Goltermann go...@openbsd.org To: Andre Keller a...@list.ak.cx Cc: misc@openbsd.org Sent: Tue, September 21, 2010 12:21:28 AM Subject: Re: em(4) ierrs On 20.09.2010 19:15, Andre Keller wrote: Hi I have some odd packet loss on a openbsd based router (running -current as of the beginning of september) . The router has 6 physical interfaces (all em, Intel 82575EB), 4 of them have traffic (about 10-20 Mbps). which packet rate do you expect on the interfaces? Do you see livelocks (systat -b mbuf)? - Joerg livelocks are seen on my em interfaces as well. I also have livelocks on my far less busy bge1 management interface. See below IFACE LIVELOCKS SIZE ALIVE LWM HWM CWM System 256 11684 2k 92 504 lo0 em0 29363 2k37 4 25637 em1 101742k37 4 256 37 bge0 bge142k 17 17 512 17 enc0 vlan300 bridge0 pflog0 pflow0 I should mention that these might have been made prior to some recent tuning. However, for the purpose of following this thread I will keep an eye on it to be sure. I am in bridging mode and I too, am indeed seeing a slow increase in livelocks on my em0 interfaces. Traffic has been quite low over the past week or so, so it certainly shouldn't be an issue. The only modifications I have made thus far are to the net.inet.ip.ifq.maxlen bumped to 2048. If you want any other info please let me know. #sysctl -b mbuf 1 usersLoad 0.13 0.09 0.08 Tue Sep 21 20:22:30 2010 IFACE LIVELOCKS SIZE ALIVE LWM HWM CWM System25698 84 2k74 504 lo0 em0 298912k29 4 25629 em1 103812k28 4 25628 bge0 bge1 42k1717 51217 enc0 vlan300 bridge0 pflog0 pflow0 # netstat -m 100 mbufs in use: 95 mbufs allocated to data 1 mbuf allocated to packet headers 4 mbufs allocated to socket names and addresses 74/1008/6144 mbuf 2048 byte clusters in use (current/peak/max) 0/8/6144 mbuf 4096 byte clusters in use (current/peak/max) 0/8/6144 mbuf 8192 byte clusters in use (current/peak/max) 0/8/6144 mbuf 9216 byte clusters in use (current/peak/max) 0/8/6144 mbuf 12288 byte clusters in use (current/peak/max) 0/8/6144 mbuf 16384 byte clusters in use (current/peak/max) 0/8/6144 mbuf 65536 byte clusters in use (current/peak/max) 2544 Kbytes allocated to network (6% in use) 0 requests for memory denied 0 requests for memory delayed 0 calls to protocol drain routines # --- James A. Peltier james_a_pelt...@yahoo.ca
Re: CARP-ed dns server ?
- Original Message From: PP;Q Q P(P8P?P8QP8P= chipits...@gmail.com To: misc@openbsd.org Sent: Mon, September 20, 2010 2:04:18 AM Subject: Re: CARP-ed dns server ? hello! can you provide more details ? 1. what is dns software ? 2. how two copies of dns server (on master and backup) are replicated ? 3. any carp hooks on switching ? cheers, Ilia Chipitsine If BIND: read the documentation Get the book Pro DNS and BIND or the O'Rielly BIND book. If Unbound: Read the documentation If djbdns: Read the documentation There is nothing really special about running any of these on a CARP interface other than it is highly available. --- James A. Peltier james_a_pelt...@yahoo.ca
Re: em(4) ierrs
- Original Message From: Andre Keller a...@list.ak.cx To: misc@openbsd.org Sent: Mon, September 20, 2010 10:15:58 AM Subject: em(4) ierrs Hi I have some odd packet loss on a openbsd based router (running -current as of the beginning of september) . The router has 6 physical interfaces (all em, Intel 82575EB), 4 of them have traffic (about 10-20 Mbps). We did some tuning (mostly with informations from: https://calomel.org/network_performance.html) and could improve the performance: Currently we use the following sysctl tweaks: sysctl kern.maxclusters=122880 sysctl net.inet.ip.ifq.maxlen=1536 sysctl net.inet.tcp.recvspace=262144 sysctl net.inet.tcp.sendspace=262144 sysctl net.inet.udp.recvspace=262144 sysctl net.inet.udp.sendspace=262144 But still we have about 1300 Ierrs per minute... When we run a simple ping, we can see that something is strange. Where the majority of packets have a rtt of 1ms or less about every tenth package shows a rtt of 250ms... I could really use a hint of what to try next (autoneg has been disabled on all interfaces for testing, now it has been enabled again...) Thank you for your inputs Andri Keller The switches on the other and of the device are both cisco 2960G with a lacp to two interfaces on the openbsd box: em0: flags=8b43UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST mtu 1500 lladdr 00:25:90:05:54:6c priority: 0 trunk: trunkdev trunk1 media: Ethernet autoselect (1000baseT full-duplex) status: active inet6 fe80::225:90ff:fe05:546c%em0 prefixlen 64 scopeid 0x1 em1: flags=8b43UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST mtu 1500 lladdr 00:25:90:05:54:6c priority: 0 trunk: trunkdev trunk1 media: Ethernet autoselect (1000baseT full-duplex) status: active inet6 fe80::225:90ff:fe05:546d%em1 prefixlen 64 scopeid 0x2 em2: flags=8b43UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST mtu 1500 lladdr 00:25:90:05:54:6e priority: 0 trunk: trunkdev trunk0 media: Ethernet 1000baseT full-duplex status: active inet6 fe80::225:90ff:fe05:546e%em2 prefixlen 64 scopeid 0x3 em3: flags=8b43UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST mtu 1500 lladdr 00:25:90:05:54:6e priority: 0 trunk: trunkdev trunk0 media: Ethernet autoselect (1000baseT full-duplex) status: active inet6 fe80::225:90ff:fe05:546f%em3 prefixlen 64 scopeid 0x4 trunk0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:25:90:05:54:6e priority: 0 trunk: trunkproto lacp trunk id: [(8000,00:25:90:05:54:6e,4054,,), (8000,18:ef:63:bf:d7:00,0002,,)] trunkport em3 active,collecting,distributing trunkport em2 active,collecting,distributing groups: trunk media: Ethernet autoselect status: active inet ADDRESS REMOVED inet6 fe80::225:90ff:fe05:546e%trunk0 prefixlen 64 scopeid 0xa inet6 ADDRESS REMOVED trunk1: flags=8943UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST mtu 1500 lladdr 00:25:90:05:54:6c priority: 0 trunk: trunkproto lacp trunk id: [(8000,00:25:90:05:54:6c,405C,,), (8000,18:ef:63:bf:d7:00,0003,,)] trunkport em1 active,collecting,distributing trunkport em0 active,collecting,distributing groups: trunk media: Ethernet autoselect status: active inet6 fe80::225:90ff:fe05:546c%trunk1 prefixlen 64 scopeid 0xb vlan56: flags=8943UP,BROADCAST,RUNNING,PROMISC,SIMPLEX,MULTICAST mtu 1500 lladdr 00:25:90:05:54:6c priority: 0 vlan: 56 priority: 0 parent interface: trunk1 groups: vlan status: active inet6 fe80::225:90ff:fe05:546c%vlan56 prefixlen 64 scopeid 0x11 inet ADDRESS REMOVED netstat -m 9023 mbufs in use: 9003 mbufs allocated to data 11 mbufs allocated to packet headers 9 mbufs allocated to socket names and addresses 528/1970/512000 mbuf 2048 byte clusters in use (current/peak/max) 0/8/512000 mbuf 4096 byte clusters in use (current/peak/max) 0/8/512000 mbuf 8192 byte clusters in use (current/peak/max) 0/8/512000 mbuf 9216 byte clusters in use (current/peak/max) 0/8/512000 mbuf 12288 byte clusters in use (current/peak/max) 0/8/512000 mbuf 16384 byte clusters in use (current/peak/max) 0/8/512000 mbuf 65536 byte clusters in use (current/peak/max) 7060 Kbytes allocated to network (46% in use) 0
Re: trouble with symon
please disregard. i found that i wasn't capturing symon/mux stats for lo0. Since i focused too much on the physical interfaces, i overlooked the logical ones. ;) --- James A. Peltier james_a_pelt...@yahoo.ca - Original Message From: James Peltier james_a_pelt...@yahoo.ca To: OpenBSD Mail List misc@openbsd.org Sent: Thu, September 16, 2010 9:39:27 PM Subject: trouble with symon Hi All, I'm testing trying to get symon working before putting it onto my production server. I've created a OpenBSD-current KVM based virtual machine which has two interfaces (pcn0 re0). I'm now trying to get the symon pf.layout file to create a graph similar to the right side graph located in this image which shows states for each interface. The first graph showing entries, removals, inserts but the second does not appear and I'm not sure why http://www.xs4all.nl/~wpd/symon/img/sw_pf.png I essentially made a backup of the original the pf.layout file edited it to change xl0 and de0 to pcn0 and re0 respectively. I then changed www to localhost and restarted, but this didn't work. I also copied the config from http://www.xs4all.nl/~wpd/symon/examples/pf.layout which is provided as a sample but this did not work either. Any ideas? I'm using current with the symon-2.82 and syweb 0.60 packages not built from sources. --- James A. Peltierjames_a_pelt...@yahoo.ca
trouble with symon
Hi All, I'm testing trying to get symon working before putting it onto my production server. I've created a OpenBSD-current KVM based virtual machine which has two interfaces (pcn0 re0). I'm now trying to get the symon pf.layout file to create a graph similar to the right side graph located in this image which shows states for each interface. The first graph showing entries, removals, inserts but the second does not appear and I'm not sure why http://www.xs4all.nl/~wpd/symon/img/sw_pf.png I essentially made a backup of the original the pf.layout file edited it to change xl0 and de0 to pcn0 and re0 respectively. I then changed www to localhost and restarted, but this didn't work. I also copied the config from http://www.xs4all.nl/~wpd/symon/examples/pf.layout which is provided as a sample but this did not work either. Any ideas? I'm using current with the symon-2.82 and syweb 0.60 packages not built from sources. --- James A. Peltier james_a_pelt...@yahoo.ca
OpenBSD Dell Latitude E6500 built in wireless
Anyone using the Dell Latitude E6500 with the built in Broadcom wireless adaptor? I see that marco@ mentions he owns a E6500 here http://www.mail-archive.com/source-chan...@openbsd.org/msg04064.html but I don't see reference to it in the bwi device or elsewhere. I'm running -current -- James A. Peltier james_a_pelt...@yahoo.ca
Re: aucat on OpenBSD 4.8 current exits in monitoring mode
- Original Message From: pet...@schwertfisch.de pet...@schwertfisch.de To: misc@openbsd.org Sent: Mon, September 13, 2010 3:11:39 PM Subject: aucat on OpenBSD 4.8 current exits in monitoring mode Hi, I am enjoying aucat on OpenBSD 4.8 current (snapshot from end of August), but sometimes the audio server just exits, leaving the currently running audio application(s) homeless and confused. I can sort of reproduce this behavior with audacity (from ports), although it also happens with other audio applications occasionally. Opening audacity on a wave file, and clicking Play/Stop a couple of times yields the following output from aucat: $ aucat -ddd -q rmidi:1 -s default -m mon -s mon default: recording s24le4msb,0:11,44100 default: playing s24le4msb,0:9,44100 default: block size is 660 frames, using 2 blocks m...@default: mon=0:1 defa...@default: rec=0:1 play=0:1 vol=32768 audacit0: buffer size = 9240, play = s16le,0:1,44100 starting device device stopped audacit0: buffer size = 9240, play = s16le,0:1,44100 starting device device stopped audacit0: buffer size = 9240, play = s16le,0:1,44100 monitor xrun, not allowed Abort trap (core dumped) This does not seem to happen with $ aucat -ddd -q rmidi:1 -s default In both cases, midicat is not running. Am I misusing the monitoring mode? Regards, Dirk $ audioctl # while aucat is running name=Envy24 version=- config=M-Audio Audioph encodings=slinear_le:24:4:1 properties=full_duplex,independent full_duplex=1 fullduplex=1 blocksize=26400 hiwat=2 lowat=1 output_muted=0 monitor_gain=0 mode=play,record play.rate=44100 play.sample_rate=44100 play.channels=10 play.precision=24 play.bps=4 play.msb=1 play.encoding=slinear play.gain=127 play.balance=32 play.port=0x0 play.avail_ports=0x0 play.seek=0 play.samples=0 play.eof=0 play.pause=1 play.error=0 play.waiting=0 play.open=1 play.active=0 play.buffer_size=65536 play.block_size=26400 play.errors=0 record.rate=44100 record.sample_rate=44100 record.channels=12 record.precision=24 record.bps=4 record.msb=1 record.encoding=slinear record.gain=127 record.balance=32 record.port=0x0 record.avail_ports=0x0 record.seek=0 record.samples=0 record.eof=0 record.pause=1 record.error=0 record.waiting=0 record.open=1 record.active=0 record.buffer_size=65536 record.block_size=31680 record.errors=0 $ dmesg OpenBSD 4.8-current (GENERIC) #312: Tue Aug 31 21:59:22 MDT 2010 dera...@i386.openbsd.org:/usr/src/sys/arch/i386/compile/GENERIC cpu0: AMD Athlon(tm) XP 2000+ (AuthenticAMD 686-class, 256KB L2 cache) 1.68 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,MMX,FXSR,SSE real mem = 804810752 (767MB) avail mem = 781688832 (745MB) mainbus0 at root bios0 at mainbus0: AT/286+ BIOS, date 10/18/06, BIOS32 rev. 0 @ 0xfdb10, SMBIOS rev. 2.3 @ 0xf0630 (21 entries) bios0: vendor American Megatrends Inc. version P2.80 date 10/18/2006 bios0: American Megatrends Inc. K7S41GX acpi0 at bios0: rev 0 acpi0: sleep states S0 S1 S4 S5 acpi0: tables DSDT FACP APIC acpi0: wakeup devices PS2M(S4) PS2K(S4) UAR1(S4) USB1(S4) USB2(S4) EHCI(S4) LAN_(S4) MDM_(S4) AUD_(S4) PCI0(S4) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: apic clock running at 268MHz ioapic0 at mainbus0: apid 2 pa 0xfec0, version 11, 24 pins acpiprt0 at acpi0: bus 0 (PCI0) acpicpu0 at acpi0 acpipwrres0 at acpi0: URP1 acpipwrres1 at acpi0: URP2 acpipwrres2 at acpi0: FDDP acpipwrres3 at acpi0: LPTP acpibtn0 at acpi0: PWRB bios0: ROM list: 0xc/0xd000 pci0 at mainbus0 bus 0: configuration mode 1 (bios) pchb0 at pci0 dev 0 function 0 SiS 741 PCI rev 0x03 sisagp0 at pchb0 agp0 at sisagp0: aperture at 0xd000, size 0x1000 ppb0 at pci0 dev 1 function 0 SiS 648FX AGP rev 0x00 pci1 at ppb0 bus 1 vga1 at pci1 dev 0 function 0 ATI Radeon 9200 PRO rev 0x01 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) radeondrm0 at vga1: apic 2 int 16 (irq 11) drm0 at radeondrm0 pcib0 at pci0 dev 2 function 0 SiS 85C503 System rev 0x25 pciide0 at pci0 dev 2 function 5 SiS 5513 EIDE rev 0x00: 741: DMA, channel 0 wired to compatibility, channel 1 wired to compatibility wd0 at pciide0 channel 0 drive 0: WDC WD1600JB-22GVC0 wd0: 16-sector PIO, LBA48, 152627MB, 312581808 sectors wd1 at pciide0 channel 0 drive 1: WDC WD800JB-00ETA0 wd1: 16-sector PIO, LBA48, 76319MB, 156301488 sectors wd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 5 wd1(pciide0:0:1): using PIO mode 4, Ultra-DMA mode 5 atapiscsi0 at pciide0 channel 1 drive 0 scsibus0 at atapiscsi0: 2 targets cd0 at scsibus0 targ 0 lun 0: LITE-ON, DVD SOHD-167T, 9S19 ATAPI 5/cdrom removable atapiscsi1 at pciide0 channel 1 drive 1
Re: Bridge Monitoring
- Original Message From: Jason Dixon ja...@dixongroup.net To: James Peltier james_a_pelt...@yahoo.ca Cc: OpenBSD Mail List misc@openbsd.org Sent: Tue, September 7, 2010 4:03:09 AM Subject: Re: Bridge Monitoring On Mon, Sep 06, 2010 at 09:26:09PM -0700, James Peltier wrote: Hi All, Now that I have my new bridge in place and happily filtering away I would like to look at monitoring and graphing it. I'd like to setup a monitor port style so that I can send the traffic over to another box for processing. I was thinking of installing symon on the bridge itself and sending it over to another box. Additionally, I was looking at setting up a pflow device and sending it to another box and analyze using something like netflow dashboard. We currently use a Cisco sending data to a GNU/Linux box running MRTG. We use arpwatch, IP Audit and other tools. Any ideas what might be best to use in this case? What are others using to monitor their network firewalls, bridges or networks in general? Off the top of my head (probably forgetting a lot): munin, symon, cacti, reconnoiter, nfsen, netflow dashboard -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net/ Thanks for the responses. So it seems like using symon to capture the statistics and sending them to another box for processing is a workable solution. Could this also be done by using the pfsync device to mirror the traffic on another OpenBSD server. I do not want to install web server applications on the bridge or on my routers as that would increase the risk of compromise. Real-time analysis would be really nice and I think pfsync would allow for nearly that.
Re: Distribute bandwidth by IP's
- Original Message From: Hermes Ojeda Ruiz hermes@gmail.com To: misc@openbsd.org Sent: Tue, September 7, 2010 12:09:03 PM Subject: Re: Distribute bandwidth by IP's Sorry, if my explanation don't have enough details. - The internet connection is an E1 - There are ~150 users (IPs) - The company give full internet access to the clients. With no service restriction. - There only a C class LAN. E1 --- OpenBSD Firewall --- LAN with ~150 IPs The problem is to distribute equally the bandwidth to the users. My first approach is a CBQ rule by user giving a minimum bandwidth quote and using the borrow option, to use the remaining bandwidth when some users don't waste the bandwidth. But the number of rules is so big. I hope that my explanation can be useful. On 07/09/10 13:43, Johan Beisser wrote: On Tue, Sep 7, 2010 at 11:15 AM, Hermes Ojeda Ruizhermes@gmail.com wrote: Hi, Maybe this is a basic question, but I've read the man pages and the PF book and I don't know how solve this problem. - I have an E1 and the problem is how to distribute the bandwidth equally on all the ip's. There are some constraints like use DHCP, and no block ports. What exactly are you trying to accomplish. Please explain a little more, in detail. I have some simple firewalls with prioritization, but I don't know how should do that. May be with CBQ but they are a lot of rules. If you're trying to set up a fair service, remember that PF simply processes the packets as they come in. So turn off queues, or define what you're trying to accomplish first. If you're trying to ensure some kinds of traffic can always leave fairly take a look at using HFSC queuing, then define the queues based on ports and use packet tagging to define what matches each queue. http://cvs.openbsd.org/faq/pf/tagging.html jb Why are you trying to do this? It seems overly complex to setup a queue for each IP on the network just to allow them to borrow bandwidth from each other which they would be doing anyway. It would seem more manageable to either segment the network (DMZ, IT Staff, Users) such that you can assign a segment to respective queues or in a different method to queue based on traffic type (http/ftp/ssh,etc). Filtering rules would also be incredibly more simplified. --- James A. Peltier james_a_pelt...@yahoo.ca
Re: Distribute bandwidth by IP's
- Original Message From: Hermes Ojeda Ruiz hermes@gmail.com To: misc@openbsd.org Sent: Tue, September 7, 2010 1:38:41 PM Subject: Re: Distribute bandwidth by IP's Yes, It's a little complex but is a requirement to guarantee a little bandwidth to the user. (and of course use the remaining unused bandwidth). There is another way? Thanks for the reply On 07/09/10 15:14, James Peltier wrote: - Original Message From: Hermes Ojeda Ruizhermes@gmail.com To: misc@openbsd.org Sent: Tue, September 7, 2010 12:09:03 PM Subject: Re: Distribute bandwidth by IP's Sorry, if my explanation don't have enough details. - The internet connection is an E1 - There are ~150 users (IPs) - The company give full internet access to the clients. With no service restriction. - There only a C class LAN. E1 --- OpenBSD Firewall --- LAN with ~150 IPs The problem is to distribute equally the bandwidth to the users. My first approach is a CBQ rule by user giving a minimum bandwidth quote and using the borrow option, to use the remaining bandwidth when some users don't waste the bandwidth. But the number of rules is so big. I hope that my explanation can be useful. On 07/09/10 13:43, Johan Beisser wrote: On Tue, Sep 7, 2010 at 11:15 AM, Hermes Ojeda Ruizhermes@gmail.com wrote: Hi, Maybe this is a basic question, but I've read the man pages and the PF book and I don't know how solve this problem. - I have an E1 and the problem is how to distribute the bandwidth equally on all the ip's. There are some constraints like use DHCP, and no block ports. What exactly are you trying to accomplish. Please explain a little more, in detail. I have some simple firewalls with prioritization, but I don't know how should do that. May be with CBQ but they are a lot of rules. If you're trying to set up a fair service, remember that PF simply processes the packets as they come in. So turn off queues, or define what you're trying to accomplish first. If you're trying to ensure some kinds of traffic can always leave fairly take a look at using HFSC queuing, then define the queues based on ports and use packet tagging to define what matches each queue. http://cvs.openbsd.org/faq/pf/tagging.html jb Why are you trying to do this? It seems overly complex to setup a queue for each IP on the network just to allow them to borrow bandwidth from each other which they would be doing anyway. It would seem more manageable to either segment the network (DMZ, IT Staff, Users) such that you can assign a segment to respective queues or in a different method to queue based on traffic type (http/ftp/ssh,etc). Filtering rules would also be incredibly more simplified. --- James A. Peltier james_a_pelt...@yahoo.ca Well since you're talking service level agreements it is understandable that you might want to do such a thing and in such case you would have no choice but to create the individual queues/rules manually or by script. Still, likely you will run into other issues, such as the number of queues available by default in the code that may need to be tweaked. See a post earlier this month to misc@ about how to do that. Also, perhaps there will be a performance hit in the evaluation of all the queues that might be more hindering than helpful? Best to let the devs speak to that though. --- James A. Peltier james_a_pelt...@yahoo.ca
Bridge Monitoring
Hi All, Now that I have my new bridge in place and happily filtering away I would like to look at monitoring and graphing it. I'd like to setup a monitor port style so that I can send the traffic over to another box for processing. I was thinking of installing symon on the bridge itself and sending it over to another box. Additionally, I was looking at setting up a pflow device and sending it to another box and analyze using something like netflow dashboard. We currently use a Cisco sending data to a GNU/Linux box running MRTG. We use arpwatch, IP Audit and other tools. Any ideas what might be best to use in this case? What are others using to monitor their network firewalls, bridges or networks in general? --- James A. Peltier james_a_pelt...@yahoo.ca
Re: MTA choice
- Original Message From: open...@e-solutions.re open...@e-solutions.re To: misc@openbsd.org Cc: Christer Solskogen christer.solsko...@gmail.com Sent: Fri, August 13, 2010 12:41:36 AM Subject: Re: MTA choice I only want to know what is better (easiest way, most secure) to use. And have your advice. On Fri, 13 Aug 2010 09:04:01 +0200, Christer Solskogen christer.solsko...@gmail.com wrote: On Fri, Aug 13, 2010 at 8:55 AM, open...@e-solutions.re wrote: Hi, I want to install a mailserver. What is the easiest and the most secure solution ? OpenBSD comes with Sendmail. I seen a lot of people use Postfix instead Sendmail. Is there someone to advice me about the choice of the MTA ? Why do you think OpenBSD ships with (a custom and secure) sendmail by default? Do you think it is because that is the easiest and most secure option or do you think by installing postfix you'll be all secure and stuff? The one that you are most familiar with will always be the most secure solution. If you think choosing a particular product will ensure security you are wrong from the start. I happen to like sendmail and use it still --- James A. Peltier james_a_pelt...@yahoo.ca
OpenBSD performance numbers
Hello fellow OpenBSD'ers. I would just like to share some information with the list about our new firewall/bridge and perhaps get some input as to where I might be able to look to squeeze some additional performance improvements. I must say though, I am very impressed with the performance improvements of networking/PF in the snapshots. Parameters: === bridge: OpenBSD 4.8-BETA (snapshot Aug 5, 2010) server: CentOS 5.5 w/Updates as of Aug 5, 2010 - head3) client: Ubuntu 10.04 w/Updates as of today - buckeye) iperf options on server/client == server: --- iperf -s client: --- for count in 1 2 3 4 5; do iperf -i 1 -t 60 -c head3 sleep 15; done Transfer indicates the amount of data transferred throughout the duration of the test. Bandwidth indicates the average bandwidth consumed for the test. [ ID] Interval Transfer Bandwidth [ 1] 0.0-60.0 sec 5.28 GBytes756 Mbits/sec [ 2] 0.0-60.0 sec 5.20 GBytes744 Mbits/sec [ 3] 0.0-60.0 sec 5.12 GBytes733 Mbits/sec [ 4] 0.0-60.0 sec 5.30 GBytes759 Mbits/sec [ 5] 0.0-60.0 sec 5.08 GBytes727 Mbits/sec So as can be seen here we are seeing data transfer rates of between 85 and 90MBps. Pretty impressive for an first pass, untweaked configuration. However, there are some unfortunates. During these tests the system was running at between 80 and 95% interrupt, with the inverse being idle. This means that either there are some tweaks that I can add to counteract the interrupts, perhaps a tweak for interrupt mitigation, or that the hardware is currently not able to handle more than a single gigabit link running at full capacity. In any case I would like to know what the developers see if better hardware would help as well as any performance tweaks that may help. These unfortunates are not really bad news. The box is certainly up to the task of dealing with our network traffic. Some tweaking may help and for a first pass test it is a good baseline to work from and understand where the bottlenecks are. Obligitory Configuration Information: === # cat /etc/pf.conf # See pf.conf(5) for syntax and examples. # Remember to set net.inet.ip.forwarding=1 and/or net.inet6.ip6.forwarding=1 # in /etc/sysctl.conf if packets are to be forwarded between interfaces. set skip on lo # Table definitions table bad_hosts persist pass in on vlan300 # to establish keep-state # block any host deemed for whatever reason to be bad block quick from bad_hosts pass out on vlan300 # if a host is found to be connecting more than 100 times within 10 minutes # add them to bad_hosts table so they can be blocked pass in proto tcp to any port ssh keep state \ (max-src-conn-rate 15/5, overload bad_hosts flush global) # By default, do not permit remote connections to X11 #block in on ! lo0 proto tcp to port 6000:6010 # cat /etc/sysctl.conf # $OpenBSD: sysctl.conf,v 1.47 2009/06/09 11:52:54 sthen Exp $ # # This file contains a list of sysctl options the user wants set at # boot time. See sysctl(3) and sysctl(8) for more information on # the many available variables. # net.inet.ip.forwarding=1# 1=Permit forwarding (routing) of IPv4 packets #net.inet.ip.mforwarding=1 # 1=Permit forwarding (routing) of IPv4 multicast packets #net.inet.ip.multipath=1# 1=Enable IP multipath routing #net.inet.icmp.rediraccept=1# 1=Accept ICMP redirects #net.inet6.icmp6.rediraccept=0 # 0=Don't accept IPv6 ICMP redirects #net.inet6.ip6.forwarding=1 # 1=Permit forwarding (routing) of IPv6 packets #net.inet6.ip6.mforwarding=1# 1=Permit forwarding (routing) of IPv6 multicast packets #net.inet6.ip6.multipath=1 # 1=Enable IPv6 multipath routing #net.inet6.ip6.accept_rtadv=1 # 1=Permit IPv6 autoconf (forwarding must be 0) #net.inet.tcp.rfc1323=0 # 0=Disable TCP RFC1323 extensions (for if tcp is slow) #net.inet.tcp.rfc3390=0 # 0=Disable RFC3390 for TCP window increasing #net.inet.esp.enable=0 # 0=Disable the ESP IPsec protocol #net.inet.ah.enable=0 # 0=Disable the AH IPsec protocol #net.inet.esp.udpencap=0# 0=Disable ESP-in-UDP encapsulation #net.inet.ipcomp.enable=1 # 1=Enable the IPCOMP protocol #net.inet.etherip.allow=1 # 1=Enable the Ethernet-over-IP protocol #net.inet.tcp.ecn=1 # 1=Enable the TCP ECN extension #net.inet.carp.preempt=1# 1=Enable carp(4) preemption #net.inet.carp.log=1# 1=Enable logging of carp(4) packets #ddb.panic=0# 0=Do not drop into ddb on a kernel panic #ddb.console=1 # 1=Permit entry of ddb from the console #fs.posix.setuid=0 # 0=Traditional BSD chown() semantics #vm.swapencrypt.enable=0# 0=Do not encrypt pages that go to swap #vfs.nfs.iothreads=4# Number of nfsio kernel threads #net.inet.ip.mtudisc=0 # 0=Disable tcp mtu discovery
Re: OpenBSD performance numbers
- Original Message From: Robert info...@die-optimisten.net To: misc@openbsd.org Sent: Wed, August 11, 2010 12:18:24 PM Subject: Re: OpenBSD performance numbers On Wed, 11 Aug 2010 11:12:02 -0700 (PDT) James Peltier james_a_pelt...@yahoo.ca wrote: I would just like to share some information with the list about our new firewall/bridge and perhaps get some input as to where I might be able to look to squeeze some additional performance improvements. I must say though, I am very impressed with the performance improvements of networking/PF in the snapshots. You might want to read this: https://calomel.org/network_performance.html regards, Robert I am already familiar with these works. Thanks. ;)
Re: OpenBSD performance numbers
- Original Message From: Christiano F. Haesbaert haesba...@haesbaert.org To: James Peltier james_a_pelt...@yahoo.ca Cc: OpenBSD Mail List misc@openbsd.org Sent: Wed, August 11, 2010 12:29:48 PM Subject: Re: OpenBSD performance numbers snip Could you perform the same test using tcpbench between two openbsd boxes ? I never had the chance to test it under a heavy load like yours. /snip Henning made the same request off list. I'm going to be performing these tests later this week.
Re: OpenBSD performance numbers
- Original Message From: Daniel Melameth dan...@melameth.com To: OpenBSD Mail List misc@openbsd.org Sent: Wed, August 11, 2010 12:42:13 PM Subject: Re: OpenBSD performance numbers On Wed, Aug 11, 2010 at 12:12 PM, James Peltier james_a_pelt...@yahoo.ca wrote: Hello fellow OpenBSD'ers. I would just like to share some information with the list about our new firewall/bridge and perhaps get some input as to where I might be able to look to squeeze some additional performance improvements. I must say though, I am very impressed with the performance improvements of networking/PF in the snapshots. Parameters: === bridge: OpenBSD 4.8-BETA (snapshot Aug 5, 2010) ... So as can be seen here we are seeing data transfer rates of between 85 and 90MBps. Pretty impressive for an first pass, untweaked configuration. However, there are some unfortunates. During these tests the system was running at between 80 and 95% interrupt, with the inverse being idle. This means that either there are some tweaks that I can add to counteract the interrupts, perhaps a tweak for interrupt mitigation, or that the hardware is currently not able to handle more than a single gigabit link running at full capacity. In any case I would like to know what the developers see if better hardware would help as well as any performance tweaks that may help. These unfortunates are not really bad news. The box is certainly up to the task of dealing with our network traffic. Some tweaking may help and for a first pass test it is a good baseline to work from and understand where the bottlenecks are. I imagine you'll see better performance if you do not use OpenBSD as a bridge. I am aware of the limitations of OpenBSDs bridge code. However, OpenBSD is often in this position and sometimes you just don't have a choice, so posting numbers for OpenBSD in this mode of operation in the hopes of getting tweaks, good ideas or in general the code fixed for this role is always good. Just trying to make the project better. ;) --- James A. Peltier james_a_pelt...@yahoo.ca
Re: which monitoring do you use (on OpenBSD)
- Original Message From: Jason Dixon ja...@dixongroup.net To: C. Bensend be...@bennyvision.com Cc: misc@openbsd.org Sent: Tue, August 10, 2010 12:58:50 PM Subject: Re: which monitoring do you use (on OpenBSD) On Tue, Aug 10, 2010 at 12:41:26PM -0500, C. Bensend wrote: nagios is shit. misdesigned, horrible code, and someone who obviously doesn't understand blocking semantics of sockets writing that part of the code... that said, I use it, too. and as almost every other serious user with at least a little bit of standards left I hate it. I cannot speak to the quality of code; I couldn't code my way out of a wet paper bag and am horribly unqualified to comment. Henning is completely accurate (*). Nagios code is shite and reflects poorly on the engineering skills of the creator. Its near-monopoly position in the community is based on two factors: 1) Price. Although you pay dearly in time spent setting it up, maintaining it, and in outages caused by it (keep reading). 2) It's the least crappy of all crappy open-source monitoring options. However, this is a majority of my job where I am now, and I don't dislike it. It's infinitely extensible, makes it simple to write plugins for stuff that you can't already find one for, and has a fairly large community. We used it for a very long time on a very large scale. While it is extensible, it promotes poor design choices and puts no limitations on the style or number of shite extensions. But my biggest beef is on some of the design choices that allow you to shoot yourself in the foot. As my therapist would say, Nagios is an enabler. Take for example, Nagios acknowledgments. They never expire, so it's very easy to ack something and forget about it. For days. Or better yet, the idea of flapping. At face value, this seems like a good idea. But whatever happened to actually *responding* to an alert when something goes wrong. Let me get this straight... you WANT your monitoring system to stop alerting you when your shit goes down? What am I missing here? It's a *helluva* lot better than Mon or Big Brother, both of which I've used in the past, and both of which made me weep tears of blood. See above. (*) I should disclose that I'm the Prod. Mgr. for Circonus, a SaaS version of Reconnoiter with trending, fault detection and notifications. Circonus is not free, but is based on Reconnoiter which is actively developed as an open-source BSD-licensed project. Both were engineered to directly address the pain we've experienced over the years working with solutions like Nagios and Cacti. So although it's fair to consider me biased towards our software, suffice it to say that if Nagios didn't suck so badly we never would have developed either Reconnoiter or Circonus. There are some OpenBSD-Reconnoiter users in the community; if you're interested in finding out more about Reconnoiter, ask around or check out the project website. http://labs.omniti.com/labs/reconnoiter -- Jason Dixon DixonGroup Consulting http://www.dixongroup.net/ Being as I have never used Reconnoiter or Circonus, would you care to elaborate as to where these products suck less then Nagios or other solutions? I am looking into replacing out very aged monitoring system now and Nagios is the one that seems to stand out the most, although Zabbix and Munin look good in their own rights. Guidance is always appreciated. :)
Re: CARP technical paper
- Original Message
From: Henning Brauer lists-open...@bsws.de
To: misc@openbsd.org
Sent: Thu, July 29, 2010 3:32:01 AM
Subject: Re: CARP technical paper
* Steven Moncayo ste...@infoquality.com.ec [2010-07-29 08:30]:
My request goes for a tech paper with specifications for the CARP protocol,
just like a RFC. I Google 'd quite a long time with no luck. Wish you could
help with this.
/usr/src/sys/netinet/ip_carp.c
/usr/src/sys/netinet/ip_carp.h
--
Henning Brauer, h...@bsws.de, henn...@openbsd.org
BS Web Services, http://bsws.de
Full-Service ISP - Secure Hosting, Mail and DNS Services
Dedicated Servers, Rootservers, Application Hosting
You forgot a batch processing step.
gen_tech_paper -in
{/usr/src/sys/netinet/ip_carp.c,/usr/src/sys/netinet/ip_carp.h} -out
tech_paper.pdf
---
James A. Peltier james_a_pelt...@yahoo.ca
Re: traffic management
Ouch. I like IRIX. ex-SGI employee 43951. :) --- James A. Peltier james_a_pelt...@yahoo.ca - Original Message From: Theo de Raadt dera...@cvs.openbsd.org To: Jan Stary h...@stare.cz Cc: misc@openbsd.org Sent: Tue, June 1, 2010 2:40:37 PM Subject: Re: traffic management Hello Misc, Are there any plans have changed in the system of traffic control? For example removal of code altq from pf and make a separate management interface traffic other than pf. Or replace altq to something else, more fast, simple and functional. Or revision of an existing traffic management system. obvious troll is obvious no kidding. As we've told irix before, it will not happen.
Confirmation of trunk configuration
I'm trying to configure OpenBSD with trunking using LACP but I can't
seem to get it to work correctly. I have an HP Procurve 5304XL connected to a
Dell 1750 with an Intel PRO/1000MT QP (82546EB). I am unable to get trunking
and LACP to work together for some reason. Any help would be greatly
appreciated.
HP Ports B1 and B2 are connected to Dell 1750 em0 and em1
HP Ports B3 and B4 are connected to Dell 1750 em2 and em3
ProCurve Switch 5304XL# show lacp
LACP
no LACP ports found.
ProCurve Switch 5304XL# show trunk
Load Balancing
Port | Name Type | Group Type
+ - + - -
ProCurve Switch 5304XL(config)# trunk b3-b4 trk2 lacp
ProCurve Switch 5304XL(config)# show trunk
Load Balancing
Port | Name Type | Group Type
+ - + - -
B3 | 100/1000T | Trk2 LACP
B4 | 100/1000T | Trk2 LACP
ProCurve Switch 5304XL(config)# show lacp
LACP
PORT LACP TRUNK PORT LACP LACP
NUMB ENABLED GROUP STATUSPARTNER STATUS
--- --- --- --- ---
B3 ActiveTrk2 UpNoSuccess
B4 ActiveTrk2 UpNoSuccess
ProCurve Switch 5304XL(config)# vlan 303 name NAT
ProCurve Switch 5304XL(config)# vlan 303 tagged trk2
ProCurve Switch 5304XL(config)# show vlan 303
Status and Counters - VLAN Information - Ports - VLAN 303
VLAN ID : 303
Name : NAT
Status : Port-based
Voice : No
Port Information Mode Unknown VLAN Status
--
Trk2 Tagged LearnUp
ProCurve Switch 5304XL(config)# vlan 303 ip address 10.0.0.253 255.0.0.0
ProCurve Switch 5304XL(config)# wr mem
OpenBSD box
cat /etc/hostname.em2
up
cat /etc/hostname.em3
up
cat /etc/hostname.trunk1
trunkproto lacp trunkport em2 trunkport em3 up
cat /etc/hostname.vlan303
vlan 303 vlandev trunk1 descr NAT Network 10.0.0.254/8
# ifconfig em2
em2: flags=8b43UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST mtu
1500
lladdr 00:04:23:45:de:e6
priority: 0
trunk: trunkdev trunk1
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet6 fe80::204:23ff:fe45:dee6%em2 prefixlen 64 scopeid 0x3
# ifconfig em3
em3: flags=8b43UP,BROADCAST,RUNNING,PROMISC,ALLMULTI,SIMPLEX,MULTICAST mtu
1500
lladdr 00:04:23:45:de:e6
priority: 0
trunk: trunkdev trunk1
media: Ethernet autoselect (1000baseT full-duplex)
status: active
inet6 fe80::204:23ff:fe45:dee7%em3 prefixlen 64 scopeid 0x4
# ifconfig trunk1
trunk1: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500
lladdr 00:04:23:45:de:e6
priority: 0
trunk: trunkproto lacp
trunk id: [(8000,00:04:23:45:de:e6,404C,,),
(,00:00:00:00:00:00,,,)]
trunkport em3 active,collecting,distributing
trunkport em2 active,collecting,distributing
groups: trunk
media: Ethernet autoselect
status: active
inet6 fe80::204:23ff:fe45:dee6%trunk1 prefixlen 64 scopeid 0x9
# ifconfig vlan303
vlan303: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500
lladdr 00:04:23:45:de:e6
description: NAT Network
priority: 0
vlan: 303 priority: 0 parent interface: trunk1
groups: vlan
inet6 fe80::204:23ff:fe45:dee6%vlan303 prefixlen 64 scopeid 0xa
inet 10.0.0.254 netmask 0xff00 broadcast 10.255.255.255
# ping 10.0.0.253
PING 10.0.0.253 (10.0.0.253): 56 data bytes
ping: sendto: Host is down
ping: wrote 10.0.0.253 64 chars, ret=-1
ping: sendto: Host is down
ping: wrote 10.0.0.253 64 chars, ret=-1
ping: sendto: Host is down
ping: wrote 10.0.0.253 64 chars, ret=-1
ping: sendto: Host is down
ping: wrote 10.0.0.253 64 chars, ret=-1
--- 10.0.0.253 ping statistics ---
13 packets transmitted, 0 packets received, 100.0% packet loss
--
James A. Peltier james_a_pelt...@yahoo.ca
Re: VLANs and security (was:network performance problems)
--- On Tue, 2/16/10, Corey clinge...@gmail.com wrote: From: Corey clinge...@gmail.com Subject: VLANs and security (was:network performance problems) To: misc@openbsd.org Received: Tuesday, February 16, 2010, 8:54 PM I did put all interfaces (in,out,pfsync,management) through VLANs in msk0 Throwing out a topic for discussion...I have seen a couple of posts on here regarding use of VLANs to segregate traffic that I would usually use separate interfaces for. I am just curious what the thoughts of the list are on this practice. I haven't ever set up VLANs on anything large or serious, and do not claim to know the security implications, other than switch/interface misconfiguration possibly getting one into trouble, and awareness of (but no experience with) tools like dsniff. There is quite a bit of stuff out there on Google, of course, but I trust this list more :^) Thanks in advance. We use VLANs quite extensively and are now looking at deploying VRF-ish solutions for the campus. We still use multiple interfaces in order to spread the interrupt load for really busy VLANs. Security is not really a factor in VLANs, as they don't provide any inherent increase in security. Misconfigurations would equate to the same compromises really. --- James A. Peltier james_a_pelt...@yahoo.ca __ Looking for the perfect gift? Give the gift of Flickr! http://www.flickr.com/gift/
Re: routing and pf at 10Gbps
--- On Thu, 2/11/10, Claudio Jeker cje...@diehard.n-r-g.com wrote: From: Claudio Jeker cje...@diehard.n-r-g.com Subject: Re: routing and pf at 10Gbps To: misc@openbsd.org Received: Thursday, February 11, 2010, 5:24 PM On Thu, Feb 11, 2010 at 03:07:28PM -0500, Daniel Ouellet wrote: On 2/11/10 2:46 PM, Henning Brauer wrote: disk i/o is irrelevant. you will need a very very very fast opengl capable graphics card with loads of memory of course. ??? I am sure I am missing something big here, but Fast Video Card with OpenGL for router? Are you trying to look live every packets routed here? If I may asked Henning, please give me a clue stick as that part I really do not understand what so ever. No bunt intended, I just do not understand that at all, please help me get it? What Video have to do with routing? Henning, I told you, we should not talk about unfinsihed projects. We planned to announce this in exactly 7 weeks. Anyway, to late, the cat is out of the bag. So Henning and Oga are working at offloading pf into the graphic card cores by using the DRI interface. The shader will evaluate the ruleset and packets in parallel and use the graphic memory for the state table. Additionally if the speed of one card is not enough you can use SLI or crossfire to use multiple cards in parallel. -- :wq Claudio It is just a 3-line diff You have *got* to be kidding me. - head explodes - --- James A. Peltier james_a_pelt...@yahoo.ca
Re: Download rate and sysctl settings
--- On Sat, 2/6/10, Claudio Jeker cje...@diehard.n-r-g.com wrote: From: Claudio Jeker cje...@diehard.n-r-g.com Subject: Re: Download rate and sysctl settings To: misc@openbsd.org Received: Saturday, February 6, 2010, 9:12 AM On Sat, Feb 06, 2010 at 01:27:12PM +0100, Sebastiano Pomata wrote: Il 06/02/10 03:55, Stuart Henderson ha scritto: I really can understand this, for the sake of system portability and so on. Anyway, I really hardly understand why, without touching any of the default settings, download rate from every server would never overcome the value of 400 kB/s. Is it all due to the tcp windows size? Yes. Thank you for the clear answer. Anyway, trying to act on tcp.sendspace isn't affecting the upload capabilities of my OpenBSD server. I tried downloading a file through httpd, via ftp but results are still disappointing: 60-70 kbps between two boxes on the same switch. The box is going to become a webserver, could you please give me more hints about tuning network performance? Check your links. This sounds like a full-duplex issue between switch and machines. On a LAN even with default tcp send/recvspace you should get easily get up to 200Mbps. -- :wq Claudio If the firewall is on try turning it off or go to a very simple rule set. Perhaps there is a problem with your filtering rules and not the network settings. --- James A. Peltier james_a_pelt...@yahoo.ca __ Make your browsing faster, safer, and easier with the new Internet Explorer. 8. Optimized for Yahoo! Get it Now for Free! at http://downloads.yahoo.com/ca/internetexplorer/
Re: Download rate and sysctl settings
--- On Sat, 2/6/10, Kenneth R Westerback kwesterb...@rogers.com wrote: From: Kenneth R Westerback kwesterb...@rogers.com Subject: Re: Download rate and sysctl settings To: Sebastiano Pomata sebastianopom...@tiscali.it Cc: misc@openbsd.org Received: Saturday, February 6, 2010, 11:33 AM On Sat, Feb 06, 2010 at 04:09:08PM +0100, Sebastiano Pomata wrote: Il 06/02/10 15:12, Claudio Jeker ha scritto: On Sat, Feb 06, 2010 at 01:27:12PM +0100, Sebastiano Pomata wrote: Il 06/02/10 03:55, Stuart Henderson ha scritto: I really can understand this, for the sake of system portability and so on. Anyway, I really hardly understand why, without touching any of the default settings, download rate from every server would never overcome the value of 400 kB/s. Is it all due to the tcp windows size? Yes. Thank you for the clear answer. Anyway, trying to act on tcp.sendspace isn't affecting the upload capabilities of my OpenBSD server. I tried downloading a file through httpd, via ftp but results are still disappointing: 60-70 kbps between two boxes on the same switch. The box is going to become a webserver, could you please give me more hints about tuning network performance? Check your links. This sounds like a full-duplex issue between switch and machines. On a LAN even with default tcp send/recvspace you should get easily get up to 200Mbps. Just logged through ssh on the server, ifconfig reports: re0: flags=8843UP,BROADCAST,RUNNING,SIMPLEX,MULTICAST mtu 1500 lladdr 00:b0:c2:02:5e:a0 priority: 0 groups: egress media: Ethernet autoselect (100baseTX full-duplex,rxpause,txpause) status: active inet 192.167.132.99 netmask 0xff00 broadcast 192.167.132.255 inet6 fe80::2b0:c2ff:fe02:5ea0%re0 prefixlen 64 scopeid 0x2 As from the name, nic is a common Realtek card (OpenBSD just got it without need of doing anything). So I suppose nic is running in full duplex. Hints? Optimally you now need to check what the switch port is configured/negotiated to. e.g. if it has ended up in 10/half you have a problem. Ditto for the connections for the other device. If you have no access to the switch you can try every manual media setting to force your OpenBSD boxen to the different possibilities and see if any work better. Ken Have you tried another network card, like an Intel (em) based card? The Realtek cards have, at least in the past, been poor performers for me. __ The new Internet Explorer. 8 - Faster, safer, easier. Optimized for Yahoo! Get it Now for Free! at http://downloads.yahoo.com/ca/internetexplorer/
Re: Building a High-performance Computing Cluster Using OpenBSD
--- On Sat, 2/6/10, Predrag Punosevac punoseva...@gmail.com wrote: From: Predrag Punosevac punoseva...@gmail.com Subject: Re: Building a High-performance Computing Cluster Using OpenBSD To: misc@openbsd.org, list-...@designtools.org Received: Saturday, February 6, 2010, 10:56 AM J.C. Roberts list-...@designtools.org wrote: On Fri, 05 Feb 2010 23:39:19 -0500 Predrag Punosevac punoseva...@gmail.com wrote: Dear All, Could anybody kindly point me to any literature regarding building a high-performance computing cluster using OpenBSD. I am not interested in FreeBSD and NetBSD related papers on this topics. I can find them easily. I am specifically interested in OpenBSD. Applications I am planning to run are related to Bifurcation Theory. Thank You, Predrag Punosevac Pendrag, At one point in time, the phrase High Performance Computing (HPC) actually meant something fairly specific, but over the years it has degraded to an exceedingly vague buzzword. In the classic sense of HPC where you're doing significant amounts of computation on problems requiring tightly coupled nodes (i.e. hard parallelization), That is exactly what I have in mind. I have computations which can be parallelized and which currently require in upward of a week to preform. Usually, after a week we see that we didn't get quite right the initial conditions and we are repeated the thing. After half dozen iteration we usually get things right. That takes about 2 months. We have a pile of blades (i386/amd64) laying around and my idea (even that I have never done that) before is that we tightly couple and try to reduce the computation time to less then a day per computation. asking for OpenBSD specific papers on this topic is the equivalent of asking for papers on using a hammer to trun a screw. In the case of using classic HPC on hard parallelization problems, OpenBSD is the wrong tool for the job. The reason is OpenBSD does not support vast amounts of RAM, and it doesn't have support for fast memory interconnects (Myrinet, SCI, ...). I had a hunch that OpenBSD is a wrong tool but I wanted to make sure that I am not missing anything. That is why I posted the question. C.J. which OS would you pick. A main FreeBSD paper on cluster computing is from 2003 when SMP support was immature. Now they have ULE, good SMP I would have to check for other things. NetBSD mailing list tech-cluster is dead. NetBSD amd64 does support lots of RAM. They seem to have a great SMP support now. I see that NetBSD was used in the past for those things. I would still go with GNU/Linux unless you're dead set on a BSD in which case FreeBSD would be your best choice from a performance standpoint. If it has to be Linux would you go with a RedHat? Or a RedHat derivative such as CentOS or Scientific Linux. Please tell me little bit more. If the problems you're trying to solve do not have intensive memory requirements and qualify as easy parallelization (a.k.a. Embarrassingly Parallel), then you do not need a tightly coupled cluster and OpenBSD could be a good choice. In essence, it comes down to the specific problem(s) *YOU* are trying to solve, so you *REALLY* need to elaborate on your problem domain(s) and how you are trying to solve them. Well I said it is computation of Bifurcations around homoclinic orbits as well as computing of fast responding curves. I just got in into the team. At this point I am not even sure if the simulations by co-workers want to do are events of positive measure. I am thinking about it. I am more of a guy who is proving theorems rather than trying to compute something but as you can see I do not mind getting my hands dirty. Embarrassingly Parallel can be interpreted two different ways as the term parallel can be interpreted to mean, I have a job that takes 1-n parameters and I want to run as many tests in parallel as possible or, I have a job that can easily have its problem domain split across 1-n nodes. Which does your target? Does your problem set fit within the confines of a single nodes memory? If not inter-node interconnect is going to become an issue especially if there is a lot of inter-node communication taking place. We have a Torque+Maui+CentOS 5 cluster with 68 nodes and 628 cores which is GigE connected and since the majority of our jobs are serial, thousands of jobs with different parameters simultaneously or little inter-node communication it works just fine. For weather simulations it certainly would be much better to have Infiniband. I'm not entirely familiar with your problem set but based on some, albeit rudimentary reading, inter-node *could* be an issue for you. __ The new Internet Explorer. 8 - Faster, safer, easier. Optimized for Yahoo! Get it Now for Free! at
Re: Building a High-performance Computing Cluster Using OpenBSD
--- On Sat, 2/6/10, Daniel Dickman didick...@gmail.com wrote: From: Daniel Dickman didick...@gmail.com Subject: Re: Building a High-performance Computing Cluster Using OpenBSD To: misc@openbsd.org Cc: punoseva...@gmail.com Received: Saturday, February 6, 2010, 1:01 AM Could anybody kindly point me to any literature regarding building a high-performance computing cluster using OpenBSD. I am not interested in FreeBSD and NetBSD related papers on this topics. I can find them easily. I am specifically interested in OpenBSD. Applications I am planning to run are related to Bifurcation Theory. You'll probably want to provide just a bit more detail about what you have in mind. But you can take a look at devel/lam and sysutils/clusterit if you haven't already... You may want to consider looking at GNU/Linux and not be stuck on using OpenBSD. I'll probably get flamed, but really GNU/Linux is the dominant HPC platform and the application set is far greater. Not that I don't like OpenBSD, but HPC isn't its forte so to speak. Of course feel free to try. Look into MPICH, MPICH2, OpenMPI (my choice). In the end it's the applications that matter, not the OS. --- James A. Peltier james_a_pelt...@yahoo.ca
Re: -CURRENT, VLANs, NAT
--- On Tue, 2/2/10, David Gwynne l...@animata.net wrote: match out on vlan301 from vlan303:network nat-to vlan301 all the cool kids are going: match out on vlan301 nat-to vlan301 received-on vlan303 You've got to be kidding me. This makes me all giddy inside! Woot! Woot! --- James A. Peltier james_a_pelt...@yahoo.ca __ Make your browsing faster, safer, and easier with the new Internet Explorer. 8. Optimized for Yahoo! Get it Now for Free! at http://downloads.yahoo.com/ca/internetexplorer/
Re: pf and apache: to stop a scripter
--- On Tue, 2/2/10, Lars Nooden lars.cura...@gmail.com wrote: From: Lars Nooden lars.cura...@gmail.com Subject: Re: pf and apache: to stop a scripter To: Cc: Jacob Yocom-Piatt j...@fixedpointgroup.com, OpenBSD general usage list misc@openbsd.org Received: Tuesday, February 2, 2010, 6:58 AM Jacob Yocom-Piatt wrote: there is a website protected by pf and running apache on a recent openbsd snapshot that needs to be protected against scripting attacks. i can configure both pf and apache to help block this behavior but am not familiar with the best practices for such configurations. the situation is that a user who authenticates to apache via htpasswd has run a script a number of times in an attempt to mine a database. all of the user activity is already logged by apache and it is crystal clear that scripting is going on. i would like to stop this scripting in its tracks and here is what i am already looking at: Jacob, what was their response when you spoke with them in person (or on the phone) about the scripting? How, exactly, did you word your request for them to stop? /Lars Stop! Or I'll say stop again! :) __ Looking for the perfect gift? Give the gift of Flickr! http://www.flickr.com/gift/
Re: -CURRENT, VLANs, NAT
--- On Tue, 2/2/10, David Gwynne l...@animata.net wrote: all the cool kids are going: match out on vlan301 nat-to vlan301 received-on vlan303 I just got around to testing this rule and it didn't work for me as I would have expected. The output of pfctl -nv -f /etc/pf.conf expanded to the inet6 address of VLAN 301 interface by default. When I changed the line to read match out on vlan301 inet nat-to vlan301 received-on vlan303 it expanded to the inet address I would have expected to see by default. Is this intended or a bug? I would assume that you would want to expand to inet by default and not inet6. This is of course just a matter of opinion. --- James A. Peltier james_a_pelt...@yahoo.ca
-CURRENT, VLANs, NAT
Hi All, I'm trying to setup a new router/firewall for multiple VLANs including one VLAN that must be NAT and I seem to be running into an odd issue. OS is OpenBSD 4.7-BETA; Jan 27, 2010 snapshot from ftp.openbsd.org /etc/hostname.em0 -- up /etc/hostname.em0 -- up /etc/hostname.vlan301 -- inet 1.2.3.4 255.255.255.0 vlan 301 vlandev em0 description Uplink /etc/hostname.vlan303 -- inet 10.0.0.254 255.255.255.0 vlan 303 vlandev em0 description NAT /etc/pf.conf -- #skip filtering on loopback set skip on lo # NAT VLAN 303 traffic on our Uplink VLAN nat on vlan301 from vlan303:network to any - (vlan301) pass# to establish keep-state So, starting with a very simple rule set, however, pfctl -nf /etc/pf.conf complains that the nat on line is incorrect. I used the similar example from http://www.openbsd.org/cgi-bin/man.cgi?query=pf.confsektion=5arch=i386apropos=0manpath=OpenBSD+Current Am I missing something here? It would seem that this would map all VLAN 303 (10.0.0.0/24) addresses to VLAN 301 (1.2.3.4) address. Has the syntax changed and even -current documentation isn't correct? --- James A. Peltier james_a_pelt...@yahoo.ca
Correction: -CURRENT, VLANs, NAT
--- On Mon, 2/1/10, James Peltier james_a_pelt...@yahoo.ca wrote: From: James Peltier james_a_pelt...@yahoo.ca Subject: -CURRENT, VLANs, NAT To: OpenBSD Mail List misc@openbsd.org Received: Monday, February 1, 2010, 7:27 PM Hi All, I'm trying to setup a new router/firewall for multiple VLANs including one VLAN that must be NAT and I seem to be running into an odd issue. OS is OpenBSD 4.7-BETA; Jan 27, 2010 snapshot from ftp.openbsd.org /etc/hostname.em0 -- up /etc/hostname.em0 -- up /etc/hostname.vlan301 -- inet 1.2.3.4 255.255.255.0 vlan 301 vlandev em0 description Uplink /etc/hostname.vlan303 -- inet 10.0.0.254 255.255.255.0 vlan 303 vlandev em0 description NAT Please note a mistype. The VLAN device for this VLAN is em1 and not em0. It should read this inet 10.0.0.254 255.255.255.0 vlan 303 vlandev em1 description NAT /etc/pf.conf -- #skip filtering on loopback set skip on lo # NAT VLAN 303 traffic on our Uplink VLAN nat on vlan301 from vlan303:network to any - (vlan301) pass# to establish keep-state So, starting with a very simple rule set, however, pfctl -nf /etc/pf.conf complains that the nat on line is incorrect. I used the similar example from http://www.openbsd.org/cgi-bin/man.cgi?query=pf.confsektion=5arch=i386apro pos=0manpath=OpenBSD+Current Am I missing something here? It would seem that this would map all VLAN 303 (10.0.0.0/24) addresses to VLAN 301 (1.2.3.4) address. Has the syntax changed and even -current documentation isn't correct? --- James A. Peltier james_a_pelt...@yahoo.ca bookmark your favourite sites. Download it now http://ca.toolbar.yahoo.com. __ Make your browsing faster, safer, and easier with the new Internet Explorer. 8. Optimized for Yahoo! Get it Now for Free! at http://downloads.yahoo.com/ca/internetexplorer/
Re: -CURRENT, VLANs, NAT
--- On Mon, 2/1/10, Scott Learmonth sc...@moosepile.net wrote: From: Scott Learmonth sc...@moosepile.net Subject: Re: -CURRENT, VLANs, NAT To: misc@openbsd.org Received: Monday, February 1, 2010, 10:04 PM On Mon, Feb 01, 2010 at 06:02:07PM -0800, Scott Learmonth wrote: On Mon, Feb 01, 2010 at 04:27:12PM -0800, James Peltier wrote: Hi All, I'm trying to setup a new router/firewall for multiple VLANs including one VLAN that must be NAT and I seem to be running into an odd issue. OS is OpenBSD 4.7-BETA; Jan 27, 2010 snapshot from ftp.openbsd.org /etc/hostname.em0 -- up /etc/hostname.em0 -- up /etc/hostname.vlan301 -- inet 1.2.3.4 255.255.255.0 vlan 301 vlandev em0 description Uplink /etc/hostname.vlan303 -- inet 10.0.0.254 255.255.255.0 vlan 303 vlandev em0 description NAT /etc/pf.conf -- #skip filtering on loopback set skip on lo # NAT VLAN 303 traffic on our Uplink VLAN nat on vlan301 from vlan303:network to any - (vlan301) pass # to establish keep-state So, starting with a very simple rule set, however, pfctl -nf /etc/pf.conf complains that the nat on line is incorrect. I used the similar example from http://www.openbsd.org/cgi-bin/man.cgi?query=pf.confsektion=5arch=i386apro pos=0manpath=OpenBSD+Current Am I missing something here? It would seem that this would map all VLAN 303 (10.0.0.0/24) addresses to VLAN 301 (1.2.3.4) address. Has the syntax changed and even -current documentation isn't correct? --- James A. Peltier james_a_pelt...@yahoo.ca Yes, the syntax has changed. I only briefly looked, but the faq seems dated. The man page is correct. You'd want something like pass out on vlan301 from vlan303:network nat-to vlan301 Cheers I stand somewhat corrected. The link you provided doesn't seem to jive with what my system gives me. I'm not going to comment further on that though without doing my homework and/or supplying a diff lest I look like even more of a fool. Nonetheless, pass out on vlan301 from vlan303:network to ! vlan301 nat-to vlan301 should work for you. You may want to look at match instead/as well. p.s. my last note was missing the to I did end up finding that the documentation had changed and match out did correct the problem. match out on vlan301 from vlan303:network nat-to vlan301 as could be found in http://www.openbsd.org/faq/current.html#20090901 Just needed to look harder.. Move along, nothing to see here. ;) __ Get a sneak peak at messages with a handy reading pane with All new Yahoo! Mail: http://ca.promos.yahoo.com/newmail/overview2/
Re: Maximizing File/Network I/O
--- On Thu, 1/14/10, Jean-Francois jfsimon1...@gmail.com wrote: From: Jean-Francois jfsimon1...@gmail.com Subject: Re: Maximizing File/Network I/O To: misc@openbsd.org Received: Thursday, January 14, 2010, 12:53 PM Le mardi 05 janvier 2010 09:04:53, nixlists a icrit : On Tue, Jan 5, 2010 at 1:45 AM, Bret S. Lambert blamb...@openbsd.org wrote: Start with mount_nfs options, specifically -r and -w; I assume that you would have mentioned tweaking those if you had already done so. Setting -r and -w to 16384, and jumbo frames to 9000 yields just a couple of MB/s more. Far from 10 MB/s more the network can do ;( For some reasone, when I mount NFS drives with -r=4096 and -w=4096 I reach the best transfer rates. This is possibly because the OS is able to match the request to a single memory page for your architecture. Other architectures offer larger page sizes. Not saying that's the case, but a possibility. __ Looking for the perfect gift? Give the gift of Flickr! http://www.flickr.com/gift/
Re: VLANs, OpenBSD, Cisco HP
--- On Thu, 1/14/10, Graeme Lee gra...@omni.net.au wrote: Check that you are not tagging the incoming traffic as vlan 301. The ports need to be in trunk mode. It so funny that you should mention this, yesterday we had a 7 hour outage due to our Cisco 6506 failing to route anything on our network. It took Cisco engineers 5 of those 7 hours to restore service. Once everything was back up and running I noticed that the port that I configured for VLAN 301 was the native VLAN on the Cisco trunk and thus was not tagged. Even the Cisco guys didn't notice this. I think everything should work fine now but I haven't gotten back to working on it because I have several hundred RT tickets to attend to this morning due to the outage. ;( Sorry for the noise and thanks for the help guys. __ Be smarter than spam. See how smart SpamGuard is at giving junk email the boot with the All-new Yahoo! Mail. Click on Options in Mail and switch to New Mail today or register for free at http://mail.yahoo.ca
Re: VLANs, OpenBSD, Cisco HP
--- On Thu, 1/14/10, Graeme Lee gra...@omni.net.au wrote: From: Graeme Lee gra...@omni.net.au Subject: Re: VLANs, OpenBSD, Cisco HP To: misc@openbsd.org Received: Thursday, January 14, 2010, 3:27 AM inet 1.2.3.4 255.255.255.0 NONE vlan 301 vlandev em0 description Uplink Like this: # cat /etc/hostname.vlan0 vlan 301 vlandev em0 inet 192.168.1.2 255.255.255.0 192.168.1.255 description Uplink # cat /etc/hostname.em0 up From everything I have read in the man pages, FAQ and the great oracle Google, my chosen syntax works too. See http://www.openbsd.org/faq/faq6.html Or, you may want to use special flags specific to a certain interface. The format of the hostname file doesn't change much! $ cat /etc/hostname.vlan0 inet 172.21.0.31 255.255.255.0 NONE vlan 2 vlandev fxp1
VLANs, OpenBSD, Cisco HP
Hi have an OpenBSD -current installation as of today that I'm trying to get VLANs working on. I have an link from a Cisco 6506 (interface 5/8) to a HP ProCurve 5408XL port B4. The Cisco port 5/8 is configured to the following set trunk 5/8 on dot1q vlan 301 on the HP ProCurve I have added the VLANs to the switch and ports and it works but not the way I would expect. Port B4 has VLAN 301 tagged and A1 is the port on which the OpenBSD box is connected which is also tagged VLAN 301. On the OpenBSD box I have /etc/hostname.em0 -- up /etc/hostname.vlan301 -- inet 1.2.3.4 255.255.255.0 NONE vlandev em0 description Uplink /etc/mygate -- 1.2.3.254 So, here's what I don't expect and maybe my expectations are wrong, but anyways. This configuration doesn't work?!? If I have inet 1.2.3.4 255.255.255.0 NONE description Uplink in /etc/hostname.em0 it works. Since the port is tagged on the Cisco and both HP ports I would have thought that you needed to have the VLAN 301 configuration on OpenBSD as well to properly untag the ports? Any help would be extremely useful as I'm trying to deploy this as a VLAN router. I'm sure it's something really simple that I'm missing here. --- James A. Peltier james_a_pelt...@yahoo.ca __ Looking for the perfect gift? Give the gift of Flickr! http://www.flickr.com/gift/
Re: VLANs, OpenBSD, Cisco HP
--- On Thu, 1/14/10, James Peltier james_a_pelt...@yahoo.ca wrote: /etc/hostname.vlan301 -- inet 1.2.3.4 255.255.255.0 NONE vlandev em0 description Uplink Please note that I've typed this wrong and it actually has inet 1.2.3.4 255.255.255.0 NONE vlan 301 vlandev em0 description Uplink in /etc/hostname.em0 and doesn't work. Just wanted to make sure people don't jump to the your sytax is wrong theory. ;)
Re: OT: Have you hugged your local OpenBSD dev lately?
--- On Wed, 11/18/09, Bryan bra...@gmail.com wrote: From: Bryan bra...@gmail.com Subject: OT: Have you hugged your local OpenBSD dev lately? To: Misc OpenBSD misc@openbsd.org Received: Wednesday, November 18, 2009, 7:05 PM So glad we don't have these kinds of issues... https://bugzilla.redhat.com/show_bug.cgi?id=534047 This is a blatant ID10T error. Comments 9 and 10 are my favorite. Last I looked it *was* insecure to let non-root users install software let alone do it by default and without a password! --- James A. Peltier james_a_pelt...@yahoo.ca __ Looking for the perfect gift? Give the gift of Flickr! http://www.flickr.com/gift/
Re: Relayd relayctl reload error on 4.6
--- On Thu, 11/12/09, Brent Jones br...@servuhome.net wrote: From: Brent Jones br...@servuhome.net Subject: Relayd relayctl reload error on 4.6 To: misc@openbsd.org Received: Thursday, November 12, 2009, 6:55 PM It seems the 'relayctl' command returns an error code when used on several systems of mine (all i386 4.6) # relayctl reload command failed Found this bug files in January with similar issue with relayctl: http://cvs.openbsd.org/cgi-bin/query-pr-wrapper?full=yesnumbers=6046 If I can provide any additional details, or if anyone would like me to try anything to get it going, I'm more than happy to help. Regards, -- Brent Jones br...@servuhome.net Doesn't work here either. I issue a pkill relayd; relayd to restart
Re: Simpliest issue tracking software?
--- On Tue, 9/22/09, Gregory Edigarov g...@bestnet.kharkov.ua wrote: From: Gregory Edigarov g...@bestnet.kharkov.ua Subject: Simpliest issue tracking software? To: misc@openbsd.org Received: Tuesday, September 22, 2009, 5:09 AM Hello everybody, I am looking for an advice of which issue tracking system to use for a small team of admins (4 members)? OTRS, RT - are an overhead for our purposes. so we don't need anything fancy, all we need is to make sure all requests coming from our abonent department and users will be properly processed. -- With best regards, Gregory Edigarov I use Request Tracker, but ticgit might be of use. Extremely simple but I haven't used it. __ Looking for the perfect gift? Give the gift of Flickr! http://www.flickr.com/gift/
Re: router/firewall
http://openbsd.org/faq/pf/index.html --- James A. Peltier james_a_pelt...@yahoo.ca --- On Fri, 9/4/09, Sha'ul pbap...@gmail.com wrote: From: Sha'ul pbap...@gmail.com Subject: router/firewall To: misc@openbsd.org Received: Friday, September 4, 2009, 4:33 AM Where can I find some information or some sort of guide for how to setup and configure OpenBSD to install on an old PC to use as a router and firewall? __ Looking for the perfect gift? Give the gift of Flickr! http://www.flickr.com/gift/
Re: Recommendation for Beowulf/Apache Setup
--- On Thu, 5/7/09, Vivek Ayer vivek.a...@gmail.com wrote: From: Vivek Ayer vivek.a...@gmail.com Subject: Recommendation for Beowulf/Apache Setup To: misc misc@openbsd.org Received: Thursday, May 7, 2009, 12:36 PM Hey guys, This is a very general question, but I'm sure not exactly sure how to proceed. I'll be getting a lot of hardware soon to be clustered and I was wondering what was your take on the setup. My setup was going to be: 1 OpenBSD Router running 4.5 routing to a subnet of 13 nodes running FreeBSD 7.2. Of the 13 nodes, 1 node is a master mysql server and the 12 nodes will run apache running LAMP-like services. The router will round-robin using hoststated for load-balancing. hoststated? What is that? I think you mean relayd! ;) However, they will serve an additional task: The master mysql server will be head node for MPI jobs delivered to the 12 nodes. Basically, this setup will double up as a beowulf and web server. Is this efficient? I imagine the MPI jobs won't be running all the time and while they're up, might as well do something. I think you are going to be heading for a world of hurt here. I am the HPC director at a university supporting 3 faculties. Once people begin to use the resource the *will* crash nodes. Having any critical services running on HPC compute nodes is *not advisable*. Firstly, would you recommend BSD or Linux for this. The router is a given to have OpenBSD of course, but what about the others? OS doesn't matter! It's all about the tools. We use GNU/Linux (CentOS 5) for our HPC cluster because there are more tools available natively for it. This is an unfortunate fact. More and more applications out there are becoming GNU/Linux specific and just don't work properly or at all on other OSs. Evaluate your tools and make a decision. AFAIK, Open-MPI, MPICH and MPICH2 compile and run fine on the BSDs. Other tools and libs, well, YMMV. I figured it makes sense to parallelize as much as possible so that the HTTP/MPI load can be shared among as many computers as possible. Let me know your thoughts. Unless you have hard memory and CPU provisioning limiting what the cluster nodes can do, alah XEN/VMWare. Forget about it. Trust me. I've rebooted enough deadlocked/crash nodes due to user error to know better. If you have to... well... NO CARRIER...
Calomel.org
There was mention of calomel.org recently. This is a great resource, however, it needs to be a bit more updated. For example the following page advises *not* to use the GENERIC.MP kernel, however, considering how much work has gone into the MP work and fact that MP will become default I think it should be updated. ;) https://calomel.org/network_performance.html --- James A. Peltier james_a_pelt...@yahoo.ca __ Make your browsing faster, safer, and easier with the new Internet Explorer. 8. Optimized for Yahoo! Get it Now for Free! at http://downloads.yahoo.com/ca/internetexplorer/
Re: Problem with slow disk I/O
--- On Thu, 4/23/09, Thomas Pfaff tpf...@tp76.info wrote: From: Thomas Pfaff tpf...@tp76.info Subject: Problem with slow disk I/O To: misc@openbsd.org Received: Thursday, April 23, 2009, 9:27 AM I'm getting horrible disk performance compared to Ubuntu on my system. I noticed this when extracting ports.tar.gz on the same machine with different OSs (this is something I did a while back to check for a possible hardware problem when OpenBSD crashed upon extracting ports.tar.gz). OpenBSD (ffs): $ time tar -zxf ports.tar.gz 0m59.90s real 0m1.00s user 0m6.95s system Ubuntu (ext3): $ time tar -zxf ports.tar.gz real0m18.440s user0m1.212s sys0m2.596s 1 minute on OpenBSD and 18.5 seconds on Ubuntu, doing the exact same thing on the exact same hardware! Why the huge difference? Both are default installations, except softdep is turned on. Thanks for any pointers or advice. Thomas OpenBSD 4.5-current (GENERIC.MP) #13: Thu Apr 23 13:00:36 CEST 2009 tpf...@ws.tp76.info:/usr/src/sys/arch/amd64/compile/GENERIC.MP real mem = 3152609280 (3006MB) avail mem = 3045097472 (2904MB) mainbus0 at root bios0 at mainbus0: SMBIOS rev. 2.4 @ 0xf06b0 (76 entries) bios0: vendor American Megatrends Inc. version 1704 date 11/27/2007 bios0: ASUSTeK Computer INC. P5B-E acpi0 at bios0: rev 2 acpi0: tables DSDT FACP APIC MCFG OEMB HPET acpi0: wakeup devices P0P2(S4) P0P1(S4) UAR1(S4) PS2K(S4) PS2M(S4) EUSB(S4) USBE(S4) P0P4(S4) P0P5(S4) P0P6(S4) P0P7(S4) P0P8(S4) P0P9(S4) USB0(S4) USB1(S4) USB2(S4) USB3(S4) USB4(S4) USB5(S4) acpitimer0 at acpi0: 3579545 Hz, 24 bits acpimadt0 at acpi0 addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz, 2135.29 MHz cpu0: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16, xTPR,NXE,LONG cpu0: 2MB 64b/line 8-way L2 cache cpu0: apic clock running at 266MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Intel(R) Core(TM)2 CPU 6400 @ 2.13GHz, 2135.04 MHz cpu1: FPU,VME,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,PSE36,CFLUS H,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,DS-CPL,VMX,EST,TM2,CX16, xTPR,NXE,LONG cpu1: 2MB 64b/line 8-way L2 cache ioapic0 at mainbus0 apid 2 pa 0xfec0, version 20, 24 pins acpihpet0 at acpi0: 14318179 Hz acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 1 (P0P2) acpiprt2 at acpi0: bus 5 (P0P1) acpiprt3 at acpi0: bus 4 (P0P4) acpiprt4 at acpi0: bus -1 (P0P5) acpiprt5 at acpi0: bus -1 (P0P6) acpiprt6 at acpi0: bus 3 (P0P7) acpiprt7 at acpi0: bus 2 (P0P8) acpicpu0 at acpi0 acpicpu1 at acpi0 acpibtn0 at acpi0: PWRB pci0 at mainbus0 bus 0 pchb0 at pci0 dev 0 function 0 Intel 82G965 Host rev 0x02 ppb0 at pci0 dev 1 function 0 Intel 82G965 PCIE rev 0x02: apic 2 int 16 (irq 11) pci1 at ppb0 bus 1 mem address conflict 0xc000/0x1000 vga1 at pci1 dev 0 function 0 NVIDIA GeForce 7600 GT rev 0xa1 wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) uhci0 at pci0 dev 26 function 0 Intel 82801H USB rev 0x02: apic 2 int 16 (irq 11) uhci1 at pci0 dev 26 function 1 Intel 82801H USB rev 0x02: apic 2 int 17 (irq 5) ehci0 at pci0 dev 26 function 7 Intel 82801H USB rev 0x02: apic 2 int 18 (irq 15) usb0 at ehci0: USB revision 2.0 uhub0 at usb0 Intel EHCI root hub rev 2.00/1.00 addr 1 azalia0 at pci0 dev 27 function 0 Intel 82801H HD Audio rev 0x02: apic 2 int 22 (irq 3) azalia0: codecs: Analog Devices AD1988A audio0 at azalia0 ppb1 at pci0 dev 28 function 0 Intel 82801H PCIE rev 0x02: apic 2 int 16 (irq 11) pci2 at ppb1 bus 4 ppb2 at pci0 dev 28 function 3 Intel 82801H PCIE rev 0x02: apic 2 int 19 (irq 10) pci3 at ppb2 bus 3 age0 at pci3 dev 0 function 0 Attansic Technology L1 rev 0xb0: apic 2 int 19 (irq 10), address 00:18:f3:9d:7d:04 atphy0 at age0 phy 0: F1 10/100/1000 PHY, rev. 5 ppb3 at pci0 dev 28 function 4 Intel 82801H PCIE rev 0x02: apic 2 int 16 (irq 11) pci4 at ppb3 bus 2 jmb0 at pci4 dev 0 function 0 JMicron JMB363 IDE/SATA rev 0x02 ahci0 at jmb0: apic 2 int 16 (irq 11), AHCI 1.0 scsibus0 at ahci0: 32 targets pciide0 at jmb0: DMA, channel 0 wired to native-PCI, channel 1 wired to native-PCI pciide0: using apic 2 int 16 (irq 11) for native-PCI interrupt atapiscsi0 at pciide0 channel 0 drive 0 scsibus1 at atapiscsi0: 2 targets cd0 at scsibus1 targ 0 lun 0: PLEXTOR, DVDR PX-740A, 1.00 ATAPI 5/cdrom removable cd0(pciide0:0:0): using PIO mode 4, Ultra-DMA mode 2 pciide0: channel 1 disabled (no drives) uhci2 at pci0 dev 29 function 0 Intel 82801H USB rev 0x02: apic 2 int 23 (irq 7) uhci3 at pci0 dev 29 function 1 Intel 82801H USB rev 0x02: apic 2 int 19 (irq 10) uhci4 at pci0 dev 29 function 2 Intel 82801H USB rev 0x02: apic 2 int 18 (irq 15) ehci1 at pci0
Re: OpenBSD relayd and public addresses
--- On Tue, 4/21/09, FRLinux frli...@gmail.com wrote:
From: FRLinux frli...@gmail.com
Subject: Re: OpenBSD relayd and public addresses
To: James Peltier james_a_pelt...@yahoo.ca
Cc: misc@openbsd.org
Received: Tuesday, April 21, 2009, 6:20 PM
On Tue, Apr 21, 2009 at 9:32 PM,
James Peltier james_a_pelt...@yahoo.ca
wrote:
I hate to say this but correction to your syntax attached
to your
response would also be a nice addition to the list :)
Steph
Here is the final working configuration
ext_addr=1.2.3.4
#
# Global Options
#
interval 2
timeout 1000
prefork 5
table rthosts { 1.2.3.5 1.2.3.6 }
http protocol http_rt {
header append $REMOTE_ADDR to X-Forwarded-For
header append $SERVER_ADDR:$SERVER_PORT to X-Forwarded-By
# Performance related options
tcp { nodelay, sack, socket buffer 65536, backlog 128 }
}
relay rt {
listen on $ext_addr port 80
protocol http_rt
forward to rthosts port 80 mode loadbalance check http / code 200
}
---
James A. Peltier james_a_pelt...@yahoo.ca
Re: OpenBSD relayd and public addresses
--- On Wed, 4/22/09, James Records james.reco...@gmail.com wrote: From: James Records james.reco...@gmail.com Subject: Re: OpenBSD relayd and public addresses To: FRLinux frli...@gmail.com Cc: James Peltier james_a_pelt...@yahoo.ca, misc@openbsd.org Received: Wednesday, April 22, 2009, 1:25 PM Just curious, does this work when you use the transparent keyword? The server will see the connection as coming from the relayd box in this case correct? Not that it matters but for logging purposes you may want to know. For note: I'm running the Apr 20, 2009 current code and when I issue a relayctl reload it comes back and states that the command fails with this configuration. If I pkill relayd; sleep 2; relayd relayd starts just fine but issues a warning about no redirections nothing to do. Not sure if this is expected behaviour, I suspect not. When I change it to transparent forward blah... it simply won't start and bitches about missing interface.
OpenBSD relayd and public addresses
Hi All,
I'm trying to setup an OpenBSD HTTP load balancer and am failing miserably. I
think this is because I am trying to setup a load balancer that uses public IP
addresses for all the hosts including the load balancer which is not supported.
Is this true? Can I not use public IP addresses with OpenBSD relayd?
I've basically taken the supplied relayd.conf and modified it to use
ext_if=em0
ext_addr=1.2.3.4
webhost1=1.2.3.5
webhost2=1.2.3.6
table webhosts { $webhost1 $webhost2 }
and tried to configure a relay using modified the protocol and relay options
but it didn't work.
http protocol httpbalance {
header append $REMOTE_ADDR to X-Forwarded-For
header append $SERVER_ADDR:$SERVER_PORT to X-Forwarded-By
header change Connection to close
# Various TCP Performance Options
tcp { nodelay, sack, socket buffer 65536, backlog 128 }
}
relay wwwbalance {
listen on $ext_if port 80
protocol httpbalance
# forward to real host in webhosts table
forward to webhosts port http mode loadbalance check http / code 200
}
---
James A. Peltier james_a_pelt...@yahoo.ca
__
Looking for the perfect gift? Give the gift of Flickr!
http://www.flickr.com/gift/
Re: OpenBSD relayd and public addresses
I hate it when I have to reply to my own e-mail. I was able to get it to work
and it was due to syntax. I've now gotten it working and am very excited at
the possibilities.
---
James A. Peltier james_a_pelt...@yahoo.ca
---
On Tue, 4/21/09, James Peltier james_a_pelt...@yahoo.ca wrote:
From:
James Peltier james_a_pelt...@yahoo.ca
Subject: OpenBSD relayd and public
addresses
To: misc@openbsd.org
Received: Tuesday, April 21, 2009, 2:12 PM
Hi All,
I'm trying to setup an OpenBSD HTTP load balancer and am
failing miserably. I think this is because I am trying
to setup a load
balancer that uses public IP addresses for
all the hosts including the load
balancer which is not
supported. Is this true? Can I not use public IP
addresses with OpenBSD relayd?
I've basically taken the supplied
relayd.conf and modified
it to use
ext_if=em0
ext_addr=1.2.3.4
webhost1=1.2.3.5
webhost2=1.2.3.6
table webhosts { $webhost1
$webhost2 }
and tried to configure a relay using modified the protocol
and relay options but it didn't work.
http protocol httpbalance {
header append $REMOTE_ADDR to X-Forwarded-For
header append
$SERVER_ADDR:$SERVER_PORT to
X-Forwarded-By
header change
Connection to close
# Various TCP Performance Options
tcp {
nodelay, sack, socket buffer 65536, backlog
128 }
}
relay wwwbalance
{
listen on $ext_if port 80
protocol httpbalance
# forward to
real host in webhosts table
forward to webhosts port http mode
loadbalance check http / code 200
}
---
James A. Peltier
james_a_pelt...@yahoo.ca
__
Looking
for the perfect gift? Give the gift of Flickr!
http://www.flickr.com/gift/
__
The new
Internet Explorer. 8 - Faster, safer, easier. Optimized for Yahoo! Get it
Now for Free! at http://downloads.yahoo.com/ca/internetexplorer/
Replacing a Cisco 6506 with OpenBSD
Hi All, I'm looking at replacing a Cisco 6506 with an OpenBSD machine serving a university network. The current Cisco setup is basically providing routing and VLAN trunks to our HP ProCurve switches with some basic firewall services. I'd like to look at replacing it with an OpenBSD based solution but I am unsure as to whether OpenBSD is up to the task. Does anyone have any hard evidence that a high quality machine running OpenBSD would be sufficent to replace such a unit? Anything I may want to investigate further prior to pitching this to my manager. He's aware of the benefits to OpenBSD such as the multitude of features available in the stock system, but is a bit worried that it will not be able to keep up. We're only pushing about 50-60M during peak times and are only providing services over a gigabit link between buildings so I think it will be able to keep up. PPS and memory latency are the key issues to tackle I think. Any hints, direction, or yeah, I've done it here.. style cases are greatly appreciated. --- James A. Peltier [EMAIL PROTECTED]
Re: Editing C with...
http://xkcd.com/378/ --- James A. Peltier [EMAIL PROTECTED] http://www.site-fx.net --- On Wed, 5/7/08, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote: From: [EMAIL PROTECTED] [EMAIL PROTECTED] Subject: Re: Editing C with... To: [EMAIL PROTECTED] Cc: misc@openbsd.org Received: Wednesday, May 7, 2008, 11:42 PM --- Matthew Szudzik [EMAIL PROTECTED] wrote: And anyway, I'm a minimalist (that's why I run OpenBSD). nvi is fine--vim and emacs just have too much bloat. Which is why we have mg in tree: emacs without the bloat.
