Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
On Wed, 9 May 2007, Daniel Ouellet wrote: Otto Moerbeek wrote: Where are the OS bottleneck that I can may be improve here? Loks at the memory usage. 300 httpd procces could take up 3000M easily, especially with stuff like php. In that case, the machine starts swapping and your hit the roof. As a general rul, do not allow more httpd procces than our machine can handle without swapping. Also, a long KeepAliveTmeout can works against you, by holding slots. Thanks Otto, I am still doing tests and tweak, but as far as swap, I checked that and same for keep alive in httpd.conf and I even changed it in: net.inet.tcp.keepinittime=10 net.inet.tcp.keepidle=30 net.inet.tcp.keepintvl=30 These parameters do not have a lot to do with what you are seeing. I was talking abouty the KeepAliveTimeout of apache. It's by default 15s. WIth a long timout, any processs that has served a request will wait 15s to see if the client issues more requests on the same connection before it becomes available to serve other requests. For more details, see http://httpd.apache.org/docs/1.3/mod/core.html#keepalivetimeout For testing only. I am not saying the value above are any good, but I am testing multiple things and reading a lot on sysctl and what each one does. KeepAliveTmeout is at 5 seconds. Try lowering it even more. No swapping is happening, even with 1000 httpd running. load averages: 123.63, 39.74, 63.3285 01:26:47 1064 processes:1063 idle, 1 on processor CPU states: 0.8% user, 0.0% nice, 3.1% system, 0.8% interrupt, 95.4% idle Memory: Real: 648M/1293M act/tot Free: 711M Swap: 0K/4096M used/tot
Re: Thecus N2100 and RAID 1
On May 8, 2007, at 6:44 AM, Aaron Poffenberger wrote: Raidframe is really easy to use. The man pages for raidctl(8) will give you step-by-step instructions. In a nutshell, though: 1) enable raidframe in your kernel (search for RAIDframe in GENERIC to get find the line), 2) create the raidn.conf (where n is a number for the array) following the man page -- see the examples section, 3) create the raid -- again, see the examples section in the man page, 4) copy the raidn.conf file to /etc if you want auto configuration during reboots (this part didn't leap out at me from the manpage), 5) enjoy. Thanks for the feedback. Once the N2100 arrives (which should be tomorrow), I will try it. Bryan
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
Here is more tests with always repeated results. I increase the number of contiguous connection only by 5, from 305 to 310, and you get 3 times slower response for always the same thing and repeated all the time. Very consistent and from different clients as well. You can do any variation of 10 to 300 connections and you will always get the same results, or very close to it. See that at the end as well for proof. So, I know I am hitting a hard limit someplace, but can't find where. Note that I use a difference of 5 here, but I can reproduce the results almost all the time, just by increasing the number of connections by 1. From 307 to 308 I get 75% of the time the same results as below, meaning times it;'s 6.7 seconds for the same transfer and other is 18.1 seconds. See below. Always the same transfer size, always the same amount of requests, always 100% success, but 3x slower. Also, if I continue to increase it more, then I start to also get drop in replies, etc. So, far I have played with 26 different sysctl setting that may affect that based on various possibility and from the man page and Google, but I can improve it some, not to the point of be able to use 500 connections or more for example. What is it that really limit the number of connection that badly and that hard? === 305 parallel # http_load -parallel 305 -fetches 500 -timeout 30 /tmp/test 500 fetches, 305 max parallel, 6.549e+06 bytes, in 6.71609 seconds 13098 mean bytes/connection 74.4481 fetches/sec, 975121 bytes/sec msecs/connect: 1813.57 mean, 6007.53 max, 0.418 min msecs/first-response: 509.309 mean, 1685.92 max, 3.606 min HTTP response codes: code 200 -- 500 # http_load -parallel 305 -fetches 500 -timeout 30 /tmp/test 500 fetches, 305 max parallel, 6.549e+06 bytes, in 6.8586 seconds 13098 mean bytes/connection 72.9012 fetches/sec, 954860 bytes/sec msecs/connect: 1957.35 mean, 6007.17 max, 0.445 min msecs/first-response: 485.676 mean, 1559.27 max, 3.317 min HTTP response codes: code 200 -- 500 # http_load -parallel 305 -fetches 500 -timeout 30 /tmp/test 500 fetches, 305 max parallel, 6.549e+06 bytes, in 6.81823 seconds 13098 mean bytes/connection 73.3328 fetches/sec, 960513 bytes/sec msecs/connect: 1825.19 mean, 6007.11 max, 0.484 min msecs/first-response: 508.281 mean, 1646.53 max, 3.422 min HTTP response codes: code 200 -- 500 = 310 parallel # http_load -parallel 310 -fetches 500 -timeout 30 /tmp/test 500 fetches, 310 max parallel, 6.549e+06 bytes, in 18.0998 seconds 13098 mean bytes/connection 27.6245 fetches/sec, 361826 bytes/sec msecs/connect: 2281.39 mean, 18008.3 max, 0.434 min msecs/first-response: 456.326 mean, 1555.78 max, 3.328 min HTTP response codes: code 200 -- 500 # http_load -parallel 310 -fetches 500 -timeout 30 /tmp/test 500 fetches, 310 max parallel, 6.549e+06 bytes, in 18.1142 seconds 13098 mean bytes/connection 27.6027 fetches/sec, 361540 bytes/sec msecs/connect: 2245.47 mean, 18011.4 max, 0.565 min msecs/first-response: 460.068 mean, 1495.42 max, 3.32 min HTTP response codes: code 200 -- 500 # http_load -parallel 310 -fetches 500 -timeout 30 /tmp/test 500 fetches, 310 max parallel, 6.549e+06 bytes, in 18.1635 seconds 13098 mean bytes/connection 27.5278 fetches/sec, 360559 bytes/sec msecs/connect: 2485.7 mean, 18011.9 max, 0.598 min msecs/first-response: 455.163 mean, 1573.78 max, 3.471 min HTTP response codes: code 200 -- 500 # === 10 parallel # http_load -parallel 10 -fetches 500 -timeout 30 /tmp/test 500 fetches, 10 max parallel, 6.549e+06 bytes, in 6.01266 seconds 13098 mean bytes/connection 83.1579 fetches/sec, 1.0892e+06 bytes/sec msecs/connect: 24.6605 mean, 6002.47 max, 0.349 min msecs/first-response: 28.6373 mean, 798.5 max, 3.23 min HTTP response codes: code 200 -- 500 == 20 parallel # http_load -parallel 20 -fetches 500 -timeout 30 /tmp/test 500 fetches, 20 max parallel, 6.549e+06 bytes, in 7.12896 seconds 13098 mean bytes/connection 70.1365 fetches/sec, 918648 bytes/sec msecs/connect: 48.676 mean, 6003.58 max, 0.342 min msecs/first-response: 58.1521 mean, 1249.71 max, 3.216 min HTTP response codes: code 200 -- 500 === 50 parallel # http_load -parallel 50 -fetches 500 -timeout 30 /tmp/test 500 fetches, 50 max parallel, 6.549e+06 bytes, in 8.00917 seconds 13098 mean bytes/connection 62.4285 fetches/sec, 817688 bytes/sec msecs/connect: 84.686 mean, 6003.49 max, 0.418 min msecs/first-response: 174.045 mean, 1950.98 max, 3.349 min HTTP response codes: code 200 -- 500 100 parallel # http_load -parallel 100 -fetches 500 -timeout 30 /tmp/test 500 fetches, 100 max parallel, 6.549e+06 bytes, in 7.90241 seconds 13098 mean bytes/connection 63.2718 fetches/sec, 828735 bytes/sec msecs/connect: 72.8683 mean, 6003.78 max, 0.417 min msecs/first-response: 379.736 mean, 1964.26 max, 3.366 min HTTP response codes: code 200 -- 500
Re: Softupdates question
Still curious how they would work on, say, /var/mysql or /var/postgresql, but I can play with this on my own. Has anyone already tried? Care to comment? FWIW I run softdep on ALL partitions except / and /var and have for many years. I exclude /var because on a crash I want the best chance items logged in /var/log/ to show up. // marc
Re: [OT] language tricks (was: creating menu's)
On Tue, May 08, 2007 at 09:34:35PM -0400, Douglas Allan Tutty wrote:
On Tue, May 08, 2007 at 01:22:10PM -0700, Bryan Irvine wrote:
I need a fairly simple menu, and have thought about just simple
selects but figured now would also be a good time to learn something
new as well. It's nothing so complex that I need to go ncurses to do.
Just a basic option 1 then option 3 then run some command
thing.
My front-ends I do in python. It doesn't have a case/select. I just
use if/then/elif/
Then there's Fortran with computed gotos; very slick. I forget the
syntax but is something like goto (10+choice)
11 ch1()
...
12 ch2()
...
13 ch3()
...
It means that only one computation takes place instead of one comparison
for each choice until one matches.
Just pointing out: if Python can do the job at all, you almost certainly
don't need that kind of micro-optimization in Fortran code. Also, this
is a menu. Efficiency is not exactly a big goal.
However, and this is where I go completely off-topic, while we're at it,
you don't need Fortran for this, most languages have equivalent
constructs (C):
switch(option) {
case 1:
...
case 2:
...
case 3:
...
default:
/* error! */
...
}
or even
void (*dispatch[])(void) = {
proc_opt1,
proc_opt2,
proc_opt3
}
void
proc_opt1(void)
{
...
}
void
proc_opt2(void)
{
...
}
void
proc_opt3(void)
{
...
}
In languages with higher order-functions, this can be written even more
concisely (Scheme):
(define dispatch
(vector
(lambda () ...)
(lambda () ...)
(lambda () ...)))
A suiteable make-menu macro could even make something like
(define toplevel-menu
(make-menu
(opt1 (lambda () ...))
(opt2 (lambda () ...))
(another menu another-menu)))
(define another-menu
(make-menu
(opt3 (lambda () ...))
(opt4 (lambda () ...))
(top toplevel-menu)))
do what it looks like it should do.
However, all of this is massively overkill. Just use a shell script.
Joachim
--
TFMotD: mirroring-ports (7) - how to build a mirror for ports distfiles
Re: VNC server on OpenBSD (error allocating memory)
On 2007/05/08 16:17, Daniel Bolgheroni wrote: Applied the patch succesfully against 4.1 net/tightvnc, but getting the following error: Sorry, I missed -P when I generated the diff. It's fixed now. http://spacehopper.org/openbsd/tightvnc-1.3.8-update.txt I suggest any more discussion on this is moved to ports@
Re: Softupdates question
mickey wrote: On Tue, May 08, 2007 at 07:06:06AM -0400, Nick Holland wrote: George C wrote: ... Is it always best to mount /, /tmp, /usr, /var, /home with softdep? Under what curcumstances would it not be appropriate? If your app makes assumptions about write ordering, softdeps can negate the care the app author took. For example, some mail programs don't ack the receipt of a message until it has been safely written to disk, the idea being that if the power goes out or the machine crashes, if the message has been acknowledged, IT HAS BEEN RECEIVED and will be there when the machine comes back up. Softdeps promises that what is on your disk is coherent, but coherent usually means the last few files written to disk may be just removed when the system comes back up. Not desired in this case. this is not true. fsync() works as specified. Apparently, not all apps use fsync, or don't use it properly. At least qmail advises against the use of softdeps: http://cr.yp.to/qmail/faq/reliability.html#filesystems I also found a reference to another mail program which had people making similar advisories, but not sure if they are still applicable. Softdeps don't do anything for you if you are mostly reading from disk, or if the partition is mounted read-only. It's about writing. of course they do. there are still atime updates for example that will be handled if not mount read-only. yeah, no idea why I phrased it in such absolute terms. duh. Softdeps is much more complex than conventional disk access. While I have not personally seen a softdep-related bug in some time, and that one was quickly fixed, you HAVE to assume it is more likely to have bugs than the non-softdep systems. this is also not exactly true -- there are softdep bugs fixed at the rate of ten per year if not more. most of them are bugs that been there forever. I (apparently) phrased this poorly, having seen at least two unintended interpretations... I have only *experienced* one softdep bug in many years of using it on virtually all partitions of virtually all systems I have installed. After providing the PS and TRACE, I think Pedro had me a patch within an hour. :) Yes, certainly, bugs have been spotted, and there are most likely other bugs that remain. Some people have apps which expose bugs better than mine... If it was not obvious from my comments, I love softdeps. I have a siteXX.tgz file which does a few simple things, one of which is to change all mount points to use softdeps. One really does have to hunt a bit for relevant reasons not to use it. About the only place I can think of where I deliberately don't use it is on an e-mail archive system on the filled partitions which are mounted read-only. I can't tell you how many times I have forgot to install my siteXX file, started loading up /usr/src, and realized, Dang, obviously no softdeps. At which point, I stop the checkout, fix the problem, reboot, and try again. Yes, the performance difference is that obvious, and it is faster to reboot than it is to wait it out. Nick.
postfix-2.2.8 with sasl2 and ldap support
Hi, How can i install postfix-2.2.8 with ldap and sasl2 support? I've searched the web but couldnt find any usefull information. Can you please guide me? thanks atno
Re: Sun Netra and DAS
Kevin wrote: Hello all, I'm about out of space on a Sun Netra T1 that has been happily running OpenBSD for some time. I'd rather keep this server in action and add space to it, but both internal drive slots are occupied, so that means the only choice (short of reloading on bigger disks, which for a variety of reasons I'd rather avoid) is adding external storage. It seems like the logical choice would be a Direct Attached Storage box like a D1000 plugged into the external SCSI port or a PCI RAID card. So: 1.) Is the D1000 supported in 4.1 when attached to a Netra T1 either via the external SCSI or via a RAID card? (http://www.openbsd.org/sparc64.html#hardware doesn't mention it either way) 2.) Given the various supported RAID cards, is a more generic RAID enclosure attached to a 3rd party RAID card a better way to go? 3.) Are there better alternatives that I'm just overlooking? As always, many thanks. Kevin Kevin, I am not sure about the OpenBSD support but, you cannot use the onboard SCSI port to connect to a D1000. You need an HVD/Differential SCSI card. These are easy to find, but needed. James
Re: postfix-2.2.8 with sasl2 and ldap support
1. cd /usr/ports/mail/postfix/snapshot 2. export FLAVOR=sasl2 ldap 3. make install clean 4. ? 5. PROFIT -Mensagem original- De: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Em nome de Antonis Faragitakis Enviada: quarta-feira, 9 de Maio de 2007 12:20 Para: misc@openbsd.org Assunto: postfix-2.2.8 with sasl2 and ldap support Hi, How can i install postfix-2.2.8 with ldap and sasl2 support? I've searched the web but couldnt find any usefull information. Can you please guide me? thanks atno
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
On 5/9/07, Daniel Ouellet [EMAIL PROTECTED] wrote: I increase the number of contiguous connection only by 5, from 305 to 310, and you get 3 times slower response for always the same thing and repeated all the time. Very consistent and from different clients as well. You can do any variation of 10 to 300 connections and you will always get the same results, or very close to it. See that at the end as well for proof. So, I know I am hitting a hard limit someplace, but can't find where. You've assumed that Apache is the bottleneck, but perhaps your benchmark tool could be limited in some way. I suggest you try with apache benchmark or some other tool just to verify the results. Apache (especially in the prefork model) is known to have concurrency issues. I doubt that there are knobs you can twist OpenBSD-wise that will compensate for Apache and somehow magically make it scale.
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
Srebrenko Sehic wrote: On 5/9/07, Daniel Ouellet [EMAIL PROTECTED] wrote: I increase the number of contiguous connection only by 5, from 305 to 310, and you get 3 times slower response for always the same thing and repeated all the time. Very consistent and from different clients as well. You can do any variation of 10 to 300 connections and you will always get the same results, or very close to it. See that at the end as well for proof. So, I know I am hitting a hard limit someplace, but can't find where. You've assumed that Apache is the bottleneck, but perhaps your benchmark tool could be limited in some way. I suggest you try with apache benchmark or some other tool just to verify the results. Apache (especially in the prefork model) is known to have concurrency issues. I doubt that there are knobs you can twist OpenBSD-wise that will compensate for Apache and somehow magically make it scale. Actually I have found a few things that fix it tonight. I spend the last 24 hours reading like crazy and all night testing and reading more. I can now have two clients using 1000 parallel connections to one i386 850MHz server, my old one that I was testing with and I get all that no problem now. No delay and I can even push it more, but I figure at 2000 parallel connections I should be able to get some breathing time now. I will send the results soon. All only in sysctl.conf Now, I am still having some drop, not much, but some when I put pf in actions. So, that would be the next step I guess, but not now. I need some sleep. Thanks Daniel
Re: postfix-2.2.8 with sasl2 and ldap support
thank you Pedro hope it'll work :) atn On 09/05/07, Pedro de Oliveira [EMAIL PROTECTED] wrote: 1. cd /usr/ports/mail/postfix/snapshot 2. export FLAVOR=sasl2 ldap 3. make install clean 4. ? 5. PROFIT -Mensagem original- De: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] Em nome de Antonis Faragitakis Enviada: quarta-feira, 9 de Maio de 2007 12:20 Para: misc@openbsd.org Assunto: postfix-2.2.8 with sasl2 and ldap support Hi, How can i install postfix-2.2.8 with ldap and sasl2 support? I've searched the web but couldnt find any usefull information. Can you please guide me? thanks atno
Wireless NIC questions
Greetings I was wondering if anyone here had any experience setting up a wireless access point. I am running OpenBSD 4.0 with Z-COM WLAN PC Card but can not bring up the card in access point mode. Bret
Performance: OpenVPN vs IPsec
Hello,
I've got two networks connected with OpenVPN right now, the setup is
like this.
{Network_A}-{OpenVPN_Server}--{Network_B}
NetworkA is a real network where the router (with dynamic IP) is
connected directly to a dedicated OpenVPN server with a static IP.
NetworkB is just a single host within another network which is
connected to the OpenVPN server to be able to directly access NetworkA
over the central OpenVPN server.
Now, as I understand it, it isn't possible to create an IPsec connection
from a single host within a NATed network to an external server but
OpenVPN works great here. Please correct me if I am wrong. (I have no
access to the NAT router here.)
Even though the NetworkA router just got a dynamic IP it would still be
possible to set up the VPN with IPsec. At the moment I use OpenVPN here
but I consider the pros/cons about switching to IPsec at the moment. One
important part would be the overall performance.
The NetworkA router is a Soekris net4801 with vpn1411. Both NetworkA
router, the host in NetworkB and the central server run OpenBSD 4.x-stable.
I now did some speed testing. Both OpenVPN and IPsec use keys of the
same size.
When using the OpenVPN connection I can download a file from the central
server using scp with approx 200kB/s to the Soekris memory file system,
getting around or more than 1000 interrupts on the vpn1411 card when
examining it with systat vmstat.
When using the IPsec connection I can download the same file at around
the same speed but am only getting around 300 interrupts so it seems to
me the overall performance should be better because the system is
stressed a lot less.
When downloading the file directly to the Soekris mfs without any VPN I
get something like =400kB/s.
I have no clue about the VPN traffic overhead differences between
OpenVPN and IPsec but I would guess that IPsec would be faster/less
ressource consumning/more performant since it is a protocol extension
and is not running in userspace.
Anyone got more experience on this or got an explanation why there is no
visible gain (ie. transfer speed), except the lesser system and memory
usage which is already nice enough, when using IPsec.
Michael
Re: Softupdates question
On Wed, May 09, 2007 at 06:46:19AM -0400, Nick Holland wrote: mickey wrote: On Tue, May 08, 2007 at 07:06:06AM -0400, Nick Holland wrote: George C wrote: ... Is it always best to mount /, /tmp, /usr, /var, /home with softdep? Under what curcumstances would it not be appropriate? If your app makes assumptions about write ordering, softdeps can negate the care the app author took. For example, some mail programs don't ack the receipt of a message until it has been safely written to disk, the idea being that if the power goes out or the machine crashes, if the message has been acknowledged, IT HAS BEEN RECEIVED and will be there when the machine comes back up. Softdeps promises that what is on your disk is coherent, but coherent usually means the last few files written to disk may be just removed when the system comes back up. Not desired in this case. this is not true. fsync() works as specified. Apparently, not all apps use fsync, or don't use it properly. oh so now you are saying that softdeps are broken because applications are not calling fsync() ? At least qmail advises against the use of softdeps: http://cr.yp.to/qmail/faq/reliability.html#filesystems I also found a reference to another mail program which had people making similar advisories, but not sure if they are still applicable. you whole above statement is wrong and is not based on facts. now you are trying to back it up w/ somebody elses opinion that is also not based on facts. now it is also in the archives and peoples will refer to it as some sort of truth. the damage has been done. cu -- paranoic mickey (my employers have changed but, the name has remained)
Re: creating menu's
Bryan Irvine sparctacus at gmail.com writes: It's been years (just shy of a decade IIRC) since the last time I needed to create a menu-shell type of thing. But now I need to. I'm wondering what people are using these days. Is there something neat in ports I should be trying out? /usr/ports/misc/mshell
Re: Wireless NIC questions
Bret [EMAIL PROTECTED] writes: I was wondering if anyone here had any experience setting up a wireless access point. I am running OpenBSD 4.0 with Z-COM WLAN PC Card but can not bring up the card in access point mode. Unfortunately not all wireless cards support Host AP mode (that's what you want to look for in the man page). I've had good experience with ath, ral and rum cards myself. -- Peter N. M. Hansteen, member of the first RFC 1149 implementation team http://www.blug.linux.no/rfc1149/ http://www.datadok.no/ http://www.nuug.no/ First, we kill all the spammers The Usenet Bard, Twice-forwarded tales delilah spamd[29949]: 85.152.224.147: disconnected after 42673 seconds.
Re: Softupdates question
* mickey [EMAIL PROTECTED] [2007-05-09 15:15]: On Wed, May 09, 2007 at 06:46:19AM -0400, Nick Holland wrote: mickey wrote: On Tue, May 08, 2007 at 07:06:06AM -0400, Nick Holland wrote: George C wrote: ... Is it always best to mount /, /tmp, /usr, /var, /home with softdep? Under what curcumstances would it not be appropriate? If your app makes assumptions about write ordering, softdeps can negate the care the app author took. For example, some mail programs don't ack the receipt of a message until it has been safely written to disk, the idea being that if the power goes out or the machine crashes, if the message has been acknowledged, IT HAS BEEN RECEIVED and will be there when the machine comes back up. Softdeps promises that what is on your disk is coherent, but coherent usually means the last few files written to disk may be just removed when the system comes back up. Not desired in this case. this is not true. fsync() works as specified. Apparently, not all apps use fsync, or don't use it properly. oh so now you are saying that softdeps are broken because applications are not calling fsync() ? Nick never said softdeps were broken. he said that using them with certain applications is not a good idea - that is different. The application is to blame tho. At least qmail advises against the use of softdeps: http://cr.yp.to/qmail/faq/reliability.html#filesystems I also found a reference to another mail program which had people making similar advisories, but not sure if they are still applicable. you whole above statement is wrong and is not based on facts. now you are trying to back it up w/ somebody elses opinion that is also not based on facts. now it is also in the archives and peoples will refer to it as some sort of truth. the damage has been done. the softdeps are incompatible with qmail truth is as old as softdeps, the damage is long done. -- Henning Brauer, [EMAIL PROTECTED], [EMAIL PROTECTED] BS Web Services, http://bsws.de Full-Service ISP - Secure Hosting, Mail and DNS Services Dedicated Servers, Rootservers, Application Hosting - Hamburg Amsterdam
Re: OT: GUI programming languages
Jacob Yocom-Piatt jy-p at fixedpointgroup.com writes: have been coding touchscreen-driven applications using visual basic lately and am sick of VB. i would much rather be using openbsd with another programming language that allows me to accomplish the same sort of stuff. glade (from ports), perl (in base) plus the perl Gtk bindings (from ports). The GTK bindings now include support for Glade Designer files. Glade should be easy enough to work with as a (former) VB programmer. Add the EPIC plugin for Eclipse (also in ports) to help with the actual coding and you're set. If you really need compiled language support, C/C++ (with the Gtk+ port) can be used to write Glade based apps instead.
Binary upgrade of mozilla-thunderbird fails on OpenBSD 4.1
Suspected line reads: Checking for collisions with .libs-mozilla-thunderbird-1.5.0.10... some found Could anyone explain what to do next ? Thanks! Tired of spam? Yahoo! Mail has the best spam protection around http://mail.yahoo.com # pkg_add -uivvv mozilla-thunderbird Candidates for updating mozilla-thunderbird-1.5.0.9p1 - mozilla-thunderbird-1.5.0.9p1 mozilla-thunderbird-1.5.0.10 Ambiguous: choose package for mozilla-thunderbird-1.5.0.9p1 0: None 1: mozilla-thunderbird-1.5.0.10 2: mozilla-thunderbird-1.5.0.9p1 Your choice: 1 No need to update jpeg-6bp3 No need to update hicolor-icon-theme-0.9 No need to update glib2-2.10.3p0 No need to update png-1.2.14p0 No need to update cairo-1.2.6p0 No need to update expat-2.0.0 No need to update gettext-0.14.6 No need to update nspr-4.6.5p0 No need to update tiff-3.8.2p0 No need to update libiconv-1.9.2p3 No need to update libaudiofile-0.2.6p0 No need to update esound-0.2.34p0 No need to update glitz-0.5.6 No need to update atk-1.10.3p2 No need to update gtk+2-2.8.20p4 No need to update pango-1.12.3p0 Running the equivalent of pkg_add -r mozilla-thunderbird-1.5.0.10 parsing mozilla-thunderbird-1.5.0.10 New package mozilla-thunderbird-1.5.0.10 contains potentially unsafe operations @exec rm -rf /tmp/.mozilla @exec cd /usr/local/mozilla-thunderbird env HOME=/tmp LD_LIBRARY_PATH=/usr/local/mozilla-thunderbird ./regxpcom @exec rm -rf /tmp/.mozilla proceed with update anyways? [y/N/a] y Checking for collisions with .libs-mozilla-thunderbird-1.5p2... none found Checking for collisions with .libs-mozilla-thunderbird-1.5.0.8... none found Checking for collisions with .libs-mozilla-thunderbird-1.5.0.10... some found Checking for collisions with .libs-mozilla-thunderbird-1.5.0.7... none found Checking for collisions with .libs-mozilla-thunderbird-1.5.0.2... none found Can't update to mozilla-thunderbird-1.5.0.10 because of collision with old libs /usr/sbin/pkg_add: mozilla-thunderbird-1.5.0.10:Fatal error
Re: Wireless NIC questions
Hello Bret, Bret schrieb: I was wondering if anyone here had any experience setting up a wireless access point. I am running OpenBSD 4.0 with Z-COM WLAN PC Card but can not bring up the card in access point mode. I recently set up an access point using OpenBSD 4.0 and now upgraded it to 4.1. The ath card (CM9) I had just works in 11b mode as hostap with OpenBSD though but the other ral card I got works great with 11g. I would really suggest getting a Ralink card, they just work. You might want to check out kd85.com if you are from Europe. ral_abg is the one I got. Michael
Re: Softupdates question
I had always assumed the use of softupdates was safe as long as you could have reasonable assurances that the machine would not be shutdown without warning. (i.e. no loss of power or reset being hit). So if you had a UPS, good hardware, and no vandals it's good to use.
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
On Wed, May 09, 2007 at 01:30:41AM -0400, Daniel Ouellet wrote: No swapping is happening, even with 1000 httpd running. load averages: 123.63, 39.74, 63.3285 01:26:47 1064 processes:1063 idle, 1 on processor CPU states: 0.8% user, 0.0% nice, 3.1% system, 0.8% interrupt, 95.4% idle Memory: Real: 648M/1293M act/tot Free: 711M Swap: 0K/4096M used/tot How does this server do with 1000 non-httpd processes running? Perhaps I need a newer Nemeth et al, but in my 3rd edition, pg 759 middle of the page says Modern systems do not deal welll with load averages over about 6.0. Could your bottleneck be in context-switching between so many processes? With so many, the memory cache will be faulting during the context switching and have to be retreived from main memory. I don't think that such slow-downs appear in top, and I don't know about vmstat. I don't know if there's a tool to measure this on i386. I've never run httpd but it looks to me like a massivly parralized problem where each connection is trivial to serve (hense low CPU usage, no disk-io waiting) but there are just so many of them. How does the server do with other connection services, e.g. pop or ftp? Doug.
Re: Sun Netra and DAS
On 5/8/07, Kevin [EMAIL PROTECTED] wrote: Hello all, I'm about out of space on a Sun Netra T1 that has been happily running OpenBSD for some time. I'd rather keep this server in action and add space to it, but both internal drive slots are occupied, so that means the only choice (short of reloading on bigger disks, which for a variety of reasons I'd rather avoid) is adding external storage. It seems like the logical choice would be a Direct Attached Storage box like a D1000 plugged into the external SCSI port or a PCI RAID card. So: ... 3.) Are there better alternatives that I'm just overlooking? A StorEdge S1 would be a nice alternative, only 1U and will work off the external SCSI port on your Netra T1. Paul. -- Paul D. Ouderkirk Senior UNIX System Administrator JadedPixel Technologies [EMAIL PROTECTED] -- laughing, in the mechanism -- William Gibson
Re: Redirected packet from pf is lost
Andreas Hdber wrote:
Hi all,
I've got a Dell SC1435, running OpenBSD 4.0, with two Ethernet interfaces
(bge0 and bge1) working as a gateway and firewall for our internal network.
bge0 is the external connection (with a class B IPv4 address), and bge1 is
the internal connection (private IP network, class C). They are both part of
a bridge, bridge0:
# cat /etc/bridgename.bridge0
add bge0
add bge1
blocknonip bge0
blocknonip bge1
up
#
Our pf-config has worked fine for normal Internet access, so internal
computers can access external hosts fine (through NAT).
However, now we need to redirect packets from an external host
(external.sip.proxy.example below, using a normal class B IPv4 address) to
one of our internal hosts (internal.sip.proxy.test below, which is part of
the same private network as bge1 on our gateway). This is the first rdr rule
below. I've also used rdr pass instead of the explicit pass as shown
below, obviously with no success.
The pf-config looks like this (rules related to IPSec, SSH-access are
removed):
ext_if=bge0 # External interface
int_if=bge1 # Internal interface
set block-policy return
set loginterface $ext_if
set skip on { lo enc0 }
scrub in
rdr on $ext_if proto udp from external.sip.proxy.example port sip to any
port 6060 \
tag VoIP - internal.sip.proxy.test port 6060
nat on $ext_if from !($ext_if) to any - ($ext_if)
nat-anchor ftp-proxy/*
rdr-anchor ftp-proxy/*
rdr on $int_if proto tcp from any to any port ftp - 127.0.0.1 port 8021
block in log all
pass out keep state
anchor ftp-proxy/*
antispoof quick for { lo enc0 $int_if }
# Does NOT work (see tag on rdr-rule above)
pass in log tagged VoIP
# Does work, according to pflog. Tag is nowhere to be seen, though.
pass in log on {$ext_if $int_if} proto udp from external.sip.proxy.example
port sip to internal.sip.proxy.test port 6060 tag VoIP2 keep state
pass quick on { $int_if, enc0 }
# -- end pf.conf --
As you can see above, I'm logging blocked packets and also the relevant
packets passed in. I've found these two packets in pflog0 related to this.
The first one is a SIP request sent out from internal.sip.proxy.test to
external.sip.proxy.example:
Frame 205258 (1458 bytes on wire, 1458 bytes captured)
Arrival Time: May 8, 2007 16:58:45.715379000
[Time delta from previous packet: 679.119839000 seconds]
[Time since reference or first frame: 8590.343581000 seconds]
Frame Number: 205258
Packet Length: 1458 bytes
Capture Length: 1458 bytes
[Frame is marked: True]
[Protocols in frame: pflog:ip:udp:sip:sdp]
PF Log IPv4 passed on bge1 by rule 46
Header Length: 61
Address Family: IPv4 (2)
Action: passed (0)
Reason: match (0)
Interface: bge1
Ruleset:
Rule Number: 46
Sub Rule Number: -1
Direction: Unknown (255)
Internet Protocol, Src: internal.sip.proxy.test (192.168.1.7), Dst:
external.sip.proxy.example (external.sip.proxy.example)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x10 (DSCP 0x04: Unknown DSCP; ECN: 0x00)
0001 00.. = Differentiated Services Codepoint: Unknown (0x04)
..0. = ECN-Capable Transport (ECT): 0
...0 = ECN-CE: 0
Total Length: 1394
Identification: 0x (0)
Flags: 0x04 (Don't Fragment)
0... = Reserved bit: Not set
.1.. = Don't fragment: Set
..0. = More fragments: Not set
Fragment offset: 0
Time to live: 64
Protocol: UDP (0x11)
Header checksum: 0x622c [correct]
[Good: True]
[Bad : False]
Source: internal.sip.proxy.test (192.168.1.7)
Destination: external.sip.proxy.example (external.sip.proxy.example)
User Datagram Protocol, Src Port: 6060 (6060), Dst Port: 5060 (5060)
Source port: 6060 (6060)
Destination port: 5060 (5060)
Length: 1374
Checksum: 0x1eac [correct]
Session Initiation Protocol
Request-Line: INVITE sip:[EMAIL PROTECTED] SIP/2.0
Method: INVITE
[Resent Packet: False]
[Snipped away rest of the SIP-content!]
The external.sip.proxy.example sends the following response back
Frame 205259 (805 bytes on wire, 805 bytes captured)
Arrival Time: May 8, 2007 16:58:45.716547000
[Time delta from previous packet: 0.001168000 seconds]
[Time since reference or first frame: 8590.344749000 seconds]
Frame Number: 205259
Packet Length: 805 bytes
Capture Length: 805 bytes
[Frame is marked: True]
[Protocols in frame: pflog:ip:udp:sip]
PF Log IPv4 passed on bge0 by rule 14
Header Length: 61
Address Family: IPv4 (2)
Action: passed (0)
Reason: match (0)
Interface: bge0
Ruleset:
Rule Number: 14
Sub Rule Number: -1
Direction: Unknown (255)
Internet Protocol, Src: external.sip.proxy.example
(external.sip.proxy.example), Dst: internal.sip.proxy.test (192.168.1.7)
Version: 4
Header length: 20 bytes
Differentiated Services Field: 0x10 (DSCP 0x04:
Re: Performance: OpenVPN vs IPsec
Michael wrote:
Hello,
I've got two networks connected with OpenVPN right now, the setup is
like this.
{Network_A}-{OpenVPN_Server}--{Network_B}
NetworkA is a real network where the router (with dynamic IP) is
connected directly to a dedicated OpenVPN server with a static IP.
NetworkB is just a single host within another network which is
connected to the OpenVPN server to be able to directly access NetworkA
over the central OpenVPN server.
Now, as I understand it, it isn't possible to create an IPsec connection
from a single host within a NATed network to an external server but
OpenVPN works great here. Please correct me if I am wrong. (I have no
access to the NAT router here.)
[snip]
Hi,
From MY experience it is possible to use an IPSEC VPN through NAT, with
some conditions!!
1. There can only be 1 IPSEC connection through the NAT router UNLESS
the router supports NAT-T.
2. The IPSEC connection cannot be doing AH, only ESP. If you do not
understand this statement, man(4) ipsec will be our friend.
Someone else may correct me, but these are my empirical findings and my
understanding from doing LOTS of reading. I'm very much a beginner at
this stuff though.
The rest I have no idea about.
Good Luck,
Steve Williams
Re: Redirected packet from pf is lost
On Wed, May 09, 2007 at 09:08:58AM -0600, Steve Williams wrote: Check out a (very) recent thread initiated by myself with the subject rdr on bridge interface possible? (squid transparent proxy on bridge). There are a few suggestions there, none of which have worked for me. I have no idea why it's not working for me. Let me know if you get it working! Steve, I only posted a single rule before. Here are all the relevant parts... ext_if=de0 # this if has an IP address rdr on $ext_if inet proto tcp from spamd to port smtp \ - 127.0.0.1 port spamd pass in on $ext_if route-to lo0 inet proto tcp to 127.0.0.1 port spamd Note that the pass/route-to rule targets the *destination* of the rdr... -- Darrin Chandler| Phoenix BSD User Group | MetaBUG [EMAIL PROTECTED] | http://phxbug.org/ | http://metabug.org/ http://www.stilyagin.com/ | Daemons in the Desert | Global BUG Federation
OpenBSD roadtrip: Ede Netherlands 20070510, Krakow Poland 20070512-13
Hey, I'm happy to announce that all (wel, most) OpenBSD 4.1 orders have been shipped out and that we are back on the road attending conferences. For those in Netherlands, tonight (after 22h :-) and tomorrow, we'll be in Ede, at the NLUUG Voorjaarsconferentie 2007 http://www.nluug.nl/events/vj07/index.html This weekend, Felix, Henning and I will be in Krakow, Poland for Confidence 2007 http://2007.confidence.org.pl/ I've been told Jacek is alive and will be giving us info about his missing books this weekend, so I hope to have that all cleared about by next week. Feel free to drop by and say hello Wim. -- =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-= https://kd85.com/notforsale.html --
Re: [OT] language tricks (was: creating menu's)
On Wed, May 09, 2007 at 10:56:57AM +0200, Joachim Schipper wrote: On Tue, May 08, 2007 at 09:34:35PM -0400, Douglas Allan Tutty wrote: On Tue, May 08, 2007 at 01:22:10PM -0700, Bryan Irvine wrote: I need a fairly simple menu, and have thought about just simple selects but figured now would also be a good time to learn something new as well. It's nothing so complex that I need to go ncurses to do. Just a basic option 1 then option 3 then run some command thing. My front-ends I do in python. It doesn't have a case/select. I just use if/then/elif/ Then there's Fortran with computed gotos; very slick. I forget the syntax but is something like goto (10+choice) for each choice until one matches. Just pointing out: if Python can do the job at all, you almost certainly don't need that kind of micro-optimization in Fortran code. Also, this is a menu. Efficiency is not exactly a big goal. I don't do enough programming to want to keep track of multiple languages. If I have to read a program in 10 years I want to know what its trying to do. C has too much punctuation everywhere. So I only program in Python and Fortran. However, and this is where I go completely off-topic, while we're at it, you don't need Fortran for this, most languages have equivalent constructs (C): In languages with higher order-functions, this can be written even more concisely (Scheme): However, all of this is massively overkill. Just use a shell script. Shell is too much like C (punctuation and spacing matter). (sorry if this sounds anti-unix). I use shell if its like a dos bat file, sequential. Once I have to test conditions and branch I switch to python. Then if something takes a long time (or I know it will before hand), I use fortran 77. Unfortunaly, I can't get my head around regex either. Two hours after I'v written it I can't understand it. So I code it in python or fortran. Doug.
mpi(4): any way to view WWN?
I have a remotely located system with an mpi(4) HBA (LSI FC929X) where it would be useful to display the WWN. Is there currently a way to retrieve this that's more convenient than watching for BIOS messages at boot?
Re: OpenBSD 4.1 install issue??
I had the same problem installing OpenBSD 4.1 on an Intel D945GCcr motherboard and the snapshot worked just fine! But i noticed that it is not possible to install gd package due to lack of libfontconfig.3.0 on xbase41.tgz of the snapshot. But the libfontconfig.3.0 is on the xbase41.tgz of the release. So i installed the xbase41.tgz of the release over the snapshot installation, and i could install gd smoothly. Now i just have to move on to stable and i'm ready to go! Thanks ! Marcos Laufer - Original Message - From: Rob Waite [EMAIL PROTECTED] To: misc@openbsd.org Sent: Wednesday, May 02, 2007 9:52 AM Subject: Re: OpenBSD 4.1 install issue?? Yep.. the snapshot worked... I did not get a chance to try 3.9... I spent all last night making a new release... I was pretty sure that I only needed cd41.iso and the kernel but I went ahead and did the whole thing anyway. So now it is up and running and everything seems fine. Thanks for your help! Rob - Original Message - From: Tom Cosgrove [EMAIL PROTECTED] To: Rob Waite [EMAIL PROTECTED] Sent: Tuesday, May 01, 2007 11:58 AM Subject: Re: OpenBSD 4.1 install issue?? Rob Can you try a snapshot? VM_PHYSSEG_MAX was upped from 5 to 16 at the end of March; this should help. (This change was too late for 4.1 though.) I was also serious in asking you to try 3.9: I think you will find that this does not work either. In other words, 4.0 is the only recent release that will have worked for you - there were changes in 4.0 which were reverted (undone) for 4.1 because they did not work for everyone. However, they did include a higher value for VM_PHYSSEG_MAX. Thanks Tom Rob Waite 1-May-07 14:43 Yeah I am sorry about the first hand written dmesg with the blah blah in it. Below are two links to images of my screen.. some intermediate messages were lost because they scrolled by too fast. To recap... it works on i386 4.0 (the clock_battery message shows up there too) it works on amd64 4.1 (clock_message also) and the screenshots for i386 4.1 are below. There is a message in the secondScreen image where fxp0 has a fault. If I disable the intel ethernet (it is onboard the motherboard) I get an error at about the same place about the USB having a fault. If I disable the USB... another item has a fault and so on (sorry.. I don't remember the item... if someone thinks that is an important clue I will try again and let you know). It seems like the initial avm_page_physload is a big part of this issue. It seems so strange how the other releases will work. Okay... so here are the links to the screenshots and the dmesg for amd64 was in an earlier post. Thanks again everyone. http://i175.photobucket.com/albums/w132/winstonwaite/firstScreen.jpg http://i175.photobucket.com/albums/w132/winstonwaite/secondScreen.jpg
Re: Softupdates question
Well, which would you prefer, Peter? I've had systems that have had their power yanked from them several times now, and I've yet to have seen a screwed filesystem. Yes, files created or deleted with 30(?) seconds of the outage might be inconsisten or whatever, I'll take that any day over a damaged filesystem. I think there are bugs in the softdep code. I know of one really busy system that has crashed because of softdeps being on, but only one and I've never been able to pin it down. I would say it works well and gets better with each release. --STeve Andre' On Wednesday 09 May 2007 12:03:40 Peter Fraser wrote: I did read the papers. There is a difference between the file system being screwed and data lost. Softupdates hopefully stops the files system from being in a bad state, but it is amazing how much user data can be lost on a power failure while using softupdates. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of mickey Sent: Wednesday, May 09, 2007 11:49 AM To: Peter Fraser Cc: misc@openbsd.org Subject: Re: Softupdates question On Wed, May 09, 2007 at 10:45:15AM -0400, Peter Fraser wrote: I had always assumed the use of softupdates was safe as long as you could have reasonable assurances that the machine would not be shutdown without warning. (i.e. no loss of power or reset being hit). So if you had a UPS, good hardware, and no vandals it's good to use. actually if you bother to read the papers whole idea behind softdeps is to ensure better recoverability from crashes/power/etc. cu -- paranoic mickey (my employers have changed but, the name has remained)
Re: Performance: OpenVPN vs IPsec
On Wed, May 09, 2007 at 02:51:35PM +0200, Michael wrote: Now, as I understand it, it isn't possible to create an IPsec connection from a single host within a NATed network to an external server but OpenVPN works great here. Please correct me if I am wrong. (I have no access to the NAT router here.) If the router allows UDP traffic on ports 500 and 4500, isakmpd will fall back to NAT-traversal automatically if it decides it's necessary.
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
On 5/9/07, Daniel Ouellet [EMAIL PROTECTED] wrote: I can now have two clients using 1000 parallel connections to one i386 850MHz server, my old one that I was testing with and I get all that no problem now. No delay and I can even push it more, but I figure at 2000 parallel connections I should be able to get some breathing time now. I've spent considerable time with tuning apache on openbsd to consume all available resources in OpenBSD. Here's the relevant httpd.conf sections: Timeout 300 KeepAlive On MaxKeepAliveRequests 5000 KeepAliveTimeout 15 MinSpareServers 20 MaxSpareServers 30 StartServers 50 MaxClients 5000 MaxRequestsPerChild 0 I had staticlly compiled php into my httpd binary and obviously raised HARD_LIMIT to 5000, using OpenBSD's apache. This netted me an ability to serve about a max of 3000 requests per second on a 1.6ghz athlon with 256MB of memory. hth.
Re: 4.0 locked up over the weekend
Update: I've experienced 3 more hard lockups. No messgaes on the console screen. Nothing unusual in any of the log file that I've found. Make running in /upr/ports/x11/kde was interrupted at different tasks each time, (downloading, compiling, and running a configure script). System recovered each time with no problems after a powercycle. Are there some system monitoring tools I should be running to keep track of various resources? On 5/8/07, Bruce Bauer [EMAIL PROTECTED] wrote: Initial results: complied bonnie++ from ports make is running in ports/x11/kde 2 video streams passsing through VPN tunnel at abou 32 fps total output from bonnie++: Version 1.03 --Sequential Output-- --Sequential Input- --Random- -Per Chr- --Block-- -Rewrite- -Per Chr- --Block-- --Seeks-- MachineSize K/sec %CP K/sec %CP K/sec %CP K/sec %CP K/sec %CP /sec %CP roadrunner.for 300M 50379 46 49432 6 6322 1 25376 41 34974 4 130.7 0 --Sequential Create-- Random Create -Create-- --Read--- -Delete-- -Create-- --Read--- -Delete-- files /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP /sec %CP 16 2542 5 + +++ 5113 8 2898 7 + +++ 5478 9 roadrunner.fortechsw.com,300M,50379,46,49432,6,6322,1,25376,41,34974,4,130.7,0,16,2542,5,+,+++,5113,8,2898,7,+,+++,5478,9 ran uptime after bonnie++ finished 11:21AM up 1 day, 2:15, 2 users, load averages: 4.08, 3.15, 2.55 Everything seems to be running smoothly Bruce On 5/8/07, Joachim Schipper [EMAIL PROTECTED] wrote: On Tue, May 08, 2007 at 09:05:44AM -0700, Bruce Bauer wrote: Probably a good idea to put some load on the sytem anyway. See how the VPN data transfer holds up. Downloading ports.tar.gz now Running make in ports/www/kde should keep it busy for a while Not familiar with bonnie++, I'll check it out Bonnie++ just generates a lot of I/O. The 'ghetto' version involves running 'tar xzf srf.tar.gz; rm -rf src' in a loop. Let us know how it goes... Joachim -- TFMotD: tht, thtc (4) - Tehuti Networks 10Gb Ethernet device
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
Douglas Allan Tutty wrote: On Wed, May 09, 2007 at 01:30:41AM -0400, Daniel Ouellet wrote: No swapping is happening, even with 1000 httpd running. load averages: 123.63, 39.74, 63.3285 01:26:47 1064 processes:1063 idle, 1 on processor CPU states: 0.8% user, 0.0% nice, 3.1% system, 0.8% interrupt, 95.4% idle Memory: Real: 648M/1293M act/tot Free: 711M Swap: 0K/4096M used/tot How does this server do with 1000 non-httpd processes running? Perhaps I need a newer Nemeth et al, but in my 3rd edition, pg 759 middle of the page says Modern systems do not deal welll with load averages over about 6.0. Be careful when reading these numbers here. Don't forget that I am doing this in labs with abuse, etc. I am trying to push the server as much as I can here. In production, I do see some server reaching 10, 18 and some time I saw up to 25, but all these were in extreme cases, most of the time, it's always below 10. I can't answer this question with proper knowledge here as I don't pretend to know that answer. May be someone else can speak knowingly about it? Could your bottleneck be in context-switching between so many processes? With so many, the memory cache will be faulting during the context switching and have to be retreived from main memory. I don't think that such slow-downs appear in top, and I don't know about vmstat. I don't know if there's a tool to measure this on i386. Wasn't. However yes there is and I can see faulting. I check both the vmstat and iostat to see what's up. Obviously the number are higher on older hardware as it run out of horse power obviously. But the problem was the be able to handle more then 300 parallel connections and why it just 3x when only 2 more process were added. So, no, I don't think the context-switching had anything to do with it here. You will see when I post the changes I did and the test I did. Some are surprising! I've never run httpd but it looks to me like a massivly parralized problem where each connection is trivial to serve (hense low CPU usage, no disk-io waiting) but there are just so many of them. One multi core and multi processor hardware with proper memory, it shouldn't be a problem I think, but will know soon! How does the server do with other connection services, e.g. pop or ftp? I only run one application per servers, always did and most likely always will. So, any mail server is a mail server, and a web server is only a web server here anyway. Even DNS are only running DNS as well, etc.
backup DNS server for OpenBSD in Europe for free
Hi, I had reported today a wrong DNS record for www.openbsd.org on IRC. Next to this I had also offered a free backup server for openbsd domain in Europe. cmihai pointed me to these email addresses. If this is interesting for you, I'm running a tinydns server on Debian GNU/Linux in Europe (Czech Republic) and I can set my server as a backup for your records for free. My machine is in a hosting centre. You can check the connectivity against b.ns.navratil.cz for for example navratil.cz Best regards, Jiri **-- Jiri Navratil, http://www.navratil.cz, +420 777 224 245
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
Karsten McMinn wrote: On 5/9/07, Daniel Ouellet [EMAIL PROTECTED] wrote: I can now have two clients using 1000 parallel connections to one i386 850MHz server, my old one that I was testing with and I get all that no problem now. No delay and I can even push it more, but I figure at 2000 parallel connections I should be able to get some breathing time now. I've spent considerable time with tuning apache on openbsd to consume all available resources in OpenBSD. Here's the relevant httpd.conf sections: Thanks. My configuration is more aggressive them yours and I can tell you for a fact that the problem and limitations where not in the httpd configuration, but in the OS part in my case anyway. Some of your value I think would/could crash your system. Specially the: MaxKeepAliveRequests 5000 MaxClients 5000 I don't think you could reach that high. Why, simply on a memory usage stand point. That was my next exploration, but it's possible that one apache process could take as much as 11MB 6035 www20 11M 9392K sleepnetcon 0:56 0.00% httpd Obviously not all process would use that much. The question is really depending on content. If small images and lots of them, then each process use less memory. But if it is to serve all big files, then it's possible to use a good amount of memory per process. Now I don't have that answer here and I am not sure how to come with some logic on that, but even if each process was using only 1MB, then 5000 would give you 5GB or RAM with is more then what OpenBSD was supporting until not so long ago, so you will start to swap and god knows what will happen then. So, I think the these two value are not realistic and safe to us. Timeout 300 KeepAlive On MaxKeepAliveRequests 5000 KeepAliveTimeout 15 I use KeepAliveTimeout 5 and I am considering to reduce it. If you think aboiut your suggestion here, you have KeepAliveTimeout 15 and then MaxKeepAliveRequests 5000, don't you see the paradox here? If your server is really busy, and lots of images on one page for example, then you would have a lots of process stuck in KeepAliveTimeout time out stage, so that's why you most likely increase your MaxClients 5000 to compensate for that, but that's wrong I believe. It makes your server use more resources and be slower to react. I use a logic here for the value on how to fix it. MaxKeepAliveRequests I think should be set based on how many possible additional requests a URL from a browser that support keep alive and multiple requests at once could have. How many, well I think it's based on how many elements your web page can have. That's the idea here isn't it? Many browsers will call the URL and when images for example are on that page they will fire up an additional request to the web server. So, in theory the maximum number of requests you should allow should be the maximum possible of elements one page could have on it no? Assuming a users can click a few pages in a few seconds may be, I think anything above 1000 is not good. I could be wrong, but that's how I see it. I use 250 and it serve me well and allow to protect the server from abuse from one source. I have some setup that allow 100 max here for the MaxKeepAliveRequests. But I would think that 1000 should be plenty and more then that may not be good. Unless my thinking above is wrong? I can do more tests on that to know more obviously. For testing reason in my lab I put MaxKeepAliveRequests 0, but I wouldn't use that in production for sure. Your value may be good, I just think not, but that's open to discussion. One thing for sure having the same number for MaxKeepAliveRequests and for MaxClients, I think is wrong as you open yourself to have one attacker from one source to bring your server down and huge it all for himself. I believe that MaxKeepAliveRequests should definitely be lower then your MaxClients, not the same for sure. MinSpareServers 20 MaxSpareServers 30 StartServers 50 I also thing that if you want to run a so busy server, that you should have more StartServers and for sure have a bigger margin between the min and max as it will always kill process and start new one where as you fork a lots and that's a pretty slow process and costly as well.Again here I use some logic and based that on what the traffic is like. If you allow multiple requests per connection, wouldn't it make sense for you to be sure that you could serve that connection and all it's requests without having to fork new process? Meaning if you have 50 elements on your page, then it's possible that some browser will send you 50 requests, so why not make sure you have 50 minimum process to serve them? Again, that's logical to me. I have some setup that I keep a minimum of 50 spare and maximum of 100 spare. Not always, but some cases yes. But it's better then the defautl one for sure. (; MaxClients 5000 To high I think based on the above explications.
Re: Softupdates question
On 5/9/07, Nick Holland [EMAIL PROTECTED] wrote: If it was not obvious from my comments, I love softdeps. I have a siteXX.tgz file which does a few simple things, one of which is to change all mount points to use softdeps. One really does have to hunt a bit for relevant reasons not to use it. About the only place I can think of where I deliberately don't use it is on an e-mail archive system on the filled partitions which are mounted read-only. I can't tell you how many times I have forgot to install my siteXX file, started loading up /usr/src, and realized, Dang, obviously no softdeps. At which point, I stop the checkout, fix the problem, reboot, and try again. Yes, the performance difference is that obvious, and it is faster to reboot than it is to wait it out. I'm still curious about the issue of using softdep's when you have a raid card with write-cache (and battery)... I thought I'd do a simple test unpacking the ports.tar.gz with softdeps disabled/enabled, to see for myself. Without softdep enabled, I have the following: [EMAIL PROTECTED] time tar xzf ports.tar.gz 0.970u 2.120s 1:00.62 5.0% 0+0k 9821+210784io 6pf+0w [EMAIL PROTECTED] time rm -r ports 0.160u 1.390s 1:01.65 2.5% 0+0k 14994+126181io 17pf+0w About a minute to unpack and another minute to remove. With softdep enabled, I have the following: [EMAIL PROTECTED] time tar xzf ports.tar.gz 1.270u 2.100s 0:45.62 7.3% 0+0k 9874+66318io 59pf+0w [EMAIL PROTECTED] time rm -r ports 0.210u 1.230s 0:14.59 9.8% 0+0k 15741+22055io 17pf+0w 45 seconds to unpack and 15 seconds to remove. (I've repeated this a few times each way, and I always have roughly the same results.) With softdep enabled, there was more cpu time, but a noticeable decrease in total time. So, fair to say that even with raid+write-cache+battery that softdep's are beneficial (in terms of less disk time)? I'm more interested in maintaining disk-consistency, and with this setup, it looks like softdeps will still help with that also. Thanks again for all the info! -George
Re: revenge of stupid vlan questions
Clint Pachl wrote: Jon wrote: Greetings everybody, So I've set up what I thought should be a proper vlan configuration however something is clearly still not correct. Traffic passes fine to the vlan devices from the external side of the router (I can ping them) however traffic does not seem to pass bewteen the vlan devices and their parent device - I cannot ping stuff connected to the vlans on the switch. Is the switch VLAN-aware? Are the end-nodes in each VLAN VLAN- and tag-aware? The switch is vlan aware and the hosts connected to it are plugged into ports which are assigned to vlans configured on the switch with the same numbers that I am putting in the /etc/hostname.vlan* vlan option fields. I'm not sure what you mean by end-nodes being vlan and tag-aware though. Have I made an incorrect assumption that this vlan thing is something that happens transparently to the individual hosts in the cluster (excluding the switch and the firewall)? Using tcpdump on the vlan parent device DOES show all kinds of arp requests and such noise marked as 802.1Q coming from the hosts on the various vlans (mostly unanswered arp requests for the vlan device which is their gateway) and using tcpdump on the various vlan devices on the router shows only unanswered arp requests for the various other hosts. * I've uncommented net.inet.ip.forwarding=1 in /etc/sysctl.conf * packetfilter is off * hostname.em0: inet 172.18.1.2 255.255.255.0 NONE (external side of the router, local to my desktop lan - pings go through this to the vlan devices and return just fine) * hostname.em1: up mtu 1518 (the mtu 1518 part is just cause a man page seemed to be suggesting I should set it to this) The 802.1Q protocol will increase the frame by 4 bytes, but if your destination cannot interpret the VLAN protocol ID, the packet will be dropped. So in this case that isn't just the switch and the firewall? I'm confused. :( I thought the wrapping and unwrapping of the network packets in the vlan protocol packets was handled solely by the switch and firewall. You're telling me this is not the case? * hostname.vlan0: inet 172.17.1.1 255.255.255.0 172.17.1.255 vlan 1 vlandev em1 * hostname.vlan1: inet 172.17.2.1 255.255.255.0 172.17.2.255 vlan 2 vlandev em1 * hostname.vlan2: inet 172.17.3.1 255.255.255.0 172.17.3.255 vlan 3 vlandev em1 * hostname.vlan3: inet 172.17.4.1 255.255.255.0 172.17.4.255 vlan 4 vlandev em1
Re: Chances of this hardware running OpenBSD?
Timo Schoeler wrote: I was disappointed quite often by vaporware in the Amiga universe, However, as this really might become reality Don't hold your breath. $1500 for a system that is meant to cator to the amiga crowd. *shrug* If you want to start on a port, get in contact with P.A.Semi, and buy their SDK board. The amiga board looks like a 100% knock-off of it. -- [100~Plax]sb16i0A2172656B63616820636420726568746F6E61207473754A[dZ1!=b]salax
Re: revenge of stupid vlan questions
Jon wrote: Clint Pachl wrote: Jon wrote: Greetings everybody, So I've set up what I thought should be a proper vlan configuration however something is clearly still not correct. Traffic passes fine to the vlan devices from the external side of the router (I can ping them) however traffic does not seem to pass bewteen the vlan devices and their parent device - I cannot ping stuff connected to the vlans on the switch. Is the switch VLAN-aware? Are the end-nodes in each VLAN VLAN- and tag-aware? The switch is vlan aware and the hosts connected to it are plugged into ports which are assigned to vlans configured on the switch with the same numbers that I am putting in the /etc/hostname.vlan* vlan option fields. I'm not sure what you mean by end-nodes being vlan and tag-aware though. Have I made an incorrect assumption that this vlan thing is something that happens transparently to the individual hosts in the cluster (excluding the switch and the firewall)? he might mean other switches connected to the one with vlan tagging enabled. if you want to pass tagged packets between several switches you need to make sure the vlan setting match across the switches and that the ports linking them accept tagged packets. the hosts that are on switch ports that are NOT tagged and have a certain vlan assigned to them will transparently end up in the right vlan. Using tcpdump on the vlan parent device DOES show all kinds of arp requests and such noise marked as 802.1Q coming from the hosts on the various vlans (mostly unanswered arp requests for the vlan device which is their gateway) and using tcpdump on the various vlan devices on the router shows only unanswered arp requests for the various other hosts. * I've uncommented net.inet.ip.forwarding=1 in /etc/sysctl.conf * packetfilter is off * hostname.em0: inet 172.18.1.2 255.255.255.0 NONE (external side of the router, local to my desktop lan - pings go through this to the vlan devices and return just fine) * hostname.em1: up mtu 1518 (the mtu 1518 part is just cause a man page seemed to be suggesting I should set it to this) The 802.1Q protocol will increase the frame by 4 bytes, but if your destination cannot interpret the VLAN protocol ID, the packet will be dropped. So in this case that isn't just the switch and the firewall? I'm confused. :( I thought the wrapping and unwrapping of the network packets in the vlan protocol packets was handled solely by the switch and firewall. You're telling me this is not the case? see above. the 4 byte tag is only applied when packets need to pass to a tagged port on the switch, AFAIK. if you have an untagged port that is assigned to a vlan and the traffic passes to a tagged port, the tag will be applied to the traffic going out the tagged port. i am no authority on vlans but i have them setup and working throughout the company network at my workplace. cheers, jake * hostname.vlan0: inet 172.17.1.1 255.255.255.0 172.17.1.255 vlan 1 vlandev em1 * hostname.vlan1: inet 172.17.2.1 255.255.255.0 172.17.2.255 vlan 2 vlandev em1 * hostname.vlan2: inet 172.17.3.1 255.255.255.0 172.17.3.255 vlan 3 vlandev em1 * hostname.vlan3: inet 172.17.4.1 255.255.255.0 172.17.4.255 vlan 4 vlandev em1
Re: revenge of stupid vlan questions
On 2007/05/09 14:08, Jon wrote:
The switch is vlan aware and the hosts connected to it are plugged
into ports which are assigned to vlans configured on the switch with
the same numbers that I am putting in the /etc/hostname.vlan* vlan
option fields.
Usually you can configure a switch port to have one untagged vlan,
and zero or more tagged vlans.
Tagged vlans simply have the ethernet frame marked with the number
of the vlan, and need support from the connected device. You would use
this on the connection to the router.
Frames sent to untagged vlans have any tags, if present, stripped off
by the switch and passed on. You can connect normal equipment to an
untagged port, it doesn't need to know anything about VLANs. The cheap
fanless managed switch I have at home gives a dropdown list to choose
'PVID', which is the untagged vlan. Procurve and Extreme switches
just use tagged/untagged. $DEITY knows what Cisco use, they like to
make up their own names.
You can have both types running on a port together.
Using tcpdump on the vlan parent device DOES show
all kinds of arp requests and such noise marked as 802.1Q coming
from the hosts on the various vlans (mostly unanswered arp requests
for the vlan device which is their gateway) and using tcpdump on the
various vlan devices on the router shows only unanswered arp requests
for the various other hosts.
Run two copies of tcpdump, one on em1, one on a vlan. All the frames
marked for that vlan should be shown twice, one on parent, one on the
vlan. Is that working ok?
* I've uncommented net.inet.ip.forwarding=1 in /etc/sysctl.conf
You did either also set this manually, or reboot to activate it, I take
it? Please check output of 'sysctl net.inet.ip.forwarding' if you are not
absolutely certain.
* packetfilter is off
ok ('Status: disabled' in pfctl -si?)
* hostname.em0: inet 172.18.1.2 255.255.255.0 NONE (external side of the
router, local to my desktop lan - pings go through this to the vlan
devices and return just fine)
* hostname.em1: up mtu 1518 (the mtu 1518 part is just cause a man page
seemed to be suggesting I should set it to this)
Which man page seemed to be suggesting that? Maybe the wording needs
some adjustment. You should reset to 1500 and remove the setting from
hostname.em1. MTU is the maximum size of IP packets. Apart from on
extremely crappy nics, vlans do not affect that unless you stack them
on top of each other (e.g. vlandev vlanXXX).
The 802.1Q protocol will increase the frame by 4 bytes, but if your
destination cannot interpret the VLAN protocol ID, the packet will be
dropped.
Not necessarily; there is definitely some kit out there which just
strips the vlan tags and passes them through, I have a pseudowire WAN
circuit which does just that. (neos networks, for those in .uk-land
who are interested in such things: don't know whether it's them or
the modems on the telewest tail that's doing it...)
So in this case that isn't just the switch and the firewall? I'm
confused. :( I thought the wrapping and unwrapping of the network
packets in the vlan protocol packets was handled solely by the switch
and firewall.
picking nits: ethernet has frames, IP has packets.
You're telling me this is not the case?
It *is* the case, at least on untagged ports. I think that must be
how you have things configured, or you wouldn't see tags coming in to
the firewall on the parent interface.
From your earlier post,
Traffic passes fine to the vlan devices from the external
side of the router (I can ping them) however traffic does not
seem to pass bewteen the vlan devices and their parent device
- I cannot ping stuff connected to the vlans on the switch.
this isn't quite clear. can you try rewriting it more completely?
I cannot ping stuff - what stuff, where from? It may be simpler to
just copy-and-paste bits of a terminal session. Also include
arp -an, netstat -rnfinet, ifconfig -A from the firewall.
If you can describe exactly how the switch is configured, that may
help too.
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
Hi, I am passing my finding around for the configuration of sysctl.conf to remove bottleneck I found in httpd as I couldn't get more then 300 httpd process without crapping out badly and above that, the server simply got out of wack. All is default install and the tests are done with a server that is an old one. dmesg at the end in case you are interested. This is on OpenBSD 4.0 and I pick that server just to see what's possible as it's not really a very powerful one. You can also see the iostat output and the vmstat as well with the changes in place. You sure can see a few page fault as I am really pushing the server much, but even then I get decent results and the bottleneck was remove, even with 2000 parallel connections. In that case I had to use two different clients as the http_load only support up to 1021 parallel connections, so to test pass that, I use more then one clients to push the server more. But in all, the results are much better then a few days ago and now looks like we get more for the buck and adding more powerful hardware will be use better now instead of suffering the same limitations. I put also the value changed in sysctl.conf to come to this final setup. I am not saying the value are the best possible choice, but they work well in the test situation and there is many as you will see. Some are very surprising to me, like the change in net.inet.ip.portfirst. Yes I know, but if I leave it as default, then I can't get full success in the test below and get time out, some errors and efficiency is not as good. May be that's because of the random ports range calculations, I can't say, but in any case, the effect is there and tested. I try to stay safe in my choices and comments are welcome, but I have to point out as well that ALL the values below needs to be changes to that new value to get working well. If even only one of them is not at the level below, the results in the tests start to be affected pretty bad at times. So, not only one value needs to be changed or address the issues, but ALL of them below. I am still working on finding may be more restrictive value to keep the system as stable and safe and close to the default as possible, but below is a very good setup in y tests and all the results are below as well. As for the value in httpd.conf, they are still in progress to make them more normal, but for this test they are: Timeout 300 KeepAlive On MaxKeepAliveRequests 0 (shouldn't stay like this as limits needs to be in place) KeepAliveTimeout 5 MinSpareServers 40 MaxSpareServers 80 StartServers 40 MaxClients 2048 MaxRequestsPerChild 0 Also, the httpd use .so module like php and is not compile statically. For the value above, I think a more reasonable (still in progress as well) would be for a very busy server: Timeout 300 KeepAlive On MaxKeepAliveRequests 100 KeepAliveTimeout 5 MinSpareServers 50 MaxSpareServers 100 StartServers 75 MaxClients 768 MaxRequestsPerChild 0 However, I am not settle on them fully yet. I send an earlier email with explication for why some value should be pick. http://marc.info/?l=openbsd-miscm=117874246431437w=2 Any comments on any parts or caution I have overlooked? Thanks and hope this help some others that may suffer from the same problem I did. Daniel === sysctl.conf changes. kern.seminfo.semmni=1024 kern.seminfo.semmns=4096 kern.shminfo.shmall=16384 kern.maxclusters=12000 kern.maxproc=2048 # Increase for the process limits. kern.maxfiles=5000 kern.shminfo.shmmax=67108864 kern.somaxconn=2048 net.bpf.bufsize=524288 net.inet.ip.maxqueue=1278 net.inet.ip.portfirst=32768 net.inet.ip.redirect=0 net.inet.tcp.keepinittime=10 net.inet.tcp.keepidle=30 net.inet.tcp.keepintvl=30 net.inet.tcp.mssdflt=1452 net.inet.tcp.recvspace=65535 net.inet.tcp.rstppslimit=400 net.inet.tcp.sendspace=65535 net.inet.tcp.synbucketlimit=420 net.inet.tcp.syncachelimit=20510 === Test with multiple parallel connections, from 10 to 1000. As expected, the results gets better as we go and I was able to go up to 2000, but I limit the server at 2048 in the recompile version. At 2000, I get close to 2x the delay, meaning it's start to go back up before that, but still get full completed without errors in less then the time out of 30 seconds, witch I couldn't do before at 300 parallel connections anyway. # http_load -parallel 10 -fetches 1500 -timeout 30 /tmp/test 1500 fetches, 10 max parallel, 1.9647e+07 bytes, in 19.8742 seconds 13098 mean bytes/connection 75.4747 fetches/sec, 988568 bytes/sec msecs/connect: 84.6428 mean, 6003.03 max, 0.347 min msecs/first-response: 17.6985 mean, 1698.64 max, 3.236 min HTTP response codes: code 200 -- 1500 # http_load -parallel 20 -fetches 1500 -timeout 30 /tmp/test 1500 fetches, 20 max parallel, 1.9647e+07 bytes, in 20.824 seconds 13098 mean bytes/connection 72.0324 fetches/sec, 943480 bytes/sec msecs/connect:
Re: revenge of stupid vlan questions
This was very informative. Thank you very much. After re-evaluating
the vlan/tagging settings on the 3com switch ports we noticed that
they were all set to hybrid mode (so some could be on multiple
vlans) but the connection to the router was set to trunking mode
instead of hybrid. Changing it to hybrid fixed everything.
Stuart Henderson wrote:
On 2007/05/09 14:08, Jon wrote:
The switch is vlan aware and the hosts connected to it are plugged
into ports which are assigned to vlans configured on the switch with
the same numbers that I am putting in the /etc/hostname.vlan* vlan
option fields.
Usually you can configure a switch port to have one untagged vlan,
and zero or more tagged vlans.
Tagged vlans simply have the ethernet frame marked with the number
of the vlan, and need support from the connected device. You would use
this on the connection to the router.
Frames sent to untagged vlans have any tags, if present, stripped off
by the switch and passed on. You can connect normal equipment to an
untagged port, it doesn't need to know anything about VLANs. The cheap
fanless managed switch I have at home gives a dropdown list to choose
'PVID', which is the untagged vlan. Procurve and Extreme switches
just use tagged/untagged. $DEITY knows what Cisco use, they like to
make up their own names.
You can have both types running on a port together.
Using tcpdump on the vlan parent device DOES show
all kinds of arp requests and such noise marked as 802.1Q coming
from the hosts on the various vlans (mostly unanswered arp requests
for the vlan device which is their gateway) and using tcpdump on the
various vlan devices on the router shows only unanswered arp requests
for the various other hosts.
Run two copies of tcpdump, one on em1, one on a vlan. All the frames
marked for that vlan should be shown twice, one on parent, one on the
vlan. Is that working ok?
* I've uncommented net.inet.ip.forwarding=1 in /etc/sysctl.conf
You did either also set this manually, or reboot to activate it, I take
it? Please check output of 'sysctl net.inet.ip.forwarding' if you are not
absolutely certain.
* packetfilter is off
ok ('Status: disabled' in pfctl -si?)
* hostname.em0: inet 172.18.1.2 255.255.255.0 NONE (external side of the
router, local to my desktop lan - pings go through this to the vlan
devices and return just fine)
* hostname.em1: up mtu 1518 (the mtu 1518 part is just cause a man page
seemed to be suggesting I should set it to this)
Which man page seemed to be suggesting that? Maybe the wording needs
some adjustment. You should reset to 1500 and remove the setting from
hostname.em1. MTU is the maximum size of IP packets. Apart from on
extremely crappy nics, vlans do not affect that unless you stack them
on top of each other (e.g. vlandev vlanXXX).
The 802.1Q protocol will increase the frame by 4 bytes, but if your
destination cannot interpret the VLAN protocol ID, the packet will be
dropped.
Not necessarily; there is definitely some kit out there which just
strips the vlan tags and passes them through, I have a pseudowire WAN
circuit which does just that. (neos networks, for those in .uk-land
who are interested in such things: don't know whether it's them or
the modems on the telewest tail that's doing it...)
So in this case that isn't just the switch and the firewall? I'm
confused. :( I thought the wrapping and unwrapping of the network
packets in the vlan protocol packets was handled solely by the switch
and firewall.
picking nits: ethernet has frames, IP has packets.
You're telling me this is not the case?
It *is* the case, at least on untagged ports. I think that must be
how you have things configured, or you wouldn't see tags coming in to
the firewall on the parent interface.
From your earlier post,
Traffic passes fine to the vlan devices from the external
side of the router (I can ping them) however traffic does not
seem to pass bewteen the vlan devices and their parent device
- I cannot ping stuff connected to the vlans on the switch.
this isn't quite clear. can you try rewriting it more completely?
I cannot ping stuff - what stuff, where from? It may be simpler to
just copy-and-paste bits of a terminal session. Also include
arp -an, netstat -rnfinet, ifconfig -A from the firewall.
If you can describe exactly how the switch is configured, that may
help too.
www.openbsd.org (and vs openbsd.org)
Two small things: 1. www.openbsd.org replies with Forbidden at the moment -- but I guess most people already know. 2. Long time ago I was told that I shouldn't use openbsd.org, as it wasn't/isn't the official site. I was told to always use the www subdomain. Maybe this was just some people pulling my chain, however, I remember having discovered small differences between the two sites (for more than a year ago, though). The two names point to different addresses (this may mean nothing or everything). Please enlighten me :) Sorry for the noise. Martin [demime 1.01d removed an attachment of type application/pgp-signature which had a name of signature.asc]
Re: www.openbsd.org (and vs openbsd.org)
You can still get to the FAQ. I have a search box setup in FF and I was able to get to it... So the whole site ain't down, probably a permissions issue??? On 5/9/07, Martin Toft [EMAIL PROTECTED] wrote: Two small things: 1. www.openbsd.org replies with Forbidden at the moment -- but I guess most people already know. 2. Long time ago I was told that I shouldn't use openbsd.org, as it wasn't/isn't the official site. I was told to always use the www subdomain. Maybe this was just some people pulling my chain, however, I remember having discovered small differences between the two sites (for more than a year ago, though). The two names point to different addresses (this may mean nothing or everything). Please enlighten me :) Sorry for the noise. Martin [demime 1.01d removed an attachment of type application/pgp-signature which had a name of signature.asc]
Re: www.openbsd.org (and vs openbsd.org)
1. www.openbsd.org replies with Forbidden at the moment -- but I guess most people already know. www.openbsd.org is a mirror on a good network connection. at the moment it is recovering from having eaten itself.
Re: revenge of stupid vlan questions
Incidentally, it was the vlan(4) man page that gave me the idea to
set the mtu to 1518:
Some Ethernet chips will either discard or truncate Ethernet frames
that are larger than 1514 bytes. This causes a problem as 802.1Q
tagged frames can be up to 1518 bytes. Most controller chips can be
told not to discard large frames and/or to increase the allowed frame
size. Refer to the hardware manual for your chip to do this.
For some reason I thought that meant it would be a good idea to up
the mtu to 1518.
On 2007/05/09 14:08, Jon wrote:
The switch is vlan aware and the hosts connected to it are plugged
into ports which are assigned to vlans configured on the switch with
the same numbers that I am putting in the /etc/hostname.vlan* vlan
option fields.
Usually you can configure a switch port to have one untagged vlan,
and zero or more tagged vlans.
Tagged vlans simply have the ethernet frame marked with the number
of the vlan, and need support from the connected device. You would use
this on the connection to the router.
Frames sent to untagged vlans have any tags, if present, stripped off
by the switch and passed on. You can connect normal equipment to an
untagged port, it doesn't need to know anything about VLANs. The cheap
fanless managed switch I have at home gives a dropdown list to choose
'PVID', which is the untagged vlan. Procurve and Extreme switches
just use tagged/untagged. $DEITY knows what Cisco use, they like to
make up their own names.
You can have both types running on a port together.
Using tcpdump on the vlan parent device DOES show
all kinds of arp requests and such noise marked as 802.1Q coming
from the hosts on the various vlans (mostly unanswered arp requests
for the vlan device which is their gateway) and using tcpdump on the
various vlan devices on the router shows only unanswered arp requests
for the various other hosts.
Run two copies of tcpdump, one on em1, one on a vlan. All the frames
marked for that vlan should be shown twice, one on parent, one on the
vlan. Is that working ok?
* I've uncommented net.inet.ip.forwarding=1 in /etc/sysctl.conf
You did either also set this manually, or reboot to activate it, I take
it? Please check output of 'sysctl net.inet.ip.forwarding' if you are not
absolutely certain.
* packetfilter is off
ok ('Status: disabled' in pfctl -si?)
* hostname.em0: inet 172.18.1.2 255.255.255.0 NONE (external side of the
router, local to my desktop lan - pings go through this to the vlan
devices and return just fine)
* hostname.em1: up mtu 1518 (the mtu 1518 part is just cause a man page
seemed to be suggesting I should set it to this)
Which man page seemed to be suggesting that? Maybe the wording needs
some adjustment. You should reset to 1500 and remove the setting from
hostname.em1. MTU is the maximum size of IP packets. Apart from on
extremely crappy nics, vlans do not affect that unless you stack them
on top of each other (e.g. vlandev vlanXXX).
The 802.1Q protocol will increase the frame by 4 bytes, but if your
destination cannot interpret the VLAN protocol ID, the packet will be
dropped.
Not necessarily; there is definitely some kit out there which just
strips the vlan tags and passes them through, I have a pseudowire WAN
circuit which does just that. (neos networks, for those in .uk-land
who are interested in such things: don't know whether it's them or
the modems on the telewest tail that's doing it...)
So in this case that isn't just the switch and the firewall? I'm
confused. :( I thought the wrapping and unwrapping of the network
packets in the vlan protocol packets was handled solely by the switch
and firewall.
picking nits: ethernet has frames, IP has packets.
You're telling me this is not the case?
It *is* the case, at least on untagged ports. I think that must be
how you have things configured, or you wouldn't see tags coming in to
the firewall on the parent interface.
From your earlier post,
Traffic passes fine to the vlan devices from the external
side of the router (I can ping them) however traffic does not
seem to pass bewteen the vlan devices and their parent device
- I cannot ping stuff connected to the vlans on the switch.
this isn't quite clear. can you try rewriting it more completely?
I cannot ping stuff - what stuff, where from? It may be simpler to
just copy-and-paste bits of a terminal session. Also include
arp -an, netstat -rnfinet, ifconfig -A from the firewall.
If you can describe exactly how the switch is configured, that may
help too.
Re: www.openbsd.org (and vs openbsd.org)
On 5/9/07, Theo de Raadt [EMAIL PROTECTED] wrote: 1. www.openbsd.org replies with Forbidden at the moment -- but I guess most people already know. www.openbsd.org is a mirror on a good network connection. at the moment it is recovering from having eaten itself. Like the snake on the new Rush CD, Snakes and Arrows? ;)
Keyboard interrupt problem ('lag')
Hello all; I asked a while back about a 'lag' in keyboard response on my laptop (Gateway MT3705) that runs -current. Someone responded off-list and noted that it was an interrupt issue. I was wondering if there is any known workaround / fix for it; the machine can be very hard to use for day to day operation with this happening. Looking through the dmesg, it doesn't look like an irq sharing problem necessarily? I'm currently working on building a new -current and a new xenocara to test with since this one is a little dated. dmesg is as follows: OpenBSD 4.1-current (EXTRO.acpi-mp) #1: Tue Apr 17 09:34:02 EDT 2007 [EMAIL PROTECTED]:/usr/src/sys/arch/i386/compile/EXTRO.acpi-mp cpu0: Genuine Intel(R) CPU T2060 @ 1.60GHz (GenuineIntel 686-class) 1.60 GHz cpu0: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,EST,TM2,xTPR real mem = 1004621824 (981076K) avail mem = 910696448 (889352K) using 4278 buffers containing 50356224 bytes (49176K) of memory mainbus0 (root) bios0 at mainbus0: AT/286+ BIOS, date 12/07/06, BIOS32 rev. 0 @ 0xfd5fd, SMBIOS rev. 2.4 @ 0xdc010 (41 entries) bios0: Gateway MT3705 pcibios0 at bios0: rev 2.1 @ 0xfd580/0xa80 pcibios0: PCI IRQ Routing Table rev 1.0 @ 0xfde90/336 (19 entries) pcibios0: no compatible PCI ICU found: ICU vendor 0x1002 product 0x4372 pcibios0: Warning, unable to fix up PCI interrupt routing pcibios0: PCI bus #8 is the last bus bios0: ROM list: 0xc/0xd000 0xdc000/0x4000! acpi0 at mainbus0: rev 0 acpi0: tables DSDT FACP SLIC APIC MCFG SSDT acpitimer0 at acpi0: 3579545 Hz, 32 bits acpi device at acpi0 from table DSDT not configured acpi device at acpi0 from table FACP not configured acpi device at acpi0 from table SLIC not configured acpimadt0 at acpi0 table APIC addr 0xfee0: PC-AT compat cpu0 at mainbus0: apid 0 (boot processor) cpu0: apic clock running at 133 MHz cpu1 at mainbus0: apid 1 (application processor) cpu1: Genuine Intel(R) CPU T2060 @ 1.60GHz (GenuineIntel 686-class) 1.60 GHz cpu1: FPU,V86,DE,PSE,TSC,MSR,PAE,MCE,CX8,APIC,SEP,MTRR,PGE,MCA,CMOV,PAT,CFLUSH,DS,ACPI,MMX,FXSR,SSE,SSE2,SS,HTT,TM,SBF,SSE3,MWAIT,EST,TM2,xTPR ioapic0 at mainbus0: apid 2 pa 0xfec0, version 21, 24 pins acpi device at acpi0 from table MCFG not configured acpi device at acpi0 from table SSDT not configured acpiprt0 at acpi0: bus 0 (PCI0) acpiprt1 at acpi0: bus 0 (PB2_) acpiprt2 at acpi0: bus 0 (PB3_) acpiprt3 at acpi0: bus 2 (PB4_) acpiprt4 at acpi0: bus 5 (PB5_) acpiprt5 at acpi0: bus 0 (PB6_) acpiprt6 at acpi0: bus 0 (PB7_) acpiprt7 at acpi0: bus 8 (P2P_) acpiprt8 at acpi0: bus 1 (AGP_) acpiec0 at acpi0: EC__ acpicpu0 at acpi0: CPU0: acpicpu0: C3 not supported acpicpu1 at acpi0: CPU1: acpicpu1: C3 not supported acpitz0 at acpi0, critical temperature: 100 degC acpiac0 at acpi0: AC unit offline acpibat0 at acpi0: BAT0: model: 6MSB serial: type: Li oem: SMP-P acpibtn0 at acpi0: PWRB acpibtn1 at acpi0: LID0 acpibtn2 at acpi0: SLPB vesabios0 at mainbus0: version 2.0, ATI Technologies Inc. MS4 pci0 at mainbus0 bus 0: configuration mode 1 (no bios) pchb0 at pci0 dev 0 function 0 vendor ATI, unknown product 0x5a31 rev 0x01 ppb0 at pci0 dev 1 function 0 ATI RS480 PCIE rev 0x00 pci1 at ppb0 bus 1 vga1 at pci1 dev 5 function 0 ATI Radeon XPRESS 200M rev 0x00, vesafb wsdisplay0 at vga1 mux 1: console (80x25, vt100 emulation) wsdisplay0: screen 1-5 added (80x25, vt100 emulation) ppb1 at pci0 dev 4 function 0 ATI RS480 PCIE rev 0x00 pci2 at ppb1 bus 2 mskc0 at pci2 dev 0 function 0 Marvell Yukon 88E8038 rev 0x14, Yukon-2 FE (0x1): apic 2 int 16 (irq 10) msk0 at mskc0 port A, address 00:03:25:3f:d0:32 eephy0 at msk0 phy 0: Marvell 88E3082 10/100 PHY, rev. 3 ukphy0 at msk0 phy 3: Generic IEEE 802.3u media interface, rev. 0: OUI 0x121012, model 0x0004 ukphy0: no media present ukphy1 at msk0 phy 6: Generic IEEE 802.3u media interface, rev. 0: OUI 0x004c00, model 0x0013 ukphy1: no media present ppb2 at pci0 dev 5 function 0 ATI RS480 PCIE rev 0x00 pci3 at ppb2 bus 5 ohci0 at pci0 dev 19 function 0 ATI IXP400 USB rev 0x80: apic 2 int 19 (irq 11), version 1.0, legacy support ohci1 at pci0 dev 19 function 1 ATI IXP400 USB rev 0x80: apic 2 int 19 (irq 11), version 1.0, legacy support ehci0 at pci0 dev 19 function 2 ATI IXP400 USB2 rev 0x80: apic 2 int 19 (irq 11) usb0 at ehci0: USB revision 2.0 uhub0 at usb0 uhub0: ATI EHCI root hub, rev 2.00/1.00, addr 1 uhub0: 8 ports with 8 removable, self powered ATI IXP400 SMBus rev 0x83 at pci0 dev 20 function 0 not configured pciide0 at pci0 dev 20 function 1 ATI IXP400 IDE rev 0x80: DMA, channel 0 configured to compatibility, channel 1 configured to compatibility wd0 at pciide0 channel 0 drive 0: HTS421210H9AT00 wd0: 16-sector PIO, LBA48, 95396MB, 195371568 sectors atapiscsi0 at pciide0 channel 0 drive 1 scsibus0 at atapiscsi0: 2 targets cd0 at scsibus0 targ 0 lun 0: HL-DT-ST, DVDRAM GSA-T10N, PG02 SCSI0 5/cdrom removable wd0(pciide0:0:0): using PIO mode 4,
Re: revenge of stupid vlan questions
Jon wrote:
This was very informative. Thank you very much. After re-evaluating
the vlan/tagging settings on the 3com switch ports we noticed that
they were all set to hybrid mode (so some could be on multiple
vlans) but the connection to the router was set to trunking mode
instead of hybrid. Changing it to hybrid fixed everything.
Ok, that's weird, that has nothing to do with VLAN. I'm still not
convinced that everything is setup correctly, even though it is working.
Are you using VLANs for the purpose of creating separate Ethernet
domains? Do the end-nodes (hosts connected to a VLAN-aware switch port)
belong to multiple VLANs?
Stuart Henderson wrote:
On 2007/05/09 14:08, Jon wrote:
The switch is vlan aware and the hosts connected to it are plugged
into ports which are assigned to vlans configured on the switch with
the same numbers that I am putting in the /etc/hostname.vlan* vlan
option fields.
Usually you can configure a switch port to have one untagged vlan,
and zero or more tagged vlans.
Tagged vlans simply have the ethernet frame marked with the number
of the vlan, and need support from the connected device. You would use
this on the connection to the router.
Frames sent to untagged vlans have any tags, if present, stripped off
by the switch and passed on. You can connect normal equipment to an
untagged port, it doesn't need to know anything about VLANs. The cheap
fanless managed switch I have at home gives a dropdown list to choose
'PVID', which is the untagged vlan. Procurve and Extreme switches
just use tagged/untagged. $DEITY knows what Cisco use, they like to
make up their own names.
If you are just using VLANs for Ethernet domain segmentation only, the
end-nodes do not need to speak 802.1Q. You could disable VLAN on your
hosts if your 3COM switch accepts _only_ untagged frames (as opposed to
accepting tagged only and both) from hosts in a VLAN. Then, before the
switch forwards frames to these hosts, it will strip the 802.1Q protocol
bytes. Also, when the switch receives a VLAN tagged frame from a host on
an untagged frames only configured switch port, it will discard it.
This can be a security feature because it will make it more difficult
for a cracker to VLAN hop by injecting spoofed VLAN IDs into frames.
Such a frame could be forwarded by your switch to a normally unreachable
segment, thus bypassing your routing firewall.
You can have both types running on a port together.
Using tcpdump on the vlan parent device DOES show
all kinds of arp requests and such noise marked as 802.1Q coming
from the hosts on the various vlans (mostly unanswered arp requests
for the vlan device which is their gateway) and using tcpdump on the
various vlan devices on the router shows only unanswered arp requests
for the various other hosts.
Run two copies of tcpdump, one on em1, one on a vlan. All the frames
marked for that vlan should be shown twice, one on parent, one on the
vlan. Is that working ok?
* I've uncommented net.inet.ip.forwarding=1 in /etc/sysctl.conf
You did either also set this manually, or reboot to activate it, I take
it? Please check output of 'sysctl net.inet.ip.forwarding' if you are
not
absolutely certain.
* packetfilter is off
ok ('Status: disabled' in pfctl -si?)
* hostname.em0: inet 172.18.1.2 255.255.255.0 NONE (external side
of the router, local to my desktop lan - pings go through this to
the vlan devices and return just fine)
* hostname.em1: up mtu 1518 (the mtu 1518 part is just cause a man
page seemed to be suggesting I should set it to this)
Which man page seemed to be suggesting that? Maybe the wording needs
some adjustment. You should reset to 1500 and remove the setting from
hostname.em1. MTU is the maximum size of IP packets. Apart from on
extremely crappy nics, vlans do not affect that unless you stack them
on top of each other (e.g. vlandev vlanXXX).
See the diagnostics section of vlan(4).
The 802.1Q protocol will increase the frame by 4 bytes, but if your
destination cannot interpret the VLAN protocol ID, the packet will
be dropped.
Not necessarily; there is definitely some kit out there which just
strips the vlan tags and passes them through, I have a pseudowire WAN
circuit which does just that. (neos networks, for those in .uk-land
who are interested in such things: don't know whether it's them or
the modems on the telewest tail that's doing it...)
Well, then that device would be VLAN-aware. VLAN-aware devices can
transparently forward 802.1Q enhanced frames or strip the protocol
bytes, recalculate the FCS and forward.
Tag-aware devices manipulate and/or create 802.1Q enhanced frames.
So in this case that isn't just the switch and the firewall? I'm
confused. :( I thought the wrapping and unwrapping of the network
packets in the vlan protocol packets was handled solely by the switch
and firewall.
picking nits: ethernet has frames, IP has packets.
Not if you involve a tag-aware end-node (a
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
Daniel, Try the same test with this changes Timeout 60 KeepAlive Off If my guess is right, you'll notice big improvement. Tell me how it goes Marcos Laufer - Original Message - From: Daniel Ouellet [EMAIL PROTECTED] To: misc@openbsd.org Sent: Wednesday, May 09, 2007 7:41 PM Subject: Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections Hi, I am passing my finding around for the configuration of sysctl.conf to remove bottleneck I found in httpd as I couldn't get more then 300 httpd process without crapping out badly and above that, the server simply got out of wack. All is default install and the tests are done with a server that is an old one. dmesg at the end in case you are interested. This is on OpenBSD 4.0 and I pick that server just to see what's possible as it's not really a very powerful one. You can also see the iostat output and the vmstat as well with the changes in place. You sure can see a few page fault as I am really pushing the server much, but even then I get decent results and the bottleneck was remove, even with 2000 parallel connections. In that case I had to use two different clients as the http_load only support up to 1021 parallel connections, so to test pass that, I use more then one clients to push the server more. But in all, the results are much better then a few days ago and now looks like we get more for the buck and adding more powerful hardware will be use better now instead of suffering the same limitations. I put also the value changed in sysctl.conf to come to this final setup. I am not saying the value are the best possible choice, but they work well in the test situation and there is many as you will see. Some are very surprising to me, like the change in net.inet.ip.portfirst. Yes I know, but if I leave it as default, then I can't get full success in the test below and get time out, some errors and efficiency is not as good. May be that's because of the random ports range calculations, I can't say, but in any case, the effect is there and tested. I try to stay safe in my choices and comments are welcome, but I have to point out as well that ALL the values below needs to be changes to that new value to get working well. If even only one of them is not at the level below, the results in the tests start to be affected pretty bad at times. So, not only one value needs to be changed or address the issues, but ALL of them below. I am still working on finding may be more restrictive value to keep the system as stable and safe and close to the default as possible, but below is a very good setup in y tests and all the results are below as well. As for the value in httpd.conf, they are still in progress to make them more normal, but for this test they are: Timeout 300 KeepAlive On MaxKeepAliveRequests 0 (shouldn't stay like this as limits needs to be in place) KeepAliveTimeout 5 MinSpareServers 40 MaxSpareServers 80 StartServers 40 MaxClients 2048 MaxRequestsPerChild 0 Also, the httpd use .so module like php and is not compile statically. For the value above, I think a more reasonable (still in progress as well) would be for a very busy server: Timeout 300 KeepAlive On MaxKeepAliveRequests 100 KeepAliveTimeout 5 MinSpareServers 50 MaxSpareServers 100 StartServers 75 MaxClients 768 MaxRequestsPerChild 0 However, I am not settle on them fully yet. I send an earlier email with explication for why some value should be pick. http://marc.info/?l=openbsd-miscm=117874246431437w=2 Any comments on any parts or caution I have overlooked? Thanks and hope this help some others that may suffer from the same problem I did. Daniel === sysctl.conf changes. kern.seminfo.semmni=1024 kern.seminfo.semmns=4096 kern.shminfo.shmall=16384 kern.maxclusters=12000 kern.maxproc=2048 # Increase for the process limits. kern.maxfiles=5000 kern.shminfo.shmmax=67108864 kern.somaxconn=2048 net.bpf.bufsize=524288 net.inet.ip.maxqueue=1278 net.inet.ip.portfirst=32768 net.inet.ip.redirect=0 net.inet.tcp.keepinittime=10 net.inet.tcp.keepidle=30 net.inet.tcp.keepintvl=30 net.inet.tcp.mssdflt=1452 net.inet.tcp.recvspace=65535 net.inet.tcp.rstppslimit=400 net.inet.tcp.sendspace=65535 net.inet.tcp.synbucketlimit=420 net.inet.tcp.syncachelimit=20510 === Test with multiple parallel connections, from 10 to 1000. As expected, the results gets better as we go and I was able to go up to 2000, but I limit the server at 2048 in the recompile version. At 2000, I get close to 2x the delay, meaning it's start to go back up before that, but still get full completed without errors in less then the time out of 30 seconds, witch I couldn't do before at 300 parallel connections anyway. # http_load -parallel 10 -fetches 1500 -timeout 30 /tmp/test 1500 fetches, 10 max parallel, 1.9647e+07 bytes, in 19.8742 seconds 13098 mean bytes/connection 75.4747 fetches/sec, 988568 bytes/sec
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
Marcos Laufer wrote: Daniel, Try the same test with this changes Timeout 60 KeepAlive Off If my guess is right, you'll notice big improvement. Tell me how it goes Neither apply to the issue that was at hand. Timeout 60, or 300 like in this case have nothing to do with the connections rate or limit, but in some cases where processing from php scripts takes a long time, doing less then timeout 60 will stop the script for finishing. Plus timeout 60 is the time it will wait for an answer on the client side. The issue here is not a lack of reply, or a delay in it. See: http://httpd.apache.org/docs/1.3/mod/core.html#timeout For more details. As for KeepAlive Off, that would simply increase the number of required connections to the server with would have the opposite effect of helping. http://httpd.apache.org/docs/1.3/mod/core.html#keepalive I appreciate you looking at it, but that really have nothing to do with the problem as it was describe and demonstrated as well. Thanks Daniel
Re: Bottleneck in httpd. I need help to address capacity issues on max parallel and rate connections
On 5/9/07, Daniel Ouellet [EMAIL PROTECTED] wrote: I try to stay safe in my choices and comments are welcome, but I have to point out as well that ALL the values below needs to be changes to that new value to get working well. If even only one of them is not at the level below, the results in the tests start to be affected pretty bad at times. net.bpf.bufsize=524288 net.inet.ip.redirect=0 never mind the rest, but these two really make no sense. none.
