Re: Multithreaded server performance
From [EMAIL PROTECTED] Tue Apr 25 06:02:09 2000 Date: Tue, 25 Apr 2000 05:23:39 +0300 From: A G F Keahan [EMAIL PROTECTED] To: "Richard Seaman, Jr." [EMAIL PROTECTED] Cc: Jason Evans [EMAIL PROTECTED], [EMAIL PROTECTED] Subject: Re: Multithreaded server performance Jason and Richard, thank you very much for your explanations of libc_r and LinuxThreads. Due to the significant processing time of each request (typically between 50-800ms, averaging 100ms), I doubt that FreeBSD's threads would perform any worse than if I forked a separate process for each connection (memory usage would go sky high), or even if I had a single process and called poll() myself to check each descriptor for being readable/writable. What worries me though is that by design client connections are kept alive (although the server is allowed to disconnect a client after a period of inactivity), hence after a while there will be a lot of idle descriptors, and continuously polling them might slow down the processing of the active ones. I have to There is a way to save from slowing down on lot of connections. If we assume that response time is not consired to be too fast, for example, 500-5000 msec, we can poll() not all sockets, but just delay inserting some sockets in polling set. Look how it works: 1. we've made socket action (accept, read etc) 2. insert socket into poll/select waiting set after N msec 3. if socket has any activity within M msec, go to 1. 4. delay socket insertion on L msec If some of connections require most interactivity, they must use its own L,M,N. Only overhead we have is supporting two queues of active and frozen sockets. So, by variation L,M,N we have array of only 32 descriptors for poll() and really work with about 1000 sockets, working in single process without treads and forks. Sincerely your, Cyril A. Vechera To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: floating point exceptions
I am running FreeBSD 4.0-RELEASE on x86 with gcc 2.95.2 and the
httperf-0.6 port gives a SIGFPE and dumps core when run against a system
that has no web server running. (The default behavior is to measure
localhost when no arguments are specified).
It seems this is caused by a divide by zero error since the delta between
connections ends up being zero. The author suggest that the divide
should return a defined value, Inf, according to the IEEE floating point
standard. FreeBSD generates SIGFPE. I temporarily patched the code
locally to check for a delta of zero and arbitrarily set it to 1.0 so
that the divide succeeds and everything comes out ok without crashing.
Is FreeBSD's behavior correct? Why or why not? You can use the included
code snippet to verify that this occurs.
Thanks,
Nate
-- Forwarded message --
Date: Mon, 24 Apr 2000 15:54:38 -0700 (PDT)
From: David Mosberger [EMAIL PROTECTED]
To: Nate Lawson [EMAIL PROTECTED]
Subject: Re: patch to httperf-0.6
On Mon, 24 Apr 2000 15:37:21 -0700 (PDT), Nate Lawson [EMAIL PROTECTED] said:
Nate On x86. Your code generates SIGFPE, not SIGBUS sorry. Here
Nate is a code snippet which generates SIGFPE as well.
Nate main() { float a, b, c;
Nate a = 1.0; b = 0.0; c = a/b; }
Nate This seems to be correct behavior.
No, IEEE says that the exceptions should be disabled by default. You
should get Inf instead of SIGFPE.
--david
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: floating point exceptions
On Mon, Apr 24, 2000 at 11:44:59PM -0700, Nate Lawson wrote:
I am running FreeBSD 4.0-RELEASE on x86 with gcc 2.95.2 and the
httperf-0.6 port gives a SIGFPE and dumps core when run against a system
that has no web server running. (The default behavior is to measure
localhost when no arguments are specified).
It seems this is caused by a divide by zero error since the delta between
connections ends up being zero. The author suggest that the divide
should return a defined value, Inf, according to the IEEE floating point
standard. FreeBSD generates SIGFPE. I temporarily patched the code
locally to check for a delta of zero and arbitrarily set it to 1.0 so
that the divide succeeds and everything comes out ok without crashing.
Is FreeBSD's behavior correct? Why or why not? You can use the included
code snippet to verify that this occurs.
FreeBSD has traditionaly violated the IEEE FP standard in this regard.
This is fixed in 5.0 and I think in 4.0-STABLE (though I can't remember
what file this is in so I can't check.) If upgrading to -stable isn't
an option you can add a call to fpsetmask(3) in the application as
follows (this will work on Solaris and Irix as well):
#include ieeefp.h
...
main() {
declare things
fpsetmask(0);
...
}
-- Brooks
--
Any statement of the form "X is the one, true Y" is FALSE.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: commit MAKE_SHELL?
On Sun, Apr 23, 2000 at 06:51:16PM -0400, Brian Fundakowski Feldman wrote:
I certainly don't mind adding more shells to the ${MAKE_SHELL} logic, but
so far have only done ksh because using pdksh as the ${MAKE_SHELL} does,
for me, result in about 10% faster make world time, and speeds port
building enormously
Do you have any guesses about what causes this speed increase? What does
our shell suck at, in terms of speed? Maybe we could try and speed it up.
(this is not meant to counter your proposal).
--
Anatoly Vorobey,
[EMAIL PROTECTED] http://pobox.com/~mellon/
"Angels can fly because they take themselves lightly" - G.K.Chesterton
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: floating point exceptions
OK, having to call fpsetmask(0) is an acceptable workaround. So if I
do:
#ifdef __freebsd___
fpsetmask(0);
#endif
Then this should work on all versions of freebsd?
--david
On Tue, 25 Apr 2000 00:05:23 -0700, Brooks Davis [EMAIL PROTECTED]
said:
Brooks On Mon, Apr 24, 2000 at 11:44:59PM -0700, Nate Lawson wrote:
I am running FreeBSD 4.0-RELEASE on x86 with gcc 2.95.2 and the
httperf-0.6 port gives a SIGFPE and dumps core when run against a
system that has no web server running. (The default behavior is
to measure localhost when no arguments are specified).
It seems this is caused by a divide by zero error since the delta
between connections ends up being zero. The author suggest that
the divide should return a defined value, Inf, according to the
IEEE floating point standard. FreeBSD generates SIGFPE. I
temporarily patched the code locally to check for a delta of zero
and arbitrarily set it to 1.0 so that the divide succeeds and
everything comes out ok without crashing.
Is FreeBSD's behavior correct? Why or why not? You can use the
included code snippet to verify that this occurs.
Brooks FreeBSD has traditionaly violated the IEEE FP standard in
Brooks this regard. This is fixed in 5.0 and I think in 4.0-STABLE
Brooks (though I can't remember what file this is in so I can't
Brooks check.) If upgrading to -stable isn't an option you can add
Brooks a call to fpsetmask(3) in the application as follows (this
Brooks will work on Solaris and Irix as well):
Brooks #include ieeefp.h ... main() { declare things
Brooks fpsetmask(0); ... }
Brooks -- Brooks
Brooks -- Any statement of the form "X is the one, true Y" is
Brooks FALSE.
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: floating point exceptions
On Tue, Apr 25, 2000 at 07:47:01AM -0700, David Mosberger wrote: OK, having to call fpsetmask(0) is an acceptable workaround. So if I do: #ifdef __freebsd___ fpsetmask(0); #endif Then this should work on all versions of freebsd? #ifdef __FreeBSD__ fpsetmask(0); #endif BTW: if you update httperf, let me know so I can update the port (I'm the maintainer for the httperf port). -- Will Andrews [EMAIL PROTECTED] GCS/E/S @d- s+:++:- a---+++ C++ UB P+ L- E--- W+++ !N !o ?K w--- ?O M+ V-- PS+ PE++ Y+ PGP t++ 5 X++ R+ tv+ b++ DI+++ D+ G+ e- h! r--+++ y? To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: commit MAKE_SHELL?
Anatoly Vorobey:
On Sun, Apr 23, 2000 at 06:51:16PM -0400, Brian Fundakowski Feldman wrote:
I certainly don't mind adding more shells to the ${MAKE_SHELL} logic, but
so far have only done ksh because using pdksh as the ${MAKE_SHELL} does,
for me, result in about 10% faster make world time, and speeds port
building enormously
Do you have any guesses about what causes this speed increase? What does
our shell suck at, in terms of speed? Maybe we could try and speed it up.
I compared some complex shell scripts on different platforms, albeit
quite a while back (FreeBSD 2.something, I think). FreeBSD didn't do
too well, speedwise, mainly due to not having a built-in "test", thus
having to execute "/bin/test" (or "/bin/[") a lot.
$.02,
/Mikko
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Re: Multithreaded server performance
Linux runs into problems at less than 4000 threads because of a limit on the total number of processes, even if the thread stack size is decreased. 16xxx if you use a 2.3.99pre-x kernel? At least I thought that that was being mentioned as one of the major new things in 2.4.x kernels. Marco van de Voort ([EMAIL PROTECTED]) http://www.stack.nl/~marcov/xtdlib.htm To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Fwd: socket.h and _POSIX_SOURCE
On Mon, Apr 24, 2000 at 04:39:43PM +0200, Jeroen Ruigrok/Asmodai wrote: -On [2420 20:02], Arun Sharma ([EMAIL PROTECTED]) wrote: Comments ? $ cat test.c #include sys/types.h #include sys/socket.h $ cc -D_POSIX_SOURCE -c test.c In file included from test.c:2: /usr/include/sys/socket.h:47: syntax error before `sa_family_t' /usr/include/sys/socket.h:47: warning: data definition has no type or storage class sys/socket.h declares sa_family_t as a u_char. If we look at sys/types.h we see: #ifndef _POSIX_SOURCE typedef unsigned char u_char; [snip] #endif Which combined with D_POSIX_SOURCE causes [logical] problems. Would it be fair to say this is a (POSIX non-compliance) bug in the header files ? -Arun To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Turning on a relay.
I'd like to turn on a relay to the power for my laserprinter 3 rooms away where the server is located. I have an i/o board with a 8255 24 bit i/o port.(IIRC) So I wrote a simple userland program to do inb/outb, but it dumped core with BUSERR, I presume because userland is not supposed to do i/o to the hardware. I guess I have these options: A: write a driver/kernel module to access the port. B: use an extra parallel port. (I use 2 at the moment) C: use a serial port; I have 3-4 available. What would be the simplest to interface from a shellscript, i.e. the spooler to turn on and off the printer? (The relay has a turn-off delay, so I don't have to worry about turning off the power after everything has been sent, but the printer not finished, or turning off/on between printjobs) Leif To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
RE: Turning on a relay.
I think you need to have a fd open on /dev/io to do inb/outb. Jason Young Access US(tm) Chief Network Engineer -Original Message- From: Leif Neland [mailto:[EMAIL PROTECTED]] Sent: Tuesday, April 25, 2000 1:51 PM To: [EMAIL PROTECTED] Subject: Turning on a relay. I'd like to turn on a relay to the power for my laserprinter 3 rooms away where the server is located. I have an i/o board with a 8255 24 bit i/o port.(IIRC) So I wrote a simple userland program to do inb/outb, but it dumped core with BUSERR, I presume because userland is not supposed to do i/o to the hardware. I guess I have these options: A: write a driver/kernel module to access the port. B: use an extra parallel port. (I use 2 at the moment) C: use a serial port; I have 3-4 available. What would be the simplest to interface from a shellscript, i.e. the spooler to turn on and off the printer? (The relay has a turn-off delay, so I don't have to worry about turning off the power after everything has been sent, but the printer not finished, or turning off/on between printjobs) Leif To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Turning on a relay.
On 25/04, Leif Neland wrote: | I guess I have these options: | A: write a driver/kernel module to access the port. | B: use an extra parallel port. (I use 2 at the moment) | C: use a serial port; I have 3-4 available. D: use i386_set_ioperm to get access to the I/O port space To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Turning on a relay.
* Leif Neland [EMAIL PROTECTED] [000425 12:24] wrote: I'd like to turn on a relay to the power for my laserprinter 3 rooms away where the server is located. I have an i/o board with a 8255 24 bit i/o port.(IIRC) So I wrote a simple userland program to do inb/outb, but it dumped core with BUSERR, I presume because userland is not supposed to do i/o to the hardware. I guess I have these options: A: write a driver/kernel module to access the port. B: use an extra parallel port. (I use 2 at the moment) C: use a serial port; I have 3-4 available. D: open(/dev/io) -Alfred To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: Turning on a relay.
Yet another way.
I used
#include sys/kbio.h
...
io_fd = open("/dev/console", O_RDWR, 0);
ioctl(io_fd, KDENABIO, 0);
and
ioctl(io_fd, KDDISABIO, 0);
to turn it off again.
Is there a "right" way of doing it? Linux has a iopl call that sets the
i/o privilege level, it seems much easier and at least better documented.
(I was porting glx code, that was using iopl)
-Charlie
On Tue, Apr 25, 2000 at 12:30:21PM -0700, Alfred Perlstein wrote:
* Leif Neland [EMAIL PROTECTED] [000425 12:24] wrote:
I'd like to turn on a relay to the power for my laserprinter 3 rooms away
where the server is located.
I have an i/o board with a 8255 24 bit i/o port.(IIRC)
So I wrote a simple userland program to do inb/outb, but it dumped core with
BUSERR, I presume because userland is not supposed to do i/o to the
hardware.
I guess I have these options:
A: write a driver/kernel module to access the port.
B: use an extra parallel port. (I use 2 at the moment)
C: use a serial port; I have 3-4 available.
D: open(/dev/io)
-Alfred
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
--
Charles Anderson[EMAIL PROTECTED]
No quote, no nothin'
To Unsubscribe: send mail to [EMAIL PROTECTED]
with "unsubscribe freebsd-hackers" in the body of the message
Broken Kernel??
Warner thinks he may have broken the kernel in his last few commits. If so, the fix is to replicate card_if.m line in conf/files change pccard to card... (Warner was on vacation when he called me and asked me to post this e-mail for him...He appologizes for the mistake, and said he'll be back in 2-3 days...) Ryan [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
Re: bind and the limit of serial number ???
The generally accepted method (AFAIR) is mmddxx, where xx starts at 00 and is incremented for each change during that day. This allows for multiple updates in a single day without causing problems for situations such as 3 updates today followed by one update each day for the next 4 days ... Of course, if you never manually update the zone file, this isn't really an issue ;) On 23-Apr-00 Matthew Dillon wrote: For manual updates of the zone file, I recommend using mmdd rather then mmddhhmm, and if you modify it twice in one day just increment the day (and hopefully real time will catch up to it again). For automatic updates (i.e. scripts that update zone files), I recommend simply starting the serial number at 1 and incrementing it on every update. Trying to make the serial number into a date for viewing ease is overrated. Cheers, Colin To Unsubscribe: send mail to [EMAIL PROTECTED] with "unsubscribe freebsd-hackers" in the body of the message
