Linux-Development-Sys Digest #192, Volume #7 Tue, 14 Sep 99 03:14:00 EDT
Contents:
Re: major.minor.patch numbering (Tristan Wibberley)
Re: threads (Leslie Mikesell)
Re: Linux standards compliance (Kaz Kylheku)
kernel_lock (David Lee)
Re: Figure Out The MS Source Code Yourself (David Fox)
Re: Linux + C + 2 serial port (Vern Hoxie)
Device driver changes for new Kernel versions... (Shane Luttrell)
Re: kernel_lock (Karlo Szabo)
Re: threads (David Schwartz)
Re: Linux for SGI (Bryan)
----------------------------------------------------------------------------
From: Tristan Wibberley <[EMAIL PROTECTED]>
Subject: Re: major.minor.patch numbering
Date: Mon, 13 Sep 1999 05:13:47 +0100
Reply-To: [EMAIL PROTECTED]
Carl Karsten wrote:
>
> Where can I find the GNU version numbering standard? I am trying to propose
> using it, but I don't have a good description of what "it" is.
It's a common sense numbering scheme.
The major number is changed when there is a major change, the minor
number is changed when there is a minor change, the patch level is
changed when a bug is fixed.
With the name of a package, the numbering is fairly independent of the
actual changes in the software. With a binary that is supposed to be
used by a program, the major number has a real accepted meaning - If you
make a change that can break software using the binary, you change the
major number.
This way, a program which expects the interface of liblib.so.2 gets that
interface when it tries to load liblib.so.2 (even though it may get
liblib.so.2.1.0). New features/non-breaking additions to the interface
add to the minor number.
There is an often used sub-scheme where development releases have an odd
minor number and stable releases have an even one - Personally I don't
recommend this, it can be hell for people who want to try devel
versions, I personally prefer that development versions are considered
different libraries (eg, liblib-devel.so.2). This part has no standard -
and none is needed.
--
Tristan Wibberley
------------------------------
From: [EMAIL PROTECTED] (Leslie Mikesell)
Subject: Re: threads
Date: 13 Sep 1999 23:10:08 -0500
In article <[EMAIL PROTECTED]>,
David Schwartz <[EMAIL PROTECTED]> wrote:
>> I'm missing something here... How can 2 system calls be cheaper
>> than one? The only reason to select() is if you have something
>> to do when there is no input, and a well-designed server program
>> often doesn't. A blocking read lets the kernel schedule some
>> equally important other process at a time when this one is
>> idle.
>
> Because a read that blocks is equivalent to two system calls plus a
>context switch. There is the 'entry' half of the system call (up to
>where we decide to block), the block itself and reschedule, and then the
>'return' part of the system call after the wakeup (when we decide what
>to return).
Are you guessing that the select won't block?
> Imagine if ten connections are ready for I/O. Consider first a case
>where ten processes (each in a blocking read) handle them:
That takes a lot of imagination. What has the CPU been doing to let
ten accumulate?
> We have:
> 1) The ten reads that blocked.
> 2) Ten wakeups/context switches when the data comes in
> 3) The ten reads that now complete.
> Now imagine if one process blocks in 'poll' or 'select' and then does
>all ten I/Os in a non-blocking fashion.
>
> We have:
> 1) The one poll or select that decided to block.
> 2) One context switch to that process.
> 3) The 'other half' of that poll or select (when it returns).
> 3) Ten non-blocking reads that now complete.
Huh? Why did select not return when the first read could complete?
> See? 10+10+10 is MUCH greater than 1+1+1+10. In general, the context
>switches are more expensive than the system calls, so it's really
>comparing 10 to 1.
Don't all system calls involve a context switch into the kernel, and
isn't that a good time to come back directly into the program that
is ready to complete it's read()? I run lots of different programs
at once and there is no reason to think this one is going to complete
some operation before another equally important program.
>(Although exactly how the math works out is
>platform-dependent, but the second case will nearly always be far better
>than the first.)
Only if you are CPU bound. The only way I've every gotten a machine
to be CPU bound was forking a perl cgi at rates over 10 a second.
(But I don't do any number crunching...)
> Note that a 'thread per connection' model is more like the first case
>than the second! This is part of why using on thread per connection is
>bad.
They are bad if your select() wakes them all up when any i/o is ready.
>> How do you do ten i/o's at once when you are processing faster than
>> the i/o? You'll service the first one, complete it, sit in select,
>> service the next, and so on, doing 2 system calls on every input
>> instead of the one it would take seperate processes doing blocking
>> reads.
>
> Yes, this is what will happen when you aren't under stress. But really,
>in a good design, you optimize for the hard cases, not the easy ones.
Usually reading the input stream is the least of a server's worries. If
anything needs to be optimized it will be at the other end of the service
(i.e. processing the request after reading it).
>And your '2 to 1' math is way off because blocking system calls are more
>expensive than non-blocking ones.
But I expect select() to block every time.
>> I'm not conviced, but would like to know why you think this is true.
>> Are there any benchmarks that test both approaches to see if there
>> is some level of i/o where the select() method makes it likely
>> that you can read() bigger chunks and start saving system calls?
>
> The short answer is, if you don't care about performance, do whatever
>you want. If you do care about performance, you can't have one
>thread/process blocking in read for every client. It just WILL NOT
>SCALE.
My question is, at what point can we see a difference, and what
software will demonstrate it? I've run squid and apache as reverse
proxies to accelerate access to the static portions of a web site
(somewhere over a million hits a day, mostly in a 4 hour period)
and couldn't see a dramatic difference between the machine load
of 200 httpd's with 1 or 2 runnable vs 1 squid sitting in select.
The actual setup wasn't exactly the same so it isn't a great
comparison, but still, I don't think there was any real advantage
with that kind of load.
Les Mikesell
[EMAIL PROTECTED]
------------------------------
From: [EMAIL PROTECTED] (Kaz Kylheku)
Subject: Re: Linux standards compliance
Reply-To: [EMAIL PROTECTED]
Date: Sun, 12 Sep 1999 19:03:11 GMT
On 12 Sep 1999 18:32:17 +0100, Nix <$}xinix{[email protected]> wrote:
>Tristan Wibberley <[EMAIL PROTECTED]> writes:
>
>> I am not suggesting that for pay developers are any worse at writing
>> software than volunteers
>
>I am.
>
>When you pay people to do something, you will attract people whose
>primary interest is in that money. Not the code quality.
True, but you don't necessarily attract *only* such people.
Many good freeware writers are professional programmers who get paid in their
other life. There are programmers who take pride in providing good quality
code even though they also depend on getting paid.
>And those people will bugger up the code turned out by the rest of us
>(even if just slightly).
What about all those ``rest of us'' who have to work to pay the bills?
I guess any employed programmer will bugger up free software, due to $$$
motivation, right? Leave the development exclusively to the bums on social
assistance, or retired tycoons. ;)
>Unlinking (even partially) code quality and remuneration *will*
>compromise code quality. (And does.)
What evidence do you have? Most commercial software is also proprietary. So
it's not clear whether the poor quality comes from the closed nature or from
wrong motivation on the part of the developers. That is to say, it's not clear
whether the traditional problems inherent in proprietary software also extend
to free software that is commercially developed.
------------------------------
From: [EMAIL PROTECTED] (David Lee)
Subject: kernel_lock
Date: 14 Sep 1999 04:12:11 GMT
Hi there,
Does anyone know the purpose of kernel_lock(). It is used for
pervent task switching in kernel or to prevent other
processes to enter kernel.
Is it used in SMP kernel only?
-- David Lee
------------------------------
From: d s f o x @ c o g s c i . u c s d . e d u (David Fox)
Crossposted-To: comp.os.linux.development.apps,comp.os.linux.misc
Subject: Re: Figure Out The MS Source Code Yourself
Date: 13 Sep 1999 22:25:43 -0700
Dave Newton <[EMAIL PROTECTED]> writes:
> d s f o x @ c o g s c i . u c s d . e d u (David Fox) wrote:
> > How does what you expect enter into this?
>
> From a legal standpoint, obviously it doesn't.
>
> > Putting a law on the books means that whenever someone takes a fancy
> > to putting you in jail for breaking it, they probably will.
>
> As I said.
>
> > What you "expect" is of little importance.
>
> To you and the law, perhaps, but not to me.
>
> It's simple-I expect common sense to prevail. If I want to reverse
> engineer something and play with it I expect people to leave me alone;
> I'm not hurting anyone. If someone wants to put me in jail for that,
> fine, they can try. I will be civilly disobediant, and if that doesn't
> work, I'll be noisy about it.
What I'm still not understanding is what you mean when you say "I
expect" this and that. It sounds like you aren't concerned about the
law because you don't expect that it will be enforced. I'm concerned
about such a law, whether it makes common sense or not - especially if
it doesn't.
--
David Fox http://hci.ucsd.edu/dsf xoF divaD
UCSD HCI Lab baL ICH DSCU
------------------------------
From: [EMAIL PROTECTED] (Vern Hoxie)
Subject: Re: Linux + C + 2 serial port
Date: Tue, 14 Sep 1999 01:37:22 GMT
In article <[EMAIL PROTECTED]>,
Gert van der Knokke <[EMAIL PROTECTED]> wrote:
>Erol ELISABETH wrote:
deletion
>> I want to setup the serial port whith my C prog or someting else whiout
>> lunching minicom
>> please help...
>
>from teh comand line try:
>
>stty < /dev/ttySx
This is a dumb and hazardous way to set up a port. Some other program
can reconfigure the 'struct termios' before you get around from
running your application.
vern
--
Vernon C. Hoxie [EMAIL PROTECTED]
3975 W. 29th Ave. uucp: 303-455-2670
Denver, Colo., 80212 voice: 303-477-1780
I intend to live forever - so far, so good
------------------------------
From: Shane Luttrell <[EMAIL PROTECTED]>
Subject: Device driver changes for new Kernel versions...
Date: Tue, 14 Sep 1999 00:38:40 -0500
Hello,
I am new to Linux development. I am coming over from windows
development to write some device drivers for a V2.2.5 kernel. I have
been using the book Linux Device Drivers (author Alessandro Rubini) as a
reference (Linux seems to be easier to write hardware specific code for,
especially in the area of DMA's). Pretty good book, but unfortunately
the most recent kernel mentioned was the 2.1 version. Does anyone have
a FAQ or link that has a good summary of all the differences in the new
2.2 kernels as related to writing device drivers.
All help is appreciated.
Thanks,
Shane
------------------------------
From: Karlo Szabo <[EMAIL PROTECTED]>
Subject: Re: kernel_lock
Date: Tue, 14 Sep 1999 14:40:42 +1000
>
> Is it used in SMP kernel only?
>
Maybe it's used to lock out the other CPU?
------------------------------
From: David Schwartz <[EMAIL PROTECTED]>
Subject: Re: threads
Date: Mon, 13 Sep 1999 23:35:14 -0700
Leslie Mikesell wrote:
>
> In article <[EMAIL PROTECTED]>,
> David Schwartz <[EMAIL PROTECTED]> wrote:
>
> >> I'm missing something here... How can 2 system calls be cheaper
> >> than one? The only reason to select() is if you have something
> >> to do when there is no input, and a well-designed server program
> >> often doesn't. A blocking read lets the kernel schedule some
> >> equally important other process at a time when this one is
> >> idle.
> >
> > Because a read that blocks is equivalent to two system calls plus a
> >context switch. There is the 'entry' half of the system call (up to
> >where we decide to block), the block itself and reschedule, and then the
> >'return' part of the system call after the wakeup (when we decide what
> >to return).
>
> Are you guessing that the select won't block?
No, the idea is for the select or poll to block. The select or poll is
the blocking point for the I/O thread. It's where it waits until there
is work to do.
> > Imagine if ten connections are ready for I/O. Consider first a case
> >where ten processes (each in a blocking read) handle them:
>
> That takes a lot of imagination. What has the CPU been doing to let
> ten accumulate?
Quite a few things. Running other tasks. Perhaps doing the work
necessary to service the I/O that is currently active.
We are talking about optimization here, and you don't optimize for the
"the CPU is doing nothing when all of a sudden one request comes in"
case. In that case, who cares what you do -- anything will work. We care
about what happens when the CPU and the system are overwhelmed.
We try to design, when possible, so that the more overwhelmed the
system is, the _more_ efficient we are. This helps to prevent
catastrophic collapse. Designs that become better under load tend to be
much more stable and implosion-resistant than designs that get worse as
load increases.
With a blocking poll/select loop, the more overloaded and strained we
are, the more work we can do per context switch. This is a very good
thing.
> > We have:
> > 1) The ten reads that blocked.
> > 2) Ten wakeups/context switches when the data comes in
> > 3) The ten reads that now complete.
>
> > Now imagine if one process blocks in 'poll' or 'select' and then does
> >all ten I/Os in a non-blocking fashion.
> >
> > We have:
> > 1) The one poll or select that decided to block.
> > 2) One context switch to that process.
> > 3) The 'other half' of that poll or select (when it returns).
> > 3) Ten non-blocking reads that now complete.
>
> Huh? Why did select not return when the first read could complete?
Because the receiving of a packet that generates I/O that satisfies the
select and thus makes to process ready to run takes place before the
process actually gets scheduled. In some cases, way before. On
uniprocessor, we may be busy doing work for other connections that are
part of the same server.
The only case where we wakeup instantly is when the CPU isn't busy. And
who cares about that case? It's senseless to optimize for that case
since anything is equally good when there is CPU to burn.
All you care about in that case, perhaps, is saving memory. And to save
memory, you want as few processes/threads as possible since that reduces
the number of stacks. So a small number of threads is better in that
case too.
> > See? 10+10+10 is MUCH greater than 1+1+1+10. In general, the context
> >switches are more expensive than the system calls, so it's really
> >comparing 10 to 1.
>
> Don't all system calls involve a context switch into the kernel, and
> isn't that a good time to come back directly into the program that
> is ready to complete it's read()? I run lots of different programs
> at once and there is no reason to think this one is going to complete
> some operation before another equally important program.
If your asking if all system calls are equally expensive, no. System
calls that don't require an extra pass through the scheduler are cheaper
than those that do. We're talking about the total number of process or
thread context switches here. That's usually what you need to minimize
to boost performance.
> >(Although exactly how the math works out is
> >platform-dependent, but the second case will nearly always be far better
> >than the first.)
>
> Only if you are CPU bound. The only way I've every gotten a machine
> to be CPU bound was forking a perl cgi at rates over 10 a second.
> (But I don't do any number crunching...)
Something has to be the limiting factor in the server. Whatever that
is, that's what you need to deal with.
> > Note that a 'thread per connection' model is more like the first case
> >than the second! This is part of why using on thread per connection is
> >bad.
>
> They are bad if your select() wakes them all up when any i/o is ready.
Well, that's a straw man, because anybody dumb enough to do that
deserves what he or she gets. Obviously, spurious wakeups are bad. You
want to try to avoid waking a thread or process that can't do anything
about a particular situation. Fortunately, the tools to do that are
readily available.
> >> How do you do ten i/o's at once when you are processing faster than
> >> the i/o? You'll service the first one, complete it, sit in select,
> >> service the next, and so on, doing 2 system calls on every input
> >> instead of the one it would take seperate processes doing blocking
> >> reads.
> >
> > Yes, this is what will happen when you aren't under stress. But really,
> >in a good design, you optimize for the hard cases, not the easy ones.
>
> Usually reading the input stream is the least of a server's worries. If
> anything needs to be optimized it will be at the other end of the service
> (i.e. processing the request after reading it).
Right. That's why it's petty common to hand off a request to a back end
of worker threads. This way your I/O thread will queue as much work as
it can, and then your worker thread will do as much work as it can (on
uniprocessor). This way, you can do any amount of work on any number of
connections with only one context switch.
It would be nice if you could avoid the context switch entirely, and on
NT there are ways to do this. But generally, you do the best you can.
> >And your '2 to 1' math is way off because blocking system calls are more
> >expensive than non-blocking ones.
>
> But I expect select() to block every time.
It depends, there are cases where it won't. For example, if you just
added a new file descriptor to the set you are selecting on for writing.
But, yes, the idea is to have 'select' (or poll) be the blocking point
for your I/O thread(s).
> >> I'm not conviced, but would like to know why you think this is true.
> >> Are there any benchmarks that test both approaches to see if there
> >> is some level of i/o where the select() method makes it likely
> >> that you can read() bigger chunks and start saving system calls?
> >
> > The short answer is, if you don't care about performance, do whatever
> >you want. If you do care about performance, you can't have one
> >thread/process blocking in read for every client. It just WILL NOT
> >SCALE.
>
> My question is, at what point can we see a difference, and what
> software will demonstrate it? I've run squid and apache as reverse
> proxies to accelerate access to the static portions of a web site
> (somewhere over a million hits a day, mostly in a 4 hour period)
> and couldn't see a dramatic difference between the machine load
> of 200 httpd's with 1 or 2 runnable vs 1 squid sitting in select.
> The actual setup wasn't exactly the same so it isn't a great
> comparison, but still, I don't think there was any real advantage
> with that kind of load.
200 is nothing. Try 5,000. Or 16,000.
DS
------------------------------
From: Bryan <Bryan@[EMAIL PROTECTED]>
Subject: Re: Linux for SGI
Date: Tue, 14 Sep 1999 05:39:28 GMT
www.linux.sgi.com
yes, its that obvious ;-)
Hong Hu <[EMAIL PROTECTED]> wrote:
: --------------D7F6852C43275F51C7BF2D40
: Content-Type: text/plain; charset=gb2312
: Content-Transfer-Encoding: 7bit
: Does anyone know if it is possible to install linux on a old SGI machine? (Say SGI
:Indigo)--
: ===============================================
: Hong Hu *
: Nieuwland Science Hall 251 *
: Department of Chemistry&Biochemistry *
: University of Notre Dame *
: Notre Dame, IN 46556 *
: -----------------------------------------------
: Ph: 219-6313904(office) *
: Email [EMAIL PROTECTED] *
: ===============================================
: --------------D7F6852C43275F51C7BF2D40
: Content-Type: text/html; charset=gb2312
: Content-Transfer-Encoding: 7bit
: <!doctype html public "-//w3c//dtd html 4.0 transitional//en">
: <html>
: <pre>Does anyone know if it is possible to install linux on a old SGI machine? (Say
:SGI Indigo)--
: ===============================================
: Hong
:Hu
: *
: Nieuwland Science Hall
:251
: *
: Department of
:Chemistry&Biochemistry *
: University of Notre
:Dame
: *
: Notre Dame, IN
:46556
: *
: -----------------------------------------------
: Ph:
:219-6313904(office)
: *
: Email
:[EMAIL PROTECTED]
: *
: ===============================================</pre>
: </html>
: --------------D7F6852C43275F51C7BF2D40--
--
Bryan, http://www.Grateful.Net - Linux/Web-based Network Management
->->-> to email me, you must hunt the WUMPUS and kill it.
------------------------------
** FOR YOUR REFERENCE **
The service address, to which questions about the list itself and requests
to be added to or deleted from it should be directed, is:
Internet: [EMAIL PROTECTED]
You can send mail to the entire list (and comp.os.linux.development.system) via:
Internet: [EMAIL PROTECTED]
Linux may be obtained via one of these FTP sites:
ftp.funet.fi pub/Linux
tsx-11.mit.edu pub/linux
sunsite.unc.edu pub/Linux
End of Linux-Development-System Digest
******************************
