Linux-Development-Sys Digest #753, Volume #8 Sun, 27 May 01 16:13:29 EDT
Contents:
Re: Preprocessor symbols automatically defined by gcc (Nix)
Re: Preprocessor symbols automatically defined by gcc (Nix)
is there a way to password protect LILO? (Anonymous)
Re: Please read... (Roger Leigh)
thundering herd problem: the REAL scoop ([EMAIL PROTECTED])
Re: is there a way to password protect LILO? ("Michael Faurot")
Debug kernel (Zhiyong Xu)
----------------------------------------------------------------------------
From: Nix <$}xinix{$@esperi.demon.co.uk>
Subject: Re: Preprocessor symbols automatically defined by gcc
Date: 27 May 2001 17:10:38 +0100
On 24 May 2001, Paul Kimoto gibbered:
> In article <[EMAIL PROTECTED]>, Nix wrote:
>> (GCC-3.0 loses the __i386__ crock
>
> I still seem to have it defined, though. (Is my installation broken?)
No, I can't read. It doesn't lose it; it's just defined in the
CPP_CPU_SPEC in i386.h instead of in the OS-specific places. (It was
being defined in both places before GCC-3.0.)
CPP_CPU_SPEC is not subject to magic underscoring, so i386.h defines
i386, __i386, and __i386__ separately.
(In many ways this whole area needs a rewrite. Zack Weinberg's been
giving it a good clean, but it's still horrible.)
--
`Technology is meaningless. What matters is how people _think_
of it.' --- Linus Torvalds
------------------------------
From: Nix <$}xinix{$@esperi.demon.co.uk>
Subject: Re: Preprocessor symbols automatically defined by gcc
Date: 27 May 2001 17:17:18 +0100
On 25 May 2001, [EMAIL PROTECTED] yowled:
> What I am trying to do is to come up with a header file that contains
> typedefs for various fundamental scalar types ie 32 bit signed/unsigned
> ints, 64 bit signed/unsigned ints, etc.
Use AC_CHECK_SIZEOF; autoconf-2.50, just released, can make this work
even for cross-compilation, so there's no reason not to use it :)
(Unless, that is, there's some other reason why you can't use it.)
> The system that we are developing may possibly require certain portions
> of code to be backported to Windows and may possibly be 'ported' to UNIX
> platforms other than Linux.
The point of autoconf is to keep such porting to a minimum :)
AC_CHECK_SIZEOF is a good way of avoiding porting needs for integral
type-size-variation reasons.
> I was originally using the types defined in <stdint.h> and <sys/types.h>
> but we need to keep in mind systems (ie Windows) that don't have ISO C99
> or POSIX headers.
Of course. (If they don't have C89 headers I think we can let them rot
in this day and age.)
--
`Technology is meaningless. What matters is how people _think_
of it.' --- Linus Torvalds
------------------------------
Date: Sun, 27 May 2001 13:09:56 -0400
Subject: is there a way to password protect LILO?
From: Anonymous <[EMAIL PROTECTED]>
Is there a way to password protect LILO
so some random person can't just
type
linux 1
at the prompt and get into single user mode?
--------== Posted Anonymously via Newsfeeds.Com ==-------
Featuring the worlds only Anonymous Usenet Server
-----------== http://www.newsfeeds.com ==----------
------------------------------
From: Roger Leigh <[EMAIL PROTECTED]>
Subject: Re: Please read...
Date: 27 May 2001 17:34:48 +0100
"psy" <[EMAIL PROTECTED]> writes:
> Hi!
>
> Since I am new to Linux, please be patient with me. I have had problems
> with Linux recognizing my two ethernet cards since I got the message "failed
> to bringing up interface eth0".
>
> My first ethernet card is a D-Link DFE-530tx + PCI adapter and my second one
> is a Realtek rtl8139(A) pci fast ethernet.
>
> What I did is the following: I went on D-Link website (through windows) and
> downloaded a driver for Linux which is an executable (I installed it on a
> floppy disk). Happy with that, I went back on Linux, KDE, double clicked on
> a: drive and double clicked on the file. Then, nothing happened... Back to
> square one!!
In Windows it is common to install software via self-extracting
executable archives. On Linux it is not. I would suggest that you
read the kernel-HOWTO (www.linuxdoc.org) and compile your own kernel
with support for your two network cards as loadable modules. I have
no knowledge of D-Link cards, but I would be very wary of installing
random executables off websites. If you really do need to use it, and
it is called something like dlink.o, copy it to
/lib/modules/`uname -r`/net/ where `uname -r` is the output of
'uname -r'.
You can get the latest copy of the kernel source from www.kernel.org,
or its mirrors. If you are running a 2.2.x kernel, get 2.2.19, or if
2.4.x, get 2.4.5. If you are running 2.0.x (erk!), then you need
2.0.39.
The newsgroup comp.os.linux.setup is most appropriate for your
question. Please use it in the future, and use an appropriate subject
line in your posts.
> In a previous message I got some advise such as "activate driver with
> "kudzu"", etc. Now, since I know nothing about Linux, how do I activate
> that? And how can I install this D-Link driver?
Message from what? You load modules with 'modprobe' or 'insmod'.
More usually you add entries to /etc/modules.conf so that it all
happens automatically.
Drivers come with the kernel source, not from manufacturers websites.
Even if you find one, it's best to use the source and compile your own
if you can.
> Also, when I double clicked on "linux config" I got this message "couldn't
> execute /usr/bin/kdesu"/sbin/linuxconf" Please install the linuxconf
> package". I have with me 5 cds. The first two is intall, the third one is
> called "source", the fourth one is "powertools" and 5th one is
> "documentation". I have to confess that I went through the documentation
> but was quite hard to understand...
Read the kernel-HOWTO, and the Network Administrator's Guide may also
be of some help. Not all the documentation is easy to understand, but
take it one bit at a time and you'll get there. Good Luck!
Roger
--
Roger Leigh ** Registration Number: 151826, http://counter.li.org **
Need Epson Stylus Utilities? http://gimp-print.sourceforge.net/
For GPG Public Key: finger [EMAIL PROTECTED] or see public keyservers.
------------------------------
From: [EMAIL PROTECTED]
Crossposted-To: comp.unix.programmer,comp.unix.bsd.freebsd.misc
Subject: thundering herd problem: the REAL scoop
Date: 27 May 2001 16:51:24 GMT
Over the course of the past few weeks, I have posted articles starting
a few threads trying to get more information about known solutions to
the "thundering-herd" problem. The primary reason I have sought more
information online is the apparent lack of information in the regular
documentation sources (aside from the source itself, which I usually
find very time consuming to pin down precise details of exact behaviour
considering my lack of knowledge of kernel internal logic and pattern).
When I did post these questions, what it seemed I was getting were more
"half answers" leading in different directions, many apparently not
understanding what I was asking. Even though I have worked out some
scenarios where specifics of implementation to solutions can affect the
logic, some responses were saying that things "just work" and I do not
need to even know what the solution is to use it. And finally, I was
still seeing the "thundering-herd" problem persist and the threads
would die out with no resolution.
Yesterday, I finally got the answer. It was not the answer that I was
looking for, but then, a true quest for knowledge very often does not
yield what you are expecting. That answer that I got is that there are
TWO distinct problems labeled as the "thundering herd" problem.
This also explained much of the confusion that existed in the threads
that were discussing it. I was only aware of one of the problems, and
I presume that most people answering me with what seemed to me to be
the confusing or misleading answers were either aware of only the other
problem, or at least thought that was what I was in fact asking about.
The general description of the "thundering herd" problem is that when
a network connection arrives for a port listened to by more than one
process, each process wakes up and runs to attempt to acquire the new
connection and service it. All but one will fail to get it and go back
to sleep. The problem is that this does not scale well when the number
of processes gets large. The confusion is that this description applies
to both of the "thundering herd" problems, hence the common label, but
the details do not, nor do the solutions.
The two specific problems in more detail are:
1. Multiple processes have non-blocking sockets for which listen() has
been called, and because they have other descriptors, they must use the
select() or poll() system calls to wait for an active descriptor. When
the listening socket receives a new connection, the descriptor for every
process is now technically an active descriptor, and select() or poll()
wakes up in each and every process. Every process now races to do the
accept() call, but only one process will succeed and get a new socket
descriptor and service the connection. All the rest will get an error
with an EAGAIN or EWOULDBLOCK error number, and go back to sleep in the
select() or poll() call.
2. Multiple processes have blocking sockets for which listen() has been
called, and have only this one socket to wait for events, so they simply
call accept(). While only one process will awake from accept() with a
new socket descriptor, what goes on behind the scenes in the kernel is
a different story. It seems internally, the way a UNIX-class kernel
has traditionally behaved for accept() is that the kernel thread that
is associated with the process indicates it willingness to received a
wakeup when a new connection arrives on a particular queue, then goes
to sleep. When that connection does arrive, all the internal threads
that have indicated this are now scheduled to execute. Each proceeds
to check the queue for a connection, and when none are found for all but
one of them, they go back to waiting. Although the constants in this
version of the problem are smaller, it still has the same scalability
problem as the userland process version has.
When I made my postings online, I was only aware of problem #1. I did
not even imagine the kernel would have problem #2. Even though I do
have experience in operating system internals, it has not been in UNIX,
so the specific details of how something is implemented would not have
revealed to me a specific problem. Further, much of the terms I did
read used things like function names that quite probably were well
chosen and very descriptive for internal kernel functions, but due to
that (and my lack of knowing of problem #2) I simply treated them as
abstract descriptions; I didn't realize they were probably referring to
specific kernel roles that were having the problem.
It also seems to be the case that many other people may be unaware of
the existance of problem #1, or may think it is an unimportant problem,
or think the solution they know is so obvious it is not worth even being
mentioned (but I knew of no such solution).
Apparently FreeBSD has solved problem #2 quite a while ago, and Linux
has solved it recently as well. I don't know one way or the other about
other systems like Solaris (but I would imagine on the scale it runs, it
would either have to not have it, or have solved it). Some may have
solved it ages ago (or never had it due to their design), or may even
still have problem #2.
The solution to #2 would certainly have no visible impact on userland
processes, and I now see what people meant when they said I should not
see any affect, and would not need to code my process any differently,
to work with solutions to problem #2. Unfortunately for their repsonse,
that was not the problem I have encountered.
The Apache web server has addressed this issue by using a semaphore to
control execution entry into the accept() call. This seems to only
work when there is just one socket to listen on. It is not clear to me,
yet, how Apache deals with the multiple listen socket case, as the code
is large, complex, and deals with lots of details both for the server
itself, and for the many platforms and variant locking mechanisms it
has to deal with. They do seem to have identified specific platforms
where problem #2 exists, or not, in ap_config.h.
I am still seeking a solution to problem #1.
The situation I have involves processes calling select() or poll().
They cannot do a locking mechanism like Apache does because there is
more than just listening sockets being selected. If all but one
process is blocked in a semaphore, they cannot receive messages on
other descriptors, and cannot become aware of write buffer space that
was previously full becoming available. They will not be able to carry
out a lingering shutdown/close on other sockets without delaying their
ability to accept another connection.
The solution I ultimately want is for the "wake one" semantics to be
applicable to select() and poll() themselves. Some of the posters in
past threads did seem to understand I was asking for this, but simply
asserted that it could not be done, or would violate the semantics.
I can't say that it wouldn't violate semantics and that if universally
implemented, wouldn't break some other programs. I was expecting there
to be some special call, maybe an ioctl() on the listen socket, for the
process to indicate it wanted "wake one" semantics to be applied in that
particular case. I would disagree that this cannot be done. But I do
not personally have the time to delve into the FreeBSD or Linux kernels
to learn all what's inside to the level where I could implement this to
prove that it can be done.
There is one potential problem with doing "wake one" on select() or
poll(). If one connection arrives and one process wakes up, if that
process decides to do something else instead, or otherwise fails to
execute the accept() call, there remains a pending connection on the
queue. When another connection arrives, if only one more process now
wakes up, it would likely accept() just the first connection, leaving
the second connection still pending. Now we have an "off by one" kind
of problem in the queueing. These kinds of things could be solved, but
it would take some extra kernel logic to do, possibly a lot to shut all
the exposures.
FreeBSD, interestingly, may be on the verge of solving this, although
using the kqueue() and kevent() syscalls. These syscalls allow a
process to converge events, such as descriptor readiness, in a more
orderly manner that is more efficient for both process code and kernel
code. With an addition to the kevent() call to specifically address
the acceptance of a new connection as an atomic event, and returning
a descriptor (possibly a list of them when many new connections arrive
very rapidly, or if this is the only listening process), it would be
possible to wake up exactly one process and complete the acceptance of
the connection at the same time. Since it would be done atomically,
it is as committed to serving the new connection as a process that has
returned from accept() with a decriptor in the traditional way would be
committed. If that process now dies, the connection is disconnected
either way, but at least we don't have "off by one in the queue".
I recall reading that perhaps NetBSD and/or OpenBSD also support the
kqueue() and kevent() syscalls.
I would like to see FreeBSD (and others) extend kevent() so that it can
specifically handle an atomic acceptance of a new connection when the
caller specifies it (an EVFILT_ACCEPT filter). I would also like to see
Linux adopt and implement the kqueue() and kevent() syscalls as they do
seem to be very powerful and I've already found at least one use for
them in an unrelated matter.
In the mean time, it looks like I will have to do some crazy things to
implement my way around the "thundering herd problem in select()". The
idea I have right now is to pre- fork() the children from a parent and
let them proceed to initialize (in my case they need to do some network
connections and send and receive data just to initialize to be ready to
service connections coming in to them), and have the parent doing all
the listening for new connections. Once at least one child indicates
it is ready to go, the parent adds the listen socket(s) to its list of
selected descriptors. When the parent wakes up on this descriptor, it
will accept() the connection, then send that descriptor to the child via
the sendmsg() call. I will also have the parent fork() one special
child before all this so each service child has a pipe to it. When the
service child is done with a connection, it will call sendmsg() to pass
the descriptor of the connection it is done with to the special child
which will collect all these connections and carry out the lingering
shutdown() close semantics, allowing the service child to quickly exit
and the parent to fork a new one.
I've also considered having the parent not call accept() when it wakes
up for the selected listen socket, and send a message to the child to
do this. The problem with this approach is that while the child is
waking up to discover the message, the parent will try to go back to
sleep. If the connection is still in the queue it will wake up again
if it still selects the listen socket. If it does not select the listen
socket for at least a short while, then it will not be able to rapidly
respond to a second connection to pass it to a second child. So it does
seem the parent needs to do the accept() in this model.
I rule out for my case having the parent do fork() after accept() since
these children need to do some lengthy (several seconds) initialization
before being able to respond to the connection. For most other cases
this may well be a good framework model to use (so don't rule it out in
your own just because I have ruled it out in mine).
--
=================================================================
| Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ |
| [EMAIL PROTECTED] | Texas, USA | http://phil.ipal.org/ |
=================================================================
------------------------------
From: "Michael Faurot" <[EMAIL PROTECTED]>
Subject: Re: is there a way to password protect LILO?
Date: 27 May 2001 17:22:46 GMT
Anonymous <[EMAIL PROTECTED]> wrote:
: Is there a way to password protect LILO
: so some random person can't just
: type
: linux 1
: at the prompt and get into single user mode?
man 5 lilo.conf
--
==============================================================================
Michael | mfaurot | Bedfellows make strange politicians.
Faurot | atww.org |
------------------------------
From: Zhiyong Xu <[EMAIL PROTECTED]>
Subject: Debug kernel
Date: Sun, 27 May 2001 16:00:28 -0400
Hi,
How can I debug a testing kernel, I modified kernel, when I use
my new kernel, sometiimes I got NULL pointer error, but at that time,
I can do nothing, so it is hard for me to find the reason, is it
possible to debug a running kernel?
Thanks
------------------------------
** FOR YOUR REFERENCE **
The service address, to which questions about the list itself and requests
to be added to or deleted from it should be directed, is:
Internet: [EMAIL PROTECTED]
You can send mail to the entire list by posting to the
comp.os.linux.development.system newsgroup.
Linux may be obtained via one of these FTP sites:
ftp.funet.fi pub/Linux
tsx-11.mit.edu pub/linux
sunsite.unc.edu pub/Linux
End of Linux-Development-System Digest
******************************
