Linux-Development-Sys Digest #746, Volume #8 Thu, 24 May 01 13:13:11 EDT
Contents:
Re: process monitoring (James Stephens)
Re: Trouble using large amount of memory with Redhat 7 ("Karl Heyes")
Re: thundering-herd vs wake-one ("Karl Heyes")
Re: GNU_SOURCE (Martin von Loewis)
Re: thundering-herd vs wake-one ([EMAIL PROTECTED])
Problem with sigsetjmp (Frederico Faria)
Potentially dangerous ("Mike Austin")
Re: thundering-herd vs wake-one ([EMAIL PROTECTED])
Re: thundering-herd vs wake-one (Linus Torvalds)
Re: Kernel 2.4.4 and Adaptec 1480 Slim SCSI Desperate HELP Please. David Hinds?
(bill davidsen)
Re: Major/Minor numbers under 2.4? (bill davidsen)
How can I pipe the output of a program? (Iassen Hristov)
Re: Self Installing CD ("Wayne")
Can't build 'compat-binutils' RPM in RH6.2! (Dmitry)
Re: thundering-herd vs wake-one ("Karl Heyes")
Re: How can I pipe the output of a program? (Lew Pitcher)
----------------------------------------------------------------------------
From: James Stephens <[EMAIL PROTECTED]>
Subject: Re: process monitoring
Date: Thu, 24 May 2001 08:46:12 -0400
The following url is the home page for the sysstat package (responsible for
shell utils like 'sar' and 'iostat' - really good system monitoring tools,
giving disk io usage, socket io usage, memory usage...). Go to downloads
and grab the source... be prepared for sscanf() and the like. While the
code tracks only system usages (/proc/stat) you could apply the similar
logic to /proc/self/stat (to monitor yourself). Refer to the
linux-src/Documentation/proc.txt for a key to fields AND the proc man page.
Also look into the UNIX-98 compliant getrusage() routine
http://perso.wanadoo.fr/sebastien.godard/
"������" wrote:
> I want to develop program that is monitoring process at linux platform..
> I know that I have to use /proc filesystem.
>
> I want to get sample code
> I use C language..
>
> Please...
------------------------------
From: "Karl Heyes" <[EMAIL PROTECTED]>
Subject: Re: Trouble using large amount of memory with Redhat 7
Date: Thu, 24 May 2001 15:00:24 +0100
In article <9ehsvq$7h2$[EMAIL PROTECTED]>, "J. P. Montgomery"
<[EMAIL PROTECTED]> wrote:
> I will most likely upgrade to 2.2.19 or recompile the kernel. When looking
> at present kernel directory and my config files, I note that the stock
> kernels in Redhat 7.0 are compiled with 1G and no BIGMEM. I would assume
> that I need to turn on 2G ... but am unsure of BIGMEM. Also the parameters
> or patches to make for jobs bigger than 2G (i.e. with 1.5G of ram, I would
> like to be able to use 1.5 G of the swap occasionally for a job size of 3G).
> Any pointers to documentation.
2G setting yes, not BIGMEM. Thats for over 4G.
A useful site is www.linux-mm.org
> Also, I note that there are a 'lot' of kernels now adays ... 386, 586, 686,
> etc. I assume that these have slightly different options turned on. I
> assume that with an AMD K7 I really want to use the 686 ... or is there a
> kernel more tuned to the K7 ... or is this all configurable when compiling
> the kernel (I haven't done this is a few years ... hence the questions).
I'm 100% on this as I don't have a K7 but any option should be ok, but the
686 option should be fine. It adds support for the registers specific to the
class of CPU like tsc etc.
karl.
------------------------------
From: "Karl Heyes" <[EMAIL PROTECTED]>
Crossposted-To: comp.unix.bsd.freebsd.misc
Subject: Re: thundering-herd vs wake-one
Date: Thu, 24 May 2001 15:15:12 +0100
In article <sJ4P6.3259$[EMAIL PROTECTED]>, "Cameron Kerr"
<[EMAIL PROTECTED]> wrote:
> In article <[EMAIL PROTECTED]>, "Unknown"
> <[EMAIL PROTECTED]> wrote:
>
>> Note: This is crossposted on FreeBSD and Linux because I'm asking this
>> question in the context of both of these systems. Linux recently added
>> "wake-one" capability in version 2.4 (patches exist for 2.2), and from what
>> I read so far, FreeBSD has had this well before
> Is this related to "wake on LAN"? If so, you should check out www.scyld.com.
> If not, what's wake-one? [snip]
No this is related to the select/poll call and how they don't scale to large
numbers. I don't know about FreeBSD but linux woke up all listeners even
through only one could answer the call, hence a thundering herd. Apache is
using a semaphore to restrict the number of listeners to reduce the problem.
wake-one is whats implemented in 2.4 so that only one listener is awoken and
therefore the scaling is better. There are better solutions to select/poll
as they suffer from the array scanning on large scale systems.
The actual difference in wake-one is purely from a kernel point of view, not
a userspace program. The select/poll however is a standard API that is
limiting and thats why web engines like TUX perform considerably better.
It wouldn't suprise me if FreeBSD had solved the issue before as they tend to
be used for high end systems, so the problem was encountered before linux saw
it.
karl.
------------------------------
From: Martin von Loewis <[EMAIL PROTECTED]>
Subject: Re: GNU_SOURCE
Date: 24 May 2001 16:34:25 +0200
"Ivor Cox" <[EMAIL PROTECTED]> writes:
> Looking at the header file pthread.h I notice conditional compiles based on
> __USE_GNU, __UNIX_98(?) and so on. Should I be specifying any of these to
> use pthread in linux?
No. By default, gcc will arrange defines so that all functions are
available. If -ansi is given to gcc, it will restrict the library to
provide only ISO C functions. In that case, you need to define the
proper *_SOURCE macro.
> What are they for
The __USE_* macros are not meant for direct use by the
programmer. Instead, the programmer controls which library functions
are available by defining a _SOURCE macro, e.g. _POSIX_SOURCE or
_XOPEN_SOURCE. This, in turn, will set __USE macros. See
/usr/include/features.h for a list of supported _SOURCE macros, and
the resulting __USE_ macros.
The default is _GNU_SOURCE if no option is given to gcc, and
__STRICT_ANSI__ if -ansi is given.
> and how do they affect what pthread does?
It does not at all affect what pthread does, it only affects what data
types and functions are available during compilation.
Of course, if you use a function without prototype, the compiler may
infer incorrect calling conventions for functions. I recommend to
compile your source with -Wall, and get rid of all warnings. That may
require including more header files, and perhaps setting a _SOURCE
macro.
Regards,
Martin
------------------------------
From: [EMAIL PROTECTED]
Crossposted-To: comp.unix.bsd.freebsd.misc
Subject: Re: thundering-herd vs wake-one
Date: 24 May 2001 14:28:48 GMT
In comp.os.linux.development.system Cameron Kerr <[EMAIL PROTECTED]> wrote:
| In article <[EMAIL PROTECTED]>, "Unknown"
| <[EMAIL PROTECTED]> wrote:
|
|> Note: This is crossposted on FreeBSD and Linux because I'm asking this
|> question in the context of both of these systems.
|>
|> Linux recently added "wake-one" capability in version 2.4 (patches exist
|> for 2.2), and from what I read so far, FreeBSD has had this well before
|
| Is this related to "wake on LAN"? If so, you should check out
| www.scyld.com. If not, what's wake-one?
No it's not.
I'm trying to get an exact definition of "wake-one". The approximate
definition is that it avoids the issue of having multiple processes
waking up in a race to do accept() to take the next connection. It's
the opposite of "wake-many", also know as "thundering herd" or "the
stampede effect".
I may have to start posting what appear to be "informative" articles,
but are explicitly and intentionally incorrect. That is a tactic I
have found in the past brings out the people who actually know and
don't otherwise have an interest and being informative. But they will
jump on the opportunity to flame someone for being "stupid".
I suspect there are very few people that really understand how it works.
It may end up being necessary to spend a lot of time auditing the code
of FreeBSD and Linux to find out exactly what they really do. I just
don't want commit all that time if someone who already knows can provide
a document that explains it. Unfortunately, most of the people who do
know are probably the ones who would rather code than document.
Not considering "wake-one" semantics, the whole definition of the
socket API isn't all that precise, but usually doesn't need to be.
But in some unusual cases, a precise definition is needed. I think
that "wake-one" is one of those cases.
--
=================================================================
| Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ |
| [EMAIL PROTECTED] | Texas, USA | http://phil.ipal.org/ |
=================================================================
------------------------------
From: [EMAIL PROTECTED] (Frederico Faria)
Subject: Problem with sigsetjmp
Date: 24 May 2001 08:24:23 -0700
Hi,
I use alarm() and sigsetjmp() as timeout mechanism to CORBA bind() call.
My alarm handler with siglongjmp is called at all the time because my network is down.
After any time I begin receive error messages as ( I saw in strace):
socket(....) = -1 EMFILE ( Too many open files )
open(/etc/hosts) = -1 EMFILE ( Too many open files )
.....
.......
My doubt is :
After of siglongjmp the recourses allocated by interrupted
system calls ( bind() , connect(), socket() ) are not free ?
How can I resolve it ?
Could I use an alarmHandler() without the siglongjmp() ?
What would be the risk ?
Thank you,
Faria.
------------------------------
From: "Mike Austin" <[EMAIL PROTECTED]>
Subject: Potentially dangerous
Date: Thu, 24 May 2001 11:38:26 -0400
Greetings,
I'm doing something in a kernel module that I'm not sure is exactly
safe, or what can be done to make it safe. Essentially, I have a block
device which is treated as a hard drive. I want it to show up in
/proc/partitions, so all I have done is modify gendisk_head to include my
disk in the list. After clean up, I remove it from the list. I read
somewhere that this list should not be modified by a module, but we need to
support LVM. Are there any other considerations that I should make? Is
there anything else that needs to be called or updated when doing this so
that the OS remains stable? (I haven't seen any problems yet, but that
doesn't mean their not there. I've looked through the kernel source, and
haven't found a reason there should be a problem but I would like
verification by others more comfortable with the kernel itself).
Thank you for your assistance,
Mike Austin
------------------------------
From: [EMAIL PROTECTED]
Crossposted-To: comp.unix.bsd.freebsd.misc
Subject: Re: thundering-herd vs wake-one
Date: 24 May 2001 15:11:48 GMT
In comp.unix.bsd.freebsd.misc Karl Heyes <[EMAIL PROTECTED]> wrote:
| wake-one is whats implemented in 2.4 so that only one listener is awoken and
| therefore the scaling is better. There are better solutions to select/poll
| as they suffer from the array scanning on large scale systems.
What better solutions do you have or suggest?
| The actual difference in wake-one is purely from a kernel point of view, not
| a userspace program. The select/poll however is a standard API that is
| limiting and thats why web engines like TUX perform considerably better.
I disagree. Because the wake-one change is a technical violation of the
select() semantics, there really can be subtle effects. What happens if
the one waking process exits or faults before it does accept()? Will the
kernel wake another process to deal with it? Would the parent process
have to deal with this (it will have to know that wake-one is in effect
to do this)? What happens when the next connection comes in when one
connection still sits in the queue ... will the kernel wake just one, or
wake as many as are in the queue?
| It wouldn't suprise me if FreeBSD had solved the issue before as they tend to
| be used for high end systems, so the problem was encountered before linux saw
| it.
Exactly. This is one of the reasons I want to understand exactly how
FreeBSD does it. This may well be the platform of choice, even if
Linux 2.4 also does it. There may still be differences in the details
of how it works that means one or the other will work out better for a
given process strategy. Or maybe a process strategy decision needs to
be made to function on each system.
BTW, for those ready to say "read the source", the issue is about the
subtle details, not the overall picture (though that is useful, too).
To understand this requires intimate understanding of how the kernel
does work, and this makes it not practical since achieving that level
of kernel understanding will take quite a length of time.
--
=================================================================
| Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ |
| [EMAIL PROTECTED] | Texas, USA | http://phil.ipal.org/ |
=================================================================
------------------------------
From: [EMAIL PROTECTED] (Linus Torvalds)
Crossposted-To: comp.unix.bsd.freebsd.misc
Subject: Re: thundering-herd vs wake-one
Date: 24 May 2001 08:55:00 -0700
In article <[EMAIL PROTECTED]>, <[EMAIL PROTECTED]> wrote:
>
>Linux recently added "wake-one" capability in version 2.4 (patches exist
>for 2.2),
No.
2.2 does indeed have "wake_one()" patches, but the whole concept is in
my opinion pretty broken. The waker does not know what the sleepers are
doing, and cannot safely wake up just one sleeper.
What 2.4.x has is the notion of the _waiters_ (who _do_ know what they
are doing) saying "I'm an exclusive waiter, once you wake me up there's
no point in waking any other exclusive waiters". That's very different
from "wake_one()", although it also avoids the herd behaviour when used
correctly.
>What I am asking for is for information that specifically describes how
>wake-one is supposed to work, including things like what circumstances it
>offers no advantages, and what circumstances it does. I'm also interested
>in compatibility issues with programs written with the understanding that
>wake-many would be the way.
There are no compatibility issues.
Any waiters that have "wake-many" semantics (the classic example of this
is "select()") will simply not claim to be exclusive, and behaviour does
not change (even if there are exclusive waiters _too_ - exclusive
waiters are only exclusive of each other).
So in this kind of schenario a wakeup is a wakeup, and the person who
does the wakeup doesn't know (and _shouldn't_ know) what the sleeper
actually intends to do.
Linus
------------------------------
From: [EMAIL PROTECTED] (bill davidsen)
Subject: Re: Kernel 2.4.4 and Adaptec 1480 Slim SCSI Desperate HELP Please. David
Hinds?
Date: Thu, 24 May 2001 16:08:08 +0000 (UTC)
In article <9duqdo$[EMAIL PROTECTED]>,
E-mu <[EMAIL PROTECTED]> wrote:
| It works if I choose aic7xxx as (m) rather than driver into the kernel.
|
|
| But shouldn't it work as (y) in "make menuconfig" as well?
It should, in the "I would like it to" sense, but may not in the "you
can expect it to" sense. I'm going to go back and try this trick with a
system I'm using, although I then have to insert the module in the
initrd image, since that's the boot device. Bother!
--
bill davidsen <[EMAIL PROTECTED]> CTO, TMR Associates, Inc
At LinuxExpo Sun was showing Linux applications running on Solaris.
They don't get it, the arrow points the other way. There's a reason why
there's no SolarisExpo, Solaris is a tool; Linux is a philosophy, a
religion, a way of life, and only incidentally an operating system.
------------------------------
From: [EMAIL PROTECTED] (bill davidsen)
Subject: Re: Major/Minor numbers under 2.4?
Date: Thu, 24 May 2001 16:11:22 +0000 (UTC)
In article <9dsklp$[EMAIL PROTECTED]>,
David Hawkins <[EMAIL PROTECTED]> wrote:
| I'm updating a PCI device driver for a custom board for use
| in Linux 2.4, and I'm a little confused about where the device
| driver major number is now associated with the
| device.
|
| Under 2.2, during init_module, you call register_chrdev
| with your major number. However in 2.4, you use
| the pci_register_device for the some-what
| equivalent operation.
|
| I haven't started hacking away at the code yet,
| I'm looking through kernel sources, since I haven't
| really seen much other than pci.txt in the Documentation
| directory.
|
| Anyway, I just couldn't see how to align the mknod
| parameters with the 2.4-style device drivers.
|
| If anyone can give me some example code, or
| perhaps just explain whats happening, then I'd
| be a happy driver writer!
Looking at the code for devfs, it would seem that the major device
numbers are rather more assigned on the fly. I don't know if this gives
you somewhere to start research, I certainly haven't chased it enough to
explain it to you!
--
bill davidsen <[EMAIL PROTECTED]> CTO, TMR Associates, Inc
At LinuxExpo Sun was showing Linux applications running on Solaris.
They don't get it, the arrow points the other way. There's a reason why
there's no SolarisExpo, Solaris is a tool; Linux is a philosophy, a
religion, a way of life, and only incidentally an operating system.
------------------------------
From: Iassen Hristov <[EMAIL PROTECTED]>
Subject: How can I pipe the output of a program?
Date: Thu, 24 May 2001 12:19:02 -0400
There are some program (like isql from the unixODBC suite) that write
their output to some other place than stdout, I would guess stderr.
How can I pipe the output of such app?
P.S. I could not seem to find a more appropriate NG for this question. I
would appreciate if you can point me to one.
------------------------------
From: "Wayne" <[EMAIL PROTECTED]>
Subject: Re: Self Installing CD
Date: Thu, 24 May 2001 11:15:55 -0500
> If you will tell us how you partition your harddisk,
> how you mount the partitions, and how your
> /etc/lilo.conf looks we will be able to help you
> much better.
> If you will tell us how you partition your harddisk,
> how you mount the partitions, and how your
> /etc/lilo.conf looks we will be able to help you
> much better.
error = first boot sector doesn't have valid LILO signature
executing /sbin/lilo -r /install <-- where hda is mounted
1 harddisk formated 1 - 128M swap the rest to the files
lilo.cong =
boot = /dev/hda
timeout = 50
prompt
default = linux
vga = normal
read-only
map = /boot/map
install = /boot/boot.b
image = /boot/vmlinuz
label = linux
vga = normal
root = /dev/hda1
------------------------------
From: [EMAIL PROTECTED] (Dmitry)
Crossposted-To: comp.os.linux.redhat
Subject: Can't build 'compat-binutils' RPM in RH6.2!
Date: 24 May 2001 09:33:41 -0700
Hi,
I am trying to rebuild all RPMS of RedHat6.2.
I've install all RPMS and got all SRPMS.
But now I can't rebuild
compat-binutils-5.2-2.9.1.0.23.1.src.rpm
packet. I see this protocol tail:
make[1]: Entering directory
`/usr/src/redhat/BUILD/binutils-2.9.1.0.23/binutils'
gcc -DHAVE_CONFIG_H -I. -I. -I. -D_GNU_SOURCE -I. -I. -I../bfd
-I./../bfd -I./../include -g -O2 -c size.c
gcc -DHAVE_CONFIG_H -I. -I. -I. -D_GNU_SOURCE -I. -I. -I../bfd
-I./../bfd -I./../include -g -O2 -c bucomm.c
gcc -DHAVE_CONFIG_H -I. -I. -I. -D_GNU_SOURCE -I. -I. -I../bfd
-I./../bfd -I./../include -g -O2 -c version.c
gcc -DHAVE_CONFIG_H -I. -I. -I. -D_GNU_SOURCE -I. -I. -I../bfd
-I./../bfd -I./../include -g -O2 -c filemode.c
/bin/sh ./libtool --mode=link gcc -g -O2 -o size size.o bucomm.o
version.o filemode.o ../bfd/libbfd.la ../libiberty/libiberty.a
mkdir .libs
gcc -g -O2 -o .libs/size size.o bucomm.o version.o filemode.o
-Wl,--rpath -Wl,/usr/i386-glibc20-linux/lib ../bfd/.libs/libbfd.so
../libiberty/libiberty.a
/lib/libc.so.6: undefined reference to `[EMAIL PROTECTED]'
/lib/libc.so.6: undefined reference to `[EMAIL PROTECTED]'
/lib/libc.so.6: undefined reference to
`[EMAIL PROTECTED]'
/lib/libc.so.6: undefined reference to `[EMAIL PROTECTED]'
...
What's wrong?
(glibc was installed correctly: libc.so.6 the same ver. as
ld-linux.so.2.)
Thanks in advance.
------------------------------
From: "Karl Heyes" <[EMAIL PROTECTED]>
Crossposted-To: comp.unix.bsd.freebsd.misc
Subject: Re: thundering-herd vs wake-one
Date: Thu, 24 May 2001 17:59:56 +0100
In article <[EMAIL PROTECTED]>, "Unknown"
<[EMAIL PROTECTED]> wrote:
> In comp.unix.bsd.freebsd.misc Karl Heyes
> <[EMAIL PROTECTED]> wrote: | wake-one is whats
> implemented in 2.4 so that only one listener is awoken and | therefore the
> scaling is better. There are better solutions to select/poll | as they
> suffer from the array scanning on large scale systems.
> What better solutions do you have or suggest?
If I remember correctly TUX uses sigio to process requests, although there is
other things like zerocopy TCP being thrown in for good measure.
karl.
------------------------------
From: [EMAIL PROTECTED] (Lew Pitcher)
Subject: Re: How can I pipe the output of a program?
Reply-To: [EMAIL PROTECTED]
Date: Thu, 24 May 2001 17:03:36 GMT
On Thu, 24 May 2001 12:19:02 -0400, Iassen Hristov
<[EMAIL PROTECTED]> wrote:
>There are some program (like isql from the unixODBC suite) that write
>their output to some other place than stdout, I would guess stderr.
>
>How can I pipe the output of such app?
someapp 2>file_to_put_stderr_in
Additionally, in Bash, the redirection can be n> where n is any file
descriptor number. So, 3>file_to_put_FD3_in is legal in bash. By
convention, file descriptor 0 is stdin, 1 is stdout and 2 is stderr.
The use of other file descriptors is dependant on the program.
>P.S. I could not seem to find a more appropriate NG for this question. I
>would appreciate if you can point me to one.
Lew Pitcher, Information Technology Consultant, Toronto Dominion Bank Financial Group
([EMAIL PROTECTED])
(Opinions expressed are my own, not my employer's.)
------------------------------
** FOR YOUR REFERENCE **
The service address, to which questions about the list itself and requests
to be added to or deleted from it should be directed, is:
Internet: [EMAIL PROTECTED]
You can send mail to the entire list by posting to the
comp.os.linux.development.system newsgroup.
Linux may be obtained via one of these FTP sites:
ftp.funet.fi pub/Linux
tsx-11.mit.edu pub/linux
sunsite.unc.edu pub/Linux
End of Linux-Development-System Digest
******************************
