Linux-Development-Sys Digest #775, Volume #8 Wed, 6 Jun 01 19:13:21 EDT
Contents:
Re: GNU_SOURCE (Martin von Loewis)
Re: how can I get a change to "official" linux? (Martin von Loewis)
e2compress on kernel 2.4 (Pierre Peiffer)
Re: how can I get a change to "official" linux? ("Karl Heyes")
Re: Getting CURRENT load average (Kasper Dupont)
Re: Stack size of kernel_thread (Kasper Dupont)
Re: Question (Kasper Dupont)
Re: how to force a core file dump ? (Kasper Dupont)
Re: mmap not working(2) - kernel bug?! (Kasper Dupont)
Re: memory usage info (Kasper Dupont)
Re: problem with compiling as a module (Kasper Dupont)
Re: mlock suggestions (Kasper Dupont)
Re: mmap not working(2) - kernel bug?! (Dragan Cvetkovic)
Re: module doesn't load in kernel 2.2.x SMP
Linux-2.4.5
information (garibaldi)
Re: mlock suggestions (Paul Rubin)
Re: Linux-2.4.5 ("Karl Heyes")
mmap+/dev/zero+madvise(WILLNEED) fails? (M.)
Re: thundering herd problem: the REAL scoop ([EMAIL PROTECTED])
----------------------------------------------------------------------------
From: Martin von Loewis <[EMAIL PROTECTED]>
Subject: Re: GNU_SOURCE
Date: 06 Jun 2001 12:54:40 +0200
"Ivor Cox" <[EMAIL PROTECTED]> writes:
> Where do I find out/read about the use/meaning etc for these options?
In /usr/include/features.h.
Regards,
Martin
------------------------------
From: Martin von Loewis <[EMAIL PROTECTED]>
Subject: Re: how can I get a change to "official" linux?
Date: 06 Jun 2001 13:51:47 +0200
Eric Taylor <[EMAIL PROTECTED]> writes:
> So, is there in fact an "official" linux and how would I propose
> such an enhancement?
Others have already answered, but let me try also.
Yes, there is an "official" Linux kernel; this is the one released by
Linus Torvalds. John Reiser is right that it will take many years from
the moment you propose a feature to the time the typical Redhat user
gets it installed by default.
You can propose it here, or on the linux-kernel list. Unless you
provide a *very* good rationale, and a patch, nothing will likely
happen. Of course, if you provide funding for a kernel hacker, that
might be another option.
> It would allow for the (re)-positioning of the .so libraries so I
> can have a nearly 3 gig (contiguous) array.
If you want to reposition your shared libraries, I suggest that
changing/improving ld-linux.so is a better way. Alternatively, setting
the preferred load address in the shared libraries might be good enough.
If you have a single application that needs contiguous memory, I
suggest that link a static binary, and set the load address to some
small value.
Regards,
Martin
------------------------------
From: [EMAIL PROTECTED] (Pierre Peiffer)
Subject: e2compress on kernel 2.4
Date: 6 Jun 2001 04:52:09 -0700
Hello everybody,
Does anybody know if someone is working on porting e2compress on kernel 2.4.X ?
Thanks,
Pierre
------------------------------
From: "Karl Heyes" <[EMAIL PROTECTED]>
Subject: Re: how can I get a change to "official" linux?
Date: Wed, 06 Jun 2001 13:19:19 +0100
In article <[EMAIL PROTECTED]>, "Villy Kruse"
<[EMAIL PROTECTED]> wrote:
> But it should be possible to distribute kernel modules plus patch files that
> would implement this feature. If public deman and/or Linus thinks this
> feature is a nice and usefull feature it might go into the official kernel.
> If you check the various distributions, their kernels often have extra
> features and bug fixes added when compared with the standard kernel. Some
> features started out that way, and up to at least kernel 2.2 the pcmcia
> feature was not included in the official kernel; don't know if it is
> included now in kernel 2.4. Pcmcia is a nice feature, and on some systems
> (laptops) even a vital feature. Villy
It would be relatively easy to provide a module to add an entry into proc for
getting the value but as for changing it on the fly I suspect that would be
impossible.
karl.
------------------------------
From: Kasper Dupont <[EMAIL PROTECTED]>
Subject: Re: Getting CURRENT load average
Date: Wed, 06 Jun 2001 13:11:07 +0000
Dennis Jenkins wrote:
>
> Cameron Kerr wrote:
> >
> > Hello, I've been code surfing all afternoon trying to find
> > a way to do the following...
> >
> > I need to make a program to retrieve the current laod average of the
> > system (This is for cluster computing, I want to find the least busiest
> > machine). The best I can find is the look at the stats in /proc/loadavg.
> >
> > However, the information there is rather undesirable, since its a
> > sliding/decaying average over at least 1 minute. I would like it so that
> > after the load had gone, the information I am able to give is more
> > accurate, rather than having to wait for the load average to dacay.
> >
> > Looking at the kernel source, I'm thinking about the best I could do
> > would be to write a module or supplement the kernel so that it calculates
> > the average over a smaller timeslice, say 5 seconds.
>
> Further more, how does 'top' calculate CPU busy percentage? I'm going
> to consult the source soon, I guess...
>
> > Is this the best way to proceed, or is there something better?
> >
> > Thanks in advance,
> > --
> > Cameron Kerr -- cameron.kerr @ paradise.net.nz
> > Praise Slackware, our baud and saviour!
>
> Slackware rules!
>
> --
> [EMAIL PROTECTED] Universal Savings Bank.
> Security Administrator, Unix Administrator, Alpha Geek
>
> The three most dangerous things are a programmer with a soldering
> iron, a manager who codes, and a user who gets ideas.
There is a difference between the CPU use percentage
and the load.
If a number of processes uses all the CPU time they
can get the CPU usage will be 100%. You will not be
able to tell the number of processes from the usage.
The load OTOH will tell you how many processes
actually want to use the CPU.
Another important point to notice is that the load
average tries to measure the load on both CPU and
disk. This is done by counting the number of
processes that are runing, ready to run or in a
noninterruptible sleep. Usually a process in a
noninterruptible sleep is waiting for a disk I/O
to complete, but it is not always a good meassure.
--
Kasper Dupont
------------------------------
From: Kasper Dupont <[EMAIL PROTECTED]>
Subject: Re: Stack size of kernel_thread
Date: Wed, 06 Jun 2001 13:20:43 +0000
Hanspeter Halle wrote:
>
> Hi,
> does anybody know how to increase the memory avaliable for the stack of a
> kernel_thread (kerbnel 2.2 and 2.4)?
>
> Thanks, Hanspeter
On i386 you have to change two files in the
directory include/asm-i386.
In processor.h you must change the line
#define THREAD_SIZE (2*PAGE_SIZE)
In current.h you must change the line
__asm__("andl %%esp,%0; ":"=r" (current)
: "0" (~8191UL));
Notice that the size must be a power of 2,
and that it may not be smaller than 4096.
(But making it smaller than the default of
8192 is asking for trouble.)
After changing these files recompile your
kernel and _ALL_ kernel modules.
There might be other files to change that
I don't know about.
--
Kasper Dupont
------------------------------
From: Kasper Dupont <[EMAIL PROTECTED]>
Subject: Re: Question
Date: Wed, 06 Jun 2001 13:29:25 +0000
[EMAIL PROTECTED] wrote:
>
> Hello!
>
> Excuse me, is this newsgroup a suitable place to ask questions about kernel
> programming?
>
> --
> Alexey
Yes.
--
Kasper Dupont
------------------------------
From: Kasper Dupont <[EMAIL PROTECTED]>
Subject: Re: how to force a core file dump ?
Date: Wed, 06 Jun 2001 13:46:06 +0000
Christophe Dore wrote:
>
> Hi,
>
> I'd like to write handlers error signal (such as SIGSEV, SIGFPE, ...),
> but I would like to keep core dump.
>
> How can I force a program to dump a core? (especially in signal
> handlers ?)
>
> Thanks,
>
> C.Dore
Get the sourcecode of the init program. It does
something very interesting with core dumps that
might help you get some ideas.
--
Kasper Dupont
------------------------------
From: Kasper Dupont <[EMAIL PROTECTED]>
Crossposted-To:
hr.comp.os.linux,comp.os.linux.development.apps,comp.os.linux.development,hr.comp.programiranje.c
Subject: Re: mmap not working(2) - kernel bug?!
Date: Wed, 06 Jun 2001 14:07:12 +0000
Zeljko Vrba wrote:
>
> if(lseek(fd, sz-1, SEEK_SET) < 0) {
> perror("lseek");
> exit(1);
> }
> write(fd, &c, 1);
>
That can be done a lot simpler using ftruncate()
ftruncate(fd,sz);
--
Kasper Dupont
------------------------------
From: Kasper Dupont <[EMAIL PROTECTED]>
Subject: Re: memory usage info
Date: Wed, 06 Jun 2001 14:11:41 +0000
Zhiyong Xu wrote:
>
> Hi,
> I doing some research on linux kernel, I modified some of
> buffer cache system. But it seems some problem occurs , I want to know
> how can I get info of current memory usage and display? It seems
> I consumed all memory and no process so how can I know current memory
> condition?
Enable the "Magic SysRq key" option when you
configure your kernel. Now you can press
Alt+SysRq+H to get a list of options. This
only works when in textmode, not much use
under X.
--
Kasper Dupont
------------------------------
From: Kasper Dupont <[EMAIL PROTECTED]>
Subject: Re: problem with compiling as a module
Date: Wed, 06 Jun 2001 14:14:53 +0000
Srinivas Vedula wrote:
>
> gcc -c Wall -DMODULE -D__KERNEL__ -DLINUX foo.c
>
Adding -I/usr/src/linux/include or
-I/usr/src/linux-2.2.17/include might
help.
--
Kasper Dupont
------------------------------
From: Kasper Dupont <[EMAIL PROTECTED]>
Subject: Re: mlock suggestions
Date: Wed, 06 Jun 2001 14:19:24 +0000
Paul Rubin wrote:
>
> 1) I'd like to request that the mlock system call always return
> success (even for non-root processes) when asked to lock writeable
> pages, if the system has paging/swapping disabled. After all, if
> there's no paging, mlock guarantee that the relevant block won't be
> paged. All such pages are already automatically locked. This avoids
> annoying warning messages from programs like GnuPG. I guess some
> read-only executable code pages might still not be locked in such an
> environment, but it's ok for mlock to return failure for those pages.
Sounds reasonable, there is just one minor problem.
If swaping is enabled at a later time we might be
breaking the promises we earlier made to a program.
>
> 2) A more controversial change that might require some design
> examination would be to allow non-root processes to lock a few pages
> despite being non-root. GnuPG (again) and similar programs could make
> use of that. There could be some per-user limit on the number of
> simultaneous locked pages, as well as a per-process limit, to stop
> unprivileged users from grabbing infinite RAM. I'd like to request
> that this idea be considered, but I'm not sure what the exact outcome
> I'd favor.
I believe some UNIX systems already does this so
why shouldn't Linux be able to do that too.
>
> Thanks
--
Kasper Dupont
------------------------------
Crossposted-To:
hr.comp.os.linux,comp.os.linux.development.apps,comp.os.linux.development,hr.comp.programiranje.c
Subject: Re: mmap not working(2) - kernel bug?!
From: Dragan Cvetkovic <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Date: 06 Jun 2001 10:52:41 -0400
Kasper Dupont <[EMAIL PROTECTED]> writes:
> Zeljko Vrba wrote:
> >
> > if(lseek(fd, sz-1, SEEK_SET) < 0) {
> > perror("lseek");
> > exit(1);
> > }
> > write(fd, &c, 1);
> >
>
> That can be done a lot simpler using ftruncate()
> ftruncate(fd,sz);
Yes, except that the behaviour is not guaranteed. As per Linux ftruncate(2)
man page (on my Debian Potato 2.2r3 system):
Truncate causes the file named by path or referenced by fd
to be truncated to at most length bytes in size. If the
file previously was larger than this size, the extra data
is lost. If the file previously was shorter, it is
unspecified whether the file is left unchanged or is
extended. In the latter case the extended part reads as
zero bytes. With ftruncate, the file must be open for
writing.
POSIX is also rather vague here (from the latest draft):
If the file previously was smaller than this size, ftruncate( )
shall either increase the size of the file or fail.
Bye, Dragan
--
Dragan Cvetkovic,
To be or not to be is true. G. Boole
------------------------------
From: [EMAIL PROTECTED] ()
Subject: Re: module doesn't load in kernel 2.2.x SMP
Date: Wed, 06 Jun 2001 19:10:46 -0000
In article <[EMAIL PROTECTED]>,
Jeseem <[EMAIL PROTECTED]> wrote:
> My kernel module has versioning disabled. (the kernel has versioning
>enabled.)
Wouldn't that be a big problem?
--
http://www.spinics.net/linux/
------------------------------
From: <[EMAIL PROTECTED]>
Subject: Linux-2.4.5
Date: Wed, 06 Jun 2001 19:30:09 -0000
I have been trying to upgrade from RH2.2.12-20 to kernel 2.4.5. I am
running an AMD-K6-2 processor with 64mb ram, a CreativeLabs PCI 512 sound
card, and a Lynksys NIC. The kernel freezes at boot. It only gets as for as
Uncompressing linux.... Ok boot. This not verbatum but close. I tried 2.4.3
previously with the same problem. The pathes were unsuccessful.
B.M. Richard
--
Posted via CNET Help.com
http://www.help.com/
------------------------------
From: garibaldi <[EMAIL PROTECTED]>
Subject: information
Date: Wed, 06 Jun 2001 19:30:13 -0000
Hello:
I�m looking for the progam of Vicent Larsen (autorun.exe)
anybody know where can I find it?. It�s a very good pwl.crack
progam.
--
Posted via CNET Help.com
http://www.help.com/
------------------------------
From: Paul Rubin <[EMAIL PROTECTED]>
Subject: Re: mlock suggestions
Date: 06 Jun 2001 12:55:09 -0700
Kasper Dupont <[EMAIL PROTECTED]> writes:
> Sounds reasonable, there is just one minor problem.
> If swaping is enabled at a later time we might be
> breaking the promises we earlier made to a program.
That's right, I forgot it was possible to turn on swapping without a
reboot. So I guess the feature should be controlled by a boot time
parameter. The parameter would make mlock always return true, and
also disable swapping until the next reboot. Hopefully this isn't
getting too messy yet.
------------------------------
From: "Karl Heyes" <[EMAIL PROTECTED]>
Subject: Re: Linux-2.4.5
Date: Wed, 06 Jun 2001 21:26:45 +0100
In article <[EMAIL PROTECTED]>, "Unknown"
<[EMAIL PROTECTED]> wrote:
> I have been trying to upgrade from RH2.2.12-20 to kernel 2.4.5. I am running
> an AMD-K6-2 processor with 64mb ram, a CreativeLabs PCI 512 sound card, and
> a Lynksys NIC. The kernel freezes at boot. It only gets as for as
> Uncompressing linux.... Ok boot. This not verbatum but close. I tried 2.4.3
> previously with the same problem. The pathes were unsuccessful. B.M.
> Richard
Did you build 2.4.5 with virtual console support? if not then that would
explain it.
karl.
------------------------------
From: [EMAIL PROTECTED] (M.)
Subject: mmap+/dev/zero+madvise(WILLNEED) fails?
Date: 6 Jun 2001 15:01:41 -0700
I am trying to mmap some memory from /dev/zero and that is working fine.
What's failing for me is when I try to madvise that memory with the
MADV_WILLNEED flag. It comes back with EIO error. If I do the same
thing with a regular file (not /dev/zero), madvise works fine.
Question is: is this a normal design behaviour? if so, what does the
EIO error mean? and, is there another way to madvise the OS to keep
this memory in physical ram rather than on secondary storage?
Thanks for your help.
M.
------------------------------
From: [EMAIL PROTECTED]
Crossposted-To: comp.unix.programmer,comp.unix.bsd.freebsd.misc
Subject: Re: thundering herd problem: the REAL scoop
Date: 6 Jun 2001 21:27:29 GMT
In comp.unix.programmer Kasper Dupont <[EMAIL PROTECTED]> wrote:
| [EMAIL PROTECTED] wrote:
|>
| [...]
|>
|> The two specific problems in more detail are:
|>
|> 1. Multiple processes have non-blocking sockets for which listen() has
|> been called, and because they have other descriptors, they must use the
|> select() or poll() system calls to wait for an active descriptor. When
|> the listening socket receives a new connection, the descriptor for every
|> process is now technically an active descriptor, and select() or poll()
|> wakes up in each and every process. Every process now races to do the
|> accept() call, but only one process will succeed and get a new socket
|> descriptor and service the connection. All the rest will get an error
|> with an EAGAIN or EWOULDBLOCK error number, and go back to sleep in the
|> select() or poll() call.
|>
|
| Surely the two problems you mention is both thundering
| herd problems. The difference is that #1 is a thundring
| heard problem in user space and #2 is a thundering herd
| problem in kernel space.
|
| Problem #2 can simply be solved by changing the kernel
| code where there actually is a problem.
|
| Problem #1 cannot simple be solved by chaning the kernel
| because the problem is somewhere else namely in the
| userspace program calling select() or poll().
I'm not sure if problem #1 got ignored, but based on my postings in the
past, which were about problem #1 (before I knew problem #2 existed),
responses were (I realized eventually) about problem #2.
| [...]
|>
|> There is one potential problem with doing "wake one" on select() or
|> poll(). If one connection arrives and one process wakes up, if that
|> process decides to do something else instead, or otherwise fails to
|> execute the accept() call, there remains a pending connection on the
|> queue. When another connection arrives, if only one more process now
|> wakes up, it would likely accept() just the first connection, leaving
|> the second connection still pending. Now we have an "off by one" kind
|> of problem in the queueing. These kinds of things could be solved, but
|> it would take some extra kernel logic to do, possibly a lot to shut all
|> the exposures.
|>
| [...]
|
| In this case there surely is a problem in the userspace
| program. If this is a problem common to many userspace
| programs it might be a good idea to find a way to solve
| the problem by changing the kernel. But of course it is
| not possible to have the kernel solve all thundering herd
| problems in all userspace programs. And of course we may
| not break semantics when trying to solve some of the
| problems.
In some cases, there is a workaround. Apache uses semaphores.
But I understand this is only useful when there is ONE listen
descriptor.
| Trying to use wake on semantics inside the kernel will
| solve the thundering herd problem. But it will introduce
| incorrect semantics in the case where the process being
| waked up does not make use of the file descriptor
| causing the wakeup.
Certainly true.
| I don't think we should discard the idea because it in
| some cases introduce incorrect semantics. Instead we
| can create a workaround achieving the correct semantics
| in these rare cases.
In the realm of high performance server processes, I do not
think it is that rare. I would guess that if this can be
solved for userspace processes, it will get used a lot.
| We can wakeup one process and remember the fact, that
| we have a waitqueue that must be waked up unless the
| current process does a read/write/accept or whatever
| call very soon. When the process goes to sleep or is
| preempted, and the file descriptor is still ready we
| wakeup one more process. We continue this way until
| the file descriptor is no more ready, or the waitqueue
| is empty.
That could work.
| This will require some additional information stored
| in the file structures, and will require minor changes
| to the scheduler and/or interrupt handling. But it
| surely is a feasible way to solve the problem. Can
| anybody see any way this will break any semantics. (It
| might change semantics, but as long as it remains Posix
| compliant I don't consider it to be broken.)
FreeBSD's kqueue/kevent syscalls could easily be made to solve
this problem. With the addition of a filter (which I understand
is being done) to atomically complete the acceptance of a new
connection and return the new descriptor to the caller of kevent,
then only one process ever need wake up for one new connection.
That process will be as committed as it would be if it called
accept() and got a descriptor. The only difference would be that
no other processes would be woken up (since there is nothing to
given to them).
If Linux implements kqueue/kevent (why not?) I would hope it keeps
the same semantics as in FreeBSD.
--
=================================================================
| Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ |
| [EMAIL PROTECTED] | Texas, USA | http://phil.ipal.org/ |
=================================================================
------------------------------
** FOR YOUR REFERENCE **
The service address, to which questions about the list itself and requests
to be added to or deleted from it should be directed, is:
Internet: [EMAIL PROTECTED]
You can send mail to the entire list by posting to the
comp.os.linux.development.system newsgroup.
Linux may be obtained via one of these FTP sites:
ftp.funet.fi pub/Linux
tsx-11.mit.edu pub/linux
sunsite.unc.edu pub/Linux
End of Linux-Development-System Digest
******************************
