Linux-Development-Apps Digest #382, Volume #7 Mon, 7 May 01 09:13:19 EDT
Contents:
Re: Setting a Pointer to Struct tm (time.h) (Erik Max Francis)
Re: Setting a Pointer to Struct tm (time.h) (Eric P. McCoy)
Re: Setting a Pointer to Struct tm (time.h) (Floyd Davidson)
Re: How to get a number of processors (Floyd Davidson)
Re: How to get a number of processors (Kaz Kylheku)
Re: How to get a number of processors (Nate Eldredge)
Re: How to get a number of processors (Floyd Davidson)
Re: How to get a number of processors (Dave Blake)
Re: How to get a number of processors (Chris)
Re: How to get a number of processors (Dave Blake)
Handling large numbers of sockets? ("Nir Arbel")
File locking as a mutual exclusion mechanism (Timur Aydin)
Re: File locking as a mutual exclusion mechanism (Kaz Kylheku)
Re: Handling large numbers of sockets? (Kaz Kylheku)
Re: How to get a number of processors (Floyd Davidson)
Re: MAN-pages to HTML ? ("Nils O. Selåsdal")
Re: How to get a number of processors (Stefaan A Eeckels)
----------------------------------------------------------------------------
From: Erik Max Francis <[EMAIL PROTECTED]>
Subject: Re: Setting a Pointer to Struct tm (time.h)
Date: Sun, 06 May 2001 10:27:14 -0700
Xmina wrote:
> I Have No errors at running time and at compile time, However when
> I
> check the Log I Get "strnge" Values, which means (I supose ) that i
> Must
> initialize *stime.
Yes, you must. You're using an uninitialized pointer; it's pointing
nowhere, so you get garbage (you're lucky you don't crash).
man localtime
man gmtime
--
Erik Max Francis / [EMAIL PROTECTED] / http://www.alcyone.com/max/
__ San Jose, CA, US / 37 20 N 121 53 W / ICQ16063900 / &tSftDotIotE
/ \ To endure what is unendurable is true endurance.
\__/ (a Japanese proverb)
Physics reference / http://www.alcyone.com/max/reference/physics/
A physics reference.
------------------------------
Subject: Re: Setting a Pointer to Struct tm (time.h)
From: [EMAIL PROTECTED] (Eric P. McCoy)
Date: 06 May 2001 14:11:18 -0400
You need a few extra lines, inserted by me below:
"Xmina" <[EMAIL PROTECTED]> writes:
> struct tm *stime // Defining a Pointer of tm
> char *date;
time_t t;
> date=malloc(sizeof(char)*10);
time(&t);
stime = localtime(&t);
> strftime(date, sizeof(date), "%d%m%Y", stime);
sizeof() returns the size of the variable, not the number of bytes
allocated to it. So sizeof(date) returns the size of a char*, which
is 4 on many platforms. Instead do:
strftime(date, 10, "%d%m%Y", stime);
--
Eric McCoy <[EMAIL PROTECTED]>
"Knowing that a lot of people across the world with Geocities sites
absolutely despise me is about the only thing that can add a positive
spin to this situation." - Something Awful, 1/11/2001
------------------------------
From: Floyd Davidson <[EMAIL PROTECTED]>
Subject: Re: Setting a Pointer to Struct tm (time.h)
Date: 06 May 2001 10:26:12 -0800
[EMAIL PROTECTED] (Rouben Rostamian) wrote:
>
>/* Warning: untested code follows */
>
>#include <time.h>
>
>...
> struct tm *stime;
> time_t now;
>
> time(&now);
> stime = localtime(&now);
> date = malloc(10);
> strftime(date, sizeof(date), "%d%m%Y", stime);
strftime(3) wants to know the size of the object pointed to by
date, not the size of date (a pointer to char).
(Probably other headers should be included too, and that
will result in a name conflict with stime(2) also.)
--
Floyd L. Davidson <http://www.ptialaska.net/~floyd>
Ukpeagvik (Barrow, Alaska) [EMAIL PROTECTED]
------------------------------
From: Floyd Davidson <[EMAIL PROTECTED]>
Subject: Re: How to get a number of processors
Date: 06 May 2001 10:36:57 -0800
Greg Copeland <[EMAIL PROTECTED]> wrote:
>
>Considering that each thread above the maximum number of CPU's
>is probably going to negatively impact the overall computation
>time, it does make since. Knowing the number of CPU's is often
>an important piece of information when attempting to optimize a
>parallel algorithm. Exmaple, do this:
>time make bzlilo
>time make -j2 bzlilo
>time make -j4 bzlilo
>time make -j8 bzlilo
>
>Let me know which one finishes first. Obviously, the number of
>CPU's and memory and disk will all be factors here. On my
>system, -j2 finishes almost twice as fast (I'm SMP). And, I
>bet a -j4 doesn't take my system nearly as long as it does for
>your to complete (percent wise that is), assuming you are a
>uniprocessor system. In short, knowing the number of CPU's is
>often important. As is, knowing the number of NICs and even,
>perhaps, how much I/O is still available.
However, -j8 will in fact _not_ be significantly slower than any
of the others. Even on a single processor machine.
--
Floyd L. Davidson <http://www.ptialaska.net/~floyd>
Ukpeagvik (Barrow, Alaska) [EMAIL PROTECTED]
------------------------------
From: [EMAIL PROTECTED] (Kaz Kylheku)
Crossposted-To: comp.os.linux.development.system
Subject: Re: How to get a number of processors
Reply-To: [EMAIL PROTECTED]
Date: Sun, 06 May 2001 19:59:05 GMT
On 04 May 2001 14:59:29 -0500, Greg Copeland <[EMAIL PROTECTED]> wrote:
>
>Oddly enough, my man page for sysconf doesn't show the _SC_NPROCESSORS_CONF
>option. Hmm....wonder how long it's been around. Does it exist for Linux?
I *think* this is new in glibc2.2. If you don't have this sysconf, you can do
the C equivalent of:
grep processor < /proc/cpuinfo | wc -l
:)
------------------------------
From: Nate Eldredge <[EMAIL PROTECTED]>
Subject: Re: How to get a number of processors
Date: 06 May 2001 14:11:20 -0700
Floyd Davidson <[EMAIL PROTECTED]> writes:
> However, -j8 will in fact _not_ be significantly slower than any
> of the others. Even on a single processor machine.
Depends. If you run too many jobs at once, the large amount of memory
(or swap) required and the amount of concurrent disk access starts to
slow things down unnecessarily.
I agree that the time spent switching tasks, which *is* added
overhead, is probably negligible. Caching issues can make a
difference in CPU time needed, however.
--
Nate Eldredge
[EMAIL PROTECTED]
------------------------------
From: Floyd Davidson <[EMAIL PROTECTED]>
Subject: Re: How to get a number of processors
Date: 06 May 2001 14:17:22 -0800
Nate Eldredge <[EMAIL PROTECTED]> wrote:
>Floyd Davidson <[EMAIL PROTECTED]> writes:
>
>> However, -j8 will in fact _not_ be significantly slower than any
>> of the others. Even on a single processor machine.
>
>Depends. If you run too many jobs at once, the large amount of memory
>(or swap) required and the amount of concurrent disk access starts to
>slow things down unnecessarily.
But that depends on what the discussion is about. In this case
the discussion is not about the effects on memory requirements,
but on the need to know how many cpu's are available. The
assumption is that enough RAM can be added to any motherboard,
but extra cpu's cannot.
However, to be pedantic, using a large amount of RAM (as opposed
to using a high percentage of available RAM) or having many
concurrent disk accesses will not affect it either.
Of course swapping would _dramatically_ slow things down and
reducing RAM available for cache below a given level would have
some effect, but the discussion was about the difference between
knowing how many cpu's are available or not.
>I agree that the time spent switching tasks, which *is* added
>overhead, is probably negligible. Caching issues can make a
>difference in CPU time needed, however.
Not enough memory will have the same effect regardless of how
many cpu's are available. The point was that for any given
configuration, -j8 will not be slower due to too few cpu's.
--
Floyd L. Davidson <http://www.ptialaska.net/~floyd>
Ukpeagvik (Barrow, Alaska) [EMAIL PROTECTED]
------------------------------
From: [EMAIL PROTECTED] (Dave Blake)
Crossposted-To: comp.os.linux.development.system
Subject: Re: How to get a number of processors
Date: 6 May 2001 23:46:25 GMT
Reply-To: [EMAIL PROTECTED]
Kaz Kylheku <[EMAIL PROTECTED]> wrote:
> On 04 May 2001 14:59:29 -0500, Greg Copeland <[EMAIL PROTECTED]> wrote:
> >
> >Oddly enough, my man page for sysconf doesn't show the _SC_NPROCESSORS_CONF
> >option. Hmm....wonder how long it's been around. Does it exist for Linux?
>
> I *think* this is new in glibc2.2. If you don't have this sysconf, you can do
> the C equivalent of:
>
> grep processor < /proc/cpuinfo | wc -l
It is actually recommended to use /proc/stat instead. This
came up when procps didn't work properly for SMP machines.
These sections of the /proc filesystem are considered malleable
- but /proc/stat is considered more stable than /proc/cpuinfo.
FWIW.
--
Dave Blake
[EMAIL PROTECTED]
------------------------------
From: Chris <[EMAIL PROTECTED]>
Crossposted-To: comp.os.linux.development.system
Subject: Re: How to get a number of processors
Date: Mon, 07 May 2001 00:12:26 +0000
Kaz Kylheku wrote:
>
> On 04 May 2001 14:59:29 -0500, Greg Copeland <[EMAIL PROTECTED]> wrote:
> >
> >Oddly enough, my man page for sysconf doesn't show the _SC_NPROCESSORS_CONF
> >option. Hmm....wonder how long it's been around. Does it exist for Linux?
>
> I *think* this is new in glibc2.2. If you don't have this sysconf, you can do
Nope... it's present in 2.0.7, certainly.
--
Chris Lightfoot -- chris at ex dash parrot dot com -- www.ex-parrot.com/~chris/
Never criticise somebody until you have walked a mile in their shoes.
That way, they're a mile away, and you have their shoes.
------------------------------
From: [EMAIL PROTECTED] (Dave Blake)
Subject: Re: How to get a number of processors
Date: 7 May 2001 00:49:36 GMT
Reply-To: [EMAIL PROTECTED]
Floyd Davidson <[EMAIL PROTECTED]> wrote:
> Not enough memory will have the same effect regardless of how
> many cpu's are available. The point was that for any given
> configuration, -j8 will not be slower due to too few cpu's.
FWIW.
On a dual processor PII, make -j3 is reportedly "optimized"
for maximal speed.
time make -j8 bzImage
4 minutes, 52 seconds
29.59 system
time make -j3 bzImage
4 minutes 55 seconds
29.51 system
Judging by the system time, there is a 0.023 percent
improvement for using the "correct" number of processes
concurrently during the build. Hardly worth worrying about.
--
Dave Blake
[EMAIL PROTECTED]
------------------------------
From: "Nir Arbel" <[EMAIL PROTECTED]>
Subject: Handling large numbers of sockets?
Date: Mon, 07 May 2001 03:05:28 GMT
Hey all,
I want to port a TCP server I wrote on Windows NT to Linux. Basically it
maintains alot of simultaneous TCP connections to clients, and needs to know
when data is available on each socket to read and interpret it. My question
is: what common IO strategies are there for doing something like this?
Offhand I would think of putting all the socket descriptors in one big
FD_SET and selecting on it, but I understand there's a limitation on the
number of sockets you can do that on? Is is customary to create a new
process for each maximum set and select on it? Are there any simpler
solutions?
Thanks, Nir
---
Chat and Share Files by Interest -- www.soulseek.org
------------------------------
From: Timur Aydin <[EMAIL PROTECTED]>
Subject: File locking as a mutual exclusion mechanism
Date: Sun, 6 May 2001 23:53:22 -0400
Reply-To: [EMAIL PROTECTED]
Hello everybody,
I am looking for a mutual exclusion mechanism to serialize access to a
shared memory segment. I am using the shmget, shmat, shmdt group of
functions for the shared memory.
First I looked into the linux pthreads library. Even though the POSIX
standard specifies process shared mutexes, the linux pthreads library does
not implement them. In fact, according to the linuxthreads FAQ this will
not be implemented any time soon (some people think process shared mutexes
require kernel support, Torvalds thinks this is not necessary and the
debate has been going on for quite some time).
So... I was thinking I could use file locking for this purpose using the
"fcntl" function. Every access to shared memory would require the common
file to be locked first using fcntl and when the shared memory access is
complete, the file would be unlocked.
Here are my questions:
- Conventionally, which directory would be best suited to create the file
to be locked it? Maybe /var/lock?
- Is this method of mutual exclusion efficient enough so that hudreds of
locks/unlocks per second can be done without choking the processor?
Timur.
------------------------------
From: [EMAIL PROTECTED] (Kaz Kylheku)
Subject: Re: File locking as a mutual exclusion mechanism
Reply-To: [EMAIL PROTECTED]
Date: Mon, 07 May 2001 05:34:09 GMT
On Sun, 6 May 2001 23:53:22 -0400, Timur Aydin <[EMAIL PROTECTED]> wrote:
>Hello everybody,
>
>I am looking for a mutual exclusion mechanism to serialize access to a
>shared memory segment. I am using the shmget, shmat, shmdt group of
>functions for the shared memory.
The natural thing for serializing access to a System V memory segment
would be the System V semaphore.
------------------------------
From: [EMAIL PROTECTED] (Kaz Kylheku)
Subject: Re: Handling large numbers of sockets?
Reply-To: [EMAIL PROTECTED]
Date: Mon, 07 May 2001 05:56:23 GMT
On Mon, 07 May 2001 03:05:28 GMT, Nir Arbel <[EMAIL PROTECTED]> wrote:
>Hey all,
>
>I want to port a TCP server I wrote on Windows NT to Linux. Basically it
>maintains alot of simultaneous TCP connections to clients, and needs to know
>when data is available on each socket to read and interpret it. My question
>is: what common IO strategies are there for doing something like this?
>Offhand I would think of putting all the socket descriptors in one big
>FD_SET and selecting on it, but I understand there's a limitation on the
>number of sockets you can do that on?
That limitation is certainly not 64, as in Windows. The kernel limitation is on
how many descriptors a process can have. The soft limit is tuneable with
ulimit, up to a hard-coded maximum determined by the NR_OPEN macro in the
kernel, which is by default 1024 * 1024: more than a million files.
There also is a limitation in the library implementation of fd_set
and the macros which manipulate it; I think your library fd_set is
a struct containing an array that can only represent 1024 bits!
It's better to use the poll() function, which passes a pointer to an array of
``struct pollfd'' objects. The advantages are numerous.
Firstly, sparse descriptor sets are better represented. If you want to wait for
data on sockets 1000, 1001 and 1002, you have an array of three little structs
rather than a fd_set with wasteful zero bits representing descriptors 0 through
999.
Secondly, you don't have to reinitialize the structs in your array each time
you call poll. The struct has a separate field for specifying what kinds of
events are waited for, and for storing the output results. By contrast, select
clobbers your input descriptor masks without output information, forcing you to
do inefficient bit operations to rebuild the entire request from scratch;
FD_ZERO, FD_SET, FD_SET, FD_SET ...
It's not hard to add or remove sockets; to nuke something from the poll array,
just move the last element to that position. To add a socket, add it beyond
the last element. In some programs, it's useful for some application data
structure to have a back pointer to the struct pollfd; when you move it around
this way within the array, you have to be sure to find and update that pointer.
poll() doesn't have any special ``set'' type which would lead to issues of
that type being implemented differently in the library versus kernel.
The size of the array is simply a parameter.
One last thing: if you have nonportable Windows code which assumes that fd_set
is an array of sockets and manipulates it directly, it may be easier to port to
poll() than to fix it. Some Windows programmers do it this way: after the
select() is done, rather than iterate over an independent list of structures,
and detect whether those sockets are in the output fd_set using FD_ISSET, they
instead iterate over the array in the fd_set, and map from that array to the
structures.
------------------------------
From: Floyd Davidson <[EMAIL PROTECTED]>
Subject: Re: How to get a number of processors
Date: 06 May 2001 22:40:26 -0800
[EMAIL PROTECTED] (Dave Blake) wrote:
>Floyd Davidson <[EMAIL PROTECTED]> wrote:
>
>> Not enough memory will have the same effect regardless of how
>> many cpu's are available. The point was that for any given
>> configuration, -j8 will not be slower due to too few cpu's.
>
>FWIW.
>On a dual processor PII, make -j3 is reportedly "optimized"
>for maximal speed.
>
>time make -j8 bzImage
>4 minutes, 52 seconds
>29.59 system
>
>time make -j3 bzImage
>4 minutes 55 seconds
>29.51 system
>
>Judging by the system time, there is a 0.023 percent
>improvement for using the "correct" number of processes
>concurrently during the build. Hardly worth worrying about.
Now try this command:
make -j3 MAKE="make -j3" bzImage
Just for kicks, have top running in another window too, and watch
it will the kernel is compiled.
--
Floyd L. Davidson <http://www.ptialaska.net/~floyd>
Ukpeagvik (Barrow, Alaska) [EMAIL PROTECTED]
------------------------------
Reply-To: "Nils O. Selåsdal" <[EMAIL PROTECTED]>
From: "Nils O. Selåsdal" <[EMAIL PROTECTED]>
Subject: Re: MAN-pages to HTML ?
Date: Mon, 7 May 2001 11:24:36 +0200
"Dennis Schramm" <[EMAIL PROTECTED]> wrote in message
news:9d19q0$g1kkd$[EMAIL PROTECTED]...
> Hi,
>
> I'm using SuSE Pro 7.0 and I'm wondering if there's a tool available that
> converts MAN-pages to HTML ! Any suggestions ?
one of the kde2 packages provide a man2html command
if you use konqueror, just type man:/ as the url ;)
------------------------------
From: [EMAIL PROTECTED] (Stefaan A Eeckels)
Subject: Re: How to get a number of processors
Crossposted-To: comp.os.linux.development.system
Date: Mon, 7 May 2001 14:52:39 +0200
In article <[EMAIL PROTECTED]>,
[EMAIL PROTECTED] (Dave Blake) writes:
> Eric P. McCoy <[EMAIL PROTECTED]> wrote:
>
>> This strikes me as a battle of bad ideas: I hate writing a
>> text parser to deal with /proc; I don't like using nonstandard
>> pieces of code; and no program should ever need to know how
>> many processors are in a given box. There are cases where
>> you'd want to use one or all of these bad ideas, but I, for
>> one, would need a pressing reason.
>
> Suppose I am writing a data crunching piece of software that
> parallelizes easily, and wish to run a thread on each processor.
>
> I first parse /proc/stat, and then crunch away with a thread
> on each CPU.
>
> For a web searching program, you may wish to know the number of
> NICs and CPUs, and take the lower of the two as the number of
> threads to run. And so on.
In a Unix system, the application should not need to know
anything about the hardware details. The recent obsession
with threads violates that basic tenet. If one wants to
squeeze the last ounce of performance from a box, don't
use an OS.
--
Stefaan
--
How's it supposed to get the respect of management if you've got just
one guy working on the project? It's much more impressive to have a
battery of programmers slaving away. -- Jeffrey Hobbs (comp.lang.tcl)
------------------------------
** FOR YOUR REFERENCE **
The service address, to which questions about the list itself and requests
to be added to or deleted from it should be directed, is:
Internet: [EMAIL PROTECTED]
You can send mail to the entire list by posting to the
comp.os.linux.development.apps newsgroup.
Linux may be obtained via one of these FTP sites:
ftp.funet.fi pub/Linux
tsx-11.mit.edu pub/linux
sunsite.unc.edu pub/Linux
End of Linux-Development-Apps Digest
******************************
