Linux-Development-Sys Digest #694, Volume #8 Sun, 6 May 01 00:13:10 EDT
Contents:
What is the most efficient thread library for Linux? ("Simon Stuart")
Re: Journaling Filesystem with Individual File Compression? ("Adam Warner")
Re: Journaling Filesystem with Individual File Compression? ("Adam Warner")
IPX in Kernel 2.4.3 (Paul Evans)
Re: 2.4.3 Kernel build problem (Paul Evans)
Re: How to get a number of processors (Nix)
Waking up a thread blocked on accept() ("Rosimildo daSilva")
Re: How to get a number of processors (Dave Blake)
accessing memory and IO space
Re: How to get a number of processors (Floyd Davidson)
Re: How to get a number of processors (Eric P. McCoy)
Obtaining the IP address of the eth0 interface? (Carlos Justiniano)
Re: Waking up a thread blocked on accept() ("Cameron Kerr")
Re: Journaling Filesystem with Individual File Compression? ([EMAIL PROTECTED])
Re: Obtaining the IP address of the eth0 interface?
Schedule job ("Manda Leung")
disk request ordering dependency ("Zhiyong Xu")
Re: Obtaining the IP address of the eth0 interface? (greg)
Re: Obtaining the IP address of the eth0 interface? (Carlos Justiniano)
Re: Schedule job ([EMAIL PROTECTED])
----------------------------------------------------------------------------
From: "Simon Stuart" <[EMAIL PROTECTED]>
Subject: What is the most efficient thread library for Linux?
Date: Sat, 05 May 2001 08:11:16 GMT
I am looking to develop multi-threaded C applications on Linux. I see there
are a number of alternative thread libraries available. Does the current
kernel support multi-threading natively or do you still need to add a
threading library? If not, is there any generally accepted opinion as to
which is the best library?
------------------------------
From: "Adam Warner" <[EMAIL PROTECTED]>
Subject: Re: Journaling Filesystem with Individual File Compression?
Date: Sat, 5 May 2001 23:21:53 +1200
Thanks Christopher,
<[EMAIL PROTECTED]> wrote in message
news:juMI6.211534$[EMAIL PROTECTED]...
> "Adam Warner" <[EMAIL PROTECTED]> writes:
<snip>
> Do you have some specific application in mind? If you define your
> "problem" as being:
> "I want a journaling filesystem that does data compression and
> allows tiny block sizes"
> the answer may simply be "The Answer to Your Question is NO."
>
> In contrast, in the real world, people have problems that need to be
> solved that sound more like "I'm storing _some format of file_ on a
> filesystem, and am running out of disk space. What might I do about
> this?"
>
> To which there might be much more fruitful answers.
Actually my problem is completely real-world. I was storing all my
clipart/photo/media CDs (you know, ClickArt 65,000 that sort of thing) on my
Linux box using an ext2 block size of 1kB and very compact file compression
(BTW it was superior to NTFS compression):
http://www.netspace.net.au/~reiter/e2compr/software-04x.html
But then I wanted some of the benefits of the 2.4 kernel so I moved the
files to what is now my Win2k box using small clusters and compression. I
would like to move the files back to a Linux box without throwing away hard
drive space (the files only get accessed occassionally, and each file is
accessed sequentially. On a fast computer this is a perfect use for file
compression.
ReiserFS tail merging might be a big help. I want journalling because a fsck
on such a large amount of files is quite time consuming.
I was just hoping some filesystem, even if in beta, was being ported with
file compression.
Regards,
Adam
------------------------------
From: "Adam Warner" <[EMAIL PROTECTED]>
Subject: Re: Journaling Filesystem with Individual File Compression?
Date: Sat, 5 May 2001 23:49:26 +1200
It's odd but I can't find my reply after waiting 30 minutes for it to
appear, so I'll repost:
Thanks Christopher,
<[EMAIL PROTECTED]> wrote in message
news:juMI6.211534$[EMAIL PROTECTED]...
> "Adam Warner" <[EMAIL PROTECTED]> writes:
<snip>
> Do you have some specific application in mind? If you define your
> "problem" as being:
> "I want a journaling filesystem that does data compression and
> allows tiny block sizes"
> the answer may simply be "The Answer to Your Question is NO."
>
> In contrast, in the real world, people have problems that need to be
> solved that sound more like "I'm storing _some format of file_ on a
> filesystem, and am running out of disk space. What might I do about
> this?"
>
> To which there might be much more fruitful answers.
Actually my problem is completely real-world. I was storing all my
clipart/photo/media CDs (you know, ClickArt 65,000 that sort of thing) on my
Linux box using an ext2 block size of 1kB and very compact file compression
(BTW it was superior to NTFS compression):
http://www.netspace.net.au/~reiter/e2compr/software-04x.html
But then I wanted some of the benefits of the 2.4 kernel so I moved the
files to what is now my Win2k box using small clusters and compression. I
would like to move the files back to a Linux box without throwing away hard
drive space (the files only get accessed occassionally, and each file is
accessed sequentially. On a fast computer this is a perfect use for file
compression).
ReiserFS tail merging might be a big help. I want journalling because a fsck
on such a large amount of files is quite time consuming.
I was just hoping some filesystem, even if in beta, was being ported with
file compression.
Regards,
Adam
------------------------------
From: Paul Evans <[EMAIL PROTECTED]>
Subject: IPX in Kernel 2.4.3
Date: Sat, 5 May 2001 13:37:37 +0000
Does anyone know why IPX is not working in kernel 2.4.3 and how to get it
working?
If I try to compile it in I get this at the linking stage:
net/network.o(.data+0x29c4): undefined reference to
`sysctl_ipx_pprop_broadcasting'
And if I build it as a module then it builds OK but then gives this when I
try to modprobe it:
/lib/modules/2.4.3/kernel/net/ipx/ipx.o: unresolved symbol make_EII_client
/lib/modules/2.4.3/kernel/net/ipx/ipx.o: unresolved symbol
make_8023_client
/lib/modules/2.4.3/kernel/net/ipx/ipx.o: unresolved symbol
destroy_8023_client
/lib/modules/2.4.3/kernel/net/ipx/ipx.o: unresolved symbol
destroy_EII_client
/lib/modules/2.4.3/kernel/net/ipx/ipx.o: insmod
/lib/modules/2.4.3/kernel/net/ipx/ipx.o failed
/lib/modules/2.4.3/kernel/net/ipx/ipx.o: insmod ipx failed
Paul Evans
(1st Year Maths & CompSci at Pembroke College, Cambridge, England)
[EMAIL PROTECTED]
[EMAIL PROTECTED]
ICQ number: 4135350
------------------------------
From: Paul Evans <[EMAIL PROTECTED]>
Subject: Re: 2.4.3 Kernel build problem
Date: Sat, 5 May 2001 13:43:57 +0000
On Thu, 26 Apr 2001, Cristov wrote:
> Has anybody run into a build problem when trying to compile IPX Networking
> into the kernel?
>
> I get this:
>
> net/network.o: undefined reference to 'sysctrl_ipx_pprop_broadcasting'
> make: ** [vmlinux] Error 1
>
> Anybody see this yet? Or have a patch for this?
I have this problem too. If you remove the IPX network then it will
compile OK, only removing IPX isn't really an option for me, and might not
be for you either.
I don't yet know how to fix it, but if I find out I'll let you know...
Paul Evans
(1st Year Maths & CompSci at Pembroke College, Cambridge, England)
[EMAIL PROTECTED]
[EMAIL PROTECTED]
ICQ number: 4135350
------------------------------
From: Nix <$}xinix{[email protected]>
Crossposted-To: comp.os.linux.development.apps
Subject: Re: How to get a number of processors
Date: 05 May 2001 15:47:16 +0100
On 04 May 2001, Eric P. McCoy stated:
> Greg Copeland <[EMAIL PROTECTED]> writes:
>
>> Oddly enough, my man page for sysconf doesn't show the _SC_NPROCESSORS_CONF
>> option. Hmm....wonder how long it's been around.
>
> It starts with an underscore, which probably means it's nonstandard.
It is not in the Single Unix Spec, so I agree; but *all* the sysconf
constants start `_SC_', so the underscore means little here.
--
`I like to think the situation could be likened to doing heart surgery
in a hurry with a plastic spoon after having a couple of pints.'
--- James Reeves
------------------------------
Reply-To: "Rosimildo daSilva" <[EMAIL PROTECTED]>
From: "Rosimildo daSilva" <[EMAIL PROTECTED]>
Subject: Waking up a thread blocked on accept()
Date: Sat, 05 May 2001 15:12:15 GMT
Hi guys,
I have a thread blocked on an accept() call. I'd like to wake it up from
another thread. How do I do that ?
Under WIN32, I just close the socket, and the accept() returns with an
error code.
---
Rosimildo da Silva [EMAIL PROTECTED]
ConnectTel, Inc. Austin, TX -- USA
Phone : 512-338-1111 Fax : 512-918-0449
Company Page: http://www.connecttel.com
Home Page: http://members.nbci.com/rosimildo/
------------------------------
From: [EMAIL PROTECTED] (Dave Blake)
Crossposted-To: comp.os.linux.development.apps
Subject: Re: How to get a number of processors
Date: 5 May 2001 18:05:46 GMT
Reply-To: [EMAIL PROTECTED]
Eric P. McCoy <[EMAIL PROTECTED]> wrote:
> This strikes me as a battle of bad ideas: I hate writing a
> text parser to deal with /proc; I don't like using nonstandard
> pieces of code; and no program should ever need to know how
> many processors are in a given box. There are cases where
> you'd want to use one or all of these bad ideas, but I, for
> one, would need a pressing reason.
Suppose I am writing a data crunching piece of software that
parallelizes easily, and wish to run a thread on each processor.
I first parse /proc/stat, and then crunch away with a thread
on each CPU.
For a web searching program, you may wish to know the number of
NICs and CPUs, and take the lower of the two as the number of
threads to run. And so on.
--
Dave Blake
[EMAIL PROTECTED]
------------------------------
From: <[EMAIL PROTECTED]>
Subject: accessing memory and IO space
Date: Sat, 5 May 2001 13:03:05 -0700
The need has arisen to be able to access arbritary memory and IO space while
working on driver development for an embedded project. I am hoping that
someone can tell me a way to either use gdb to read/write memory or IO space
or barring that, a debug program that will read/write any memory address. I
have read 'man gdb' and tried the 'x' command to no avail. Maybe it is as
simple as giving gdb a command line argument that I don't know yet. I would
have to believe that such a thing is easy, its just that I don't know how to
do it in this operating system. In developing drivers on other architectures
(such as vxWorks for instance), it is easy to read memory and IO space in
order to check and possibly re-configure driver hardware while developing.
------------------------------
From: Floyd Davidson <[EMAIL PROTECTED]>
Crossposted-To: comp.os.linux.development.apps
Subject: Re: How to get a number of processors
Date: 05 May 2001 11:31:07 -0800
[EMAIL PROTECTED] (Dave Blake) wrote:
>Eric P. McCoy <[EMAIL PROTECTED]> wrote:
>
>> This strikes me as a battle of bad ideas: I hate writing a
>> text parser to deal with /proc; I don't like using nonstandard
>> pieces of code; and no program should ever need to know how
>> many processors are in a given box. There are cases where
>> you'd want to use one or all of these bad ideas, but I, for
>> one, would need a pressing reason.
>
>Suppose I am writing a data crunching piece of software that
>parallelizes easily, and wish to run a thread on each processor.
>
>I first parse /proc/stat, and then crunch away with a thread
>on each CPU.
How are you going to crunch away with a "thread on each
CPU"???
--
Floyd L. Davidson <http://www.ptialaska.net/~floyd>
Ukpeagvik (Barrow, Alaska) [EMAIL PROTECTED]
------------------------------
Crossposted-To: comp.os.linux.development.apps
Subject: Re: How to get a number of processors
From: [EMAIL PROTECTED] (Eric P. McCoy)
Date: 05 May 2001 16:54:40 -0400
Followups set to c.o.l.d.apps.
[EMAIL PROTECTED] (Dave Blake) writes:
> Suppose I am writing a data crunching piece of software that
> parallelizes easily, and wish to run a thread on each processor.
> I first parse /proc/stat, and then crunch away with a thread
> on each CPU.
Well, considering that your app isn't likely to ever see more than 8
processors, and also that the overhead for 8 threads is not hugely
significant, I'd just make the threads and ignore how many CPUs there
are.
> For a web searching program, you may wish to know the number of
> NICs and CPUs, and take the lower of the two as the number of
> threads to run. And so on.
I suspect it'd be far more complex than that. And for heavy-duty
applications like this one, you use custom software, so you can just
hard-code in the relevant values.
--
Eric McCoy <[EMAIL PROTECTED]>
"Knowing that a lot of people across the world with Geocities sites
absolutely despise me is about the only thing that can add a positive
spin to this situation." - Something Awful, 1/11/2001
------------------------------
From: [EMAIL PROTECTED] (Carlos Justiniano)
Subject: Obtaining the IP address of the eth0 interface?
Date: Sat, 05 May 2001 21:29:30 GMT
I need to obtain the IP address of the eth0 interface from within my
Linux server application. I'm aware that I can do this from a
connected socket, but I would like to have the IP address prior to my
server making a connection.
I'd rather not have to depend on the hosts file or functions such as
gethostname(), gethostbyname(). The machine that my server will be
running on will obtain it's IP via DHCP.
IP and IFCONFIG both return the correct information of the interfaces
that are active. I realize that I could parse the output to obtain
the IP info that I need... but I'm more interested in a solution that
doesn't depend on a clever trick. I imagine that there must be an API
to enumerate the active interfaces?
- Carlos
------------------------------
From: "Cameron Kerr" <[EMAIL PROTECTED]>
Subject: Re: Waking up a thread blocked on accept()
Date: Sun, 06 May 2001 11:41:43 +1200
In article <jLUI6.5250$[EMAIL PROTECTED]>, "Rosimildo
daSilva" <[EMAIL PROTECTED]> wrote:
> Hi guys,
>
> I have a thread blocked on an accept() call. I'd like to wake it up from
> another thread. How do I do that ?
You can put your socket into non-blocking mode, then use select with a
timeout, checking some shared state every so often (polling)
OR
You could try signalling the thread (Look at the GNU libc manual and the
LinuxThreads documentation)
> Under WIN32, I just close the socket, and the accept() returns with an
> error code.
This should work, but I think it falls into the category of "sloppy
design"/"bad structure", from a Software Engineering viewpoint.
You might like to have a look at the O'Rielly book "PThreads
programming" (named similar)
HIH
--
Cameron Kerr -- cameron.kerr @ paradise.net.nz
Praise Slackware, our baud and saviour!
--
------------------------------
From: [EMAIL PROTECTED]
Subject: Re: Journaling Filesystem with Individual File Compression?
Crossposted-To: comp.os.linux.misc
Date: Sun, 06 May 2001 00:27:24 GMT
"Adam Warner" <[EMAIL PROTECTED]> writes:
> Thanks Christopher,
> <[EMAIL PROTECTED]> wrote in message
> news:juMI6.211534$[EMAIL PROTECTED]...
> > "Adam Warner" <[EMAIL PROTECTED]> writes:
> <snip>
> > Do you have some specific application in mind? If you define your
> > "problem" as being:
> > "I want a journaling filesystem that does data compression and
> > allows tiny block sizes"
> > the answer may simply be "The Answer to Your Question is NO."
> >
> > In contrast, in the real world, people have problems that need to be
> > solved that sound more like "I'm storing _some format of file_ on a
> > filesystem, and am running out of disk space. What might I do about
> > this?"
> >
> > To which there might be much more fruitful answers.
> Actually my problem is completely real-world. I was storing all my
> clipart/photo/media CDs (you know, ClickArt 65,000 that sort of
> thing) on my Linux box using an ext2 block size of 1kB and very
> compact file compression (BTW it was superior to NTFS compression):
> http://www.netspace.net.au/~reiter/e2compr/software-04x.html
> But then I wanted some of the benefits of the 2.4 kernel so I moved
> the files to what is now my Win2k box using small clusters and
> compression. I would like to move the files back to a Linux box
> without throwing away hard drive space (the files only get accessed
> occassionally, and each file is accessed sequentially. On a fast
> computer this is a perfect use for file compression.
> ReiserFS tail merging might be a big help. I want journalling
> because a fsck on such a large amount of files is quite time
> consuming.
> I was just hoping some filesystem, even if in beta, was being ported
> with file compression.
Compression obviously _eliminates_ random access as well as memory
mapping of files, and those being the sorts of things that filesystem
implementors are trying to support, it's not too surprising that this
wouldn't be high on the list of things to add in.
For good measure, consider that:
-> Image files are already commonly compressed internally, which
would eliminate any would-be benefits.
-> Applications that need to access this sort of thing can use zlib.
-> People have written interceptors for open() that detect if there's
compressed data inside a file, and automagically attach a gzip to
it.
I'd suggest that you take a look at ReiserFS or ext3, from the
"journalling" perspective, and look for user space solutions for
compression/decompression.
--
(concatenate 'string "cbbrowne" "@ntlug.org")
http://vip.hyperusa.com/~cbbrowne/resume.html
Rules of the Evil Overlord #190. "If my mad scientist/wizard tells me
he has almost perfected my Superweapon but it still needs more
testing, I will wait for him to complete the tests. No one ever
conquered the world using a beta version."
<http://www.eviloverlord.com/>
------------------------------
From: [EMAIL PROTECTED] ()
Subject: Re: Obtaining the IP address of the eth0 interface?
Date: Sun, 06 May 2001 03:07:26 -0000
In article <[EMAIL PROTECTED]>,
Carlos Justiniano <[EMAIL PROTECTED]> wrote:
>IP and IFCONFIG both return the correct information of the interfaces
>that are active. I realize that I could parse the output to obtain
>the IP info that I need... but I'm more interested in a solution that
>doesn't depend on a clever trick. I imagine that there must be an API
>to enumerate the active interfaces?
Why not look at the source of ifconfig? Surely that'll give you
the answer.
--
http://www.spinics.net/linux
------------------------------
From: "Manda Leung" <[EMAIL PROTECTED]>
Subject: Schedule job
Date: Sun, 6 May 2001 11:04:15 +0800
Hi,
I want to schedule a job to run every 30 mins. What can I do?
Thanks.
------------------------------
From: "Zhiyong Xu" <[EMAIL PROTECTED]>
Subject: disk request ordering dependency
Date: Sat, 5 May 2001 23:11:49 -0700
How did kernel enforce disk request dependency, for example, the rename
operation changes the name of a file by adding a link for the new name and
removing the old link. If the system goes down after the old directory block
has been written(with the link removed) but before the new one is written,
neither name for the file will exist.
I checked __make_request function in ll_rw_blk.c, but I don't where's
code deal with consistency of data block of a file and metadata of it. So
what's the linux stratege here?
Thanks!
------------------------------
From: [EMAIL PROTECTED] (greg)
Subject: Re: Obtaining the IP address of the eth0 interface?
Date: Sun, 6 May 2001 03:20:13 +0000 (UTC)
On Sun, 06 May 2001 03:07:26 -0000, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>In article <[EMAIL PROTECTED]>,
>Carlos Justiniano <[EMAIL PROTECTED]> wrote:
>
>>IP and IFCONFIG both return the correct information of the interfaces
>>that are active. I realize that I could parse the output to obtain
>>the IP info that I need... but I'm more interested in a solution that
>>doesn't depend on a clever trick. I imagine that there must be an API
>>to enumerate the active interfaces?
>
>Why not look at the source of ifconfig? Surely that'll give you
>the answer.
A quick parsing of /proc/net/dev should be enough (that's what ifconfig does
actually).
--
greg
------------------------------
From: [EMAIL PROTECTED] (Carlos Justiniano)
Subject: Re: Obtaining the IP address of the eth0 interface?
Date: Sun, 06 May 2001 03:40:48 GMT
thanks Ellis, and Greg
I've done just that.... I've looked at the code for ifconfig. I
noticed that it creates a socket prior to checking proc/net/dev!
Looks like perhaps the information isn't present in proc/net/dev
unless you actually do something net related.
- Carlos
On Sun, 6 May 2001 03:20:13 +0000 (UTC), [EMAIL PROTECTED] (greg)
wrote:
>On Sun, 06 May 2001 03:07:26 -0000, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>>In article <[EMAIL PROTECTED]>,
>>Carlos Justiniano <[EMAIL PROTECTED]> wrote:
>>
>>>IP and IFCONFIG both return the correct information of the interfaces
>>>that are active. I realize that I could parse the output to obtain
>>>the IP info that I need... but I'm more interested in a solution that
>>>doesn't depend on a clever trick. I imagine that there must be an API
>>>to enumerate the active interfaces?
>>
>>Why not look at the source of ifconfig? Surely that'll give you
>>the answer.
>A quick parsing of /proc/net/dev should be enough (that's what ifconfig does
>actually).
>
>--
>greg
------------------------------
From: [EMAIL PROTECTED]
Subject: Re: Schedule job
Date: Sun, 06 May 2001 04:02:13 GMT
"Manda Leung" <[EMAIL PROTECTED]> writes:
> I want to schedule a job to run every 30 mins. What can I do?
You'll want to set up an entry in your crontab looking something like:
7,37 * * * * /home/cbbrowne/bin/do_whatever
The 7,37 indicates that the job runs at 7 and 37 minutes after each
hour; you may want other times.
Consult the manual page for crontab for more details.
--
(reverse (concatenate 'string "gro.gultn@" "enworbbc"))
http://www.ntlug.org/~cbbrowne/resume.html
"What a depressingly stupid machine."
-- Marvin the Paranoid Android
------------------------------
** FOR YOUR REFERENCE **
The service address, to which questions about the list itself and requests
to be added to or deleted from it should be directed, is:
Internet: [EMAIL PROTECTED]
You can send mail to the entire list by posting to the
comp.os.linux.development.system newsgroup.
Linux may be obtained via one of these FTP sites:
ftp.funet.fi pub/Linux
tsx-11.mit.edu pub/linux
sunsite.unc.edu pub/Linux
End of Linux-Development-System Digest
******************************
