Linux-Development-Sys Digest #701, Volume #8 Tue, 8 May 01 06:13:13 EDT
Contents:
Re: How to get a number of processors (MindPatrol)
Re: SCSI emulation for CD-R
getting - "ioctl failed: Inappropriate ioctl for device" ([EMAIL PROTECTED])
Re: where do I see dprintk messages ([EMAIL PROTECTED])
faster way to serve: passing fds vs multiple accept ([EMAIL PROTECTED])
wait() and lost children ([EMAIL PROTECTED])
Re: How to get a number of processors (Eric P. McCoy)
Re: How to get a number of processors (Eric P. McCoy)
Re: Sun Solaris IPC (Andrew Gabriel)
Little Help ("Ryan")
Re: Q: device driver races? (Holger Eitzenberger)
user memory locked down to kernel pages? (Ralf Render)
Re: Driver functions after a Segmentation Error (Kasper Dupont)
Re: file read problem (Kasper Dupont)
Re: How to get a number of processors (Stefaan A Eeckels)
Re: How to get a number of processors (Stefaan A Eeckels)
----------------------------------------------------------------------------
From: [EMAIL PROTECTED] (MindPatrol)
Crossposted-To: comp.os.linux.development.apps
Subject: Re: How to get a number of processors
Date: Tue, 08 May 2001 03:29:43 GMT
Reply-To: [EMAIL PROTECTED]
On Mon, 7 May 2001 19:31:44 +0100, John Beardmore
<[EMAIL PROTECTED]> wrote:
[Some good stuff deleted]
>Well what you really want to do is match the number of high priority
>threads to processors.
>
>
>>The number of physical lumps of doped silicon inside the machine is an
>>irrelevant hardware detail.
>
>Nonsense.
Just to add a little spice to the mix, the Cray Inc, MTA
processor uses multithreading to hide memory latency.
Ideally, a "good" MTA program will contain dozens/hundreds
of threads. Whenever an instruction stream makes a memory
reference the hardware either switches to another thread or
stalls. Totally upsets the idea of a thread per processor.
Of course, they have a smart compiler that creates most of
the threads.
I keep reading the hardware mediated multithreading is soon
to appear in mainstream processors. IA64 ???
--
John
------------------------------
From: [EMAIL PROTECTED] ()
Subject: Re: SCSI emulation for CD-R
Date: Tue, 08 May 2001 04:12:43 -0000
In article <[EMAIL PROTECTED]>,
Philip Lee <[EMAIL PROTECTED]> wrote:
>I'm really in a bind because I have no way to tell linux to start my cd-r
>with the ide-scsi driver. The only way I can find is with lilo and editing
>the lilo.conf file to say append="hdd=ide-scsi". I use loadlin and have no
>clue what to edit. Please help.
Put it on the loadlin command line.
--
http://www.spinics.net/linux
------------------------------
From: [EMAIL PROTECTED]
Subject: getting - "ioctl failed: Inappropriate ioctl for device"
Date: Tue, 08 May 2001 04:57:54 GMT
I am getting this message: 'ioctl failed: Inappropriate ioctl for device'
Also, none of my fops are getting called including 'open'...when I look at my
kernel driver log in /var/log/messages...what am I doing wrong here ..any info
thanx,
k
------------------------------
From: [EMAIL PROTECTED]
Subject: Re: where do I see dprintk messages
Date: Tue, 8 May 2001 07:15:16 +0200
[EMAIL PROTECTED] wrote:
> In article <YHEJ6.5185$[EMAIL PROTECTED]>,
> <[EMAIL PROTECTED]> wrote:
>>I am using function calls from mm/memory.c. What do I have to do to see the
>>dprintk messages. I see my printk messages in /var/log/messages. Is there
>>special compile time flag I have to use and where should I expect them.
> You see them where you've told /etc/syslog.conf to put
> them or on the console if they are high enough priority. See
> /usr/src/linux/Documentation/proc.txt for information on setting the
> console printk priority levels.
Note that the OP writes about "dprintk". This is #defined to nothing in
mm/memory.c, at least in 2.2.18. So he should replace this by
#define dprintk printk
and then follow your instructions.
Peter
------------------------------
From: [EMAIL PROTECTED]
Subject: faster way to serve: passing fds vs multiple accept
Date: 8 May 2001 06:30:44 GMT
I'm curious what you experiences might be able to related on which
would be a better model to build a multi-process server:
1. One process doing listening, forking new processes after a new
connection arrives.
2. Pre-forking processes, doing listening, and passing connections
to the processes.
3. Pre-forking processes, but each process just does its own call
to accept().
I'm rulling out #1 for just to improve responsiveness, and want to
focus on the differences between #2 and #3. Method #2 allows doing
the listen on more than one socket, allowing specific addresses and
multiple ports. Method #3 could do that if select() or poll() is
used as well, but once you have multiple processes, doing non-blocking
is approaching pointless it might seem. Method #3 needs a "wake one"
listen/accept semantic to be useful while method #2 won't have that
issue. But then method #2 serializes more and has to pass on the
sockets it gets.
There may be no clear cut "best" and it's probably dependent on the
particular server application. Still, I'd like to get some info on
experiences with these choices to help determine what might be best
for the servers I am trying to set up. Since I'd have to do this
whole setup on each method, and the development to set this up is
going to take a lot of time, I really cannot afford to run the tests
that would ultimately be needed. So I'm going to just have to make
guesses (hopefully informed ones) about this.
--
=================================================================
| Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ |
| [EMAIL PROTECTED] | Texas, USA | http://phil.ipal.org/ |
=================================================================
------------------------------
From: [EMAIL PROTECTED]
Subject: wait() and lost children
Date: 8 May 2001 06:34:22 GMT
I'm debugging a server program I did not write which has a loop
which involves doing a wait(NULL) call if it has reached its
maximum number of children. What seems to be happening is that
if 2 or more children have exit status at the same time, the
number of wakeups is less than the number of such children, and
may even be as few as 1. This is on Linux 2.4. Is wait() allowed
to behave this way? I also tried a pointer to an int instead of
NULL for the argument, and this did not change things.
--
=================================================================
| Phil Howard - KA9WGN | Dallas | http://linuxhomepage.com/ |
| [EMAIL PROTECTED] | Texas, USA | http://phil.ipal.org/ |
=================================================================
------------------------------
Crossposted-To: comp.os.linux.development.apps
Subject: Re: How to get a number of processors
From: [EMAIL PROTECTED] (Eric P. McCoy)
Date: 08 May 2001 02:58:12 -0400
John Beardmore <[EMAIL PROTECTED]> writes:
[not sure what to snip and leave, I'll just quote this and launch into
a rant]
> Good system management usually comes from an appreciation of the
> hardware in the systems and using it appropriately.
> You seem quite reluctant to let apps have any clues that might help to
> automate any of that.
Are you objecting to the separation of Kernel and Application?
The kernel, more or less by definition, knows far more about the state
of the system (hardware and software) than any application. This
makes it the best candidate for determining how to divide up
resources. While it's certainly possible for an app to figure out
what other apps are doing, it's bad for it to base its behavior on any
of that information because it has _no idea_ what the other app is
doing. The kernel probably doesn't either, but it can certainly do
more to keep the other app under control.
If you can guarantee that your Magic Parallelizing App is going to be
the only process running at any given time, then you can damn well
guarantee you'll have an operator smart enough to know how many CPUs
are in the machine (and how to tell your app about it). If you can't
guarantee that, then you need to leave process management up to the
kernel, which can understand the situation far better than any
individual app.
Seems like what you're _really_ advocating is some powerful process
management stuff in the kernel. If so, go for it. I have no doubt
that it'd have a payoff in the workstation/desktop area as well as in
the enterprise/datacenter server area.
--
Eric McCoy <[EMAIL PROTECTED]>
"Knowing that a lot of people across the world with Geocities sites
absolutely despise me is about the only thing that can add a positive
spin to this situation." - Something Awful, 1/11/2001
------------------------------
Crossposted-To: comp.os.linux.development.apps
Subject: Re: How to get a number of processors
From: [EMAIL PROTECTED] (Eric P. McCoy)
Date: 08 May 2001 03:03:02 -0400
[EMAIL PROTECTED] (MindPatrol) writes:
> I keep reading the hardware mediated multithreading is soon
> to appear in mainstream processors. IA64 ???
IIRC, the IA64 does that, but it needs hints from the software. These
are apparently relatively straightforward, though far from easy, to
put in compilers.
--
Eric McCoy <[EMAIL PROTECTED]>
"Knowing that a lot of people across the world with Geocities sites
absolutely despise me is about the only thing that can add a positive
spin to this situation." - Something Awful, 1/11/2001
------------------------------
From: [EMAIL PROTECTED] (Andrew Gabriel)
Crossposted-To:
comp.unix.questions,comp.unix.programmer,comp.os.linux.misc,comp.unix.solaris
Subject: Re: Sun Solaris IPC
Date: 8 May 2001 00:34:34 GMT
In article <[EMAIL PROTECTED]>,
vardhan <[EMAIL PROTECTED]> writes:
>
>The project is based on a Solaris platform (version unknown) running a
>SunOS (probably 5.6). I assume that the particular SunOS itself will be
>using a specific Unix IPC standard. Please correct me if I am wrong. My
>question is, in this case (Sun on Solaris) what are the options I have
>for IPC, and where do I get the information to made a comparative study
>most suitable for me? Specifically, where do I begin to get the
>information I need? I am quite comfortable with basic UNIX IPC and sytem
>calls.
If this is only for Solaris, don't forget doors - man door_create.
--
Andrew Gabriel
Consultant Software Engineer
------------------------------
From: "Ryan" <[EMAIL PROTECTED]>
Subject: Little Help
Date: Tue, 08 May 2001 07:58:07 GMT
Hello, I have to do a uni assignment on how the Linux Kernel conducts either
its
- Processes,
- Memory Management, or
- File Systems,
Maybe one of you gurus could reccomend which topic they think I should do,
and suggest maybe a URL where I could find the appropriate resources...
Only if you want to help that is!
Thankyou
Ryan
------------------------------
From: [EMAIL PROTECTED] (Holger Eitzenberger)
Subject: Re: Q: device driver races?
Date: Tue, 8 May 2001 10:38:25 +0200
On Mon, 07 May 2001 13:17:52 +0000, [EMAIL PROTECTED] (Kasper Dupont) wrote:
>Holger Eitzenberger wrote:
>> Additionally there is a feature on the board called "reset cylcle".
>> At boot time a reset cycle is started and has to be stopped in a
>> (configurable) time period; if not, the system will be rebooted by the
>> LED board.
>
>A board with this reboot function is called a Watchdog timer.
>Linux already has drivers for some Watchdog timers, it might
>be a good idea to see if it contains anything you can use.
>If you need to write a new driver to support your board, I
>suggest you try to make it compatible with the existing
>Watchdog timer interface.
Well, i'll try that. But still the other questions prevail.
Thanks anyway.
Holger
--
+++ Debian GNU/Linux <[EMAIL PROTECTED]> +++ ICQ: 2882018 +++
++ GnuPG Key -> http://www.t-online.de/~holger.eitzenberger ++
------------------------------
From: Ralf Render <[EMAIL PROTECTED]>
Subject: user memory locked down to kernel pages?
Date: Tue, 08 May 2001 11:26:21 +0200
Hi all,
does anybody know, whether in one of the latest kernels it is possible
to lock down user memory to kernel pages?
My kernel driver needs to lock given user memory by preventing it form
beeing swapped.
I want to use the pages of this memory for a busmaster DMA of my
PCI-Card to transfer data directly from PCI memory to user memory.
After doing that the user memory should be unlocked.
Thanks for help.
Ralf.
------------------------------
From: Kasper Dupont <[EMAIL PROTECTED]>
Subject: Re: Driver functions after a Segmentation Error
Date: Tue, 08 May 2001 09:48:42 +0000
Norm Dresner wrote:
>
[...]
>
> There were two errors in the module, the first an attempted read access of
> user-space -- copy_from_user() -- and the second a write back to the same
Giving an invalid userspace pointer to copy_from_user()
is not going to break anything, being able to handle
this is the main purpose of the function. Giving an
invalid kernelspace pointer to copy_from_user() OTOH is
a disaster, you might overwrite important kernel data
or code. And even if you just get an oops you could
leave internal kernel variables in an incosistent state,
which can cause more oopses.
After executing buggy code in the kernel or a module the
only safe thing to do is reboot.
--
Kasper Dupont
------------------------------
From: Kasper Dupont <[EMAIL PROTECTED]>
Subject: Re: file read problem
Date: Tue, 08 May 2001 09:56:02 +0000
yan zhang wrote:
>
> I have a function called read_file_to_string. In this function, I tried
> to use file->f_op->read to read a text file from local directory. I can
> open it without problem but when I read it, linux system crashed.
You are being very vague. If you explain your problem in
a little more detail and post the code perhaps we can find
the bug. What exactly do you mean when you say crashed, do
you get an Oops or a Panic message? Does the system freeeze
or start behaving strange?
There are different posible reasons for the crash:
Is file a valid pointer? Is file->f_op and
file->f_op->read valid pointers? Is the buf argument
a valid pointer? Does it point to userspace or kernel
space?
Which file are you trying to read, is it a regular
file on a well tested filesystem?
--
Kasper Dupont
------------------------------
From: [EMAIL PROTECTED] (Stefaan A Eeckels)
Subject: Re: How to get a number of processors
Crossposted-To: comp.os.linux.development.apps
Date: Tue, 8 May 2001 09:34:34 +0200
In article <[EMAIL PROTECTED]>,
[EMAIL PROTECTED] (Kaz Kylheku) writes:
> On Mon, 7 May 2001 22:53:25 +0200, Stefaan A Eeckels <[EMAIL PROTECTED]>
> wrote:
>>> You sound if you find threads morally objectionable as opposed to 'just
>>> another way to get the job done'.
>>
>>No. It's a step back from the level of hardware abstraction
>>that set Unix aside from the OSes of its generation. Worrying
>>about the differences between hardware is typically something
>>that needs to be done by the OS; putting it in an application
>>is bound to yield suboptimal, or even wrong results when the
>>hardware doesn't match the assumptions made by the application.
>
> But you said that if you want the last ounce of performance,
> you should not use an OS.
Correct. The paragraph above is not about performance, but
about so-called general or portable applications that try to
"optimize" performance by matching the number of threads
they launch to the number of processors (and typically
fudge it when there is only one, or two).
>>Because on any multi-tasking box, the presence of persitent
>>service processes, and other applications, this is an example
>>in misguided micro-optimization.
>
> Dividing an application into tasks is not a form of hardware-specific
> optimization. It's an algorithmic change. There are certain things the
> application can do when it's threaded which it could not otherwise do, such as
> interrupt low priority processing to take care of a high priority input.
Sure. But we were discussing getting the number of processors (in a
portable way) to decide (without any other significant clue as to
the loading of the machine) how many threads to launch. And even
when deciding to split a program into low/high priority execution
units, the application programmer shouldn't have to worry about
synchronization primitives.
>
> Sometimes threading is done specifically with the intent that the software
> abstraction of threading will be mapped to actual parallel hardware. This is
> by no means the only way in which threading is used, because threads
> have other benefits.
Simplicity not being one of them.
> In such number crunching programs, it's best to leave the number of threads as
> a parameter that can be tuned without recompiling the program. This way the
> best number of threads can be discovered empirically for a given
> multiprocessor. On a uniprocessor, it will invariably be found that
> no benefit is derived in having more than one thread in such a number
> crunching program, whereas on some it may be found that six is not
> enough. So it would indeed be a hardware-specific assumption to hard-code that
> number to six.
The hardware configuration is but one of the parameters affecting
that decision. The current job mix (and their demands for speed and/or
response time) also affect it; I'd argue that the OS has more
information, and _on the whole_ a better chance of getting it right,
than a guy launching a program, or an application making a thumbsuck
based on the number of processors. It's just the same argument as
the argument pro or con file systems. In my days at school, we learnt
how an application designer should carefully lay out his data on
disk, where index blocks should go, and how to match the structure of
the index to the data to be stored.
>>What you should tell the OS
>>is that you want to run as many threads as possible. And maybe
>
> This is wishful thinking, because dividing a task into threads is not trivial.
> If you tell the OS that you want to run as many threads as possible, the
> problem is that it may just do exactly what you asked for.
Vide supra. I remember the same argument being made about delegating
responsibility for managing the disks to the OS. The application
should define what can be run as threads, and the OS can know what
resources are available to run these threads.
>
>>you shouldn't have to worry about threads either; indicating
>>what you want the OS to run simultaneously should suffice.
>
> If you indicate what you want to run simultaneously, you are then doing
> explicit threading. What you really seem to be saying is that you don't want to
> do explicit threading; in other words, leave it to the system to figure out
> what should be independently scheduled.
I don't want the OS to try to intuit what I want to do, and I don't
want to be responsible for scheduling and synchronization.
> The problem is that there are so many things that could be independently
> scheduled, the hard part is deciding what *not* to schedule this way.
>
> In principle, you could independently schedule each elementary expression in a
> program, using synchronization to ensure that the minimum sequencing
> constraints are met, and let the scheduler sort it out! ;)
Which would only be efficient when you've got an essentially
unlimited number of processors. I'd ust like to see the same
support for threads as we have for processes: "Hey OS, here's
a process with a bunch of potentially independent execution
units, you make sure the lot runs as smoothly as possible".
Maybe a "cofunction()" statement at the same level as a
"function()", and not having to worry about acquiring mutexes.
--
Stefaan
--
How's it supposed to get the respect of management if you've got just
one guy working on the project? It's much more impressive to have a
battery of programmers slaving away. -- Jeffrey Hobbs (comp.lang.tcl)
------------------------------
From: [EMAIL PROTECTED] (Stefaan A Eeckels)
Subject: Re: How to get a number of processors
Crossposted-To: comp.os.linux.development.apps
Date: Tue, 8 May 2001 09:37:46 +0200
In article <[EMAIL PROTECTED]>,
Luc Van Oostenryck <[EMAIL PROTECTED]> writes:
> Stefaan A Eeckels wrote:
>>
>> In article <[EMAIL PROTECTED]>,
>> John Beardmore <[EMAIL PROTECTED]> writes:
>> > In message <[EMAIL PROTECTED]>, Stefaan A Eeckels
>> > <[EMAIL PROTECTED]> writes
>> >
>> >>Maybe the OS should provide a service to ensure that certain
>> >>processes get a minimum of CPU time. The approach you describe
>> >>is a hack, and goes completely against the basic Unix concept
>> >>of hiding the hardware differences and specifics from the
>> >>applications.
>> >
>> > Guaranteeing resources to processes may be a good tool, but it isn't an
>> > alternative to knowing how many threads to create for a big crunching
>> > job. For that, you still need to know the number of available
>> > processors.
>>
>> Why not tell the OS you want the number of parallel instances
>> optimal for the kit and job mix on hand?
>
> And how you "specify to the machine" what is kit and job mix ????
It knows its hardware resources, and what jobs are currently
running. You, as someone writing, or launching a program,
typically haven't got the faintest idea.
--
Stefaan
--
How's it supposed to get the respect of management if you've got just
one guy working on the project? It's much more impressive to have a
battery of programmers slaving away. -- Jeffrey Hobbs (comp.lang.tcl)
------------------------------
** FOR YOUR REFERENCE **
The service address, to which questions about the list itself and requests
to be added to or deleted from it should be directed, is:
Internet: [EMAIL PROTECTED]
You can send mail to the entire list by posting to the
comp.os.linux.development.system newsgroup.
Linux may be obtained via one of these FTP sites:
ftp.funet.fi pub/Linux
tsx-11.mit.edu pub/linux
sunsite.unc.edu pub/Linux
End of Linux-Development-System Digest
******************************
