Linux-Development-Sys Digest #260, Volume #8 Tue, 7 Nov 00 09:13:17 EST
Contents:
Mouse Problem ("The Dude")
Shared Memory in Linux (Manoj Patil)
Re: Is that process a thread? (Maciej Golebiewski)
Re: Is that process a thread? (Maciej Golebiewski)
Re: Is that process a thread? (George MacDonald)
Re: Shared Memory in Linux (Maciej Golebiewski)
Re: Is that process a thread? ([EMAIL PROTECTED])
Re: ext2 undelete (MENON Jean-Francois)
Re: Is that process a thread? (Alexander Viro)
Re: Is that process a thread? (George MacDonald)
Re: >2GB process memory (Wolfram Gloger)
Raw block device interface (Manoj Patil)
harddisk and partitions (MESMEUR Philippe)
writing a module for the kernel ("mamyx")
Reading files using kHTTPd method, first byte only? ("Andy Jeffries")
Re: Is that process a thread? (Maciej Golebiewski)
Re: Is that process a thread? (Maciej Golebiewski)
----------------------------------------------------------------------------
From: "The Dude" <[EMAIL PROTECTED]>
Subject: Mouse Problem
Date: Mon, 6 Nov 2000 22:50:07 -0800
Linux won't find my mini dinn mouse. It finds my mini dinn keyboard but not
mouse. The mouse is not broken because it works in win98. I am a newbie at
linux and don't know what to do. Can anyone help?
------------------------------
From: Manoj Patil <[EMAIL PROTECTED]>
Crossposted-To: comp.os.linux.networking
Subject: Shared Memory in Linux
Date: Tue, 07 Nov 2000 11:52:04 +0530
Hello,
Is there any way in Linux to increase the size of a shared memory
segment already created. I am able to do this in AIX
Regards
Manoj Patil
------------------------------
From: Maciej Golebiewski <[EMAIL PROTECTED]>
Subject: Re: Is that process a thread?
Date: Tue, 07 Nov 2000 09:11:19 +0100
George MacDonald wrote:
> > Maybe you could modify the pthreads library, so that a special tag shows up in
> > the new thread's environment and then identify threads by searching for those
> > tags in /proc/#pid/environ ?
>
> Perhaps /proc/#pid/threads/#tid/...
>
> Then one could peer into the individual threads. Could a kernel module do
> this?
I'm not sure. If it is possible without modifying the kernel (e.g. no need to
hook into clone() or whatever it is called now) then yes. On the other hand,
the kernel evolves faster than pthreads, so having a module might mean more
work
to keep up with the new versions of the kernel...
> I was thinking that I could show threads with a different shape in the tree graph
> and then also have different menu items for threads. I suppose the other approach
> is to actually create a tree inside the "process" graphic. The former seems more
> appropriate for Linux, while the latter might be more representative for Solaris...
After reading your initial post, my first thought was about displaying threads
with increased indentation under the process symbol. Don't know if it makes
sense
to you, though.
Maciej
------------------------------
From: Maciej Golebiewski <[EMAIL PROTECTED]>
Subject: Re: Is that process a thread?
Date: Tue, 07 Nov 2000 09:24:57 +0100
Alexander Viro wrote:
> Thread == process. There is no such thing as "process containing this thread"
I prefer to think of a thread as a execution flow inside a process, with
possibility
of having multiple threads. But I agree that it is not the cases in Linux,
since
it has been implemented differently.
> - there are "other processes that may share something with this process".
> E.g. address space. Moreover, PIDs are unique (as they should be).
>
> It is not Solaris. There is no distinction between the process and LWP.
> Threads are not parts of processes.
Well, not from the point of view of the operating system. But for the
user/programmer
of a multithreaded program it might make difference if a process has been
started
via fork() or pthread_create().
IMHO, what you wrote is a good argument for avoiding kernel modifications/new
modules
and limiting the changes to the pthreads library (so that it would mark the
new threads
in a way that would show up in the existing structure of /proc/#pid tree.
Maciej
------------------------------
From: George MacDonald <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Is that process a thread?
Date: Tue, 07 Nov 2000 08:36:43 GMT
Alexander Viro wrote:
>
> In article <[EMAIL PROTECTED]>,
> George MacDonald <[EMAIL PROTECTED]> wrote:
> >
> >Perhaps /proc/#pid/threads/#tid/...
> >
> >Then one could peer into the individual threads. Could a kernel module do
> >this?
>
> Thread == process. There is no such thing as "process containing this thread"
> - there are "other processes that may share something with this process".
> E.g. address space. Moreover, PIDs are unique (as they should be).
>
> It is not Solaris. There is no distinction between the process and LWP.
> Threads are not parts of processes.
Thanks, I read some of the kernel sources today and got a slightly better
handle on Linux threads implementation. I'm still a bit fuzzy on it, I'll
read some of the thread code to see what it does. I was kinda hoping that
there was some easy way to tell. Like the processes with
The same name
Exactly the same memory maps
That are in a non-interupted hierarchy
are threads.
The reason I mentioned Solaris, is that treeps runs on Solaris and UnixWare.
Treeps actually started on those platforms long before the Linux port. I am
interested in trying to provide *both* an intuative representation and
an efficient one! Not easy, but so far I have managed with treeps. Even
though it's graphical/intuative it also incurs less load than top(1).
Anyhow, treating threads as separate entities in the tree
makes sense on Linux since they are in essence processes. On Solaris/UnixWare
it may make sense to do it differently.
--
We stand on the shoulders of those giants who coded before.
Build a good layer, stand strong, and prepare for the next wave.
Guide those who come after you, give them your shoulder, lend them your code.
Code well and live! - [EMAIL PROTECTED] (7th Coding Battalion)
------------------------------
From: Maciej Golebiewski <[EMAIL PROTECTED]>
Crossposted-To: comp.os.linux.networking
Subject: Re: Shared Memory in Linux
Date: Tue, 07 Nov 2000 09:40:56 +0100
Manoj Patil wrote:
>
> Hello,
> Is there any way in Linux to increase the size of a shared memory
> segment already created. I am able to do this in AIX
You can try to shmget () a new segment and then shmat () it at the
end of your existing segment. Be aware that there are some
restrictions (alignment) on the address at which the attachement
can occur (the restrictions are described in the man page for shmat).
For example code you can get MPICH from Argonne National Lab.
(http://www-unix.mcs.anl.gov/mpi/mpich/) and take a look at the
file mpich/mpid/ch_shmem/p2p.c - it creates a single contiguous
shmem buffer by having multiple shmget's and attaching them
via shmat at the end of previously attached buffers.
HTH.
Maciej
------------------------------
From: [EMAIL PROTECTED]
Subject: Re: Is that process a thread?
Date: Tue, 07 Nov 2000 08:47:23 -0000
On Tue, 07 Nov 2000 09:24:57 +0100 Maciej Golebiewski <[EMAIL PROTECTED]> wrote:
| Alexander Viro wrote:
|> Thread == process. There is no such thing as "process containing this thread"
|
| I prefer to think of a thread as a execution flow inside a process, with
| possibility
| of having multiple threads. But I agree that it is not the cases in Linux,
| since
| it has been implemented differently.
Before I learned that processes could share memory between each other, this
was the way I thought of threads. Now I realize that this model is simply
a special case of the vast model of all possible ways to make processes work
together. Because threading packages with built around the original model,
we tend to want to think that way.
| IMHO, what you wrote is a good argument for avoiding kernel modifications/new
| modules
| and limiting the changes to the pthreads library (so that it would mark the
| new threads
| in a way that would show up in the existing structure of /proc/#pid tree.
Why not use PID/PPID relationships?
--
| Phil Howard - KA9WGN | My current websites: linuxhomepage.com, ham.org
| phil (at) ipal.net +----------------------------------------------------
| Dallas - Texas - USA | [EMAIL PROTECTED]
------------------------------
From: MENON Jean-Francois <[EMAIL PROTECTED]>
Subject: Re: ext2 undelete
Date: Tue, 07 Nov 2000 09:04:08 +0000
[EMAIL PROTECTED] wrote:
>
> |> If your intent is to change the behaviour of unlink() to move a
> |> file to a garbage area
> |
> | only the list of data blocks
>
> How will you track reuse of the blocks? What about pointers?
no guarantee, no pointers, I just save the inode wih the list of the
blocks numbers in
a file that user can use ( with my utility) to try to reread the file (
if the datas
have not been overwritten). I consider that it uses little disk space (
the blocks are really frees) but can be usefull if you realise enough
soon the mistake.
>
> |> and later empty the garbage, you need to consider
> |> a lot of things about how this should work.
> |>
> |> 1. How will you empty the garbage?
> |
> | cron seems good for that
>
> If you are just tracking unlinked data, then it's not the garbage data
> OTOH, if you do keep a garbage area, then you could do this logically
> across all filesystem types.
>
yes , it is not the garbage data, but the garbage "list of blocks"
> |> | (my "rm" command already works) It seems to be somthing with ldconfig,
> |> | but I'm not sure.
> |>
> |> With ldconfig you can change libraries around and use a different one.
> |> Is that how you are intending to replace unlink()?
> |>
> |> --
> | something like that. I'm looking for the best way to implement my
> | undelete idea.
> | maybe directly in patching the ext2fs in the kernel but I don't know
> | very much about that. maybe someone have already done this?
>
> Decide how (many ways to do this) you are holding on to deleted files.
already done.
> One approach is to move the inodes to lost+found once final unlink
> (this relinks them) but obvious don't do this if it was unlinked from
> lost+found.
I didn't think about that ! thx :->
--
- Les propos tenus ici n'engagent que ceux qui les lisent.
"Il y a tellement de choses qu'on voudrait avoir faites
hier et si peu qu'on a envie de faire aujourd'hui."
[ Mignon MacLaughlin ]
------------------------------
From: [EMAIL PROTECTED] (Alexander Viro)
Subject: Re: Is that process a thread?
Date: 7 Nov 2000 04:15:27 -0500
In article <[EMAIL PROTECTED]>,
George MacDonald <[EMAIL PROTECTED]> wrote:
>Thanks, I read some of the kernel sources today and got a slightly better
>handle on Linux threads implementation. I'm still a bit fuzzy on it, I'll
>read some of the thread code to see what it does. I was kinda hoping that
>there was some easy way to tell. Like the processes with
>
> The same name
> Exactly the same memory maps
> That are in a non-interupted hierarchy
>
>are threads.
The last part is simply wrong - if child, parent and grandparent share VM
and parent does exec() we get child and grandparent sharing VM and parent
having a separate one. So non-interrupted hierarchy is not promised.
Don't forget, there are other parts of process. E.g. you may have A
sharing VM with B, B sharing descriptor table with C while A and C have
nothing in common.
--
"You're one of those condescending Unix computer users!"
"Here's a nickel, kid. Get yourself a better computer" - Dilbert.
------------------------------
From: George MacDonald <[EMAIL PROTECTED]>
Reply-To: [EMAIL PROTECTED]
Subject: Re: Is that process a thread?
Date: Tue, 07 Nov 2000 09:55:52 GMT
[EMAIL PROTECTED] wrote:
>
> On Mon, 06 Nov 2000 20:35:49 GMT George MacDonald <[EMAIL PROTECTED]> wrote:
> | Maciej Golebiewski wrote:
> |>
> |> George MacDonald wrote:
> |> >
> |> > Is there an easy way to figure out if a process is a thread? I'm thinking
> |> > about adding a new visual representation for threads in treeps. i.e.
> |> >
> |> > http://www.slip.net/~gmd/tps/treeps.htm
> |> >
> |> > but need a quick way to figure out if an entry in /proc is a thread.
> |>
> |> I think that the only way to check if two two entries in /proc are two threads
> |> of the same process is to verify that they share the same address space. But
> |> I'm not sure if there is enough information in /proc to do it... Maybe not.
> |>
> |> Maybe you could modify the pthreads library, so that a special tag shows up in
> |> the new thread's environment and then identify threads by searching for those
> |> tags in /proc/#pid/environ ?
> |
> | Perhaps /proc/#pid/threads/#tid/...
> |
> | Then one could peer into the individual threads. Could a kernel module do
> | this?
>
> What is initially a thread can become a non-thread by remapping its address
> space on it's own. That may not be what threads are supposed to be able to
> do, but a process can do that. It could be argued that a thread should be
> able to do this anyway.
Mighty morphing code!! So can a process become a thread? What's the difference
between a forked process and a clone(thread)? Stack?
>
> How much of the address has to overlap to be a thread? 100%? Does each thread
> really need access to another thread's stack (probably yes, though I could also
> argue that was be bad and possibly dangerous practice).
What's the best way to check for address space matching? The mem maps? (/proc/#/maps)
>
> The point is, we look at threads in black and white, but in fact the reality
> is that things are really very gray. We need to be prepared to deal with all
> of that.
>
> | I was thinking that I could show threads with a different shape in the tree graph
> | and then also have different menu items for threads. I suppose the other approach
> | is to actually create a tree inside the "process" graphic. The former seems more
> | appropriate for Linux, while the latter might be more representative for Solaris...
>
> How about a showing of how much address space is shared between them?
Hmm, like assigning each memory map a texture and creating pie slices on
a circle that backgrounds the process? Size of circle is proportional to
the process size, pie slices are proportional to the % of process space?
Put a texture map legend for memory "objects" in undockable panel? Doable
but getting the color/textures to look good won't be easy. Perhaps really
faint almost ghost like so that it does not detract from the tree...
I was thinking of using the same "background process info" to show process
load history, I call those "pulse plots". Load impulses would be like tree
rings... Hmm that makes a nice metaphor. Perhaps the program could switch
between them automatically, and allow the user to toggle the selection
when desired.
--
We stand on the shoulders of those giants who coded before.
Build a good layer, stand strong, and prepare for the next wave.
Guide those who come after you, give them your shoulder, lend them your code.
Code well and live! - [EMAIL PROTECTED] (7th Coding Battalion)
------------------------------
From: Wolfram Gloger <[EMAIL PROTECTED]>
Crossposted-To: comp.os.linux.misc
Subject: Re: >2GB process memory
Date: 07 Nov 2000 11:52:09 +0100
Andre Rohe <[EMAIL PROTECTED]> writes:
> we have a 2GB memory and 2GB swap-space machine here, and I would like
> to address about 3GB of memory in one process. We tried the standard Suse 7.0
> kernel and 2.4.0 test9 (we tried the 4GB and 64GB memory option).
> We never managed to malloc >2GB in one process.
Depending on the size of the chunks that you allocate, an upgrade to
glibc-2.1.97 (warning: that is a pre-release) or better to glibc-2.2
when it is released may help. Alternatively, try setting
MALLOC_MMAP_MAX_ to a really large number (the default, 1024, may be
too little for you).
Note that 3GB will be the absolute upper limit for memory that you can
_address_ on Intel-Linux at the moment. In practice, you will be able
to _allocate_ only somewhat less memory..
Regards,
Wolfram.
------------------------------
From: Manoj Patil <[EMAIL PROTECTED]>
Crossposted-To: comp.unix.programmer,comp.os.linux.networking
Subject: Raw block device interface
Date: Tue, 07 Nov 2000 16:39:50 +0530
Hi !
Is raw interface available in linux for block devices ?
I have a partition and need to access it through a raw interface.
Basically the aim of my program is to bypass the buffer cache and be
assured that every read/write operation results into disk activity.
Thanks in advance
Regards
Manoj Patil
------------------------------
From: MESMEUR Philippe <[EMAIL PROTECTED]>
Subject: harddisk and partitions
Date: Tue, 07 Nov 2000 12:21:39 +0100
Hello, I'm looking for a way to get the list of the hard-drives an the
list for all the partitions in each hard-disk. I think the is a system
call to to this.
Can you please help me. Thanks
--
========================================================
Oce-Industries SA
1, rue Jean Lemoine
94015 Creteil cedex France
phone: 33-1-48988000 fax: 33-1-48985450
========================================================
------------------------------
From: "mamyx" <[EMAIL PROTECTED]>
Subject: writing a module for the kernel
Date: Tue, 7 Nov 2000 13:52:31 +0100
hi
can I use IPC System V utilities in modules (then in kernel code) ????
I want for example use semaphores and messages queues .
thanks
------------------------------
From: "Andy Jeffries" <[EMAIL PROTECTED]>
Subject: Reading files using kHTTPd method, first byte only?
Date: Tue, 07 Nov 2000 13:31:18 +0000
Hi guys,
I am writing an on-the-fly-encryption module for Linux (yes, it will be
GPL'd and yes, I know that there is a Kernel patch and loopback interface
for this, but it doesn't create container files that are compatible with
a certain Windoze product) and have come accross a problem.
I am trying to use the same file reading mechanism as kHTTPd, but it only
seems to read the first byte. I am running 2.2.16 (at the moment) which
may be a problem (kHTTPd only seems to come with 2.4...), but it is
curious that the code seems to partially work (i.e read a byte).
Attached below is the relevant portions of the code, with some (hopefully
simple) explanations:
===== CODE BEGINS =====
/* slots is an array of my structures, contains a struct file *, and a
char * filename */
slots[slotno].filp = filp_open(slots[slotno].sd_mount->filename, 00, O_RDWR);
/* then it calls another function passing in slots[slotno] as sd_slot.
The code in that function is as follows */
char *pHeaderBlock;
int iNumRead;
mm_segment_t oldfs;
sd_slot->filp->f_pos = 0;
oldfs = get_fs();
set_fs(KERNEL_DS);
/* HEADER_BLOCK_SIZE is 2048, the size of the header block for the
container */
iNumRead = sd_slot->filp->f_op->read(sd_slot->filp,
pHeaderBlock, HEADER_BLOCK_SIZE, &(sd_slot->filp->f_pos));
if (iNumRead < HEADER_BLOCK_SIZE) {
printk(KERN_ERR "scramdisk: Cannot read the header block\n");
return NULL;
}
printk(KERN_ERR "scramdisk: pHeaderBlock = %X, read %d bytes\n",
*pHeaderBlock, iNumRead);
===== CODE ENDS =====
Now, I know I should have used a lower printk priority, but I was having
difficulty finding where it was logging the damn thing! :-)
It usually reads the header block OK (iNumRead == HEADER_BLOCK_SIZE) but
*pHeaderBlock has some strange contents. For example:
Running it on a blowfish encrypted container logs:
> Nov 7 10:44:17 ialtd07 kernel: scramdisk: pHeaderBlock = FFFFFF8F
While hexdump on the same file gives (notice the 8f):
> 0000000 f18f bd5c
Running it on /dev/zero logs:
> Nov 7 10:44:39 ialtd07 kernel: scramdisk: pHeaderBlock = 0
While hexdumping /dev/zero gives (duh!!):
> 0000000 0000 0000
Running it on a text file gives:
> Nov 7 10:45:21 ialtd07 kernel: scramdisk: pHeaderBlock = 72
While hexdump on the same file gives:
> 0000000 6f72 746f
Any ideas???
--
Andy Jeffries
Lead-developer of Scramdisk for Linux
Developer of original Scramdisk Delphi Component
------------------------------
From: Maciej Golebiewski <[EMAIL PROTECTED]>
Subject: Re: Is that process a thread?
Date: Tue, 07 Nov 2000 14:52:44 +0100
[EMAIL PROTECTED] wrote:
> [...] Now I realize that this model is simply
> a special case of the vast model of all possible ways to make processes work
> together.
Uh, and the mind boggles again :)
> | IMHO, what you wrote is a good argument for avoiding kernel modifications/new
> | modules
> | and limiting the changes to the pthreads library (so that it would mark the
> | new threads
> | in a way that would show up in the existing structure of /proc/#pid tree.
>
> Why not use PID/PPID relationships?
Because this does not distinguish between processes created by fork and
pthread_create, and while if they look the same for the system, the user might
still be interested in distinguishing them (I had a few situations myself
where
it would help me).
Maciej
------------------------------
From: Maciej Golebiewski <[EMAIL PROTECTED]>
Subject: Re: Is that process a thread?
Date: Tue, 07 Nov 2000 14:55:55 +0100
George MacDonald wrote:
> Mighty morphing code!! So can a process become a thread? What's the difference
> between a forked process and a clone(thread)? Stack?
After reading Phil's and Alexander's replies I come to conclusion that the
only difference is a programmer's intention :)
Maciej
------------------------------
** FOR YOUR REFERENCE **
The service address, to which questions about the list itself and requests
to be added to or deleted from it should be directed, is:
Internet: [EMAIL PROTECTED]
You can send mail to the entire list (and comp.os.linux.development.system) via:
Internet: [EMAIL PROTECTED]
Linux may be obtained via one of these FTP sites:
ftp.funet.fi pub/Linux
tsx-11.mit.edu pub/linux
sunsite.unc.edu pub/Linux
End of Linux-Development-System Digest
******************************
