Re: ipcrm/shmctl failure (fix NOT found)
[EMAIL PROTECTED] wrote:
[EMAIL PROTECTED] writes:
} I didn't know if you were talking about not incrementing when the
} process exits or when it rforked. If you rfork(RFMEM), you'd want to
} increment the vm_refcnt I'm pretty sure (and it does).
}
}No, you really don't.
I don't know or we don't want to increment the vm_refcnt when rforking?
Yopu don't want to increment the refcnt on the shared memory
segment twice.
}You have a number of references on the vm (one per RFMEM) process.
}The correct translation of these references is to have a *single*
}reference count instance to the shared memory segment itself,
}rather than incrementing the segment references, shmseg-shm_nattch.
Ok - so shmfork can not increment shm_nattch. But you still want
to increment vm_refcnt when you rfork or your second sentence is a
contradiction (one ref per RFMEM). But you are saying there is a single
vm (albeit with multiple references to it) but because it's only one vm
there is in effect a _single_ reference to the shmseg from that.
Do I understand you correctly?
Yes. Otherwise, for N processes, you end up with N^2 references,
which is clearly incorrect.
}If the VM reference counting on normal segments weren't working,
}then there'd be a huge-and-obvious-to-everyone problem. I think
}that incrementing the shmseg-shm_nattch on the vfork is definitely
}the wrong thing to do.
It's surprising what people don't notice.
Use of rfork() in combination with shmat() is uncommenm at best,
and never done in any code that ships with FreeBSD, including
not being done by any ports, at worst.
However, I can say from personal experience with a commercial
product that used both of these at the same time in the code,
that there is no bad interaction in the FreeBSD-only case, so
long as there is not an RFMEM-without-RFPROC or RFPROC-without-RFMEM,
which is the Linux threads case.
}Since your problem is a symptom of increment of shmseg-shm_nattch
}without a corresponding decrement, then the *only* code that can be
}involved is shmat() and shmfork() for the increment, and for the
}delete, shm_delete_mapping(), which is called from shmexit() and
}shmdt().
No, I don't think I said that - all I know is that shmexit never gets
called and that seems to be because vm_refcnt is incremented.
So you need to instrument the code, and print out the pric ID,
which will be different, at each increment and decrement. When
you see an increment without a corresponding decrement, you will
have found your problem.
}That basically impies that RFMEM is not set when vm_fork() is called
}from the Linux ABI code, since that's the only place that calls the
}shmfork() code.
Nah, I checked that. It does a clone(CLONEVM) in the linux threads lib
which translates to a rfork(RFMEM) in i386/linux/linux_machdep.c .
Again: you need to instrument the code. What the code says it
does, and what the code actually does may be two different things;
for all I know, you are loading the Linux ABI and the Linux threads
as a kernel module, and the header file manifest constants between
the module(s) and the kernel code are different.
} The whole bug is
} the point that vm_refcnt is never decremented and the shm_nattch is
} therefore only decremented if you explicitly detach from memory (which
} will call shm_delete_mapping). So if an rfork'd program uses shared mem
} and crashes, the vm_refcnt stays 1, the shared mem is never freed
} because shmexit - shm_delete_mapping is never called. Hopefully this
} only affects shared mem, as there is more stuff inside the if statement
} you include below other than the shmexit.
}It should not be incremented in the first place. It is erroneously
}incremented, IMO.
You mean shm_nattch is erroneously incremented, not vm_refcnt I think?
Yes.
} Don't know about the race, although one is mentioned in the cvs logs on
} the current branch. I presume you're talking SMP only though?
} As a side note, in current this reads:
} if (--vmspace-vm_refcnt == 0) {
}
}
}Yes. This doesn't have the race, because there isn't a window between
}the time of the compare and the decrement.
Perhaps what I'm really seeing is the race then? I do have a single vm
with a single ref to a shmseg, but when the process crashes all the
rforked processes exit and clobber the vm_refcnt so that shmexit never
gets called to decrement shm_nattch to zero? A new theory...
It's possible. You need to instrument the code, and count the
kernel printfs for the increment and decrement. If they are
equal, then it there should be zero references; if there are not,
then you know that you lost a race. I expect that they will not
be equal.
A big-time crash like this is really an extreme condition, but you
pointed out that the -current code doesn't have the race. I had
assumed you were running -current because of this.
} but I don't know who calls that unless it somehow happens from cpu_exit.
Re: ipcrm/shmctl failure (fix NOT found)
[EMAIL PROTECTED] writes: [ suggested fix ] [ snip ] }This looks right, to me. Sigh. With the fix as in current (decrement the vm_refcnt in the if statement) the system panics on boot with a vmspace already free message. With my version of the fix (only decrement if vm_refcnt) not 1 by calling vmspace_free(), the system ran for about 10 minutes and then hung. So for the moment I'll just leave it and avoid linux threads (or anything using rfork) and shared memory unless anyone else has other ideas. I can't afford the time to work out the vm code any further at the moment. C Callum Gibson [EMAIL PROTECTED] Global Markets IT, Deutsche Bank, Australia 61 2 9258 1620 ### The opinions in this message are mine and not Deutsche's ### To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure (fix NOT found)
[EMAIL PROTECTED] wrote: [EMAIL PROTECTED] writes: [ suggested fix ] [ snip ] }This looks right, to me. Sigh. With the fix as in current (decrement the vm_refcnt in the if statement) the system panics on boot with a vmspace already free message. With my version of the fix (only decrement if vm_refcnt) not 1 by calling vmspace_free(), the system ran for about 10 minutes and then hung. So for the moment I'll just leave it and avoid linux threads (or anything using rfork) and shared memory unless anyone else has other ideas. I can't afford the time to work out the vm code any further at the moment. Heh. That's a Biiig snip. I had a huge caveat on the code doing what it did where it did it, if you'll remember. 8-). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure (fix NOT found)
[EMAIL PROTECTED] writes: }Heh. That's a Biiig snip. I had a huge caveat on the code }doing what it did where it did it, if you'll remember. 8-). Sorry, Terry it was no slight on you (and it looked fine to me too). I just wanted to get something in the list archives in case some poor sod decided to search them and try it out. Also to warn against a simple MFC (though people always test that stuff, don't they? :-) C Callum Gibson [EMAIL PROTECTED] Global Markets IT, Deutsche Bank, Australia 61 2 9258 1620 ### The opinions in this message are mine and not Deutsche's ### To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure (fix NOT found)
[EMAIL PROTECTED] wrote:
[EMAIL PROTECTED] writes:
}Heh. That's a Biiig snip. I had a huge caveat on the code
}doing what it did where it did it, if you'll remember. 8-).
Sorry, Terry it was no slight on you (and it looked fine to me too). I just
wanted to get something in the list archives in case some poor sod decided
to search them and try it out. Also to warn against a simple MFC (though
people always test that stuff, don't they? :-)
I just wanted to make sure that someone would not stop their
back tracking. In my previous posting, I offered what I
thought would be the correct approach to solving the problem:
] You'd actually think that not incrementing in the RFMEM case, but
] then decrementing if the RFMEM reference goes from 1-0 would be
] the correct thing to do.
This should actually be a pretty trivial to fix. If you look in
/sys/kern/kern_exec.c in exec_new_vmspace(), you'll see the proper
way of exiting on the shm instance:
if (vmspace-vm_refcnt == 1) {
if (vmspace-vm_shm)
shmexit(imgp-proc);
...in other words, the resource track exit does not occur until
the reference count is about to go from 1-0. Note that there
is an implicit race here, actually, between the reference and
the detach, in which another instance could conceivably be
created. 8-(.
At this point, I think it would be wise to instrument rfork, fork,
vm_fork, shmfork, and shmexit to see what's going on with your
particular program.
It may be that your program is reattaching an already attached
shared memory segment, and expecting the behaviour to be sane.
Really, the place to look for that would be in the Linux kernel
sources, to see how it handled shares memory segments with Linux
threads... it may be that it doesn't expect them to be attached,
and that each thread is expected to do the attach. The above
instrumentation points should tell you this.
-- Terry
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure (fix NOT found)
[EMAIL PROTECTED] writes:
}] You'd actually think that not incrementing in the RFMEM case, but
}] then decrementing if the RFMEM reference goes from 1-0 would be
}] the correct thing to do.
I didn't know if you were talking about not incrementing when the
process exits or when it rforked. If you rfork(RFMEM), you'd want to
increment the vm_refcnt I'm pretty sure (and it does). The whole bug is
the point that vm_refcnt is never decremented and the shm_nattch is
therefore only decremented if you explicitly detach from memory (which
will call shm_delete_mapping). So if an rfork'd program uses shared mem
and crashes, the vm_refcnt stays 1, the shared mem is never freed
because shmexit - shm_delete_mapping is never called. Hopefully this
only affects shared mem, as there is more stuff inside the if statement
you include below other than the shmexit.
}This should actually be a pretty trivial to fix. If you look in
}/sys/kern/kern_exec.c in exec_new_vmspace(), you'll see the proper
}way of exiting on the shm instance:
}
}if (vmspace-vm_refcnt == 1) {
}if (vmspace-vm_shm)
}shmexit(imgp-proc);
}
}...in other words, the resource track exit does not occur until
}the reference count is about to go from 1-0. Note that there
}is an implicit race here, actually, between the reference and
}the detach, in which another instance could conceivably be
}created. 8-(.
Don't know about the race, although one is mentioned in the cvs logs on
the current branch. I presume you're talking SMP only though?
As a side note, in current this reads:
if (--vmspace-vm_refcnt == 0) {
However, I can't find the spot where the ref count _actually_ goes to zero
in 4.5 - I suspect it does, but the only decrement of vm_refcnt in the
code is in vmspace_free and I traced all calls to that. I suspect it
just frees all the memory associated with the process on exit
without doing the final decrement to zero. There is a comment just above
cpu_exit which says:
* The address space is released by vmspace_free(p-p_vmspace);
but I don't know who calls that unless it somehow happens from cpu_exit.
}At this point, I think it would be wise to instrument rfork, fork,
}vm_fork, shmfork, and shmexit to see what's going on with your
}particular program.
}
}It may be that your program is reattaching an already attached
}shared memory segment, and expecting the behaviour to be sane.
}
}Really, the place to look for that would be in the Linux kernel
}sources, to see how it handled shares memory segments with Linux
}threads... it may be that it doesn't expect them to be attached,
}and that each thread is expected to do the attach. The above
}instrumentation points should tell you this.
This is not limited to linux threads, it should affect anything which
increments vm_refcnt and allocates shared mem. It's obvious what should
happen, just not obvious how to implement it without causing a side effect.
Not sure that seeing how linux does it would help in this regard.
Anyway, all you volunteers step right up...
Callum Gibson [EMAIL PROTECTED]
Global Markets IT, Deutsche Bank, Australia 61 2 9258 1620
### The opinions in this message are mine and not Deutsche's ###
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure (fix NOT found)
[EMAIL PROTECTED] wrote:
I didn't know if you were talking about not incrementing when the
process exits or when it rforked. If you rfork(RFMEM), you'd want to
increment the vm_refcnt I'm pretty sure (and it does).
No, you really don't.
You have a number of references on the vm (one per RFMEM) process.
The correct translation of these references is to have a *single*
reference count instance to the shared memory segment itself,
rather than incrementing the segment references, shmseg-shm_nattch.
If the VM reference counting on normal segments weren't working,
then there'd be a huge-and-obvious-to-everyone problem. I think
that incrementing the shmseg-shm_nattch on the vfork is definitely
the wrong thing to do.
The reference to the shared memory segment is by the VM... not by
the process that references the VM.
Since your problem is a symptom of increment of shmseg-shm_nattch
without a corresponding decrement, then the *only* code that can be
involved is shmat() and shmfork() for the increment, and for the
delete, shm_delete_mapping(), which is called from shmexit() and
shmdt().
That basically impies that RFMEM is not set when vm_fork() is called
from the Linux ABI code, since that's the only place that calls the
shmfork() code.
The whole bug is
the point that vm_refcnt is never decremented and the shm_nattch is
therefore only decremented if you explicitly detach from memory (which
will call shm_delete_mapping). So if an rfork'd program uses shared mem
and crashes, the vm_refcnt stays 1, the shared mem is never freed
because shmexit - shm_delete_mapping is never called. Hopefully this
only affects shared mem, as there is more stuff inside the if statement
you include below other than the shmexit.
It should not be incremented in the first place. It is erroneously
incremented, IMO.
}...in other words, the resource track exit does not occur until
}the reference count is about to go from 1-0. Note that there
}is an implicit race here, actually, between the reference and
}the detach, in which another instance could conceivably be
}created. 8-(.
Don't know about the race, although one is mentioned in the cvs logs on
the current branch. I presume you're talking SMP only though?
As a side note, in current this reads:
if (--vmspace-vm_refcnt == 0) {
Yes. This doesn't have the race, because there isn't a window between
the time of the compare and the decrement.
However, I can't find the spot where the ref count _actually_ goes to zero
in 4.5 - I suspect it does, but the only decrement of vm_refcnt in the
code is in vmspace_free and I traced all calls to that. I suspect it
just frees all the memory associated with the process on exit
without doing the final decrement to zero. There is a comment just above
cpu_exit which says:
* The address space is released by vmspace_free(p-p_vmspace);
but I don't know who calls that unless it somehow happens from cpu_exit.
The reference is initialized to 1 when it is created. See vmspace_alloc()
in vm_map.c.
}At this point, I think it would be wise to instrument rfork, fork,
}vm_fork, shmfork, and shmexit to see what's going on with your
}particular program.
}
}It may be that your program is reattaching an already attached
}shared memory segment, and expecting the behaviour to be sane.
}
}Really, the place to look for that would be in the Linux kernel
}sources, to see how it handled shares memory segments with Linux
}threads... it may be that it doesn't expect them to be attached,
}and that each thread is expected to do the attach. The above
}instrumentation points should tell you this.
This is not limited to linux threads, it should affect anything which
increments vm_refcnt and allocates shared mem. It's obvious what should
happen, just not obvious how to implement it without causing a side effect.
Not sure that seeing how linux does it would help in this regard.
I think it is Linux specific. I think it is related to RFMEM not
being set in flags when the vm_fork() is called.
-- Terry
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure (fix NOT found)
[EMAIL PROTECTED] writes:
} I didn't know if you were talking about not incrementing when the
} process exits or when it rforked. If you rfork(RFMEM), you'd want to
} increment the vm_refcnt I'm pretty sure (and it does).
}
}No, you really don't.
I don't know or we don't want to increment the vm_refcnt when rforking?
}You have a number of references on the vm (one per RFMEM) process.
}The correct translation of these references is to have a *single*
}reference count instance to the shared memory segment itself,
}rather than incrementing the segment references, shmseg-shm_nattch.
Ok - so shmfork can not increment shm_nattch. But you still want
to increment vm_refcnt when you rfork or your second sentence is a
contradiction (one ref per RFMEM). But you are saying there is a single
vm (albeit with multiple references to it) but because it's only one vm
there is in effect a _single_ reference to the shmseg from that.
Do I understand you correctly?
}If the VM reference counting on normal segments weren't working,
}then there'd be a huge-and-obvious-to-everyone problem. I think
}that incrementing the shmseg-shm_nattch on the vfork is definitely
}the wrong thing to do.
It's surprising what people don't notice.
}Since your problem is a symptom of increment of shmseg-shm_nattch
}without a corresponding decrement, then the *only* code that can be
}involved is shmat() and shmfork() for the increment, and for the
}delete, shm_delete_mapping(), which is called from shmexit() and
}shmdt().
No, I don't think I said that - all I know is that shmexit never gets
called and that seems to be because vm_refcnt is incremented.
}That basically impies that RFMEM is not set when vm_fork() is called
}from the Linux ABI code, since that's the only place that calls the
}shmfork() code.
Nah, I checked that. It does a clone(CLONEVM) in the linux threads lib
which translates to a rfork(RFMEM) in i386/linux/linux_machdep.c .
} The whole bug is
} the point that vm_refcnt is never decremented and the shm_nattch is
} therefore only decremented if you explicitly detach from memory (which
} will call shm_delete_mapping). So if an rfork'd program uses shared mem
} and crashes, the vm_refcnt stays 1, the shared mem is never freed
} because shmexit - shm_delete_mapping is never called. Hopefully this
} only affects shared mem, as there is more stuff inside the if statement
} you include below other than the shmexit.
}It should not be incremented in the first place. It is erroneously
}incremented, IMO.
You mean shm_nattch is erroneously incremented, not vm_refcnt I think?
} }...in other words, the resource track exit does not occur until
} }the reference count is about to go from 1-0. Note that there
} }is an implicit race here, actually, between the reference and
} }the detach, in which another instance could conceivably be
} }created. 8-(.
}
} Don't know about the race, although one is mentioned in the cvs logs on
} the current branch. I presume you're talking SMP only though?
} As a side note, in current this reads:
} if (--vmspace-vm_refcnt == 0) {
}
}
}Yes. This doesn't have the race, because there isn't a window between
}the time of the compare and the decrement.
Perhaps what I'm really seeing is the race then? I do have a single vm
with a single ref to a shmseg, but when the process crashes all the
rforked processes exit and clobber the vm_refcnt so that shmexit never
gets called to decrement shm_nattch to zero? A new theory...
} without doing the final decrement to zero. There is a comment just above
} cpu_exit which says:
}
} * The address space is released by vmspace_free(p-p_vmspace);
}
} but I don't know who calls that unless it somehow happens from cpu_exit.
}The reference is initialized to 1 when it is created. See vmspace_alloc()
}in vm_map.c.
But where does vm_refcnt go to zero (in 4.5)?
} This is not limited to linux threads, it should affect anything which
} increments vm_refcnt and allocates shared mem. It's obvious what should
} happen, just not obvious how to implement it without causing a side
}effect.
} Not sure that seeing how linux does it would help in this regard.
}I think it is Linux specific. I think it is related to RFMEM not
}being set in flags when the vm_fork() is called.
As best I could tell, RFMEM is, in fact, set by the library and by the
kernel.
Callum Gibson [EMAIL PROTECTED]
Global Markets IT, Deutsche Bank, Australia 61 2 9258 1620
### The opinions in this message are mine and not Deutsche's ###
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure
[EMAIL PROTECTED] wrote: } You could say that X11 shouldn't use SHMs the way it does now yeah. =) }The real problem is that over the UNIX domain socket, it doesn't }get client disconnect notificiations necessary for resource tracking, }AND browser use of these resources is practically the degenerate case. I think you're right about the linux stuff. It looks that you _will_ get an EINVAL on a valid shmid if it has already been marked for deletion (but presumably still exists due to references). Using the -o to ipcs I see there are existing references (why didn't anyone suggest that before) - but these are by non existent processes. I'm now pretty sure this is caused by the linux implementation of threads which uses multiple processes instead of a single proces. Now vm_fork() (vm/vm_glue.c) calls shmfork which increments the shm ref count for the forked process but this _shouldn't_ get called when rfork is called with RFMEM (which is what linux_clone() does). So, turning to the exit side of things, sys_exit() calls exit1() which only calls shmexit to decrement the count if vm_refcnt == 1. Ok, rfork(RFMEM) increments vm_refcnt in vm_fork(), and it may be decremented in vmspace_unshare() or vmspace_free(). However as best I can see, nothing in exit1() calls any vm stuff unless vm_refcnt is already 1. So unless someone can point out where I've missed the call, I think the fix is something like: [ ... ] This looks right, to me. You'd actually think that not incrementing in the RFMEM case, but then decrementing if the RFMEM reference goes from 1-0 would be the correct thing to do. However... 1) This is actually simpler, since it avoids the need to check on the RFMEM case, when there is an out of order close 2) The System V shm case is special, in that the shared memory segments actually come out of the kernel virtual memory address space, not the process (there are good reasons for this, actually). linux-mozilla would be a pretty heavy thread user. If only linux threads was posix compliant, eh (I know Terry has commented on this before). If you scould send-pr this, I think it would be more likely to be committed (or send it to a committer you acn work with, like Alfred... 8-)). -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure
I replied privately to Terry, but I perhaps should have mentioned also that some of these segments were days old and they were definitely not in use by any process. I logout at the end of every day. [EMAIL PROTECTED] writes: }All you are doing is marking the segment as removed. The segment }remains attached by the processes which have it open, and those }references don't go awaya until the processes in question detach }the segments, and the reference count goes to zero. } }In other words, shared memory segments are like files; you can't }delete them out from under programs that still hold references }to them. [ snip ] Callum Gibson [EMAIL PROTECTED] Global Markets IT, Deutsche Bank, Australia 61 2 9258 1620 ### The opinions in this message are mine and not Deutsche's ### To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure
On Tue, 9 Apr 2002 [EMAIL PROTECTED] wrote: I replied privately to Terry, but I perhaps should have mentioned also that some of these segments were days old and they were definitely not in use by any process. I logout at the end of every day. Have you tried ipcs -p? Andrew To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure
[EMAIL PROTECTED] wrote: I replied privately to Terry, but I perhaps should have mentioned also that some of these segments were days old and they were definitely not in use by any process. I logout at the end of every day. Logging out does not necessarily stop all processes, or remove the mapping. Note that there was some confusion about shm_open(3), which is totally unrelated to the ipcs/ipcrm/shmat/shmdt/etc. code. I think this is probably a bug in the Linux emulator resource tracking on abort/exit for system V shared memory, since it's a Linux program causing the problem. In any case, disabling the use of the MIT SHM extension for X will make the problem go away by making it not use shared memory segments. You can either do this golbally, with an option to the X server, or you can do it on a program-by-program basis by setting the DISPLAY environment variable so that it uses a real network connection, instead of a UNIX domain socket (and thus allows the use of the MIT SHM extension). E.g., per the previous post: setenv DISPLAY `hostname`:0.0 -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure
Andrew wrote: On Tue, 9 Apr 2002 [EMAIL PROTECTED] wrote: I replied privately to Terry, but I perhaps should have mentioned also that some of these segments were days old and they were definitely not in use by any process. I logout at the end of every day. Have you tried ipcs -p? I believe this will give incorrect information, as it's pretty obvious that any bug here has to be in the Linux ABI treatment of the _exit() resource tracking cleanup of the segments when a proce (or Linux thread) exits. The information -p gives is the last access. However, if what has happened is that all the programs have been stopped, and the reference count was not decremented by the Linux ABI code, then the last reference you will see is the already exited X server. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure
At 8:45 PM +1000 4/9/02, [EMAIL PROTECTED] wrote: Please don't make me use the kernel debugger... waa I believe Terry suggested: ...set the DISPLAY environment variable so that it uses a real network connection, instead of a UNIX domain socket (and thus allows the use of the MIT SHM extension). E.g., per the previous post: setenv DISPLAY `hostname`:0.0 I (personally would try that, and see what effect it had, before I would dive into a kernel debugger! Maybe it will have no effect, but even then you will at least have eliminated this issue from the discussion. -- Garance Alistair Drosehn= [EMAIL PROTECTED] Senior Systems Programmer or [EMAIL PROTECTED] Rensselaer Polytechnic Instituteor [EMAIL PROTECTED] To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure
At 22:06 8-4-2002 -0700, Terry Lambert wrote: All you are doing is marking the segment as removed. The segment remains attached by the processes which have it open, and those references don't go awaya until the processes in question detach the segments, and the reference count goes to zero. It turns out that these segments are not proerly reference counted and tracked, so they are not deleted when the client goes away. This is a bug in the MIT shared memory extension for X design, and can't be fixed for long running programs with lots of bitmaps. I'd like to take a step further and say it's in SYSVSHM design. All a program has to do is forget to do a shm_detach() and you're f#$%ed. Be glad it's just a few bitmaps, and not a 250 meg segment like I had with a certain version of Oracle. It's an X11 question, and it's been that way since at least 1994, so it's a long standing X11 FAQ. You could say that X11 shouldn't use SHMs the way it does now yeah. =) To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure
Rogier R. Mulhuijzen wrote: This is a bug in the MIT shared memory extension for X design, and can't be fixed for long running programs with lots of bitmaps. I'd like to take a step further and say it's in SYSVSHM design. All a program has to do is forget to do a shm_detach() and you're f#$%ed. Heh. I could make the same argument about open... Be glad it's just a few bitmaps, and not a 250 meg segment like I had with a certain version of Oracle. It's an X11 question, and it's been that way since at least 1994, so it's a long standing X11 FAQ. You could say that X11 shouldn't use SHMs the way it does now yeah. =) The real problem is that over the UNIX domain socket, it doesn't get client disconnect notificiations necessary for resource tracking, AND browser use of these resources is practically the degenerate case. -- Terry To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure
I have a theory (and a patch) if you're willing to hear it.
[EMAIL PROTECTED] writes:
} I'd like to take a step further and say it's in SYSVSHM design. All a
} program has to do is forget to do a shm_detach() and you're f#$%ed.
}Heh. I could make the same argument about open...
Except that the kernel seems to keep track of file handles ok.
} You could say that X11 shouldn't use SHMs the way it does now yeah. =)
}The real problem is that over the UNIX domain socket, it doesn't
}get client disconnect notificiations necessary for resource tracking,
}AND browser use of these resources is practically the degenerate case.
I think you're right about the linux stuff. It looks that you _will_ get
an EINVAL on a valid shmid if it has already been marked for deletion
(but presumably still exists due to references). Using the -o to ipcs
I see there are existing references (why didn't anyone suggest that
before) - but these are by non existent processes.
I'm now pretty sure this is caused by the linux implementation of
threads which uses multiple processes instead of a single proces. Now
vm_fork() (vm/vm_glue.c) calls shmfork which increments the shm ref count
for the forked process but this _shouldn't_ get called when rfork is
called with RFMEM (which is what linux_clone() does). So, turning to
the exit side of things, sys_exit() calls exit1() which only calls
shmexit to decrement the count if vm_refcnt == 1. Ok, rfork(RFMEM)
increments vm_refcnt in vm_fork(), and it may be decremented in
vmspace_unshare() or vmspace_free(). However as best I can see, nothing
in exit1() calls any vm stuff unless vm_refcnt is already 1. So unless
someone can point out where I've missed the call, I think the fix is
something like:
--- kern_exit.c.origFri Dec 14 14:33:50 2001
+++ kern_exit.c Wed Apr 10 13:16:01 2002
@@ -218,6 +218,8 @@
VM_MAXUSER_ADDRESS);
(void) vm_map_remove(vm-vm_map, VM_MIN_ADDRESS,
VM_MAXUSER_ADDRESS);
+ } else {
+ vmspace_free(vm);
}
if (SESS_LEADER(p)) {
This will decrement the vm_refcnt for exiting rfork'd processes and allow
the final exit to do all the appropriate cleanup, including the shmexit.
linux-mozilla would be a pretty heavy thread user. If only linux threads
was posix compliant, eh (I know Terry has commented on this before).
Callum Gibson [EMAIL PROTECTED]
Global Markets IT, Deutsche Bank, Australia 61 2 9258 1620
### The opinions in this message are mine and not Deutsche's ###
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure (fix found)
FWIW, This has been fixed in current, kern/kern_exit.c revision 1.147. Someone should MFC it. Callum Gibson [EMAIL PROTECTED] Global Markets IT, Deutsche Bank, Australia 61 2 9258 1620 ### The opinions in this message are mine and not Deutsche's ### To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
ipcrm/shmctl failure
Hi, I'm running FreeBSD 4.5-RELEASE on 2 machines. Yesterday one of them ran out of shared memory segments (used by wine, X11 and mozilla). Fine, clean them up. However although I could list them with ipcs, the vast majority could not be removed, even as root with ipcrm failing with Invalid argument. I also have a program which implements ipcrm (en masse) with shmctl(2) and it fails similarly (EINVAL). In short I could only clear these by rebooting. My other machine has been up only 13 hours and already there are a number of shared memory segments I can't get rid of. The only other thing that could be relevant is that I do use the linux mozilla so perhaps it has something to do with creation of the shm under linux emulation? (I did try running a linux ipcrm binary just in case - same error.) This may be a red herring because as far as I can tell the nonremovable ones this morning were created by X (I only ran a few xterms, xclock, xload). Also the manpage mentions a file-based implementation. Where are the files kept? With this in mind, the only other thing I might mention is that I have an mfs /tmp (on both machines). I can't see any previous mention of this on questions, hackers or current nor in the PR database. Has anyone else seen this? It should be easy to repeat I would have thought as I'm not doing anything unusual. C Callum Gibson [EMAIL PROTECTED] Global Markets IT, Deutsche Bank, Australia 61 2 9258 1620 ### The opinions in this message are mine and not Deutsche's ### To Unsubscribe: send mail to [EMAIL PROTECTED] with unsubscribe freebsd-hackers in the body of the message
Re: ipcrm/shmctl failure
[EMAIL PROTECTED] wrote:
I'm running FreeBSD 4.5-RELEASE on 2 machines. Yesterday one of them
ran out of shared memory segments (used by wine, X11 and mozilla).
Fine, clean them up. However although I could list them with ipcs,
the vast majority could not be removed, even as root with ipcrm failing
with Invalid argument. I also have a program which implements ipcrm
(en masse) with shmctl(2) and it fails similarly (EINVAL). In short I
could only clear these by rebooting.
Use the source:
case IPC_RMID:
error = ipcperm(p, shmseg-shm_perm, IPC_M);
if (error)
return error;
shmseg-shm_perm.key = IPC_PRIVATE;
shmseg-shm_perm.mode |= SHMSEG_REMOVED;
if (shmseg-shm_nattch = 0) {
shm_deallocate_segment(shmseg);
shm_last_free = IPCID_TO_IX(uap-shmid);
}
break;
All you are doing is marking the segment as removed. The segment
remains attached by the processes which have it open, and those
references don't go awaya until the processes in question detach
the segments, and the reference count goes to zero.
In other words, shared memory segments are like files; you can't
delete them out from under programs that still hold references
to them.
The only other thing that could be relevant is that I do use the linux
mozilla so perhaps it has something to do with creation of the shm
under linux emulation? (I did try running a linux ipcrm binary just
in case - same error.) This may be a red herring because as far as I
can tell the nonremovable ones this morning were created by X (I only
ran a few xterms, xclock, xload).
This is probably because of the shared memory segments that
are established for bitmaps, using the shared memory extension
to the X server.
It turns out that these segments are not proerly reference
counted and tracked, so they are not deleted when the client
goes away.
This is a bug in the MIT shared memory extension for X design,
and can't be fixed for long running programs with lots of
bitmaps.
You can either quit your Netscape (Mozilla; whetever) and then
restart it, or you can tell it not to use the shared memory
extension to communicate with the X server. The way to do this
is to set your DISPLAY environment variable so that clients use
network connections, not local (UNIX) domain sockets, to talk
to the X server... e.g.:
setenv DISPLAY `hostname`:0.0
Also the manpage mentions a file-based implementation. Where are the
files kept? With this in mind, the only other thing I might mention is
that I have an mfs /tmp (on both machines).
It does not use a file based implementation; your Linux man page
is wrong.
I can't see any previous mention of this on questions, hackers or current
nor in the PR database. Has anyone else seen this? It should be easy to
repeat I would have thought as I'm not doing anything unusual.
It's an X11 question, and it's been that way since at least 1994,
so it's a long standing X11 FAQ.
-- Terry
To Unsubscribe: send mail to [EMAIL PROTECTED]
with unsubscribe freebsd-hackers in the body of the message
