SMP Threads (was: Re: SMP and SO5.0)
On Wed, Mar 03, 1999 at 06:13:43PM -0700, Russell L. Carter wrote: John Dyson extemporised: %Julian Elischer said: % % % On Tue, 16 Feb 1999, Luoqi Chen wrote: % You may try my patch at http://www.freebsd.org/~luoqi, which would allow % linux threads to run on SMP. % % I've gone through these patches and I can see that they are really needed % for SMP where address spaces are shared. % %I agree -- a per-cpu page directory per multithreaded process is the way that %I had implemented. Excellent!!! % So I tried these out. The examples work in the linuxthreads dir but many other things do not. For instance, Built debug ACE+TAO with egcs-2.91.62, March 2 -current, newest LinuxThreads port. Uniprocessor has no errors. SMP fails, for instance: rcar...@mazatzal:~/ace/ACE_wrappers/tests [82] ./TSS_Test TSS_Test in malloc(): warning: recursive call. TSS_Test in malloc(): warning: recursive call. Abort trap (core dumped) rcar...@mazatzal:~/ace/ACE_wrappers/tests [83] gdb TSS_Test -core TSS_Test.coreGDB is free software and you are welcome to distribute copies of it under certain conditions; type show copying to see the conditions. There is absolutely no warranty for GDB; type show warranty for details. GDB 4.16 (i386-unknown-freebsd), Copyright 1996 Free Software Foundation, Inc... Core was generated by `TSS_Test'. Program terminated with signal 6, Abort trap. Reading symbols from /home/rcarter/ace/ACE_wrappers/ace/libACE.so...done. Reading symbols from /usr/local/lib/liblthread.so.0...done. Reading symbols from /usr/lib/libm.so.2...done. Reading symbols from /usr/lib/libc.so.3...done. Reading symbols from /usr/libexec/ld-elf.so.1...done. #0 0x18243bf4 in sched_yield () (gdb) where #0 0x18243bf4 in sched_yield () #1 0x181eb051 in sched_yield () at sched.c:58 #2 0x181eac79 in _spinlock (lck=0x18281738) at spinlock.c:88 #3 0x18278a02 in free () #4 0x181a8765 in __builtin_delete (ptr=0x8054170) #5 0x804ada5 in worker (c=0x64) at TSS_Test.cpp:222 #6 0x180fc4e2 in ACE_Thread_Adapter::invoke (this=0x8051540) at OS.cpp:2075 #7 0x180fc556 in ace_thread_adapter (args=0x8051540) at OS.cpp:2194 #8 0x181ebe8c in pthread_start_thread (arg=0xeb5ffd04) at manager.c:160 #9 0x181ec49d in _clone () at clone.S:1 #10 0x in ?? () #11 0x1805ec5c in .curbrk () Cannot access memory at address 0xf9dc. Comments (but no solution): 1) gdb is tricky to use with threads. What you see is not always where you are. It doesn't always follow the threads very well, if at all. 2) AFAIK, there are only two reasons you should ever see the malloc(): warning: recursive call message. One reason is that the libc global variable __isthreaded is not set. I don't think this is the case here, since the gdb trace seems to show _spinlock being called within free(). This would not happen if __isthreaded is not set. 3) The other reason for the warning is that malloc/free have been called recursively within the same thread. AFAIK, this can only happen if malloc/free are called within a signal handler (which is an error), and one of malloc/free is interrupted by a signal. Looking briefly at the ACE TSS_Test code, I suspect that the signal handler calls free(). 4) I'm not sure what the real problem is, whether it is a bad signal handler, or more likely, the real problem is whatever generated the signal within malloc/free, and the bad signal handler is a misdirect. -- Richard Seaman, Jr. email: d...@tar.com 5182 N. Maple Lanephone: 414-367-5450 Chenequa WI 53058 fax: 414-367-5852 To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-current in the body of the message
Re: SMP and SO5.0
John Dyson extemporised: %Julian Elischer said: % % % On Tue, 16 Feb 1999, Luoqi Chen wrote: % You may try my patch at http://www.freebsd.org/~luoqi, which would allow % linux threads to run on SMP. % % I've gone through these patches and I can see that they are really needed % for SMP where address spaces are shared. % %I agree -- a per-cpu page directory per multithreaded process is the way that %I had implemented. Excellent!!! % So I tried these out. The examples work in the linuxthreads dir but many other things do not. For instance, Built debug ACE+TAO with egcs-2.91.62, March 2 -current, newest LinuxThreads port. Uniprocessor has no errors. SMP fails, for instance: rcar...@mazatzal:~/ace/ACE_wrappers/tests [82] ./TSS_Test TSS_Test in malloc(): warning: recursive call. TSS_Test in malloc(): warning: recursive call. Abort trap (core dumped) rcar...@mazatzal:~/ace/ACE_wrappers/tests [83] gdb TSS_Test -core TSS_Test.coreGDB is free software and you are welcome to distribute copies of it under certain conditions; type show copying to see the conditions. There is absolutely no warranty for GDB; type show warranty for details. GDB 4.16 (i386-unknown-freebsd), Copyright 1996 Free Software Foundation, Inc... Core was generated by `TSS_Test'. Program terminated with signal 6, Abort trap. Reading symbols from /home/rcarter/ace/ACE_wrappers/ace/libACE.so...done. Reading symbols from /usr/local/lib/liblthread.so.0...done. Reading symbols from /usr/lib/libm.so.2...done. Reading symbols from /usr/lib/libc.so.3...done. Reading symbols from /usr/libexec/ld-elf.so.1...done. #0 0x18243bf4 in sched_yield () (gdb) where #0 0x18243bf4 in sched_yield () #1 0x181eb051 in sched_yield () at sched.c:58 #2 0x181eac79 in _spinlock (lck=0x18281738) at spinlock.c:88 #3 0x18278a02 in free () #4 0x181a8765 in __builtin_delete (ptr=0x8054170) #5 0x804ada5 in worker (c=0x64) at TSS_Test.cpp:222 #6 0x180fc4e2 in ACE_Thread_Adapter::invoke (this=0x8051540) at OS.cpp:2075 #7 0x180fc556 in ace_thread_adapter (args=0x8051540) at OS.cpp:2194 #8 0x181ebe8c in pthread_start_thread (arg=0xeb5ffd04) at manager.c:160 #9 0x181ec49d in _clone () at clone.S:1 #10 0x in ?? () #11 0x1805ec5c in .curbrk () Cannot access memory at address 0xf9dc. (gdb) Russell To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-current in the body of the message
Re: SMP and SO5.0
You may try my patch at http://www.freebsd.org/~luoqi, which would allow linux threads to run on SMP. I've gone through these patches and I can see that they are really needed for SMP where address spaces are shared. I agree -- a per-cpu page directory per multithreaded process is the way that I had implemented. Excellent!!! Ugh. I'm sorry, but this is just evil(tm). The kernel stack and the process address space reference need to be divorced from the traditional concept of process (and point back at the process). As such, the sleep and scheduling is done by the context, not the process. These contexts could then be used either as thread contexts (bad idea) or async call contexts (good idea). This solves the seperation problem, the process ID problem, the thread identification problem, the thread grouping problem, the N:M mapping problem, the quantum churning problem, the kernel preemption problem, and the threads kernel stack overflow problem. This also resolves the pessimal scheduling for concurrent threads on an SMP system problem. Ideally, a thread is still a user space construct, and all call conversion scheduler in user space is used, as well as placing the POSIX AIO stuff in a library. Protection domain crossing costs are next to NIL, given that the unified VM and buffer cache get rid of the bmap costs. Oh yeah. This also solves the hybrization problem, which Solaris solves by running an extra thread for spawning new threads. Comments? Terry Lambert te...@lambert.org --- Any opinions in this posting are my own and not those of my present or previous employers. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-current in the body of the message
Re: SMP and SO5.0
I've gone through these patches and I can see that they are really needed for SMP where address spaces are shared. There are details I didn't get, such as where is the per-processor pde pointed, (i.e. where is the per processor KVM range) and is there a single page table for each processor that is always mapped into the processor specific slot for that process. I didn't change any of these (that's main reason I chose to have multiple page directories over a single page directory with a different slot for each processor and reference the per-processor data through a pointer in processor local storage, e.g. %fs, which is simply too complicated without compiler support. In fact I like the 2nd way better), so the process specific slot is still MPPTDI (KVM 0xff80-0xffbf?). another question that is raised I guess is how do we tell gdb to switch between processors when reading core-dumps :-). Tor has sent me a patch which implements a cpu command allowing you to switch among cpus. I've never actually used it, in most of the cases, you don't really care on which cpu the crash occurred. If you want to try it, I can send a copy of the gdb patch to you. -lq To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-current in the body of the message
Re: SMP and SO5.0
Julian Elischer said: On Tue, 16 Feb 1999, Luoqi Chen wrote: You may try my patch at http://www.freebsd.org/~luoqi, which would allow linux threads to run on SMP. I've gone through these patches and I can see that they are really needed for SMP where address spaces are shared. I agree -- a per-cpu page directory per multithreaded process is the way that I had implemented. Excellent!!! -- John | Never try to teach a pig to sing, dy...@iquest.net | it makes one look stupid jdy...@nc.com | and it irritates the pig. To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-current in the body of the message
Re: SMP and SO5.0
Hi,
I downloaded Star Office 5 and only THEN realised that the code for doing
linux thread
emulation is #ifndef SMP :) Still, after downloading 70 meg over a 56k modem
and paying
19c/meg I was gonna try the sucker regardless.. And well, it works!
The install hung at the end, after its done everything, so I killed it, but
after that I
can run it with (seemingly) no problems.. (Except for the 2000 odd 'shared
address space
fork attempted' messages in my syslog)
I only had a quick fiddle, but it started up everything fine and ran quite
well..
The install was a pain tho, as I had to unpack the setup program (its a self
extracting
zip) and rename the libs in it to lower case and then add an LD_LIBRARY_PATH
to point to
them, but apart from that it was OK.
Its worth noting though, that eMusic which uses Linux threads doesn't work
under
emulation (it just hangs) I think I'll boot a non-SMP kernel and have a go
('cause it
took me an hour to find all of the $...@$ dependancies it needs because Linux
ldd doesn't
work anymore)
---
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
The nice thing about standards is that there
are so many of them to choose from.
-- Andrew Tanenbaum
You may try my patch at http://www.freebsd.org/~luoqi, which would allow
linux threads to run on SMP.
-lq
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message
Re: SMP and SO5.0
On Tue, 16 Feb 1999, Luoqi Chen wrote:
Hi,
I downloaded Star Office 5 and only THEN realised that the code for doing
linux thread
emulation is #ifndef SMP :) Still, after downloading 70 meg over a 56k
modem and paying
19c/meg I was gonna try the sucker regardless.. And well, it works!
The install hung at the end, after its done everything, so I killed it, but
after that I
can run it with (seemingly) no problems.. (Except for the 2000 odd 'shared
address space
fork attempted' messages in my syslog)
I only had a quick fiddle, but it started up everything fine and ran quite
well..
The install was a pain tho, as I had to unpack the setup program (its a
self extracting
zip) and rename the libs in it to lower case and then add an
LD_LIBRARY_PATH to point to
them, but apart from that it was OK.
Its worth noting though, that eMusic which uses Linux threads doesn't work
under
emulation (it just hangs) I think I'll boot a non-SMP kernel and have a go
('cause it
took me an hour to find all of the $...@$ dependancies it needs because
Linux ldd doesn't
work anymore)
---
Daniel O'Connor software and network engineer
for Genesis Software - http://www.gsoft.com.au
The nice thing about standards is that there
are so many of them to choose from.
-- Andrew Tanenbaum
You may try my patch at http://www.freebsd.org/~luoqi, which would allow
linux threads to run on SMP.
-lq
With any form of kernel threads not working, and aio being highly unstable for
SMP because of the VM system, is it not worthwhile to actually implement or
merge the changes like this?
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message
Brian Feldman_ __ ___ ___ ___
gr...@unixhelp.org _ __ ___ | _ ) __| \
http://www.freebsd.org/ _ __ ___ | _ \__ \ |) |
FreeBSD: The Power to Serve! _ __ ___ _ |___/___/___/
To Unsubscribe: send mail to majord...@freebsd.org
with unsubscribe freebsd-current in the body of the message
Re: SMP and SO5.0
I'm not sure why you need a different page directory for each processor. what's your thinking on this? You might add some comments in your patches so that if becomes more obvious what you are doing... To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-current in the body of the message
Re: SMP and SO5.0
On Tue, 16 Feb 1999, Luoqi Chen wrote: You may try my patch at http://www.freebsd.org/~luoqi, which would allow linux threads to run on SMP. I've gone through these patches and I can see that they are really needed for SMP where address spaces are shared. There are details I didn't get, such as where is the per-processor pde pointed, (i.e. where is the per processor KVM range) and is there a single page table for each processor that is always mapped into the processor specific slot for that process. another question that is raised I guess is how do we tell gdb to switch between processors when reading core-dumps :-). To Unsubscribe: send mail to majord...@freebsd.org with unsubscribe freebsd-current in the body of the message
