Re: Locks and threads

2009-03-26 Thread Neil Jerram
Linas Vepstas linasveps...@gmail.com writes: I reviewed patch2 as best I could, Thanks! it looks to me like new_bucket is never released. I assume that the scm_acons did a malloc, but maybe not Yes, but the garbage collection will find and free it, so it's OK. Regards, Neil

Re: Locks and threads

2009-03-26 Thread Ludovic Courtès
Hi Neil, Neil Jerram n...@ossau.uklinux.net writes: That's an interesting idea, but to be honest I'd prefer to finish this work and move onto something else. Would it be OK just to reduce the default runtime to 2 seconds? (It wouldn't make any practical difference, because 10 seconds isn't

Re: Locks and threads

2009-03-26 Thread Linas Vepstas
2009/3/26 Neil Jerram n...@ossau.uklinux.net: - and that scm_c_eval_string consists of reading (= symbol lookup) followed by evaluation (= module obarray lookup), and that each scm_c_eval_string call is fast enough to be completed well within one time slice. One way to stress test such

Re: Locks and threads

2009-03-25 Thread Neil Jerram
Neil Jerram n...@ossau.uklinux.net writes: I've been running Linas's define-race test program, and hitting the `throw from within critical section' quite a lot. We've already discussed this a couple of times: http://www.mail-archive.com/bug-gu...@gnu.org/msg04613.html

Re: Locks and threads

2009-03-25 Thread Neil Jerram
Neil Jerram n...@ossau.uklinux.net writes: Yes, it's an unrelated bug. All of the places that raise errors (and so exit non-locally) should exit the critical section first. You're absolutely right. I'll leave this part out, and generate a separate patch for it. Here's the separate

Re: Locks and threads

2009-03-25 Thread Ludovic Courtès
Hi Neil, Neil Jerram n...@ossau.uklinux.net writes: Neil Jerram n...@ossau.uklinux.net writes: Subject: [PATCH] Avoid throw from critical section, given invalid sigaction call * libguile/scmsigs.c (scm_sigaction_for_thread): Exit critical section before raising out-of-range error. *

Re: Locks and threads

2009-03-25 Thread Linas Vepstas
2009/3/25 Neil Jerram n...@ossau.uklinux.net: Linas Vepstas linasveps...@gmail.com writes: Well, once you identify the section that needs locking, you'll want to use an rwlock instead of a mutex.  The rwlock (pthread_rwlock_rdlock) allows multiple simultaneous readers.  The writers, however,

Re: Locks and threads

2009-03-25 Thread Neil Jerram
Neil Jerram n...@ossau.uklinux.net writes: Many thanks for your input on this. I'll go ahead with a mutex or rwlock. First thing is to see if it fixes the problem; if it does, I'll try to measure the performance impact. Attached are 3 patches relating to thread-safe define. #1 is Linas's

Re: Locks and threads

2009-03-16 Thread Andy Wingo
On Sat 14 Mar 2009 15:23, l...@gnu.org (Ludovic Courtès) writes: Still, I would measure the impact on `module-define!' and `module-ref' because that's at least an additional function call. You probably know, but at least in the VM (and I thought in the interpreter too), the locations are

Re: Locks and threads

2009-03-13 Thread Neil Jerram
Andy Wingo wi...@pobox.com writes: Hey Neil, On Thu 12 Mar 2009 01:53, Neil Jerram n...@ossau.uklinux.net writes: Thanks to a hint from helgrind, I think the problem might be that the symbols obarray is not thread-safe. But surely using a mutex for every symbol that Guile reads would be

Re: Locks and threads

2009-03-12 Thread Andy Wingo
Hey Neil, On Thu 12 Mar 2009 01:53, Neil Jerram n...@ossau.uklinux.net writes: Thanks to a hint from helgrind, I think the problem might be that the symbols obarray is not thread-safe. But surely using a mutex for every symbol that Guile reads would be terrible for performance... Dunno, in

Re: Locks and threads

2009-03-11 Thread Neil Jerram
Neil Jerram n...@ossau.uklinux.net writes: #module (guile) 40376f00 #directory (guile-user) 40379680 ERROR: ERROR: Unbound variable: x1-100499 Unbound variable: define ERROR: Unbound variable: x4-100596 ERROR: Unbound variable: define ERROR: Unbound variable: define ERROR: Unbound

Re: Locks and threads

2009-03-11 Thread Linas Vepstas
2009/3/11 Neil Jerram n...@ossau.uklinux.net: Neil Jerram n...@ossau.uklinux.net writes: #module (guile) 40376f00 #directory (guile-user) 40379680 ERROR: ERROR: Unbound variable: x1-100499 Unbound variable: define ERROR: Unbound variable: x4-100596 ERROR: Unbound variable: define ERROR:

Re: Locks and threads

2009-03-11 Thread Clinton Ebadi
Linas Vepstas linasveps...@gmail.com writes: An alternative idea is to try to apply some principles from functional programming, e.g. copy on write (COW): when the obarray needs to be updated, make a copy of it. Any readers in other threads continue to safely use the original version. When

Re: Locks and threads

2009-03-10 Thread Neil Jerram
Neil Jerram n...@ossau.uklinux.net writes: next it's on to the real problem, threadsafe define. I've been running Linas's define-race test program, and hitting the `throw from within critical section' quite a lot. We've already discussed this a couple of times:

Re: Locks and threads

2009-03-08 Thread Neil Jerram
Linas Vepstas linasveps...@gmail.com writes: Perhaps I'm naive, perhaps some naming convention could be used to indicate that SCM_OUT_OF_RANGE will never return? None of the functions in the call stack gave any real hint that they might now return; they mostly looked liked ordinary

Re: Locks and threads

2009-03-06 Thread Andy Wingo
Hi Linas, On Thu 05 Mar 2009 22:56, Linas Vepstas linasveps...@gmail.com writes: Perhaps I'm naive, perhaps some naming convention could be used to indicate that SCM_OUT_OF_RANGE will never return? None of the functions in the call stack gave any real hint that they might now return; they

Re: Locks and threads

2009-03-05 Thread Neil Jerram
Neil Jerram n...@ossau.uklinux.net writes: Yes, it's an unrelated bug. All of the places that raise errors (and so exit non-locally) should exit the critical section first. You're absolutely right. I'll leave this part out, and generate a separate patch for it. Here's the separate

Re: Locks and threads

2009-03-05 Thread Linas Vepstas
2009/3/5 Neil Jerram n...@ossau.uklinux.net: Yes, it's an unrelated bug.  All of the places that raise errors (and so exit non-locally) should exit the critical section first. You're absolutely right.  I'll leave this part out, and generate a separate patch for it. Here's the separate

Re: Locks and threads

2009-03-05 Thread Linas Vepstas
2009/3/5 Neil Jerram n...@ossau.uklinux.net: Yes, it's an unrelated bug.  All of the places that raise errors (and so exit non-locally) should exit the critical section first. You're absolutely right.  I'll leave this part out, and generate a separate patch for it. Here's the separate

Re: Locks and threads

2009-03-05 Thread Neil Jerram
Linas Vepstas linasveps...@gmail.com writes: I don't understand the patch. libguile/scmsigs.c has a SCM_CRITICAL_SECTION_START at line 339, which seems to be balanced by SCM_CRITICAL_SECTION_END; at lines 442 and 461, right before the return from the subroutine. So why insert this

Re: Locks and threads

2009-03-05 Thread Ludovic Courtès
Hello Neil, Thanks for the patches and tests! Ludo'.

Re: Locks and threads

2009-03-05 Thread Linas Vepstas
2009/3/5 Neil Jerram n...@ossau.uklinux.net: Linas Vepstas linasveps...@gmail.com writes: I don't understand the patch. libguile/scmsigs.c has a SCM_CRITICAL_SECTION_START at line 339, which seems to be balanced by SCM_CRITICAL_SECTION_END; at lines 442 and 461, right before the return

Re: Locks and threads

2009-03-04 Thread Neil Jerram
Neil Jerram n...@ossau.uklinux.net writes: - first to address problems reported by helgrind (since I think we should take advantage of external tools before adding debug code to Guile internally) Here's another lock ordering fix. (For 1.8.x at least, I haven't checked if it's relevant to

Re: Locks and threads

2009-03-04 Thread Linas Vepstas
2009/3/4 Neil Jerram n...@ossau.uklinux.net: Here's another lock ordering fix.  (For 1.8.x at least, I haven't checked if it's relevant to master yet.) I skimmed it quickly, looks reasonable, except for this: else - SCM_OUT_OF_RANGE (2, handler); + { +

Re: Locks and threads

2009-02-14 Thread Ludovic Courtès
Hi, Ken Raeburn raeb...@raeburn.org writes: Quite a number of the projects at scan.coverity.com are GNU projects, including emacs and gcc; any idea if anyone from the GNU Project has talked to Coverity in an official capacity about getting the terms changed? No idea. I used to use Splint,

Re: Locks and threads

2009-02-12 Thread Ken Raeburn
On Feb 12, 2009, at 07:55, Greg Troxel wrote: Does that sound sensible; have I missed anything? Also run tests on other than Linux, with as many different OS threading implementations as possible. For all systems, set the threading libraries to the most restrictive settings, specifically

Re: Locks and threads

2009-02-12 Thread Ludovic Courtès
Hello, Neil Jerram n...@ossau.uklinux.net writes: (Is something like this actually _ever_ a problem? If locks are always _acquired_ in the right order, how can the order of _releasing_ ever cause a problem?) It can't be a problem, AFAIUI. I don't think Helgrind check the releasing order,

Re: Locks and threads

2009-02-12 Thread Ludovic Courtès
Hello, Ken Raeburn raeb...@raeburn.org writes: As helgrind and NetBSD's pthread checking code work by instrumenting the executable and watching progress, I'd suggest also trying out static analysis tools if anyone has them handy. I see guile is already listed with Coverity's open-source

Locks and threads

2009-02-11 Thread Neil Jerram
I've started working through the lock ordering and threading issues that we have. My plan is (with 1.8.x) - first to address problems reported by helgrind (since I think we should take advantage of external tools before adding debug code to Guile internally) - then to run Linas's

Re: Locks and threads

2009-02-11 Thread Neil Jerram
Neil Jerram n...@ossau.uklinux.net writes: - first to address problems reported by helgrind (since I think we should take advantage of external tools before adding debug code to Guile internally) Here's the next one. Neil From 76f55c5796f1fc7aca6c36bc57f06bab72300a94 Mon Sep 17

Re: Locks and threads

2009-02-11 Thread Ludovic Courtès
Hi Neil, Neil Jerram n...@ossau.uklinux.net writes: I've started working through the lock ordering and threading issues that we have. My plan is (with 1.8.x) - first to address problems reported by helgrind (since I think we should take advantage of external tools before adding debug

Re: Locks and threads

2009-02-11 Thread Ludovic Courtès
Neil Jerram n...@ossau.uklinux.net writes: * libguile/async.c (scm_async_click): Don't leave Guile mode when locking async_mutex. We don't need to, because none of the code that has async_mutex locked can block, and doing so may lead to lock ordering problems between async_mutex and a

Re: Locks and threads

2009-02-11 Thread Neil Jerram
Linas Vepstas linasveps...@gmail.com writes: Err, sort of, yes, unless I misunderstand. Guile 1.8 makes a certain basic assumption that is splattered throughout the code; it rather intentionally re-orders the order in which one of the locks is taken. If I remember correctly, its the in