Bugs item #924909, was opened at 2004-03-28 17:11
Message generated for change (Comment added) made by mlkersten
You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=924909&group_id=56967

Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Core
Group: (zombie: MonetDB 4.3.16-rc05)
>Status: Closed
Resolution: None
Priority: 6
Private: No
Submitted By: Martin Kersten (mlkersten)
Assigned to: Peter Boncz (boncz)
Summary: Race/Queuing/coordination issues

Initial Comment:
Running the DBtapestry programs repeatedly show the
following behaviour on
~/crackers/experiments/run{MIL,SQL}Tapestry
During the run after 30M/20M both client and server hang.

The client is waiting for output.
The server is still active, i.e. I can perform at least
2 successive 'print(1);' commands and get a response.
[I can not connect with another MapiClient though]

However, if I start the command 'clients()' or
'help("sigs")' the server  hangs too. Forcefully
quiting the server tells me that 7 threads had to be
killed. 

Looking into the issue further, the error is reported
when I am executing the command:

MapiClient  -h $HOST -s 'persists(BAT("R1"),false);
commit;' </dev/null

which is about to remove a bat prepared in the previous
experiment.

Note, this behaviour has not been seen before on these
test runs. Running the experiments in isolation  does
not cause these problems, which seems to suggest a race
condition or queuing issue.
Moreover, both experimental runs stop at different
places, which indicate an administration related bug.

----------------------------------------------------------------------

>Comment By: Martin Kersten (mlkersten)
Date: 2007-02-20 17:53

Message:
Logged In: YES 
user_id=490798
Originator: YES

The DBtapestry code is dormant for too long to
revive it and make a (stress) test out of it.
Also, too many changes in the concurrent handling
seems this issue (if still existent) will have been
tested.
So, closed with no test file left behind.

----------------------------------------------------------------------

Comment By: Sjoerd Mullender (sjoerd)
Date: 2007-02-20 17:26

Message:
Logged In: YES 
user_id=43607
Originator: NO

Nice that you accepted the explanation and resolution, but what about a
test?

----------------------------------------------------------------------

Comment By: Martin Kersten (mlkersten)
Date: 2006-12-04 18:53

Message:
Logged In: YES 
user_id=490798
Originator: YES

accepted the explanation and resolution

----------------------------------------------------------------------

Comment By: Stefan Manegold (stmane)
Date: 2006-11-29 10:06

Message:
Logged In: YES 
user_id=572415
Originator: NO

over-ruling "sf-robot":
re-set to "Pending" since adding a proper test has neither been done nor
finally been discarded, yet.

User (submitter) & developer (assignee),
please check, again.


----------------------------------------------------------------------

Comment By: SourceForge Robot (sf-robot)
Date: 2006-11-22 04:20

Message:
Logged In: YES 
user_id=1312539
Originator: NO

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 365 days (the time period specified by
the administrator of this Tracker).

----------------------------------------------------------------------

Comment By: SourceForge Robot (sf-robot)
Date: 2005-11-21 04:20

Message:
Logged In: YES 
user_id=1312539

This Tracker item was closed automatically by the system. It was
previously set to a Pending status, and the original submitter
did not respond within 14 days (the time period specified by
the administrator of this Tracker).

----------------------------------------------------------------------

Comment By: Stefan Manegold (stmane)
Date: 2005-11-06 10:52

Message:
Logged In: YES 
user_id=572415

BugDay_2005-10-06: "left-overs" are now marked as "pending"


----------------------------------------------------------------------

Comment By: Stefan Manegold (stmane)
Date: 2005-10-06 17:33

Message:
Logged In: YES 
user_id=572415

BugDay_2005-10-06: CLAIMED BY stmane

BugDay_2005-10-06: NO TEST / POSTPONED?
- race condition tests are not (easily) possible in Mtest (yet?)
- set priority to 9 as reminder


----------------------------------------------------------------------

Comment By: Peter Boncz (boncz)
Date: 2004-03-31 17:47

Message:
Logged In: YES 
user_id=591107

BUGFIX 924909

this is a race condition. BBPfree saves a BAT, and then 
removes it
from the BBPcache (BBPclear, requiring a GDKCachelock 
and/or BBPuncacheit,
requiring a GDKswapLock). After BBPfree, unloadBarrier is 
released.

(unloadBarrier is taken as long as at least one BAT is being 
unloaded --
 this can take a long time as the BAT may need to be saved 
to disk).

TMcommit first gets all the GDKcacheLock, then all 
GDKswapLocks
and then waits for unloadBarrier

The unloadBarrier exists just because in the commit we want 
to
make sure that descriptors do not disappear during the 
commit because
a BAT finishes saving. BBPcyns (ie commit) gets all locks, 
blocks BBPtrim,
and assumes that the BBP loaded state is stable while it runs.

the deadlock is as follows:
TMcommit:                         BBPtrim(b):
                                    set BBPUNLOADING bit in 
BBP_status(b)
                                    get unloadBarrier
                                    BBPfree(b)
                                    +BBPsave
 get cachelock,swaplocks            |
 get unloadbarrier ==> BLOCKS!      |
                                    +BBPclear(b)
                                     get cacheLock ==>BLOCKS!

the resolution is to release unloadBarrier from within BBPfree 
after
the save has been done but before the BBPclear

However, this would mean that BBPclear would remove the 
bat from BBP_cache
during the commit. That defeats the purpose of unloadbarrier.

Therefore a BBPuncacheit in BBPfree must happen before 
giving
back the barrier (after giving back the barrier, this BBP_cache
() may not
be modified anymore).

But, BBPuncachit took a BBPswapLock! This would have 
caused another deadlock,
as TMcommit, blocked on unloadbarrier, already has all 
swaplocks.

The way out is to realize that BBPuncacheit does not need 
the swapLock at all.
You may read from BBP_cache() without locking, but *only* 
if you have got
a BBPfix on the BAT. You can never have a BBPfix while a 
BAT is unloading.
So we know that nobody is reading BBP_cache(bid) anyway 
during a BBPuncacheit.

summary of modifications:
- give back unloadBarrier from within BBPfree instead of after 
it
- remove locking from BBPuncacheit


----------------------------------------------------------------------

You can respond by visiting: 
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=924909&group_id=56967

-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs

Reply via email to