Hi Richard,
On Tuesday 11 December 2007 13:36, Richard Taylor wrote:
> I don't think that you can rely on the threadsafety of these functions.
> Even if they are threadsafe in C Python (which I doubt that 'set' is), the
> locking in Jython in more fine grained and would likely catch you out.
It's perhaps worth noting in the version of the code I posted, in this thread,
where it said...
"""What key areas appear least threadsafe, and any general suggestions
around that."""
...I knew that set and using were not threadsafe, but wondered about other
parts. I perhaps should've been more explicit on that point. (I wanted to
simply post some ideas which showed the core logic without locking. Perhaps a
mistake :)
Anyhow, the current version is here:
https://kamaelia.svn.sourceforge.net/svnroot/kamaelia/branches/private_MPS_Scratch/Bindings/STM/Axon/STM.py
In that version, "set" now looks like this:
def set(self, key, value):
success = False
if self.lock.acquire(0):
try:
if not (self.store[key].version > value.version):
self.store[key] = Value(value.version+1,
copy.deepcopy(value.value),
self, key)
value.version= value.version+1
success = True
finally:
self.lock.release()
else:
raise BusyRetry
if not success:
raise ConcurrentUpdate
and "using" has changed to "usevar: (using now relates to a collection)
def usevar(self, key):
try:
return self.get(key)
except KeyError:
if self.lock.acquire(0):
try:
self.store[key] = Value(0, None,self,key)
finally:
self.lock.release()
else:
raise BusyRetry
return self.get(key)
Since mutations of the store rely on acquiring the lock on the store, that
should be safe(r). User code doesn't have to worry about locks however -
which is course the point of the code :-)
The reason for specifically using the acquire(0) call rather than acquire()
call is because I want it to fail hard if the lock can't be acquired. I know
it'd be nicer to have a finer grained lock here, but I'm personally primarily
going to be using this for rare operations rather than common operations.
These locks above are of course in relation to write locking. I'll think about
the read locking you've suggested.
Your locking looks incorrect on using since it both allows reading and writing
of the store. (retrieve value & if not present create & initialise)
I also think the independent locks are a misnomer, but they're useful for
thinking about it.
> I would suggest that you should routinely wrap shared datamodels like these
> in thread locks to be certain about things.
Indeed. It makes the code look worse, so for this example I was really after
suggestions (like yours :-) of "OK, where does this break badly" as well as
"does the logic look sane?".
> I would also suggest that a small change to the Value class would make it
> possible for client code to subclass it, which might make it more flexible.
I'm not convinced by the changes to Value - its there for storing arbitrary
values, rather than extending Value itself. It's probably worth noting
that .clone has changed in my version to this:
def clone(self):
return Value(self.version,
copy.deepcopy(self.value),self.store,self.key)
Which includes deepcopy on the value stored by Value. I'm beginning to think
that Value should be called "Variable" to make this clearer...
> Here is my suggestion, bare in mind that I have not tested the thread
> locking code beyond making sure that it runs :-)
The feedback is much appreciated - it's making me think more about the read
locking aspect. I suspect the GIL in CPython *may* make the reads safe, but
the lack of a GIL in jython & ironpython probably renders the reads in using &
get unsafe. (and I would like this to be safe in jython & ironpython)
It's interesting though, after having developed large amounts of code of code
based on no-shared-data & read-only/write-only pipes with data handoff and
not having had any major concurrency issues (despite mixing threads and non
threads) switching to a shared data model instantly causes problems.
The difference is really stark. One is simple, natural and easy and the other
is subtle & problematic. I'm not shocked, but find it amusing :-)
Many thanks for the feedback!
Michael.
_______________________________________________
python-uk mailing list
python-uk@python.org
http://mail.python.org/mailman/listinfo/python-uk
