Hi Andy,
On Fri, Oct 9, 2009 at 5:08 PM, Andy Freeman <[email protected]> wrote:
>
> > They are raised inside a transaction, when a conflict is detected with
> > another concurrent transaction. The transaction infrastructure will catch
> > and retry these several times, and only raise it in the external code if
> it
> > was unable to execute the transaction after several retries.
>
> Yes, but when are conflicts checked? Specifically, is the error
> always raised by the statement in the user function that runs into the
> conflict or can it be raised later, say during transaction commit.
>
Any datastore operation inside a transaction could raise this exception. It
would be a bad idea to rely on _where_ this exception will be raised.
>
> I've looked at the SDK's implementation of
> RunInTransactionCustomRetries (in google/appengine/api/datastore.py).
> The except that catches the CONCURRENT_TRANSACTION exception protects
> the commit and not the execution of the user function. That suggests
> that the user function is run to completion regardless of conflicts
> and that the conflict isn't acted upon until a commit is tried.
>
> However, your description and the documentation suggests the real
> implementation detects and acts on conflicts while running the user
> function.
>
> Here's a user function which demonstrates the difference. (Yes, I
> picked an example that I care about. I'm trying to ensure that
> memcache data is "not too stale".)
>
> def txn():
> ...
> a.put()
> memcache.set('a', a.field)
> return a
>
> If the CONCURRENT_TRANSACTION exception is raised while txn is being
> run, specifically during a.put(), the memcache.set won't happen when
> db.run_in_transaction(txn) fails. If that exception is raised after
> txn has exited and during commit (as the SDK code suggests), the
> memcache.set will happen whether or not db.run_in_transaction(txn)
> fails.
>
> If my understanding of the SDK code is correct and the real
> implementation works the same way, namely that conflicts are detected
> after the user function completes, how can I ensure that memcache data
> is not too stale? (One way is to have that data expire reasonably
> quickly, but that reduces the value of memcache.)
>
Update memcache after the transaction completes. There's still the
possibility that your script could fail between the two events, but there's
no avoiding that without transactional semantics between the datastore and
memcache.
> Also, what's the definition of "conflict"? Clearly there's a conflict
> between a user function that reads a given data store entity and one
> that writes the same entity. However, what about the following?
>
> def txn1(a, b):
> # notice - no read for a or b
> a.put()
> b.put()
> return True
>
> Does the conflict detection system detect the conflict between
> transactions with txn1 for the same datastore entities?
Yes.
> (The intent
> of transactions with txn1 is to ensure that a and b are mutually
> consistent in the datastore.)
> Speaking of "definitions of conflict", suppose that conflicts actually
> are detected/handled while the user function is being run, so that txn/
> txnw can not leave the datastore and memcache inconsistent for very
> long. Are txnw and txnr (below) seen as conflicting given the same
> key? (They're not conflicting as far as the datastore is concerned,
> but remember - I'm trying to keep memcache consistent as well.)
>
> def txnw(key, new_value):
> v = db.get(key)
> v.field = new_value
> db.put(v)
> memcache.set(str(key), v.field)
> return True
>
> def txnr(key):
> v = db.get(key)
> memcache.set(str(key), v.field)
> return True
>
> Thanks,
> -andy
>
>
>
> On Oct 9, 4:45 am, "Nick Johnson (Google)" <[email protected]>
> wrote:
> > Hi Andy,
> >
> > On Tue, Oct 6, 2009 at 8:45 PM, Andy Freeman <[email protected]>
> wrote:
> >
> > > Short version.
> >
> > > When, exactly, are apiproxy_errors.ApplicationErrors
> > > with .application_error == datastore_pb.Error.CONCURRENT_TRANSACTION
> > > raised.
> >
> > They are raised inside a transaction, when a conflict is detected with
> > another concurrent transaction. The transaction infrastructure will catch
> > and retry these several times, and only raise it in the external code if
> it
> > was unable to execute the transaction after several retries.
> >
> > -Nick Johnson
> >
> >
> >
> > --
> > Nick Johnson, Developer Programs Engineer, App Engine
> > Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration
> Number:
> > 368047
> >
>
--
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en
-~----------~----~----~----~------~----~------~--~---
