This is really weird. Actually, it's pretty unique in my experience. You clearly speak english well, and yet - even after two rounds - I have absolutely no idea what you are asking. While I don't pretend to be the smartest person in the universe, I suspect that nobody else has any idea what you are asking either.
Let's take a step back for a second. Stop worrying about put() or any other kind of operation. There's only one operation that matters in GAE-land or any other kind of optimistic-concurrency-land: commit(). You can do all kinds of crazy shit in a transaction but it only hits the fan when you try to commit it. If someone else was making sweet sweet love to your EG while you were away, you get an optimistic concurrency failure - ConcurrentModificationException in Java, some kind of annoying return value if you're in Go. The right answer in either case is to retry your idempotent (!) transaction. Eventually, no matter how many retries later, your txn will succeed and you will ride off into the sunset with the transaction of appropriate sex. There is absolutely nothing unique or novel about GAE transactions. Just search the internet for Optimistic Concurrency Control; GAE is not special in this regard. Jeff On Fri, May 15, 2015 at 3:06 PM, Francis Stephens <[email protected]> wrote: > Maybe this clarifies my question. The paragraph at bottom is a description > of a transaction from Google. It refers to checking the 'last update time' > of an entity group and then checking whether 'it has changed since our > initial check'. > > While I don't expect this short paragraph to describe all of > the complexities of App Engine transactions it does lay out a simplistic > implementation which does not cover the 'create if it doesn't exist' > transaction given on the same page. I have read a number of other > descriptions of App Engine transactions and none of them indicate that this > case is covered. My expectation is that it is, in fact, covered and these > transactions work. But I do need to be sure. > > *"When a transaction starts, App Engine uses **optimistic concurrency > control <http://en.wikipedia.org/wiki/Optimistic_concurrency_control> by > checking the last update time for the entity groups used in the > transaction. Upon commiting a transaction for the entity groups, App Engine > again checks the last update time for the entity groups used in the > transaction. If it has changed since our initial check, an error is > returned."* > > https://cloud.google.com/appengine/docs/go/datastore/transactions > > Francis > > On 15 May 2015 at 22:45, Francis Stephens <[email protected]> wrote: > >> I appreciate that the question is difficult, and I'm sure it could be >> worded differently. However, the reason I have phrased it as I did was >> because I really need to clarify this one case. It's very important to me. >> >> Perhaps it would be clearer if I try to address your description of an >> app engine transaction. >> >> *"When you start a transaction, each entity group (defined by the root >> key, whether or not an entity exists at that key) touched is enlisted in >> the transaction."* >> >> In the case I described the transaction is entered simultaneously by two >> instances. >> >> In this scenario both Get(...) calls returns an ErrNoSuchEntity we know >> (or I believe) at that point that the key does not exist anywhere in the >> datastore, attached to an entity or not. So I would like to know how the >> entity group is enlisted in this case. >> >> *"When you commit a transaction, if any of those entity groups have been >> changed by anyone anywhere, your transaction rolls back"* >> >> When the two Put(...) calls are executed one of them must fail with an >> ErrConcurrentTransaction (or ConcurrentModificationException in Java). >> So the question I have is, where the information which would allow for one >> of the Put(...) calls to fail? >> >> I have two reasons for asking for clarification on this point. >> >> 1: Data races are often very subtle, and I would like to clarify my >> understanding in this instance. >> 2: In the case given above a low probability data-race where the first >> calls might be over-written would be acceptable for most counters. That >> would be a very small counting loss and the chance of such a race, if it is >> possible at all, would have negligible impact. We don't have the same >> luxury, so I want be absolutely sure. >> >> Thanks for your response. >> >> Francis >> >> On 15 May 2015 at 01:34, Jeff Schnitzer <[email protected]> wrote: >> >>> I'm honestly having a hard time understanding your questions. >>> >>> GAE transactions are pretty simple. When you start a transaction, each >>> entity group (defined by the root key, whether or not an entity exists at >>> that key) touched is enlisted in the transaction. When you commit a >>> transaction, if any of those entity groups have been changed by anyone >>> anywhere, your transaction rolls back (in Java you get >>> ConcurrentModificationException). >>> >>> Your questions would be easier to understand without bringing up >>> implementation details like timestamps. The implementation is not >>> particularly relevant to the behavior. >>> >>> Jeff >>> >>> >>> On Thu, May 14, 2015 at 5:49 AM, Francis Stephens <[email protected]> wrote: >>> >>>> It's unclear to me where I should be posting this question. I've posted >>>> it to stack overflow in case that is more appropriate, however the nature >>>> of the question goes against the SO guidelines. >>>> >>>> >>>> http://stackoverflow.com/questions/30233093/app-engine-mechanics-of-creating-a-unique-entity-if-it-doesnt-exist >>>> >>>> >>>> On Wednesday, 13 May 2015 13:59:57 UTC+1, Francis Stephens wrote: >>>>> >>>>> We have an issue where we want to lazily create an entity if it does >>>>> not exist. There is some discussion going on about how to do this and I >>>>> would like to clarify some things around app engine transactions. I will >>>>> limit my query to single entity group transactions. >>>>> >>>>> I am using Go in my examples, but I hope the code is clear enough for >>>>> non-Go programmers. >>>>> >>>>> My understanding is that a transaction, on a single entity group, will >>>>> succeed only if the entity group is not modified externally during the >>>>> transaction. The 'entity group timestamp' indicating when an entity group >>>>> was changed is stored in the root entity of the entity group. So during a >>>>> transaction the current 'entity group timestamp' is read and the >>>>> transaction can only succeed if it hasn't changed by the end of the >>>>> transaction. >>>>> >>>>> key := datastore.NewKey(c, "Counter", "mycounter", 0, nil) >>>>> count := new(Counter) >>>>> err := datastore.RunInTransaction(c, func(c appengine.Context) error >>>>> { >>>>> err := datastore.Get(c, key, count) >>>>> if err != nil && err != datastore.ErrNoSuchEntity { >>>>> return err >>>>> } >>>>> count.Count++ >>>>> _, err = datastore.Put(c, key, count) >>>>> return err >>>>> }, nil) >>>>> >>>>> >>>>> In the example above (taken from >>>>> https://cloud.google.com/appengine/docs/go/datastore/transactions) >>>>> there are two non-error cases, I can see. >>>>> >>>>> 1: The Get succeeds and the 'entity group timestamp' on the counter >>>>> can be used to ensure no other transactions update the counter during this >>>>> transaction. >>>>> 2: The Get fails with ErrNoSuchEntity and the Put is used to store the >>>>> counter for the first time. >>>>> >>>>> In the second case it is possible that another identical transaction >>>>> is running. If both transactions' Get return ErrNoSuchEntity how does the >>>>> datastore ensure that only one put succeeds? I would expect there to be no >>>>> 'entity group timestamp' in the datastore to test against? >>>>> >>>>> Does the transaction know that it needs to test for the non-existence >>>>> of the counter in order for the Put and the entire transaction to succeed? >>>>> >>>>> Is there a chance in this case for two transactions to succeed and for >>>>> one Put to overwrite the other? >>>>> >>>>> If there is documentation, or videos etc, around the mechanism that >>>>> controls this I would love to read it. >>>>> >>>>> Thanks in advance. >>>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "Google App Engine" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To post to this group, send email to [email protected]. >>>> Visit this group at http://groups.google.com/group/google-appengine. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/google-appengine/39fce7cf-fa5a-464f-821e-0826f8fdb500%40googlegroups.com >>>> <https://groups.google.com/d/msgid/google-appengine/39fce7cf-fa5a-464f-821e-0826f8fdb500%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>>> For more options, visit https://groups.google.com/d/optout. >>>> >>> >>> -- >>> You received this message because you are subscribed to a topic in the >>> Google Groups "Google App Engine" group. >>> To unsubscribe from this topic, visit >>> https://groups.google.com/d/topic/google-appengine/NagJ97YExB0/unsubscribe >>> . >>> To unsubscribe from this group and all its topics, send an email to >>> [email protected]. >>> To post to this group, send email to [email protected]. >>> Visit this group at http://groups.google.com/group/google-appengine. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/google-appengine/CADK-0uhGRUYTb-4ehr3C9Got1oe9bFfiOHLUj3qmWCoebcYB5w%40mail.gmail.com >>> <https://groups.google.com/d/msgid/google-appengine/CADK-0uhGRUYTb-4ehr3C9Got1oe9bFfiOHLUj3qmWCoebcYB5w%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >>> For more options, visit https://groups.google.com/d/optout. >>> >> >> >> >> -- >> Francis Stephens >> Software Developer @ Belua >> > > > > -- > Francis Stephens > Software Developer @ Belua > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > Visit this group at http://groups.google.com/group/google-appengine. > To view this discussion on the web visit > https://groups.google.com/d/msgid/google-appengine/CAG1f2XCoSfD9f%2BHpnbNL8j6rQ7AD1tmw3AJ7beL%3DviwUqo8xBA%40mail.gmail.com > <https://groups.google.com/d/msgid/google-appengine/CAG1f2XCoSfD9f%2BHpnbNL8j6rQ7AD1tmw3AJ7beL%3DviwUqo8xBA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. Visit this group at http://groups.google.com/group/google-appengine. To view this discussion on the web visit https://groups.google.com/d/msgid/google-appengine/CADK-0ui35rWyVRcPLuL5aNX8Z73%3DN7qht%3DycormWX5ih_L__BQ%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
