On Thu, Jul 28, 2011 at 12:31, Stephen Johnson <[email protected]> wrote: > This does seem to be a conundrum. I'll offer up an idea for those instances > where this behavior is of concern. It's not a great solution but if you need > to guarantee that a transaction definitely succeeds and your transaction is > idempotent then maybe this or some variation will help. > First off, incrementing version numbers won't work due to the following > scenario: > 1.) open a transaction find out the current version is 78 > 2.) perform update and increment the version to 79 > 3.) commit transaction > 4.) get an exception. Now, did the transaction succeed or fail?? This is the > conundrum > 5.) so re-fetch the item and check the version number, it says 79, so > transaction must have succeed, but what if some other transaction in the > meantime was the one that updated the item and incremented the version > number. So back to conundrum!!!
This method doesn't really make sense if you read the version number in the transaction, do stuff, increment the version, then reput within the same transaction. However, I do use something like this in some cases, except I have read the current version prior to 'computing' my updated values. So I pass the expected version number into the transaction. If I get a mismatch then I know the update values might be wrong and the update either needs skipped, recomputed, or the user needs to OK the changes (and of course this happens based on the current version number). > Okay, so instead of using a version number we use a random transaction stamp > (this could be a random number plus some hashed in data that is being > transacted, etc. to make it really unique). Then instead of a version number > property, we add a transaction stamp list that holds, for example, the last > 5 or 10 stamps, or however many you want. So it goes like this: > 1.) open a transaction > 2.) perform update > 3.) add unique transaction stamp to list, if list has reached maximum remove > the oldest one (FIFO). > 4.) commit transaction > 5.) if everything is fine, go on our merry way > 6.) if exception is generated, then re-query the entity if the entity has > "our" unique timestamp in its list then the transaction really did succeed, > if not then the transaction really did fail. Again, the unique id needs generated outside the transaction so that if the transaction were to run again the id would be the same. Also, instead of the list property you could use a separate entity (in the same entity group) with no properties as a marker. That avoids the deserialization cost and is totally scalable. With some minor tweaks you would even be able to cleanup the old marker entities. Also, as Stephen notes in a later comment, if the next read is in a transaction it will force any outstanding writes to be applied -- so it is always consistent. Note that a db.get(some_key) implicitly uses a transaction. Robert > Well, that's my idea, > Stephen > CortexConnect > cortexconnect.appspot.com > > > On Thu, Jul 28, 2011 at 7:51 AM, Pol <[email protected]> wrote: >> >> I see two big problems here: >> >> 1) There appear to be absolutely no record, say in the dashboard, of >> failed transactions that eventually succeeded or really failed. For >> some type of apps, I could live with 1 in a million transaction >> failure that may or may not be eventually successful, but I need to >> know which entity got in a degenerated state. >> >> 2) The benefit of the current approach of fake-real-transactions is >> completely unknown: does it make GAE massively faster or more reliable >> or something? >> >> For instance, if you choose non-redundant storage in Amazon S3: >> 1) you get notifications for lost objects >> 2) it's cheaper >> >> On Jul 28, 4:02 pm, Joshua Smith <[email protected]> wrote: >> > The problem is that google transactions can report an exception, and >> > then go ahead and succeed anyway. >> > >> > So the docs recommend that you only write idempotent transactions, which >> > is a completely silly suggestion. I've yet to see a single example of how >> > one might write an idempotent transaction. (Unless, I suppose, you create >> > a >> > separate child model in the database which is parented by the object you >> > are >> > transacting on, and then you query the list of children every time you >> > retry >> > your transaction to see if its already in there, but that won't scale.) >> > >> > I contend that a DB that cannot tell you reliably whether a transaction >> > succeeded for failed does not support transactions. >> > >> > GAE can essentially report 3 possible results from a transaction: >> > - Definitely succeeded >> > - Definitely failed >> > - Beats me >> > >> > I contend that third possible result makes it impossible to write >> > software that relies on transactions. >> > >> > Therefore, GAE doesn't support transactions. >> > >> > On Jul 27, 2011, at 8:58 PM, Tim Hoffman wrote: >> > >> > >> > >> > >> > >> > >> > >> > > If you always get modify put within a transaction how would it be >> > > unreliable? >> > >> > > Rgds >> > >> > > Tim >> > >> > > -- >> > > You received this message because you are subscribed to the Google >> > > Groups "Google App Engine" group. >> > > To view this discussion on the web >> > > visithttps://groups.google.com/d/msg/google-appengine/-/mP_8kv_-LlMJ. >> > > To post to this group, send email to >> > > [email protected]. >> > > To unsubscribe from this group, send email to >> > > [email protected]. >> > > For more options, visit this group >> > > athttp://groups.google.com/group/google-appengine?hl=en. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
