Ok, upon further further reflection, MG is right there is an
opportunity for failure, even if MemcacheService.increment() behaves
as expected. Another description of the problem, with indentation to
distinguish the two threads:
Initial state, last value in db (and memcache) is 100
STARTING SAVE OF FIRST OBJ
value 101 obtained from memcache increment()
put() of first object with 101 begins (but not finished)
Memcache gets reset for whatever reason
STARTING SAVE OF SECOND OBJ
Querying for max value produces 100
Memcache increment() with initial value produces 101
put() of second object with 101 executes
put() of first object with 101 completes
At the end of this race you end up with duplicate objects with value of 101.
On the whole this seems pretty unlikely. It's worth putting a note in
Monotonic's documentation, but even with a small risk of occasional
duplicates the tool works effectively enough for my needs
(synchronization of rapidly changing data with a slave pseudo-database
outside of GAE).
MG, I'm curious about how you're approaching the problem, because my
gut sense right now is that there isn't a scalable way to produce a
100% guaranteed monotonically increasing number given the tools
available in GAE.
Jeff
On Sun, Jan 2, 2011 at 8:36 AM, Jeff Schnitzer <[email protected]> wrote:
> Upon deeper consideration, whether or not Monotonic currently could
> produce dups depends on the exact behavior of:
>
> MemcacheService.increment(java.lang.Object key, long delta,
> java.lang.Long initialValue)
>
> Unfortunately the documentation is ambiguous about what the return
> value will be when MemcacheService is down. Does it return null or
> does it return initialValue?
>
> If it returns null like the non-initial-value version (which is what
> the code currently assumes), the current Monotonic code is safe and
> won't produce duplicates. It will throw an IllegalStateException.
>
> If this method returns initialValue, it's still pretty easy to change
> the logic to guarantee IllegalStateException instead.
>
> Goog, any clarification of what MemcacheService.increment() does?
>
> Jeff
>
> On Sun, Jan 2, 2011 at 6:27 AM, Jeff Schwartz <[email protected]> wrote:
>> It is safe. The montonic implementation expressly handled those cases where
>> memcache is not available and throws an IllegalStateException. When memcache
>> is not available you can be almost certain that the datastore isn't
>> available as well so responding to these cases appropriately - such as
>> redirecting a user to a page explaining that the datastore isn't currently
>> available - will ensure that you maintain the integrity of the data.
>>
>> Jeff Schwartz
>>
>>
>>
>>
>> On Sun, Jan 2, 2011 at 12:56 AM, aswath satrasala
>> <[email protected]> wrote:
>>>
>>> Hi MG,
>>> I am currently using the following in my application.
>>>
>>> http://blog.appenginefan.com/2009/04/efficient-global-counters-revisited.html.
>>> It does not use Objectify
>>>
>>> But, I was planning to use the Objecify Monotic, so as to keep counter
>>> implementation simple and not worry about another library details. Now,
>>> with your valid points, it is good discussion thread with Objectify team and
>>> find out what they think.
>>>
>>> -Aswath
>>> www.AccountingGuru.in
>>>
>>> On Sat, Jan 1, 2011 at 10:13 PM, MG <[email protected]> wrote:
>>>>
>>>> Monotonic in Objectify, as defined here:
>>>>
>>>>
>>>> http://code.google.com/p/objectify-appengine/source/browse/trunk/src/com/googlecode/objectify/helper/Monotonic.java?r=620
>>>>
>>>> is not 100% safe. For example:
>>>>
>>>> 1. Memcache is off, lastMax is 100
>>>> 2. Memcache goes up for 1 sec, five requests get 101, 102, ... 105
>>>> numbers for them
>>>> 3. Memcache goes off again for 1 sec; requests from #2 are still
>>>> preparing data to insert in the related datastore model
>>>> 4. Memcache goes up, another five requests get 101, 102, ..., 105 numbers
>>>> for them
>>>> 5. now we have duplicate numbers.
>>>>
>>>> MG
>>>>
>>>> On Saturday, January 1, 2011 11:08:50 AM UTC-5, aswath wrote:
>>>>>
>>>>> I was seeing your link. You have used Objectify. Objectify already
>>>>> provides Monotonic helper class.
>>>>>
>>>>> -Aswath
>>>>> www.accountingguru.in
>>>>>
>>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google Groups
>>>> "Google App Engine for Java" group.
>>>> To post to this group, send email to
>>>> [email protected].
>>>> To unsubscribe from this group, send email to
>>>> [email protected].
>>>> For more options, visit this group at
>>>> http://groups.google.com/group/google-appengine-java?hl=en.
>>>
>>
>>
>>
>> --
>> Jeff Schwartz
>>
>>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
