Thanks Jeff,
I read Ikai Lan's blog post
[1<http://ikaisays.com/2011/01/25/app-engine-datastore-tip-monotonically-increasing-values-are-bad/>]
about monotonically increasing
values and datetime on AppEngine, which ultimately prompted me to ask my
question.
I think my scenario is pretty simple, so might help I describe it.
I have the following Entity structure:
Entity: [CommitEntity("1")]
latestVersion = 2
count = 2
Entity: [Commit("1")/Event("aaa")]
version = 1
Entity: [Commit("1")/Event("bbb")]
version = 2
I'm adding all the Event entities within a transaction and the Commit entity
group, so I should be able to assign a sequential value which is simply an
increment of the latest version (stored in the parent entity) without
contention.
Then, I can get a sorted list of Event entities given a specific Commit key:
val commitKey = KeyFactory.createKey("Commit", "1")
val query = new Query("Event", commitKey).addSort("version")
This works fine now, but I may be naive in thinking this will scale on
AppEngine.
[1]
http://ikaisays.com/2011/01/25/app-engine-datastore-tip-monotonically-increasing-values-are-bad/
On Wed, May 18, 2011 at 2:02 PM, Jeff Schnitzer <[email protected]> wrote:
> To answer your question more specifically... the answer is no.
>
> If you don't apply a sort order, ordering is undefined and may even
> change from query to query.
>
> You can order by __key__ but there is no guarantee that keys increase
> monotonically.
>
> You can create your own datestamp field but keep in mind that clocks
> can (and will) skew between instances. The amount of skew has no
> guaranteed limit. So you can pretty much only guarantee ordering for
> put()s from a single instance, and only if you ensure that the
> datestamp actually increments between writes (be careful with batch
> puts).
>
> I've found it necessary to create my own monotonically increasing
> counter using memcache increment to deal with situations like this.
> It is a nontrivial problem.
>
> Jeff
>
> On Tue, May 17, 2011 at 4:59 PM, Erick Fleming <[email protected]> wrote:
> > @Brandon
> > You are correct, I really care about the order of the records when
> > retrieving them, so I guess my question should have been "Is the result
> > order of a query consistent with the put order".
> > I know I can use DateTime (or Date in Java), but I always need these
> > Entities in a specific order and was just trying to avoid using the sort.
> > Thanks for your feedback and especially the humorous example :-)
> > On Tue, May 17, 2011 at 6:31 PM, Brandon Wirtz <[email protected]>
> wrote:
> >>
> >> Order often matters. (I don't know about the context of this app)
> >>
> >> Say you are 4th Bank Of Brandon
> >>
> >> You process transactions from your customer Strom
> >>
> >> Strom has $500 in his account.
> >>
> >> Strom hits the Che Target for $450, Cylon's Coffee Shop for $6, the Hair
> >> Stylist for $50, and Google bills him for each of his for App Engine
> >> Accounts at $9 each.
> >>
> >> Because of the way the bank does caching Strom could make any purchase
> in
> >> a
> >> 4 hour window up to $600 ($500 plus his $100 overdraft protection)
> >>
> >> If we process the charges in the above order, 4BoB charges Strom $35
> over
> >> draft on the last 5 trans actions.
> >> If we process the charges in the reverse order 4BoB only gets one over
> >> draft.
> >> Depending on the order you can have anywhere from 1 to 6 overdrafts when
> >> the
> >> daily fee assessment task runs and determines how much Strom owes.
> >>
> >> Yes if we put the time stamp in it shouldn't matter which order the
> items
> >> were added to the table... Which is likely the bit that Erick needed.
> >> (BTW
> >> Erick you are both Eric with a C and a K so I'd have thought you'd have
> >> gotten the redundancy I'm going to explain next...
> >>
> >> When you store data that will need to be ordered add the auto DateTime
> to
> >> it. Or the Time the thing took place, like my bank example someone may
> >> have
> >> made a purchase on Tuesday that doesn't show up until Thursday and you'd
> >> want both when you got the transaction and when it happened.
> >>
> >> Then sort the returned date by the DateTime, (or that other time thing
> we
> >> just discussed) then do whatever you were going to do with the data.
> >>
> >> Don't do this...
> >>
> >> Strom has
> >>
> >> $500
> >> -450
> >> = 50
> >>
> >> Google sends 4 requests for $9 in 3 seconds
> >>
> >> 50-9 = 41
> >> 50-9=41
> >> 41-9=32
> >> 32-9=23
> >>
> >> And you missed $9 because you processed each trans action rather than
> >> batching a lot of transactions but only using data older than a certain
> >> amount, and without flagging said data as "processed".
> >>
> >>
> >>
> >>
> >>
> >>
> >> -----Original Message-----
> >> From: [email protected]
> >> [mailto:[email protected]] On Behalf Of Strom
> >> Sent: Tuesday, May 17, 2011 3:55 PM
> >> To: Google App Engine
> >> Subject: [google-appengine] Re: Is the order of datastore puts
> guaranteed?
> >>
> >> First, the context needs to be a transaction, otherwise there isn't even
> a
> >> guarantee that all puts will happen.
> >> Even inside a transaction though, why does the order matter? I can only
> >> think of a situation where you use system generated ids, but those
> aren't
> >> guaranteed to be in order anyway.
> >>
> >> On May 17, 10:02 pm, Erick Fleming <[email protected]> wrote:
> >> > Within the context of an Entity group, can I guarantee that puts will
> >> > be in the order I put them?
> >> >
> >> > Thanks
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "Google App Engine" group.
> >> To post to this group, send email to [email protected].
> >> To unsubscribe from this group, send email to
> >> [email protected].
> >> For more options, visit this group at
> >> http://groups.google.com/group/google-appengine?hl=en.
> >>
> >>
> >> --
> >> You received this message because you are subscribed to the Google
> Groups
> >> "Google App Engine" group.
> >> To post to this group, send email to [email protected].
> >> To unsubscribe from this group, send email to
> >> [email protected].
> >> For more options, visit this group at
> >> http://groups.google.com/group/google-appengine?hl=en.
> >>
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to [email protected].
> > To unsubscribe from this group, send email to
> > [email protected].
> > For more options, visit this group at
> > http://groups.google.com/group/google-appengine?hl=en.
> >
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
