Ok, based on Takashi's suggestion, I now do the following: new_user_score = Score() new_user_score.member = thing new_user_score.put()
# next part is new based on Takashi's suggestion k = new_user_score.key() db.get(k) WOW... I mean... HOLY CRIPES! Time to process each request went from ~150msec to 4.5-7 SECONDS. This means I effectively need a one instance PER REQUEST. On Wednesday, August 1, 2012 2:01:43 PM UTC-4, Richard wrote: > > Yep... got that. However, a query() will still return stale data between > the put() and the (internal) commit(). Which is where I believe I am > sitting... > > On Wednesday, August 1, 2012 2:01:43 PM UTC-4, Richard wrote: > > Yep... got that. However, a query() will still return stale data between > the put() and the (internal) commit(). Which is where I believe I am > sitting... > > On Wednesday, August 1, 2012 12:39:41 PM UTC-4, Jeff Schnitzer wrote: >> >> I presume the essential line is this: >> >> "In the (now standard) High Replication Datastore, the transaction >> typically is completely applied within a few hundred milliseconds >> after the commit returns. However, even if it is not completely >> applied, subsequent reads, writes, and ancestor queries will always >> reflect the results of the commit, because these operations apply any >> outstanding modifications before executing." >> >> ...which is interesting and not something I ever thought of. I don't >> think this intermediate state was discussed in Alfred's "Under The >> Covers" I/O talk last year. Can someone describe it in more detail? >> >> Jeff >> >> On Wed, Aug 1, 2012 at 8:45 AM, Richard <[email protected]> wrote: >> > Ok, so I read that web page, and I understand that the put() has >> completed, >> > but not the commit(). >> > >> > However, I don't really see the potential solution you are suggesting >> :( >> > Can you (or someone else) please explain in a bit more detail ? >> > >> > >> > On Wednesday, August 1, 2012 10:55:35 AM UTC-4, Takashi Matsuo (Google) >> > wrote: >> >> >> >> I forgot to mention about one possible workaround which could mitigate >> >> your pain. In the frontend instances, maybe you can get those small >> >> entities with a newly created keys just after putting them in order to >> >> apply the change locally. >> >> >> >> This article describes this behavior well: >> >> https://developers.google.com/appengine/articles/transaction_isolation >> >> >> >> However, please note that >> >> * It doesn't guarantee 100% 'strong consistency'. >> >> * The latency of submitting scores will become larger. >> >> * Definitely it will cost you more. >> >> >> >> Sorry if this doesn't work well for you, but I think it's worth >> trying. >> >> >> >> -- Takashi >> >> >> >> On Wed, Aug 1, 2012 at 11:30 PM, Takashi Matsuo <[email protected]> >> >> wrote: >> >> > Hi Richard, >> >> > >> >> > I actually played your game and probably encountered the exact >> issue. >> >> > BTW, it's a cool addictive game. >> >> > >> >> > I agree that eventually consistent window should ideally be more >> >> > steady than now. I have passed your issue to the engineering team. >> >> > >> >> > I'll get back to you when I have some updates. >> >> > >> >> > -- Takashi >> >> > >> >> > On Wed, Aug 1, 2012 at 10:37 PM, Richard <[email protected]> >> wrote: >> >> >> The thing is.... GAE does a wonderful job. It has been fine for a >> year >> >> >> or >> >> >> so. Now suddenly, we are getting bitten by 'eventual consistency'. >> >> >> And not >> >> >> at peak load either! This is hitting us at the lowest load time >> and at >> >> >> the >> >> >> same time each day. >> >> >> >> >> >> So, maybe we were just lucky before this..... but it still sucks to >> >> >> have >> >> >> something work for a year or so, and then suddenly find out you >> might >> >> >> need >> >> >> to consider moving to a different stack. >> >> >> >> >> >> >> >> >> On Wednesday, August 1, 2012 7:14:19 AM UTC-4, Richard Watson >> wrote: >> >> >>> >> >> >>> On Wed, Aug 1, 2012 at 12:13 PM, Jeff Schnitzer < >> [email protected]> >> >> >>> wrote: >> >> >>> >> >> >>> > Anything that involves batching in a frontend risks orphaning >> data >> >> >>> > in >> >> >>> > the frontend... there's just no efective way to ensure that >> batching >> >> >>> > happens and that the queue is purged when "done". >> >> >>> >> >> >>> It's definitely not bulletproof, but I'd investigate whether I >> could >> >> >>> achieve "good enough" for 99% and be happy that for the rest the >> >> >>> client could re-request the score if it didn't reflect their own >> >> >>> correct value. >> >> >>> >> >> >>> As you say, if clients can connect directly to some other more >> >> >>> reliable service, maybe that's the way to go. Mixing platforms >> (GAE >> >> >>> url fetch to Amazon SQS, or whatever) would leave me worrying >> about >> >> >>> extra points of failure - you'll now fail if either GAE or >> >> >>> Amazon/Heroku go down. >> >> >>> >> >> >>> Pity. I would imaging a back-end for a game with hundreds of >> users is >> >> >>> something GAE should eat up. >> >> >>> >> >> >>> Richard >> >> >> >> >> >> -- >> >> >> You received this message because you are subscribed to the Google >> >> >> Groups >> >> >> "Google App Engine" group. >> >> >> To view this discussion on the web visit >> >> >> https://groups.google.com/d/msg/google-appengine/-/MpsNezxGjpkJ. >> >> >> >> >> >> To post to this group, send email to >> [email protected]. >> >> >> To unsubscribe from this group, send email to >> >> >> [email protected]. >> >> >> For more options, visit this group at >> >> >> http://groups.google.com/group/google-appengine?hl=en. >> >> > >> >> > >> >> > >> >> > -- >> >> > Takashi Matsuo >> >> >> >> >> >> >> >> -- >> >> Takashi Matsuo >> > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/v9XP85rmNM8J. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
