Hi Takashi & Joshua, How about using my user's id as the key ?
eg: new_score = Score(key_name='ahFkZXZ-c3Zlbi13b3JkaGVyb3ILCxIEVXNlchjFAww') I assume this would not count as monotonically increasing ? @ Joshua: At the moment, I create all new Score objects for each player of that game round and then have the backend reap them and afterwards fire a task to delete them. HOWEVER, what is I did not delete them, but instead marked them with the user's account key_name as the Score's key_name (see above) and also included the following: timestamp = db.DateTimeProperty(auto_now = True) Then I could figure out which ones were from the last round and which were from players that had last played a while ago. The assumption is that at this point, players from the previous have would have their results stored using a put() with a key name, which would be strongly consistent. Players whose first game it is, would have the chance that the DB was slow and did not yet have an entry for them ... in which case, they would just miss out on one round (unless the internal commit() takes > 3 minutes!) ... and I could update the game client to add in a user's score to the results if they were not found to 'fudge' things. Comments/thoughts ? -R Sorry it didn't work like a magic, but a single datastore get > shouldn't take that long, so there's another issue with datastore > latency. It might be tablet splitting issue which I mentioned in > another thread: > > http://ikaisays.com/2011/01/25/app-engine-datastore-tip-monotonically-increasing-values-are-bad/ > > > I'd suggest generating somewhat ramdom(well-distributed) keys for > those small entities instead of using automatic key assignment. > Besides using datastore, maybe I'll experiment with an implementation > of a memory db with some backends instances tomorrow. > > -- Takashi > > > > > > > > > On Wednesday, August 1, 2012 2:01:43 PM UTC-4, Richard wrote: > >> > >> Yep... got that. However, a query() will still return stale data > between > >> the put() and the (internal) commit(). Which is where I believe I am > >> sitting... > >> > > > > On Wednesday, August 1, 2012 2:01:43 PM UTC-4, Richard wrote: > >> > >> Yep... got that. However, a query() will still return stale data > between > >> the put() and the (internal) commit(). Which is where I believe I am > >> sitting... > >> > >> On Wednesday, August 1, 2012 12:39:41 PM UTC-4, Jeff Schnitzer wrote: > >>> > >>> I presume the essential line is this: > >>> > >>> "In the (now standard) High Replication Datastore, the transaction > >>> typically is completely applied within a few hundred milliseconds > >>> after the commit returns. However, even if it is not completely > >>> applied, subsequent reads, writes, and ancestor queries will always > >>> reflect the results of the commit, because these operations apply any > >>> outstanding modifications before executing." > >>> > >>> ...which is interesting and not something I ever thought of. I don't > >>> think this intermediate state was discussed in Alfred's "Under The > >>> Covers" I/O talk last year. Can someone describe it in more detail? > >>> > >>> Jeff > >>> > >>> On Wed, Aug 1, 2012 at 8:45 AM, Richard <steven...@gmail.com> wrote: > >>> > Ok, so I read that web page, and I understand that the put() has > >>> > completed, > >>> > but not the commit(). > >>> > > >>> > However, I don't really see the potential solution you are > suggesting > >>> > :( > >>> > Can you (or someone else) please explain in a bit more detail ? > >>> > > >>> > > >>> > On Wednesday, August 1, 2012 10:55:35 AM UTC-4, Takashi Matsuo > (Google) > >>> > wrote: > >>> >> > >>> >> I forgot to mention about one possible workaround which could > mitigate > >>> >> your pain. In the frontend instances, maybe you can get those small > >>> >> entities with a newly created keys just after putting them in order > to > >>> >> apply the change locally. > >>> >> > >>> >> This article describes this behavior well: > >>> >> > https://developers.google.com/appengine/articles/transaction_isolation > >>> >> > >>> >> However, please note that > >>> >> * It doesn't guarantee 100% 'strong consistency'. > >>> >> * The latency of submitting scores will become larger. > >>> >> * Definitely it will cost you more. > >>> >> > >>> >> Sorry if this doesn't work well for you, but I think it's worth > >>> >> trying. > >>> >> > >>> >> -- Takashi > >>> >> > >>> >> On Wed, Aug 1, 2012 at 11:30 PM, Takashi Matsuo <tmat...@google.com> > > >>> >> wrote: > >>> >> > Hi Richard, > >>> >> > > >>> >> > I actually played your game and probably encountered the exact > >>> >> > issue. > >>> >> > BTW, it's a cool addictive game. > >>> >> > > >>> >> > I agree that eventually consistent window should ideally be more > >>> >> > steady than now. I have passed your issue to the engineering > team. > >>> >> > > >>> >> > I'll get back to you when I have some updates. > >>> >> > > >>> >> > -- Takashi > >>> >> > > >>> >> > On Wed, Aug 1, 2012 at 10:37 PM, Richard <steven...@gmail.com> > >>> >> > wrote: > >>> >> >> The thing is.... GAE does a wonderful job. It has been fine for > a > >>> >> >> year > >>> >> >> or > >>> >> >> so. Now suddenly, we are getting bitten by 'eventual > consistency'. > >>> >> >> And not > >>> >> >> at peak load either! This is hitting us at the lowest load time > >>> >> >> and at > >>> >> >> the > >>> >> >> same time each day. > >>> >> >> > >>> >> >> So, maybe we were just lucky before this..... but it still sucks > to > >>> >> >> have > >>> >> >> something work for a year or so, and then suddenly find out you > >>> >> >> might > >>> >> >> need > >>> >> >> to consider moving to a different stack. > >>> >> >> > >>> >> >> > >>> >> >> On Wednesday, August 1, 2012 7:14:19 AM UTC-4, Richard Watson > >>> >> >> wrote: > >>> >> >>> > >>> >> >>> On Wed, Aug 1, 2012 at 12:13 PM, Jeff Schnitzer > >>> >> >>> <j...@infohazard.org> > >>> >> >>> wrote: > >>> >> >>> > >>> >> >>> > Anything that involves batching in a frontend risks orphaning > >>> >> >>> > data > >>> >> >>> > in > >>> >> >>> > the frontend... there's just no efective way to ensure that > >>> >> >>> > batching > >>> >> >>> > happens and that the queue is purged when "done". > >>> >> >>> > >>> >> >>> It's definitely not bulletproof, but I'd investigate whether I > >>> >> >>> could > >>> >> >>> achieve "good enough" for 99% and be happy that for the rest > the > >>> >> >>> client could re-request the score if it didn't reflect their > own > >>> >> >>> correct value. > >>> >> >>> > >>> >> >>> As you say, if clients can connect directly to some other more > >>> >> >>> reliable service, maybe that's the way to go. Mixing platforms > >>> >> >>> (GAE > >>> >> >>> url fetch to Amazon SQS, or whatever) would leave me worrying > >>> >> >>> about > >>> >> >>> extra points of failure - you'll now fail if either GAE or > >>> >> >>> Amazon/Heroku go down. > >>> >> >>> > >>> >> >>> Pity. I would imaging a back-end for a game with hundreds of > >>> >> >>> users is > >>> >> >>> something GAE should eat up. > >>> >> >>> > >>> >> >>> Richard > >>> >> >> > >>> >> >> -- > >>> >> >> You received this message because you are subscribed to the > Google > >>> >> >> Groups > >>> >> >> "Google App Engine" group. > >>> >> >> To view this discussion on the web visit > >>> >> >> https://groups.google.com/d/msg/google-appengine/-/MpsNezxGjpkJ. > > >>> >> >> > >>> >> >> To post to this group, send email to > >>> >> >> google-appengine@googlegroups.com. > >>> >> >> To unsubscribe from this group, send email to > >>> >> >> google-appengine+unsubscr...@googlegroups.com. > >>> >> >> For more options, visit this group at > >>> >> >> http://groups.google.com/group/google-appengine?hl=en. > >>> >> > > >>> >> > > >>> >> > > >>> >> > -- > >>> >> > Takashi Matsuo > >>> >> > >>> >> > >>> >> > >>> >> -- > >>> >> Takashi Matsuo > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "Google App Engine" group. > > To view this discussion on the web visit > > https://groups.google.com/d/msg/google-appengine/-/v9XP85rmNM8J. > > > > To post to this group, send email to google-appengine@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine+unsubscr...@googlegroups.com. > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > Takashi Matsuo > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/iJA7RSzgG0oJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
