On Thu, Aug 2, 2012 at 4:11 AM, Richard <[email protected]> wrote: > Hi Takashi & Joshua, > > How about using my user's id as the key ? > > eg: > new_score = Score(key_name='ahFkZXZ-c3Zlbi13b3JkaGVyb3ILCxIEVXNlchjFAww') > > I assume this would not count as monotonically increasing ?
I don't think it's ok. Probably the first few characters would be always the same. It's not good. Additionally, from your explanation bellow, a single user has multiple scores, right? If you use userid as key_name, the latest score always override the previous score. What about something like user_name + timestamp? > > @ Joshua: > At the moment, I create all new Score objects for each player of that game > round and then have the backend reap them and afterwards fire a task to > delete them. HOWEVER, what is I did not delete them, but instead marked > them with the user's account key_name as the Score's key_name (see above) > and also included the following: > timestamp = db.DateTimeProperty(auto_now = True) > > Then I could figure out which ones were from the last round and which were > from players that had last played a while ago. The assumption is that at > this point, players from the previous have would have their results stored > using a put() with a key name, which would be strongly consistent. Players > whose first game it is, would have the chance that the DB was slow and did > not yet have an entry for them ... in which case, they would just miss out > on one round (unless the internal commit() takes > 3 minutes!) ... and I > could update the game client to add in a user's score to the results if they > were not found to 'fudge' things. > > Comments/thoughts ? > > -R > > >> Sorry it didn't work like a magic, but a single datastore get >> shouldn't take that long, so there's another issue with datastore >> latency. It might be tablet splitting issue which I mentioned in >> another thread: >> >> http://ikaisays.com/2011/01/25/app-engine-datastore-tip-monotonically-increasing-values-are-bad/ >> >> I'd suggest generating somewhat ramdom(well-distributed) keys for >> those small entities instead of using automatic key assignment. >> Besides using datastore, maybe I'll experiment with an implementation >> of a memory db with some backends instances tomorrow. >> >> -- Takashi >> >> > >> > >> > >> > On Wednesday, August 1, 2012 2:01:43 PM UTC-4, Richard wrote: >> >> >> >> Yep... got that. However, a query() will still return stale data >> >> between >> >> the put() and the (internal) commit(). Which is where I believe I am >> >> sitting... >> >> >> > >> > On Wednesday, August 1, 2012 2:01:43 PM UTC-4, Richard wrote: >> >> >> >> Yep... got that. However, a query() will still return stale data >> >> between >> >> the put() and the (internal) commit(). Which is where I believe I am >> >> sitting... >> >> >> >> On Wednesday, August 1, 2012 12:39:41 PM UTC-4, Jeff Schnitzer wrote: >> >>> >> >>> I presume the essential line is this: >> >>> >> >>> "In the (now standard) High Replication Datastore, the transaction >> >>> typically is completely applied within a few hundred milliseconds >> >>> after the commit returns. However, even if it is not completely >> >>> applied, subsequent reads, writes, and ancestor queries will always >> >>> reflect the results of the commit, because these operations apply any >> >>> outstanding modifications before executing." >> >>> >> >>> ...which is interesting and not something I ever thought of. I don't >> >>> think this intermediate state was discussed in Alfred's "Under The >> >>> Covers" I/O talk last year. Can someone describe it in more detail? >> >>> >> >>> Jeff >> >>> >> >>> On Wed, Aug 1, 2012 at 8:45 AM, Richard <[email protected]> wrote: >> >>> > Ok, so I read that web page, and I understand that the put() has >> >>> > completed, >> >>> > but not the commit(). >> >>> > >> >>> > However, I don't really see the potential solution you are >> >>> > suggesting >> >>> > :( >> >>> > Can you (or someone else) please explain in a bit more detail ? >> >>> > >> >>> > >> >>> > On Wednesday, August 1, 2012 10:55:35 AM UTC-4, Takashi Matsuo >> >>> > (Google) >> >>> > wrote: >> >>> >> >> >>> >> I forgot to mention about one possible workaround which could >> >>> >> mitigate >> >>> >> your pain. In the frontend instances, maybe you can get those small >> >>> >> entities with a newly created keys just after putting them in order >> >>> >> to >> >>> >> apply the change locally. >> >>> >> >> >>> >> This article describes this behavior well: >> >>> >> >> >>> >> https://developers.google.com/appengine/articles/transaction_isolation >> >>> >> >> >>> >> However, please note that >> >>> >> * It doesn't guarantee 100% 'strong consistency'. >> >>> >> * The latency of submitting scores will become larger. >> >>> >> * Definitely it will cost you more. >> >>> >> >> >>> >> Sorry if this doesn't work well for you, but I think it's worth >> >>> >> trying. >> >>> >> >> >>> >> -- Takashi >> >>> >> >> >>> >> On Wed, Aug 1, 2012 at 11:30 PM, Takashi Matsuo >> >>> >> <[email protected]> >> >>> >> wrote: >> >>> >> > Hi Richard, >> >>> >> > >> >>> >> > I actually played your game and probably encountered the exact >> >>> >> > issue. >> >>> >> > BTW, it's a cool addictive game. >> >>> >> > >> >>> >> > I agree that eventually consistent window should ideally be more >> >>> >> > steady than now. I have passed your issue to the engineering >> >>> >> > team. >> >>> >> > >> >>> >> > I'll get back to you when I have some updates. >> >>> >> > >> >>> >> > -- Takashi >> >>> >> > >> >>> >> > On Wed, Aug 1, 2012 at 10:37 PM, Richard <[email protected]> >> >>> >> > wrote: >> >>> >> >> The thing is.... GAE does a wonderful job. It has been fine for >> >>> >> >> a >> >>> >> >> year >> >>> >> >> or >> >>> >> >> so. Now suddenly, we are getting bitten by 'eventual >> >>> >> >> consistency'. >> >>> >> >> And not >> >>> >> >> at peak load either! This is hitting us at the lowest load time >> >>> >> >> and at >> >>> >> >> the >> >>> >> >> same time each day. >> >>> >> >> >> >>> >> >> So, maybe we were just lucky before this..... but it still sucks >> >>> >> >> to >> >>> >> >> have >> >>> >> >> something work for a year or so, and then suddenly find out you >> >>> >> >> might >> >>> >> >> need >> >>> >> >> to consider moving to a different stack. >> >>> >> >> >> >>> >> >> >> >>> >> >> On Wednesday, August 1, 2012 7:14:19 AM UTC-4, Richard Watson >> >>> >> >> wrote: >> >>> >> >>> >> >>> >> >>> On Wed, Aug 1, 2012 at 12:13 PM, Jeff Schnitzer >> >>> >> >>> <[email protected]> >> >>> >> >>> wrote: >> >>> >> >>> >> >>> >> >>> > Anything that involves batching in a frontend risks orphaning >> >>> >> >>> > data >> >>> >> >>> > in >> >>> >> >>> > the frontend... there's just no efective way to ensure that >> >>> >> >>> > batching >> >>> >> >>> > happens and that the queue is purged when "done". >> >>> >> >>> >> >>> >> >>> It's definitely not bulletproof, but I'd investigate whether I >> >>> >> >>> could >> >>> >> >>> achieve "good enough" for 99% and be happy that for the rest >> >>> >> >>> the >> >>> >> >>> client could re-request the score if it didn't reflect their >> >>> >> >>> own >> >>> >> >>> correct value. >> >>> >> >>> >> >>> >> >>> As you say, if clients can connect directly to some other more >> >>> >> >>> reliable service, maybe that's the way to go. Mixing platforms >> >>> >> >>> (GAE >> >>> >> >>> url fetch to Amazon SQS, or whatever) would leave me worrying >> >>> >> >>> about >> >>> >> >>> extra points of failure - you'll now fail if either GAE or >> >>> >> >>> Amazon/Heroku go down. >> >>> >> >>> >> >>> >> >>> Pity. I would imaging a back-end for a game with hundreds of >> >>> >> >>> users is >> >>> >> >>> something GAE should eat up. >> >>> >> >>> >> >>> >> >>> Richard >> >>> >> >> >> >>> >> >> -- >> >>> >> >> You received this message because you are subscribed to the >> >>> >> >> Google >> >>> >> >> Groups >> >>> >> >> "Google App Engine" group. >> >>> >> >> To view this discussion on the web visit >> >>> >> >> https://groups.google.com/d/msg/google-appengine/-/MpsNezxGjpkJ. >> >>> >> >> >> >>> >> >> To post to this group, send email to >> >>> >> >> [email protected]. >> >>> >> >> To unsubscribe from this group, send email to >> >>> >> >> [email protected]. >> >>> >> >> For more options, visit this group at >> >>> >> >> http://groups.google.com/group/google-appengine?hl=en. >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> > -- >> >>> >> > Takashi Matsuo >> >>> >> >> >>> >> >> >>> >> >> >>> >> -- >> >>> >> Takashi Matsuo >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Google App Engine" group. >> > To view this discussion on the web visit >> > https://groups.google.com/d/msg/google-appengine/-/v9XP85rmNM8J. >> > >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> > [email protected]. >> > For more options, visit this group at >> > http://groups.google.com/group/google-appengine?hl=en. >> >> >> >> -- >> Takashi Matsuo > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/iJA7RSzgG0oJ. > > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. -- Takashi Matsuo -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
