Hi Takashi, Yeah, the first few characters would probably be very similar. I am not sure if that would be a problem ?
A single user has only one score in the leaderboard for the game. So, overwriting the score the next time is not a problem, provided we know when to NOT include a user in the leaderboards (timestamp... if they are no longer playing). We could also use the user's name as that is guaranteed unique too. Would that work ? -R On Wednesday, August 1, 2012 3:32:27 PM UTC-4, Takashi Matsuo (Google) wrote: > > On Thu, Aug 2, 2012 at 4:11 AM, Richard <[email protected]> wrote: > > Hi Takashi & Joshua, > > > > How about using my user's id as the key ? > > > > eg: > > new_score = > Score(key_name='ahFkZXZ-c3Zlbi13b3JkaGVyb3ILCxIEVXNlchjFAww') > > > > I assume this would not count as monotonically increasing ? > > I don't think it's ok. Probably the first few characters would be > always the same. It's not good. > Additionally, from your explanation bellow, a single user has multiple > scores, right? If you use userid as key_name, the latest score always > override the previous score. > > What about something like user_name + timestamp? > > > > > @ Joshua: > > At the moment, I create all new Score objects for each player of that > game > > round and then have the backend reap them and afterwards fire a task to > > delete them. HOWEVER, what is I did not delete them, but instead marked > > them with the user's account key_name as the Score's key_name (see > above) > > and also included the following: > > timestamp = db.DateTimeProperty(auto_now = True) > > > > Then I could figure out which ones were from the last round and which > were > > from players that had last played a while ago. The assumption is that > at > > this point, players from the previous have would have their results > stored > > using a put() with a key name, which would be strongly consistent. > Players > > whose first game it is, would have the chance that the DB was slow and > did > > not yet have an entry for them ... in which case, they would just miss > out > > on one round (unless the internal commit() takes > 3 minutes!) ... and I > > could update the game client to add in a user's score to the results if > they > > were not found to 'fudge' things. > > > > Comments/thoughts ? > > > > -R > > > > > >> Sorry it didn't work like a magic, but a single datastore get > >> shouldn't take that long, so there's another issue with datastore > >> latency. It might be tablet splitting issue which I mentioned in > >> another thread: > >> > >> > http://ikaisays.com/2011/01/25/app-engine-datastore-tip-monotonically-increasing-values-are-bad/ > > >> > >> I'd suggest generating somewhat ramdom(well-distributed) keys for > >> those small entities instead of using automatic key assignment. > >> Besides using datastore, maybe I'll experiment with an implementation > >> of a memory db with some backends instances tomorrow. > >> > >> -- Takashi > >> > >> > > >> > > >> > > >> > On Wednesday, August 1, 2012 2:01:43 PM UTC-4, Richard wrote: > >> >> > >> >> Yep... got that. However, a query() will still return stale data > >> >> between > >> >> the put() and the (internal) commit(). Which is where I believe I > am > >> >> sitting... > >> >> > >> > > >> > On Wednesday, August 1, 2012 2:01:43 PM UTC-4, Richard wrote: > >> >> > >> >> Yep... got that. However, a query() will still return stale data > >> >> between > >> >> the put() and the (internal) commit(). Which is where I believe I > am > >> >> sitting... > >> >> > >> >> On Wednesday, August 1, 2012 12:39:41 PM UTC-4, Jeff Schnitzer > wrote: > >> >>> > >> >>> I presume the essential line is this: > >> >>> > >> >>> "In the (now standard) High Replication Datastore, the transaction > >> >>> typically is completely applied within a few hundred milliseconds > >> >>> after the commit returns. However, even if it is not completely > >> >>> applied, subsequent reads, writes, and ancestor queries will always > >> >>> reflect the results of the commit, because these operations apply > any > >> >>> outstanding modifications before executing." > >> >>> > >> >>> ...which is interesting and not something I ever thought of. I > don't > >> >>> think this intermediate state was discussed in Alfred's "Under The > >> >>> Covers" I/O talk last year. Can someone describe it in more > detail? > >> >>> > >> >>> Jeff > >> >>> > >> >>> On Wed, Aug 1, 2012 at 8:45 AM, Richard <[email protected]> > wrote: > >> >>> > Ok, so I read that web page, and I understand that the put() has > >> >>> > completed, > >> >>> > but not the commit(). > >> >>> > > >> >>> > However, I don't really see the potential solution you are > >> >>> > suggesting > >> >>> > :( > >> >>> > Can you (or someone else) please explain in a bit more detail ? > >> >>> > > >> >>> > > >> >>> > On Wednesday, August 1, 2012 10:55:35 AM UTC-4, Takashi Matsuo > >> >>> > (Google) > >> >>> > wrote: > >> >>> >> > >> >>> >> I forgot to mention about one possible workaround which could > >> >>> >> mitigate > >> >>> >> your pain. In the frontend instances, maybe you can get those > small > >> >>> >> entities with a newly created keys just after putting them in > order > >> >>> >> to > >> >>> >> apply the change locally. > >> >>> >> > >> >>> >> This article describes this behavior well: > >> >>> >> > >> >>> >> > https://developers.google.com/appengine/articles/transaction_isolation > >> >>> >> > >> >>> >> However, please note that > >> >>> >> * It doesn't guarantee 100% 'strong consistency'. > >> >>> >> * The latency of submitting scores will become larger. > >> >>> >> * Definitely it will cost you more. > >> >>> >> > >> >>> >> Sorry if this doesn't work well for you, but I think it's worth > >> >>> >> trying. > >> >>> >> > >> >>> >> -- Takashi > >> >>> >> > >> >>> >> On Wed, Aug 1, 2012 at 11:30 PM, Takashi Matsuo > >> >>> >> <[email protected]> > >> >>> >> wrote: > >> >>> >> > Hi Richard, > >> >>> >> > > >> >>> >> > I actually played your game and probably encountered the exact > >> >>> >> > issue. > >> >>> >> > BTW, it's a cool addictive game. > >> >>> >> > > >> >>> >> > I agree that eventually consistent window should ideally be > more > >> >>> >> > steady than now. I have passed your issue to the engineering > >> >>> >> > team. > >> >>> >> > > >> >>> >> > I'll get back to you when I have some updates. > >> >>> >> > > >> >>> >> > -- Takashi > >> >>> >> > > >> >>> >> > On Wed, Aug 1, 2012 at 10:37 PM, Richard <[email protected]> > > >> >>> >> > wrote: > >> >>> >> >> The thing is.... GAE does a wonderful job. It has been fine > for > >> >>> >> >> a > >> >>> >> >> year > >> >>> >> >> or > >> >>> >> >> so. Now suddenly, we are getting bitten by 'eventual > >> >>> >> >> consistency'. > >> >>> >> >> And not > >> >>> >> >> at peak load either! This is hitting us at the lowest load > time > >> >>> >> >> and at > >> >>> >> >> the > >> >>> >> >> same time each day. > >> >>> >> >> > >> >>> >> >> So, maybe we were just lucky before this..... but it still > sucks > >> >>> >> >> to > >> >>> >> >> have > >> >>> >> >> something work for a year or so, and then suddenly find out > you > >> >>> >> >> might > >> >>> >> >> need > >> >>> >> >> to consider moving to a different stack. > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> On Wednesday, August 1, 2012 7:14:19 AM UTC-4, Richard Watson > >> >>> >> >> wrote: > >> >>> >> >>> > >> >>> >> >>> On Wed, Aug 1, 2012 at 12:13 PM, Jeff Schnitzer > >> >>> >> >>> <[email protected]> > >> >>> >> >>> wrote: > >> >>> >> >>> > >> >>> >> >>> > Anything that involves batching in a frontend risks > orphaning > >> >>> >> >>> > data > >> >>> >> >>> > in > >> >>> >> >>> > the frontend... there's just no efective way to ensure > that > >> >>> >> >>> > batching > >> >>> >> >>> > happens and that the queue is purged when "done". > >> >>> >> >>> > >> >>> >> >>> It's definitely not bulletproof, but I'd investigate whether > I > >> >>> >> >>> could > >> >>> >> >>> achieve "good enough" for 99% and be happy that for the rest > >> >>> >> >>> the > >> >>> >> >>> client could re-request the score if it didn't reflect their > >> >>> >> >>> own > >> >>> >> >>> correct value. > >> >>> >> >>> > >> >>> >> >>> As you say, if clients can connect directly to some other > more > >> >>> >> >>> reliable service, maybe that's the way to go. Mixing > platforms > >> >>> >> >>> (GAE > >> >>> >> >>> url fetch to Amazon SQS, or whatever) would leave me > worrying > >> >>> >> >>> about > >> >>> >> >>> extra points of failure - you'll now fail if either GAE or > >> >>> >> >>> Amazon/Heroku go down. > >> >>> >> >>> > >> >>> >> >>> Pity. I would imaging a back-end for a game with hundreds > of > >> >>> >> >>> users is > >> >>> >> >>> something GAE should eat up. > >> >>> >> >>> > >> >>> >> >>> Richard > >> >>> >> >> > >> >>> >> >> -- > >> >>> >> >> You received this message because you are subscribed to the > >> >>> >> >> Google > >> >>> >> >> Groups > >> >>> >> >> "Google App Engine" group. > >> >>> >> >> To view this discussion on the web visit > >> >>> >> >> > https://groups.google.com/d/msg/google-appengine/-/MpsNezxGjpkJ. > >> >>> >> >> > >> >>> >> >> To post to this group, send email to > >> >>> >> >> [email protected]. > >> >>> >> >> To unsubscribe from this group, send email to > >> >>> >> >> [email protected]. > >> >>> >> >> For more options, visit this group at > >> >>> >> >> http://groups.google.com/group/google-appengine?hl=en. > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > -- > >> >>> >> > Takashi Matsuo > >> >>> >> > >> >>> >> > >> >>> >> > >> >>> >> -- > >> >>> >> Takashi Matsuo > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "Google App Engine" group. > >> > To view this discussion on the web visit > >> > https://groups.google.com/d/msg/google-appengine/-/v9XP85rmNM8J. > >> > > >> > To post to this group, send email to > [email protected]. > >> > To unsubscribe from this group, send email to > >> > [email protected]. > >> > For more options, visit this group at > >> > http://groups.google.com/group/google-appengine?hl=en. > >> > >> > >> > >> -- > >> Takashi Matsuo > > > > -- > > You received this message because you are subscribed to the Google > Groups > > "Google App Engine" group. > > To view this discussion on the web visit > > https://groups.google.com/d/msg/google-appengine/-/iJA7RSzgG0oJ. > > > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]. > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > Takashi Matsuo > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/VLWi5mNjQYMJ. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
