Hi Stephen, If you are talking about making one indexed property, and not directly indexing a timestamp property, then yes this would help. However, if you are talking about using a composite index with auction id + a timestamp property, then it does not solve the problem since timestamp itself must be indexed.
Robert On Fri, Jan 14, 2011 at 13:44, Stephen Johnson <[email protected]> wrote: > Excuse my ignorance just seeking clarity, but if there are only 20-30 > bidders approx. for each auction and you had an index like this for bids: > Auction ID, Timestamp > Wouldn't this type of index be sharded by it's very nature that the > timestamp is second in the index? It seems that 20-30 bidders would be a > small amount of writes per auction and should require additional sharding. I > can't imagine the bidders hitting the bid button that quickly. Now, an > application for tickets to the superbowl would be different. Or have I > missed something fundamental to the way indexes are stored on GAE. > Thanks, > Steve > > On Fri, Jan 14, 2011 at 11:28 AM, supercobra <[email protected]> wrote: >> >> [email protected] says >> >> > The solution here is, again, to shard the timestamp by prefixing a value >> > to > distribute the writes. >> >> It seems that sharded counters are needed in many cases. Could Google >> develop a shared counter capability built into GAE that developers >> could use out of the box? >> >> -- [email protected] >> http://supercobrablogger.blogspot.com/ >> >> >> >> On Fri, Jan 14, 2011 at 12:05 PM, Ikai Lan (Google) >> <[email protected]> wrote: >> > You can certainly write to Memcache, but I don't think your application >> > can >> > tolerate any kind of volatility. Persistence is the price you have to >> > pay. >> > Fortunately, I think this can be done pretty cheaply. Just be aware of >> > monotonically increasing indexes like timestamps: if you have an >> > application >> > with a high write rate that has a timestamp, this will cause the >> > persistence >> > unit storing the indexes to be unable to be autosplit easily across >> > multiple >> > hardware instances and you will take a performance hit. The solution >> > here >> > is, again, to shard the timestamp by prefixing a value to distribute the >> > writes. >> > -- >> > Ikai Lan >> > Developer Programs Engineer, Google App Engine >> > Blogger: http://googleappengine.blogspot.com >> > Reddit: http://www.reddit.com/r/appengine >> > Twitter: http://twitter.com/app_engine >> > >> > >> > On Fri, Jan 14, 2011 at 6:06 AM, Richard Arrano <[email protected]> >> > wrote: >> >> >> >> I'm looking to make a silent-auction type of application where you >> >> have 20-30 users bidding on an item at a time, with potentially >> >> hundreds or thousands of auctions happening simultaneously. As soon as >> >> a high bid is made, it updates this information and sends it via the >> >> Channel API to the other users in the auction. I see two potential >> >> difficulties: >> >> >> >> 1. The limit on updating an entity group about once per second - I >> >> believe this can be solved with sharding bids amongst users in the >> >> auction and querying all shards to find the maximum bid at any given >> >> time, correct? >> >> >> >> 2. The nature of the auction lends itself to a heavy amount of >> >> writing to the datastore - this itself eats up CPU and I’m trying to >> >> figure out if it can be avoided. Is this just inevitable in this type >> >> of application? Does it matter that I would only be only updating a >> >> single IntegerProperty() in any given write? Is there some clever >> >> solution that we can apply that avoids the hammering of the datastore? >> >> >> >> Any tips or suggestions would be appreciated. >> >> >> >> Thank you, >> >> Richard >> >> >> >> -- >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "Google App Engine" group. >> >> To post to this group, send email to [email protected]. >> >> To unsubscribe from this group, send email to >> >> [email protected]. >> >> For more options, visit this group at >> >> http://groups.google.com/group/google-appengine?hl=en. >> >> >> > >> > -- >> > You received this message because you are subscribed to the Google >> > Groups >> > "Google App Engine" group. >> > To post to this group, send email to [email protected]. >> > To unsubscribe from this group, send email to >> > [email protected]. >> > For more options, visit this group at >> > http://groups.google.com/group/google-appengine?hl=en. >> > >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to [email protected]. >> To unsubscribe from this group, send email to >> [email protected]. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
