Excuse my ignorance just seeking clarity, but if there are only 20-30 bidders approx. for each auction and you had an index like this for bids:
Auction ID, Timestamp Wouldn't this type of index be sharded by it's very nature that the timestamp is second in the index? It seems that 20-30 bidders would be a small amount of writes per auction and should require additional sharding. I can't imagine the bidders hitting the bid button that quickly. Now, an application for tickets to the superbowl would be different. Or have I missed something fundamental to the way indexes are stored on GAE. Thanks, Steve On Fri, Jan 14, 2011 at 11:28 AM, supercobra <[email protected]> wrote: > [email protected] <ikai.l%[email protected]> says > > > The solution here is, again, to shard the timestamp by prefixing a value > to > distribute the writes. > > It seems that sharded counters are needed in many cases. Could Google > develop a shared counter capability built into GAE that developers > could use out of the box? > > -- [email protected] > http://supercobrablogger.blogspot.com/ > > > > On Fri, Jan 14, 2011 at 12:05 PM, Ikai Lan (Google) > <[email protected] <ikai.l%[email protected]>> wrote: > > You can certainly write to Memcache, but I don't think your application > can > > tolerate any kind of volatility. Persistence is the price you have to > pay. > > Fortunately, I think this can be done pretty cheaply. Just be aware of > > monotonically increasing indexes like timestamps: if you have an > application > > with a high write rate that has a timestamp, this will cause the > persistence > > unit storing the indexes to be unable to be autosplit easily across > multiple > > hardware instances and you will take a performance hit. The solution here > > is, again, to shard the timestamp by prefixing a value to distribute the > > writes. > > -- > > Ikai Lan > > Developer Programs Engineer, Google App Engine > > Blogger: http://googleappengine.blogspot.com > > Reddit: http://www.reddit.com/r/appengine > > Twitter: http://twitter.com/app_engine > > > > > > On Fri, Jan 14, 2011 at 6:06 AM, Richard Arrano <[email protected]> > > wrote: > >> > >> I'm looking to make a silent-auction type of application where you > >> have 20-30 users bidding on an item at a time, with potentially > >> hundreds or thousands of auctions happening simultaneously. As soon as > >> a high bid is made, it updates this information and sends it via the > >> Channel API to the other users in the auction. I see two potential > >> difficulties: > >> > >> 1. The limit on updating an entity group about once per second - I > >> believe this can be solved with sharding bids amongst users in the > >> auction and querying all shards to find the maximum bid at any given > >> time, correct? > >> > >> 2. The nature of the auction lends itself to a heavy amount of > >> writing to the datastore - this itself eats up CPU and I’m trying to > >> figure out if it can be avoided. Is this just inevitable in this type > >> of application? Does it matter that I would only be only updating a > >> single IntegerProperty() in any given write? Is there some clever > >> solution that we can apply that avoids the hammering of the datastore? > >> > >> Any tips or suggestions would be appreciated. > >> > >> Thank you, > >> Richard > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]<google-appengine%[email protected]> > . > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine?hl=en. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%[email protected]> > . > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
