Hi Robert, Yes, I forgot that in order to do the composite index of auction-id + timestamp you need to individually index the auction-id property and the timestamp property. I keep forgetting that. I think there are use cases like this that might warrant not having to have the individual properties indexed while allowing the composite index to exists. Thanks, Stephen
On Mon, Jan 17, 2011 at 12:07 PM, Robert Kluin <[email protected]>wrote: > Hi Stephen, > If you are talking about making one indexed property, and not > directly indexing a timestamp property, then yes this would help. > However, if you are talking about using a composite index with auction > id + a timestamp property, then it does not solve the problem since > timestamp itself must be indexed. > > > > > Robert > > > > > > > On Fri, Jan 14, 2011 at 13:44, Stephen Johnson <[email protected]> > wrote: > > Excuse my ignorance just seeking clarity, but if there are only 20-30 > > bidders approx. for each auction and you had an index like this for bids: > > Auction ID, Timestamp > > Wouldn't this type of index be sharded by it's very nature that the > > timestamp is second in the index? It seems that 20-30 bidders would be a > > small amount of writes per auction and should require additional > sharding. I > > can't imagine the bidders hitting the bid button that quickly. Now, an > > application for tickets to the superbowl would be different. Or have I > > missed something fundamental to the way indexes are stored on GAE. > > Thanks, > > Steve > > > > On Fri, Jan 14, 2011 at 11:28 AM, supercobra <[email protected]> > wrote: > >> > >> [email protected] <ikai.l%[email protected]> says > >> > >> > The solution here is, again, to shard the timestamp by prefixing a > value > >> > to > distribute the writes. > >> > >> It seems that sharded counters are needed in many cases. Could Google > >> develop a shared counter capability built into GAE that developers > >> could use out of the box? > >> > >> -- [email protected] > >> http://supercobrablogger.blogspot.com/ > >> > >> > >> > >> On Fri, Jan 14, 2011 at 12:05 PM, Ikai Lan (Google) > >> <[email protected] <ikai.l%[email protected]>> wrote: > >> > You can certainly write to Memcache, but I don't think your > application > >> > can > >> > tolerate any kind of volatility. Persistence is the price you have to > >> > pay. > >> > Fortunately, I think this can be done pretty cheaply. Just be aware of > >> > monotonically increasing indexes like timestamps: if you have an > >> > application > >> > with a high write rate that has a timestamp, this will cause the > >> > persistence > >> > unit storing the indexes to be unable to be autosplit easily across > >> > multiple > >> > hardware instances and you will take a performance hit. The solution > >> > here > >> > is, again, to shard the timestamp by prefixing a value to distribute > the > >> > writes. > >> > -- > >> > Ikai Lan > >> > Developer Programs Engineer, Google App Engine > >> > Blogger: http://googleappengine.blogspot.com > >> > Reddit: http://www.reddit.com/r/appengine > >> > Twitter: http://twitter.com/app_engine > >> > > >> > > >> > On Fri, Jan 14, 2011 at 6:06 AM, Richard Arrano <[email protected] > > > >> > wrote: > >> >> > >> >> I'm looking to make a silent-auction type of application where you > >> >> have 20-30 users bidding on an item at a time, with potentially > >> >> hundreds or thousands of auctions happening simultaneously. As soon > as > >> >> a high bid is made, it updates this information and sends it via the > >> >> Channel API to the other users in the auction. I see two potential > >> >> difficulties: > >> >> > >> >> 1. The limit on updating an entity group about once per second - I > >> >> believe this can be solved with sharding bids amongst users in the > >> >> auction and querying all shards to find the maximum bid at any given > >> >> time, correct? > >> >> > >> >> 2. The nature of the auction lends itself to a heavy amount of > >> >> writing to the datastore - this itself eats up CPU and I’m trying to > >> >> figure out if it can be avoided. Is this just inevitable in this type > >> >> of application? Does it matter that I would only be only updating a > >> >> single IntegerProperty() in any given write? Is there some clever > >> >> solution that we can apply that avoids the hammering of the > datastore? > >> >> > >> >> Any tips or suggestions would be appreciated. > >> >> > >> >> Thank you, > >> >> Richard > >> >> > >> >> -- > >> >> You received this message because you are subscribed to the Google > >> >> Groups > >> >> "Google App Engine" group. > >> >> To post to this group, send email to > [email protected]. > >> >> To unsubscribe from this group, send email to > >> >> [email protected]<google-appengine%[email protected]> > . > >> >> For more options, visit this group at > >> >> http://groups.google.com/group/google-appengine?hl=en. > >> >> > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "Google App Engine" group. > >> > To post to this group, send email to > [email protected]. > >> > To unsubscribe from this group, send email to > >> > [email protected]<google-appengine%[email protected]> > . > >> > For more options, visit this group at > >> > http://groups.google.com/group/google-appengine?hl=en. > >> > > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]<google-appengine%[email protected]> > . > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine?hl=en. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%[email protected]> > . > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
