Thanks Robert, I've starred it. On Mon, Jan 17, 2011 at 12:35 PM, Robert Kluin <[email protected]>wrote:
> Hey Stephen, > I agree, it would be nice if properties could only be included in > some composite indexes! > > http://code.google.com/p/googleappengine/issues/detail?id=4231 > > > > Robert > > > > > > > On Mon, Jan 17, 2011 at 14:29, Stephen Johnson <[email protected]> > wrote: > > Hi Robert, > > Yes, I forgot that in order to do the composite index of auction-id + > > timestamp you need to individually index the auction-id property and the > > timestamp property. I keep forgetting that. I think there are use cases > like > > this that might warrant not having to have the individual properties > indexed > > while allowing the composite index to exists. > > Thanks, > > Stephen > > > > On Mon, Jan 17, 2011 at 12:07 PM, Robert Kluin <[email protected]> > > wrote: > >> > >> Hi Stephen, > >> If you are talking about making one indexed property, and not > >> directly indexing a timestamp property, then yes this would help. > >> However, if you are talking about using a composite index with auction > >> id + a timestamp property, then it does not solve the problem since > >> timestamp itself must be indexed. > >> > >> > >> > >> > >> Robert > >> > >> > >> > >> > >> > >> > >> On Fri, Jan 14, 2011 at 13:44, Stephen Johnson <[email protected]> > >> wrote: > >> > Excuse my ignorance just seeking clarity, but if there are only 20-30 > >> > bidders approx. for each auction and you had an index like this for > >> > bids: > >> > Auction ID, Timestamp > >> > Wouldn't this type of index be sharded by it's very nature that the > >> > timestamp is second in the index? It seems that 20-30 bidders would be > a > >> > small amount of writes per auction and should require additional > >> > sharding. I > >> > can't imagine the bidders hitting the bid button that quickly. Now, an > >> > application for tickets to the superbowl would be different. Or have I > >> > missed something fundamental to the way indexes are stored on GAE. > >> > Thanks, > >> > Steve > >> > > >> > On Fri, Jan 14, 2011 at 11:28 AM, supercobra <[email protected]> > >> > wrote: > >> >> > >> >> [email protected] <ikai.l%[email protected]> says > >> >> > >> >> > The solution here is, again, to shard the timestamp by prefixing a > >> >> > value > >> >> > to > distribute the writes. > >> >> > >> >> It seems that sharded counters are needed in many cases. Could Google > >> >> develop a shared counter capability built into GAE that developers > >> >> could use out of the box? > >> >> > >> >> -- [email protected] > >> >> http://supercobrablogger.blogspot.com/ > >> >> > >> >> > >> >> > >> >> On Fri, Jan 14, 2011 at 12:05 PM, Ikai Lan (Google) > >> >> <[email protected] <ikai.l%[email protected]>> wrote: > >> >> > You can certainly write to Memcache, but I don't think your > >> >> > application > >> >> > can > >> >> > tolerate any kind of volatility. Persistence is the price you have > to > >> >> > pay. > >> >> > Fortunately, I think this can be done pretty cheaply. Just be aware > >> >> > of > >> >> > monotonically increasing indexes like timestamps: if you have an > >> >> > application > >> >> > with a high write rate that has a timestamp, this will cause the > >> >> > persistence > >> >> > unit storing the indexes to be unable to be autosplit easily across > >> >> > multiple > >> >> > hardware instances and you will take a performance hit. The > solution > >> >> > here > >> >> > is, again, to shard the timestamp by prefixing a value to > distribute > >> >> > the > >> >> > writes. > >> >> > -- > >> >> > Ikai Lan > >> >> > Developer Programs Engineer, Google App Engine > >> >> > Blogger: http://googleappengine.blogspot.com > >> >> > Reddit: http://www.reddit.com/r/appengine > >> >> > Twitter: http://twitter.com/app_engine > >> >> > > >> >> > > >> >> > On Fri, Jan 14, 2011 at 6:06 AM, Richard Arrano > >> >> > <[email protected]> > >> >> > wrote: > >> >> >> > >> >> >> I'm looking to make a silent-auction type of application where you > >> >> >> have 20-30 users bidding on an item at a time, with potentially > >> >> >> hundreds or thousands of auctions happening simultaneously. As > soon > >> >> >> as > >> >> >> a high bid is made, it updates this information and sends it via > the > >> >> >> Channel API to the other users in the auction. I see two potential > >> >> >> difficulties: > >> >> >> > >> >> >> 1. The limit on updating an entity group about once per second > - > >> >> >> I > >> >> >> believe this can be solved with sharding bids amongst users in the > >> >> >> auction and querying all shards to find the maximum bid at any > given > >> >> >> time, correct? > >> >> >> > >> >> >> 2. The nature of the auction lends itself to a heavy amount of > >> >> >> writing to the datastore - this itself eats up CPU and I’m trying > to > >> >> >> figure out if it can be avoided. Is this just inevitable in this > >> >> >> type > >> >> >> of application? Does it matter that I would only be only updating > a > >> >> >> single IntegerProperty() in any given write? Is there some clever > >> >> >> solution that we can apply that avoids the hammering of the > >> >> >> datastore? > >> >> >> > >> >> >> Any tips or suggestions would be appreciated. > >> >> >> > >> >> >> Thank you, > >> >> >> Richard > >> >> >> > >> >> >> -- > >> >> >> You received this message because you are subscribed to the Google > >> >> >> Groups > >> >> >> "Google App Engine" group. > >> >> >> To post to this group, send email to > >> >> >> [email protected]. > >> >> >> To unsubscribe from this group, send email to > >> >> >> [email protected]<google-appengine%[email protected]> > . > >> >> >> For more options, visit this group at > >> >> >> http://groups.google.com/group/google-appengine?hl=en. > >> >> >> > >> >> > > >> >> > -- > >> >> > You received this message because you are subscribed to the Google > >> >> > Groups > >> >> > "Google App Engine" group. > >> >> > To post to this group, send email to > >> >> > [email protected]. > >> >> > To unsubscribe from this group, send email to > >> >> > [email protected]<google-appengine%[email protected]> > . > >> >> > For more options, visit this group at > >> >> > http://groups.google.com/group/google-appengine?hl=en. > >> >> > > >> >> > >> >> -- > >> >> You received this message because you are subscribed to the Google > >> >> Groups > >> >> "Google App Engine" group. > >> >> To post to this group, send email to > [email protected]. > >> >> To unsubscribe from this group, send email to > >> >> [email protected]<google-appengine%[email protected]> > . > >> >> For more options, visit this group at > >> >> http://groups.google.com/group/google-appengine?hl=en. > >> >> > >> > > >> > -- > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "Google App Engine" group. > >> > To post to this group, send email to > [email protected]. > >> > To unsubscribe from this group, send email to > >> > [email protected]<google-appengine%[email protected]> > . > >> > For more options, visit this group at > >> > http://groups.google.com/group/google-appengine?hl=en. > >> > > >> > >> -- > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine" group. > >> To post to this group, send email to [email protected]. > >> To unsubscribe from this group, send email to > >> [email protected]<google-appengine%[email protected]> > . > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine?hl=en. > >> > > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine" group. > > To post to this group, send email to [email protected]. > > To unsubscribe from this group, send email to > > [email protected]<google-appengine%[email protected]> > . > > For more options, visit this group at > > http://groups.google.com/group/google-appengine?hl=en. > > > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
