Hey Stephen,
I agree, it would be nice if properties could only be included in
some composite indexes!
http://code.google.com/p/googleappengine/issues/detail?id=4231
Robert
On Mon, Jan 17, 2011 at 14:29, Stephen Johnson <[email protected]> wrote:
> Hi Robert,
> Yes, I forgot that in order to do the composite index of auction-id +
> timestamp you need to individually index the auction-id property and the
> timestamp property. I keep forgetting that. I think there are use cases like
> this that might warrant not having to have the individual properties indexed
> while allowing the composite index to exists.
> Thanks,
> Stephen
>
> On Mon, Jan 17, 2011 at 12:07 PM, Robert Kluin <[email protected]>
> wrote:
>>
>> Hi Stephen,
>> If you are talking about making one indexed property, and not
>> directly indexing a timestamp property, then yes this would help.
>> However, if you are talking about using a composite index with auction
>> id + a timestamp property, then it does not solve the problem since
>> timestamp itself must be indexed.
>>
>>
>>
>>
>> Robert
>>
>>
>>
>>
>>
>>
>> On Fri, Jan 14, 2011 at 13:44, Stephen Johnson <[email protected]>
>> wrote:
>> > Excuse my ignorance just seeking clarity, but if there are only 20-30
>> > bidders approx. for each auction and you had an index like this for
>> > bids:
>> > Auction ID, Timestamp
>> > Wouldn't this type of index be sharded by it's very nature that the
>> > timestamp is second in the index? It seems that 20-30 bidders would be a
>> > small amount of writes per auction and should require additional
>> > sharding. I
>> > can't imagine the bidders hitting the bid button that quickly. Now, an
>> > application for tickets to the superbowl would be different. Or have I
>> > missed something fundamental to the way indexes are stored on GAE.
>> > Thanks,
>> > Steve
>> >
>> > On Fri, Jan 14, 2011 at 11:28 AM, supercobra <[email protected]>
>> > wrote:
>> >>
>> >> [email protected] says
>> >>
>> >> > The solution here is, again, to shard the timestamp by prefixing a
>> >> > value
>> >> > to > distribute the writes.
>> >>
>> >> It seems that sharded counters are needed in many cases. Could Google
>> >> develop a shared counter capability built into GAE that developers
>> >> could use out of the box?
>> >>
>> >> -- [email protected]
>> >> http://supercobrablogger.blogspot.com/
>> >>
>> >>
>> >>
>> >> On Fri, Jan 14, 2011 at 12:05 PM, Ikai Lan (Google)
>> >> <[email protected]> wrote:
>> >> > You can certainly write to Memcache, but I don't think your
>> >> > application
>> >> > can
>> >> > tolerate any kind of volatility. Persistence is the price you have to
>> >> > pay.
>> >> > Fortunately, I think this can be done pretty cheaply. Just be aware
>> >> > of
>> >> > monotonically increasing indexes like timestamps: if you have an
>> >> > application
>> >> > with a high write rate that has a timestamp, this will cause the
>> >> > persistence
>> >> > unit storing the indexes to be unable to be autosplit easily across
>> >> > multiple
>> >> > hardware instances and you will take a performance hit. The solution
>> >> > here
>> >> > is, again, to shard the timestamp by prefixing a value to distribute
>> >> > the
>> >> > writes.
>> >> > --
>> >> > Ikai Lan
>> >> > Developer Programs Engineer, Google App Engine
>> >> > Blogger: http://googleappengine.blogspot.com
>> >> > Reddit: http://www.reddit.com/r/appengine
>> >> > Twitter: http://twitter.com/app_engine
>> >> >
>> >> >
>> >> > On Fri, Jan 14, 2011 at 6:06 AM, Richard Arrano
>> >> > <[email protected]>
>> >> > wrote:
>> >> >>
>> >> >> I'm looking to make a silent-auction type of application where you
>> >> >> have 20-30 users bidding on an item at a time, with potentially
>> >> >> hundreds or thousands of auctions happening simultaneously. As soon
>> >> >> as
>> >> >> a high bid is made, it updates this information and sends it via the
>> >> >> Channel API to the other users in the auction. I see two potential
>> >> >> difficulties:
>> >> >>
>> >> >> 1. The limit on updating an entity group about once per second -
>> >> >> I
>> >> >> believe this can be solved with sharding bids amongst users in the
>> >> >> auction and querying all shards to find the maximum bid at any given
>> >> >> time, correct?
>> >> >>
>> >> >> 2. The nature of the auction lends itself to a heavy amount of
>> >> >> writing to the datastore - this itself eats up CPU and I’m trying to
>> >> >> figure out if it can be avoided. Is this just inevitable in this
>> >> >> type
>> >> >> of application? Does it matter that I would only be only updating a
>> >> >> single IntegerProperty() in any given write? Is there some clever
>> >> >> solution that we can apply that avoids the hammering of the
>> >> >> datastore?
>> >> >>
>> >> >> Any tips or suggestions would be appreciated.
>> >> >>
>> >> >> Thank you,
>> >> >> Richard
>> >> >>
>> >> >> --
>> >> >> You received this message because you are subscribed to the Google
>> >> >> Groups
>> >> >> "Google App Engine" group.
>> >> >> To post to this group, send email to
>> >> >> [email protected].
>> >> >> To unsubscribe from this group, send email to
>> >> >> [email protected].
>> >> >> For more options, visit this group at
>> >> >> http://groups.google.com/group/google-appengine?hl=en.
>> >> >>
>> >> >
>> >> > --
>> >> > You received this message because you are subscribed to the Google
>> >> > Groups
>> >> > "Google App Engine" group.
>> >> > To post to this group, send email to
>> >> > [email protected].
>> >> > To unsubscribe from this group, send email to
>> >> > [email protected].
>> >> > For more options, visit this group at
>> >> > http://groups.google.com/group/google-appengine?hl=en.
>> >> >
>> >>
>> >> --
>> >> You received this message because you are subscribed to the Google
>> >> Groups
>> >> "Google App Engine" group.
>> >> To post to this group, send email to [email protected].
>> >> To unsubscribe from this group, send email to
>> >> [email protected].
>> >> For more options, visit this group at
>> >> http://groups.google.com/group/google-appengine?hl=en.
>> >>
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups
>> > "Google App Engine" group.
>> > To post to this group, send email to [email protected].
>> > To unsubscribe from this group, send email to
>> > [email protected].
>> > For more options, visit this group at
>> > http://groups.google.com/group/google-appengine?hl=en.
>> >
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to [email protected].
>> To unsubscribe from this group, send email to
>> [email protected].
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to [email protected].
> To unsubscribe from this group, send email to
> [email protected].
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
