If sharded entity counters can be implemented at the application level then they can be implemented at the data store level as well. Sharded counters implemented at the application level in python code have scalability limits (see referenced blog post for details) that are lower when then they would be if they implemented directly by the data store (i.e. this is a common enough problem to be pushed down into the platform).
As far as transactions go, the global coordination is already there at the entity group level (if it wasn't, then there wouldn't be any transactions at all). Sure it will be more costly at the entity level then at the group level, but it will still be less costly then layering the equivalent on top of the data store API in the python application code. Allowing transactions spanning entity groups, doesn't mean that entity groups and non-spanning transactions need be eliminated. Leave it the application developer to choose the appropriate transaction type as needed. Josh On Oct 3, 11:25 pm, "David Symonds" <[EMAIL PROTECTED]> wrote: > On Sat, Oct 4, 2008 at 4:06 PM, Josh Heitzman <[EMAIL PROTECTED]> wrote: > > * GAE can't not tell you how many total entities there are of a > > specific type and it can't count more then 1000 entities. > > * GAE limits the entities that can be processed in a single > > transaction to those in the same entity group and only one request > > processing instance can write to an entire entity group at a time. > > I fail to see how either of these could be implemented in a truly > scalable and distributed system such as App Engine. The data is > potentially spread over thousands of machines, so any kind of global > co-ordination (needed for both entity counts and transactions) would > require a single machine to be "in charge", and hence the scaling > would be limited to that machine's capacity. > > If you care about counting the number of a particular kind or all > kinds, simply increment a sharded counter each time you add an entity. > That would scale very well, and give you quick access to totals. > > Dave. --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
