Ian, We're both in agreement. I was confused by what you meant by "count" and didn't realize you were talking about the count() method. Looking back, it's clear now how you meant fetch and count methods and my response could be misinterpreted if "count" meant count().
My first instinct now is to amortize expensive operations (counting, ranking, etc) on each put. The count() method should be avoided by using counters, which can either be implemented as an entity with integer property or shards, neither of which have any limitations. So I meant to say counting is not an issue and can be implemented much more easily than Bloog's current sharded counter, since no Bloogs will probably get more than one comment per second. The next version of Bloog will use an integer property in a single entity. -Bill On Mar 2, 12:47 am, Ian Lewis <[email protected]> wrote: > Bill, > > The docs say otherwise. count() on a query object returns a maximum of > 1000.http://code.google.com/appengine/docs/python/datastore/queryclass.htm... > > Sharded counters are used to ease contention for highly updated objects but > that doesn't downplay the need for counters. You need to either put a > counter on a separate entity or create sharded counter if you want to count > more than 1000 entities. > > Ian > > > > On Mon, Mar 2, 2009 at 5:40 PM, Bill <[email protected]> wrote: > > > You can count well over 1000 objects -- the limitation is in the > > fetching, but that's not the reason for sharded counters in Bloog. > > > I wrote Bloog way back when App Engine was new and I was trying out > > different "best practices" for datastore ops. Sharded counters are a > > way to handle heavily-used counters, e.g. if the counter was getting > > updated dozens of times per second. If you have that many increments/ > > decrements on a counter, if you put the counter in a single integer > > property, you'd start getting write contention. > > > In retrospect, doing the sharded counter in Bloog was complete > > overkill. It was useful to me at the time, because I was learning. > > But the next version of Bloog will likely use a single integer > > property for the counter and scrap the sharded counter. > > -Bill > > > On Mar 1, 9:11 pm, Ian Lewis <[email protected]> wrote: > > > Krishna, > > > > The datastore can only return or count up to 1000 objects so if you have > > > more than 1000 objects in the database you cannot count them without a > > > counter. I assume the code in Bloog is keeping counts of objects in case > > it > > > runs up against this limitation. > > > > Ian > > > > On Mon, Mar 2, 2009 at 2:02 PM, Krishna <[email protected]> > > wrote: > > > > > Ian, > > > > > I'm using some open source code (Bloog) and I don't really understand > > > > how it works. So I'll trust your cautious approach although I don't > > > > quite understand why something would need to "keep counts of objects > > > > in the datastore". Best not to meddle although I was hoping to clean > > > > things up if I could. > > > > > Thank you for responding. > > > > Krishna > > > > > On Mar 1, 11:31 pm, Ian Lewis <[email protected]> wrote: > > > > > Krishna, > > > > > > Anything you see in the data viewer is created by some application > > code. > > > > It > > > > > wouldn't be put there by google appengine specifically. That said it > > > > looks > > > > > like something is using a sharded counter to keep counts of objects > > in > > > > the > > > > > datastore or something similar. This data would not likely be > > regenerated > > > > if > > > > > deleted, it would likely reset all the counters to zero so I would > > leave > > > > it > > > > > alone. > > > > > > Ian > > > > > > On Mon, Mar 2, 2009 at 9:34 AM, Krishna <[email protected]> > > > > wrote: > > > > > > > On the Data Viewer I have an entity called CounterShards. I > > considered > > > > > > deleting some extraneous data there but was afraid it would blow > > > > > > something up. If I delete data here, will it be regenerated > > > > > > automatically or should I leave it alone? > > > > > > -- > > > > > ======================================= > > > > > 株式会社ビープラウド イアン・ルイス > > > > > 〒150-0012 > > > > > 東京都渋谷区広尾1-11-2アイオス広尾ビル604 > > > > > email: [email protected] > > > > > TEL：03-5795-2707 > > > > > FAX：03-5795-2708http://www.beproud.jp/ > > > > > ======================================= > > > > -- > > > ======================================= > > > 株式会社ビープラウド イアン・ルイス > > > 〒150-0012 > > > 東京都渋谷区広尾1-11-2アイオス広尾ビル604 > > > email: [email protected] > > > TEL：03-5795-2707 > > > FAX：03-5795-2708http://www.beproud.jp/ > > > ======================================= > > -- > ======================================= > 株式会社ビープラウド イアン・ルイス > 〒150-0012 > 東京都渋谷区広尾1-11-2アイオス広尾ビル604 > email: [email protected] > TEL：03-5795-2707 > FAX：03-5795-2708http://www.beproud.jp/ > ======================================= --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
