Periodically update the 'pageX' master record. You could do this using the task queue, for example, by inserting a task to sum the shards and update the counter. Or, you could periodically map over all the shards updating the 'master' counters. Which method is best depends on your application.
Are those numbers you've posted real? If so your shards are not getting an even load, at all. The first shard is catching around 95% of the updates. You might consider not using sharded counters. Robert On Sun, Apr 24, 2011 at 21:46, Lucas <[email protected]> wrote: > let say i have sharded counter like below > > name , counting > page1_shard_1 , 5699 > page1_shard_2 , 234 > ----------------- > page2_shard_1 , 4543 > page2_shard_2 , 343 > page2_shard_3 , 0 > ----------------- > page3_shard_1 , 3434 > page3_shard_2 , 34 > ...etc > > > in order to sort by the highest value of "counting", i need to > manually get each "page1_*" from bigtable and sum up in memory. If I > have trillion of "name" records > (page1_shardX...page99999999999.._shardX) this technique is not > suitable. May i know what is the technique i can sort "order by > highest couting" without sum up each 'record' in memory ? let say > i want to find out the highest hit of page by doing just one query > from datastore. Is this possible? > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
