Yeah, messy. I'd use a backend for this. Possibly a set of backends if you need to shard the data for write volume. I'd use Memcache only to cache the count reads.
The basic entity is just an id and a count. An increment request goes to a backend, which simply tracks the change. A batch process goes through and writes any changed counts to both datastore and memcache (as an update; read increment write in a txn) every 5 minutes and clears the memory count. If write volume is too high for a single backend to handle, shard it by thingId % number of shards. You can change the shard count on the fly this way. All reads should be read from the memcache, read-through to the datastore as necessary. This should be able to handle any volume you want. If a backend crashes you'll lose its accumulated counts but I presume that's not a big deal. Jeff On Tue, Sep 27, 2011 at 10:38 AM, Peter Dev <dev133...@gmail.com> wrote: > Sorry, 100 000 000 counters > > On Sep 27, 4:53 pm, Peter Dev <dev133...@gmail.com> wrote: >> Shared counter is cool and I use it... but if you have millions of >> objects I cannot imagine how to manage them.1 000 000obj x 100 shards =10 >> 000 000counters >> >> 1. How to reset them to 0 in specified periods? >> 2. How to set the shared sum for each object to show top 100 objects? >> 3. Too much DB API Calls (each vote makes write in DB) >> >> Any ideas...? Thx > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to google-appengine-java@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
