Are you using random keynames on your entities? I was not using random keynames and I'm seeing similar behaviour on my mapper runs. I'm refactoring my datastore ( using mapper api itself ) to use random keynames to better balance the runs across several shards.
There's an open issue related to this: http://code.google.com/p/appengine-mapreduce/issues/detail?id=3 -risto On Fri, Jun 11, 2010 at 8:17 AM, Jason C <[email protected]> wrote: > We've been using MapReduce for App Engine for a couple of different > jobs. > > Typically, we use 8 shards (the default), but it seems that only 3, > sometime 4, of the shards have any items in them? E.g., we're > currently running one job and three of the shards have >218,000 items > processed, but the other 5 shards appear to have zero. > > I can understand that a particular key distribution would have > different amounts in each shard, but with so many at zero, I suspect > there is something else happening? > > BTW, we have applied the mapreduce-recommended __key__ DESC index, but > we still see this strange shard distribution. > > Is anyone else seeing this? > > j > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to [email protected]. > To unsubscribe from this group, send email to > [email protected]<google-appengine%[email protected]> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to [email protected]. To unsubscribe from this group, send email to [email protected]. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
