M/R max count

[Grant Ingersoll]

What's the best way, in a M/R program, to keep track of the number of  
items one has produced/consumed?  For instance, for the  
WikipediaDatasetCreator, how would you go about putting a maximum  
number of examples to include in an input file (or to split some  
number between train and test)?  In the old message passing days, one  
would have to communicate how many elements one has to the other nodes  
or something like that, right?  In theory, it seems like each Mapper  
could just do take max/(num mappers) but I'm not 100% certain that  
works as I am not sure that matches Hadoop's semantics, as I thought  
the value was just a suggestion.

-Grant