HI,
  We found crunch job has no counters outputted into the log. Tried to output 
logs by following code, but still, those counters emitted by InputFormat 
classes and those default map reduce counters were missing. How can I see those 
counters a normal map reduce job will produce?

public static PipelineResult printCounters(Pipeline pipeline) {
    PipelineResult result = pipeline.run();
    List<StageResult> results = result.getStageResults();
    for (StageResult partial : results) {
        String stageId = partial.getStageName();
        log.info(String.format("counters for %s :", stageId));
        for (Map.Entry<String, Set<String>> entry : 
partial.getCounterNames().entrySet()){
            String groupName = entry.getKey();
            log.info(String.format("    %s :", groupName));
            for (String counterName : entry.getValue()) {
                long value = partial.getCounterValue(groupName, counterName);
                log.info(String.format("        %s : %d", counterName, value));
            }
        }
    }
    return result;
}
I’m expecting to see counters like this, and all counters emitted from input 
format class instead of only those emitted by Crunch:
Map-Reduce Framework
                Map input records= 1234567890
                Map output records= 1234567
                Map output bytes=1234598760

Thanks!

Reply via email to