[
https://issues.apache.org/jira/browse/MAPREDUCE-6067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14122351#comment-14122351
]
Binglin Chang commented on MAPREDUCE-6067:
------------------------------------------
bq. I did saw some seeming trivival minor changes, like one or two lines of
change, will impact the CPU cache efficiency.
Never mind, I already move those counters to mapoutputcollector
bq. think operation team will want this.
In java, output file path is not printed also, but I think I can add them back
to mid-spill and final-spill log
bq. merge log, spill log, I do think we should keep them
I just move those info to mid-spill and final-spill log in a more formatted
way, not really deleting them
bq. I think more changes are better than this. Some test maven plugin allows
tests to be runned in parallel, sharing mutable stuff in Test Cases is wrong.
Sure, will make the change
bq. Yes, but you only have a declaration, but never use it?
Here, the variable name is different
{code}
+
+ _outputBytes->increase(realOutputSize);
{code}
> native-task: spilled records counter is incorrect
> -------------------------------------------------
>
> Key: MAPREDUCE-6067
> URL: https://issues.apache.org/jira/browse/MAPREDUCE-6067
> Project: Hadoop Map/Reduce
> Issue Type: Sub-task
> Components: task
> Reporter: Todd Lipcon
> Assignee: Binglin Chang
> Attachments: MAPREDUCE-6067.v1.patch, MAPREDUCE-6067.v2.patch,
> MAPREDUCE-6067.v3.patch, MAPREDUCE-6067.v4.patch, native-counters.html,
> trunk-counters.html
>
>
> After running a terasort, I see the spilled records counter at 5028651606,
> which is about half what I expected to see. Using the non-native collector I
> see the expected count of 10000000000. It seems the correct number of records
> were indeed spilled, because the job's output record count is correct.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
