[
https://issues.apache.org/jira/browse/CHUKWA-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13415820#comment-13415820
]
Hudson commented on CHUKWA-647:
-------------------------------
Integrated in Chukwa-trunk #453 (See
[https://builds.apache.org/job/Chukwa-trunk/453/])
CHUKWA-647. Spread out intermediate data with the same ReduceType into
different Reduce Tasks. Contributed by Jie Huang. (Revision 1362318)
Result = FAILURE
asrabkin :
http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1362318
Files :
*
/incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/CHUKWA_CONSTANT.java
*
/incubator/chukwa/trunk/src/main/java/org/apache/hadoop/chukwa/extraction/demux/ChukwaRecordPartitioner.java
> Spread out intermediate data with the same ReduceType into different Reduce
> Tasks
> ---------------------------------------------------------------------------------
>
> Key: CHUKWA-647
> URL: https://issues.apache.org/jira/browse/CHUKWA-647
> Project: Chukwa
> Issue Type: Improvement
> Components: Data Processors
> Affects Versions: 0.4.0, 0.6.0
> Reporter: Jie Huang
> Assignee: Ari Rabkin
> Priority: Minor
> Fix For: 0.6.0
>
> Attachments: Chukwa-647-0_4.patch
>
>
> We have found that if we partitioned the map output data according to
> ReduceType, we can see the data skew in some HiTune cases. Then one or two
> Reduce Tasks slow down the whole Demux job somehow, since those reduce tasks
> have to process more input-data.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
