[
https://issues.apache.org/jira/browse/CHUKWA-276?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jerome Boulon updated CHUKWA-276:
---------------------------------
Description:
Hourly and Daily rolling should not use more than one reducer and they are
currently relying on the default parameter to be correct.
<property>
<name>mapred.reduce.tasks</name>
<value>1</value>
<description>The default number of reduce tasks per job. Typically set
to a prime close to the number of available hosts. Ignored when
mapred.job.tracker is "local".
</description>
</property>
This should be forced within the Rolling class in order to avoid any issue.
was:
Hourly and Daily rolling should not use more than one reducer and they are
currently relying on the default parameter to be correct.
<property>
<name>mapred.reduce.tasks</name>
<value>1</value>
<description>The default number of reduce tasks per job. Typically set
to a prime close to the number of available hosts. Ignored when
mapred.job.tracker is "local".
</description>
</property>
This should be forced within the Rolling class in order to avoid any issue.
And ideally, the code should be changed to allow more than one reducer.
Summary: Hourly and Daily rolling should use a single reducer + fix
classpath issue (was: Hourly and Daily rolling should use a single reducer)
> Hourly and Daily rolling should use a single reducer + fix classpath issue
> --------------------------------------------------------------------------
>
> Key: CHUKWA-276
> URL: https://issues.apache.org/jira/browse/CHUKWA-276
> Project: Hadoop Chukwa
> Issue Type: Improvement
> Components: Data Processors
> Reporter: Jerome Boulon
> Assignee: Jerome Boulon
>
> Hourly and Daily rolling should not use more than one reducer and they are
> currently relying on the default parameter to be correct.
> <property>
> <name>mapred.reduce.tasks</name>
> <value>1</value>
> <description>The default number of reduce tasks per job. Typically set
> to a prime close to the number of available hosts. Ignored when
> mapred.job.tracker is "local".
> </description>
> </property>
> This should be forced within the Rolling class in order to avoid any issue.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
