[
https://issues.apache.org/jira/browse/HADOOP-5475?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Leitao Guo updated HADOOP-5475:
-------------------------------
Description:
The mapreduce input is a text file with only 8 lines ( filepath: /in_wc/pretty
), and we set "conf.setNumMapTasks(8)" in the program. I thought there will
generate 8 maptasks, but actually, it generated 9 maptask. Counters of map
tasks from the website show that, 0~7 maptask has "Map input records 1",
and 8 maptask has "Map input records 0"
The following is map task list information:
task_200903121214_0029_m_000000 hdfs://guoleitao:9200/in_wc/pretty:0+4
task_200903121214_0029_m_000001 hdfs://guoleitao:9200/in_wc/pretty:4+4
task_200903121214_0029_m_000002 hdfs://guoleitao:9200/in_wc/pretty:8+4
task_200903121214_0029_m_000003 hdfs://guoleitao:9200/in_wc/pretty:12+4
task_200903121214_0029_m_000004 hdfs://guoleitao:9200/in_wc/pretty:16+4
task_200903121214_0029_m_000005 hdfs://guoleitao:9200/in_wc/pretty:20+4
task_200903121214_0029_m_000006 hdfs://guoleitao:9200/in_wc/pretty:24+4
task_200903121214_0029_m_000007 hdfs://guoleitao:9200/in_wc/pretty:28+4
task_200903121214_0029_m_000008 hdfs://guoleitao:9200/in_wc/pretty:32+4
was:
The mapreduce input is a text file with only 8 lines ( filepath: /in_wc/pretty
), and we set "conf.setNumMapTasks(8)" in the program. I thought there will
generate 8 maptasks, but actually, it generated 9 maptask. Counters of map
tasks from the website show that, 0~7 maptask has "Map input records 1",
and 8 maptask has "Map input records 0"
The following is map task list information:
task_200903121214_0029_m_000000 100.00%
hdfs://guoleitao:9200/in_wc/pretty:0+4
task_200903121214_0029_m_000001 100.00%
hdfs://guoleitao:9200/in_wc/pretty:4+4
task_200903121214_0029_m_000002 100.00%
hdfs://guoleitao:9200/in_wc/pretty:8+4
task_200903121214_0029_m_000003 100.00%
hdfs://guoleitao:9200/in_wc/pretty:12+4
task_200903121214_0029_m_000004 100.00%
hdfs://guoleitao:9200/in_wc/pretty:16+4
task_200903121214_0029_m_000005 100.00%
hdfs://guoleitao:9200/in_wc/pretty:20+4
task_200903121214_0029_m_000006 100.00%
hdfs://guoleitao:9200/in_wc/pretty:24+4
task_200903121214_0029_m_000007 100.00%
hdfs://guoleitao:9200/in_wc/pretty:28+4
task_200903121214_0029_m_000008 100.00%
hdfs://guoleitao:9200/in_wc/pretty:32+4
> Split Information errors when input data volumn is trivial
> ----------------------------------------------------------
>
> Key: HADOOP-5475
> URL: https://issues.apache.org/jira/browse/HADOOP-5475
> Project: Hadoop Core
> Issue Type: Bug
> Components: mapred
> Affects Versions: 0.19.0
> Environment: CentOS 5,
> hadoop-0.19.0
> Reporter: Leitao Guo
>
> The mapreduce input is a text file with only 8 lines ( filepath:
> /in_wc/pretty ), and we set "conf.setNumMapTasks(8)" in the program. I
> thought there will generate 8 maptasks, but actually, it generated 9 maptask.
> Counters of map tasks from the website show that, 0~7 maptask has "Map input
> records 1", and 8 maptask has "Map input records 0"
> The following is map task list information:
> task_200903121214_0029_m_000000
> hdfs://guoleitao:9200/in_wc/pretty:0+4
> task_200903121214_0029_m_000001
> hdfs://guoleitao:9200/in_wc/pretty:4+4
> task_200903121214_0029_m_000002
> hdfs://guoleitao:9200/in_wc/pretty:8+4
> task_200903121214_0029_m_000003
> hdfs://guoleitao:9200/in_wc/pretty:12+4
> task_200903121214_0029_m_000004
> hdfs://guoleitao:9200/in_wc/pretty:16+4
> task_200903121214_0029_m_000005
> hdfs://guoleitao:9200/in_wc/pretty:20+4
> task_200903121214_0029_m_000006
> hdfs://guoleitao:9200/in_wc/pretty:24+4
> task_200903121214_0029_m_000007
> hdfs://guoleitao:9200/in_wc/pretty:28+4
> task_200903121214_0029_m_000008
> hdfs://guoleitao:9200/in_wc/pretty:32+4
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
