Will McQueen created SQOOP-969:
----------------------------------
Summary: sqoop export with mysql connector uses more than twice as
many mappers as the --num-mappers hint
Key: SQOOP-969
URL: https://issues.apache.org/jira/browse/SQOOP-969
Project: Sqoop
Issue Type: Bug
Components: connectors
Affects Versions: 1.4.3
Reporter: Will McQueen
sudo -u sqoop sqoop export --connect "jdbc:mysql://<hostname>/sqoop" --username
sqoop --password sqoop --table quicktest --verbose --export-dir
/user/sqoop/quicktest --num-mappers 12
...results in more than twice the number of mappers (according to the logs) as
the specified '12' hint:
13/03/26 16:20:35 INFO mapred.JobClient: Job Counters
13/03/26 16:20:35 INFO mapred.JobClient: Launched map tasks=23
...and I couldn't help notice that the logs also say:
13/03/26 16:19:03 DEBUG mapreduce.ExportInputFormat: Target numMapTasks=12
13/03/26 16:19:03 DEBUG mapreduce.ExportInputFormat: Total input bytes=23
...where # input bytes maches # launched map tasks. Coincidence, or relation?
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
