[
https://issues.apache.org/jira/browse/HBASE-3996?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13233650#comment-13233650
]
Zhihong Yu commented on HBASE-3996:
-----------------------------------
w.r.t. test failure:
https://builds.apache.org/job/PreCommit-HBASE-Build/1230//testReport/org.apache.hadoop.hbase.mapreduce/TestMultiTableInputFormat/testScanEmptyToEmpty/
You can find the following:
{code}
2012-03-20 05:32:21,778 DEBUG [pool-1-thread-1]
mapreduce.MultiTableInputFormatBase(143): getSplits: split -> 24 ->
asf011.sp2.ygridcore.net:yyy,
java.lang.NumberFormatException: For input string: "18446743988250694508"
at
java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Long.parseLong(Long.java:422)
at java.lang.Long.parseLong(Long.java:468)
at
org.apache.hadoop.util.ProcfsBasedProcessTree.constructProcessInfo(ProcfsBasedProcessTree.java:413)
at
org.apache.hadoop.util.ProcfsBasedProcessTree.getProcessTree(ProcfsBasedProcessTree.java:148)
at
org.apache.hadoop.util.LinuxResourceCalculatorPlugin.getProcResourceValues(LinuxResourceCalculatorPlugin.java:401)
at org.apache.hadoop.mapred.Task.initialize(Task.java:536)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:353)
at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:396)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1083)
{code}
We really need next release of hadoop 1.0 where MAPREDUCE-3583 is fixed.
w.r.t. Bohdan's question, there're two outstanding review comments on
https://reviews.apache.org/r/4411/ where response from Eran would help clarify.
> Support multiple tables and scanners as input to the mapper in map/reduce jobs
> ------------------------------------------------------------------------------
>
> Key: HBASE-3996
> URL: https://issues.apache.org/jira/browse/HBASE-3996
> Project: HBase
> Issue Type: Improvement
> Components: mapreduce
> Reporter: Eran Kutner
> Assignee: Eran Kutner
> Fix For: 0.94.0, 0.96.0
>
> Attachments: 3996-v2.txt, 3996-v3.txt, 3996-v4.txt, HBase-3996.patch
>
>
> It seems that in many cases feeding data from multiple tables or multiple
> scanners on a single table can save a lot of time when running map/reduce
> jobs.
> I propose a new MultiTableInputFormat class that would allow doing this.
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators:
https://issues.apache.org/jira/secure/ContactAdministrators!default.jspa
For more information on JIRA, see: http://www.atlassian.com/software/jira
