[
https://issues.apache.org/jira/browse/HBASE-1385?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12725571#action_12725571
]
Lars George commented on HBASE-1385:
------------------------------------
Wow, that was nasty. The TestTableIndex failed with the error that no index
directory was found during the verify step. But that was collateral damage. The
real error was buried in the logs
{code}
009-06-30 12:06:43,935 WARN [Thread-1100] mapred.LocalJobRunner$Job(255):
job_local_0001
java.lang.ClassCastException: org.apache.hadoop.hbase.client.Result cannot be
cast to org.apache.hadoop.hbase.mapreduce.LuceneDocumentWrapper
at
org.apache.hadoop.hbase.mapreduce.IndexRecordWriter.write(IndexRecordWriter.java:1)
at
org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at org.apache.hadoop.mapreduce.Reducer.reduce(Reducer.java:154)
at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:174)
at
org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:543)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:410)
at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:215)
2009-06-30 12:06:44,826 DEBUG [main-EventThread]
client.HConnectionManager$TableServers(213): Got ZooKeeper event, state:
SyncConnected, type: None, path: null
{code}
Now upon checking I found this in IndexTableReducer
{code}
public void reduce(ImmutableBytesWritable key, Iterator<Result> values,
Context context)
throws IOException, InterruptedException {
{code}
but when checking the stack trace above you note that Reducer.reduce() is
called! That is because while being abstract, the Reducer can run as-is and now
implements a default identity mapping. That means Result to Result. It is
imperative nearly to add "@Override" annotation because once I did I saw the
error, the method should have been
{code}
@Override
public void reduce(ImmutableBytesWritable key, Iterable<Result> values,
Context context)
throws IOException, InterruptedException {
{code}
The only difference is the use of Iterable! With the old one it simply created
a new reduce() method that was never called but the inherited default reduce().
Once I changed it to the proper signature it passed the test.
{code}
> ant -Dtestcase=mapreduce/Test* compile-test test
...
test:
[junit] Running org.apache.hadoop.hbase.mapreduce.TestTableIndex
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 75.281 sec
[junit] Running org.apache.hadoop.hbase.mapreduce.TestTableMapReduce
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 70.172 sec
[junit] Running org.apache.hadoop.hbase.mapreduce.TestTimeRangeMapRed
[junit] Tests run: 1, Failures: 0, Errors: 0, Time elapsed: 28.782 sec
{code}
All good now, patch below will have that included.
> Revamp TableInputFormat, needs updating to match hadoop 0.20.x AND remove bit
> where we can make < maps than regions
> -------------------------------------------------------------------------------------------------------------------
>
> Key: HBASE-1385
> URL: https://issues.apache.org/jira/browse/HBASE-1385
> Project: Hadoop HBase
> Issue Type: Bug
> Reporter: stack
> Fix For: 0.21.0
>
> Attachments: 1385-v5.patch, 1385-v6.patch, 1385-v7.patch, mr.patch
>
>
> Update TIF to match new MR.
> Remove the bit of logic where we will use number of configured maps as splits
> count rather than regions.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
