[
https://issues.apache.org/jira/browse/CRUNCH-549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14637243#comment-14637243
]
Josh Wills commented on CRUNCH-549:
-----------------------------------
Hey Tyler-- the HBase compatibility work was done in CRUNCH-475, which we only
committed after HBase 1.0 was released (which was after Crunch 0.12 went out.)
However, there are CDH versions of Crunch that includes the CRUNCH-475 changes
(you can see them all here:
https://repository.cloudera.com/cloudera/cloudera-repos/org/apache/crunch/crunch-core/
), but no CDH releases that include the fix from CRUNCH-540. You could also
include the changes from CRUNCH-475 (possibly including Gabriel's improvements
in CRUNCH-545 and CRUNCH-546) in your custom build.
> crunch 0.12 not compatible with hbase 1.0
> -----------------------------------------
>
> Key: CRUNCH-549
> URL: https://issues.apache.org/jira/browse/CRUNCH-549
> Project: Crunch
> Issue Type: Bug
> Affects Versions: 0.12.0
> Reporter: Tyler Kohn
>
> I'm using a patched (thanks - CRUNCH-540) version of 0.12 with CDH 5.4.3
> and getting:
> 2015-07-22 08:54:55,235 FATAL [main] org.apache.hadoop.mapred.YarnChild:
> Error running child : java.lang.NoSuchMethodError:
> org.apache.hadoop.hbase.io.hfile.HFile$Writer.append(Lorg/apache/hadoop/hbase/KeyValue;)V
> at
> org.apache.crunch.io.hbase.HFileOutputFormatForCrunch$1.write(HFileOutputFormatForCrunch.java:102)
> at
> org.apache.crunch.io.hbase.HFileOutputFormatForCrunch$1.write(HFileOutputFormatForCrunch.java:95)
> at
> org.apache.crunch.io.CrunchOutputs$OutputState.write(CrunchOutputs.java:296)
> at org.apache.crunch.io.CrunchOutputs.write(CrunchOutputs.java:175)
> at
> org.apache.crunch.impl.mr.emit.MultipleOutputEmitter.emit(MultipleOutputEmitter.java:41)
> at org.apache.crunch.MapFn.process(MapFn.java:34)
> at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
> at
> org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
> at org.apache.crunch.lib.PTables$1.process(PTables.java:163)
> at org.apache.crunch.lib.PTables$1.process(PTables.java:160)
> at org.apache.crunch.impl.mr.run.RTNode.process(RTNode.java:98)
> at
> org.apache.crunch.impl.mr.emit.IntermediateEmitter.emit(IntermediateEmitter.java:56)
> at
> looking at the code in 0.12, HFileOutputFormatForCrunch is
> trying to write a KV (writer.append(kv);) into HFile.Writer
> HFile.Writer got rid of append(KeyValue kv) and now only supports Append(Cell
> cell) in hbase 1.x.
> Am I doing something wrong? I noticed that crunch0.12 compiles agains hbase
> 0.98.1 and that version does have that interface still there. if that's the
> case, is it possible to get a cdh5 compatible version of crunch?
> thanks for the help. not sure this is a *bug* necessarily but confused.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
