[
https://issues.apache.org/jira/browse/PIG-3285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13826981#comment-13826981
]
Jarek Jarcec Cecho commented on PIG-3285:
-----------------------------------------
Thank you [~ndimiduk]! Couple of high level notes:
{code}
- TableMapReduceUtil.addDependencyJars(job.getConfiguration(), klass);
{code}
It seems that we've accidentally removed the call
{{TableMapReduceUtil.addDependencyJars()}} from method
{{addClassToJobIfExists()}}. As a result not all required jars are propagated
to the {{DistributedCache}} when running on HBase 0.95+.
{code}
+ // method doesn't exist or invocation failed. moving on.
{code}
Can we at least do a debug log message stating that
{{TableMapReduceUtil.addHBaseDependencyJars}} is not available and thus we will
revert to our own logic for propagating jars? It think that it will help to
know what logic we used for propagating jars while investigating failing pig
scripts.
> Jobs using HBaseStorage fail to ship dependency jars
> ----------------------------------------------------
>
> Key: PIG-3285
> URL: https://issues.apache.org/jira/browse/PIG-3285
> Project: Pig
> Issue Type: Bug
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Fix For: 0.13.0
>
> Attachments: 0001-PIG-3285-Add-HBase-dependency-jars.patch,
> 0001-PIG-3285-Add-HBase-dependency-jars.patch,
> 0001-PIG-3285-Add-HBase-dependency-jars.patch,
> 0001-PIG-3285-Add-HBase-dependency-jars.patch, 1.pig, 1.txt, 2.pig
>
>
> Launching a job consuming {{HBaseStorage}} fails out of the box. The user
> must specify {{-Dpig.additional.jars}} for HBase and all of its dependencies.
> Exceptions look something like this:
> {noformat}
> 2013-04-19 18:58:39,360 FATAL org.apache.hadoop.mapred.Child: Error running
> child : java.lang.NoClassDefFoundError: com/google/protobuf/Message
> at
> org.apache.hadoop.hbase.io.HbaseObjectWritable.<clinit>(HbaseObjectWritable.java:266)
> at org.apache.hadoop.hbase.ipc.Invocation.write(Invocation.java:139)
> at
> org.apache.hadoop.hbase.ipc.HBaseClient$Connection.sendParam(HBaseClient.java:612)
> at org.apache.hadoop.hbase.ipc.HBaseClient.call(HBaseClient.java:975)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine$Invoker.invoke(WritableRpcEngine.java:84)
> at $Proxy7.getProtocolVersion(Unknown Source)
> at
> org.apache.hadoop.hbase.ipc.WritableRpcEngine.getProxy(WritableRpcEngine.java:136)
> at org.apache.hadoop.hbase.ipc.HBaseRPC.waitForProxy(HBaseRPC.java:208)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.1#6144)
