[
https://issues.apache.org/jira/browse/HBASE-7934?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13587420#comment-13587420
]
Nick Dimiduk commented on HBASE-7934:
-------------------------------------
I agree that we should minimize the weight added to the user's jobs. On the
other hand, if someone wants to use some part of HBase API, I think they
shouldn't have to resolve dependencies manually the first time they run on a
real cluster. Why force the user to pretend to be maven when we can do it for
them. Better still would be some kind of automatic dependency resolution, but...
I think the logic of walking the dependency graph to decide precisely which
jars are necessary is too complex vs the pain suffered by a user for always
including everything that might be needed. As was pointed out, this operation
isn't recursive, so it doesn't have the potential of polluting the user's
classpath with incompatible versions of 3rd party jars. Worse case scenario,
they get hbase-foo.jar when they didn't need it. So long as the list of
dependencies require by hbase doesn't stray far from those provided by the
hadoop runtime, this should be painless to the consumer.
> TableMapReduceUtil doesn't include all dependency jars in new modular build
> ---------------------------------------------------------------------------
>
> Key: HBASE-7934
> URL: https://issues.apache.org/jira/browse/HBASE-7934
> Project: HBase
> Issue Type: Bug
> Components: mapreduce
> Affects Versions: 0.95.0, 0.96.0
> Reporter: Nick Dimiduk
> Assignee: Nick Dimiduk
> Attachments:
> 0001-HBASE-7934-TableMapReduceUtil-include-module-depende.patch,
> 0001-HBASE-7934-TableMapReduceUtil-include-module-depende.patch
>
>
> From post to hbase-dev:
> I'm seeing ClassNotFoundExceptions when running ImportTsv against trunk.
> Presumably the old instructions for classpath assembly need an update. I
> guess I could create a hadoop jar that ships the classpath entries in lib,
> but IMHO that's not a viable solution for users who want to run one of the
> jobs we distribute. Am I missing something?
> For you reference:
> {code}
> $ HADOOP_CLASSPATH=$(./bin/hbase classpath) hadoop jar
> ./hbase-server/target/hbase-server-0.95-SNAPSHOT.jar importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2
> -Dimporttsv.bulk.output=hdfs:///tmp/hfiles simple hdfs:///tmp/simple.tsv
> ...
> Error: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Mutate$MutateType
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at
> org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:87)
> at
> org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:70)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1069)
> at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:151)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:37)
> at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
> {code}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
