Re: hbase bulk load with map reduce error
Please let me know what am I missing here. I am using MapR hadoop. Please find the classpath which is showing all the jars with versions, mean while I will get the code snippet also. I am using bulk write approach with mapper and reducer. HBase table created with bulk load enabled as true. + HADOOP_CLASSPATH='/opt/mapr/hbase/hbase-0.98.9/bin/../ conf:/opt/mapr/java/jdk1.7.0_25/lib/tools.jar:/opt/mapr/hbase/hbase -0.98.9/bin/..:/opt/mapr/lib/zookeeper-3.4.5-mapr-1406.jar:/opt/mapr/hbase/ hbase-0.98.9/bin/../lib/activation-1.1.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/aopalliance-1.0.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/apacheds-i18n-2.0.0-M15.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/apacheds-kerberos-codec-2.0.0-M15.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/api-asn1-api-1.0.0-M20.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/api-util-1.0.0-M20.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/asm-3.1.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/commons-beanutils-1.7.0.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/commons-beanutils-core-1.8.0.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/commons-cli-1.2.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/commons-codec-1.7.jar:/opt/mapr/hbase/hbase- 0.98.9/bin/../lib/commons-collections-3.2.1.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/commons-compress-1.4.1.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/commons-configuration-1.6.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/commons-daemon-1.0.13.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/commons-digester-1.8.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/commons-el-1.0.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/commons-httpclient-3.1.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/commons-io-2.4.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/commons-lang-2.6.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/commons-logging-1.1.1.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/commons-math-2.1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/commons-math3-3.1.1.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/commons-net-3.1.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/findbugs-annotations-1.3.9-1.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/guava-12.0.1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/guice-3.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ guice-servlet-3.0.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/hamcrest-core-1.3.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase -annotations-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ hbase-checkstyle-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/hbase-client-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/hbase-common-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/hbase-common-0.98.9-mapr-1503-tests.jar:/opt/mapr/hbase/ hbase-0.98.9/bin/../lib/hbase-examples-0.98.9-mapr-1503.jar:/opt/mapr/hbase/ hbase-0.98.9/bin/../lib/hbase-hadoop2-compat-0.98.9-mapr-1503.jar:/opt/mapr/ hbase/hbase-0.98.9/bin/../lib/hbase-hadoop-compat-0.98.9-mapr-1503.jar:/ opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase-it-0.98.9- mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase- it-0.98.9-mapr-1503-tests.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/hbase -prefix-tree-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ hbase-protocol-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ hbase-rest-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ hbase-server-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase-0.98.9/bin/../lib/ hbase-server-0.98.9-mapr-1503-tests.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/hbase-shell-0.98.9-mapr-1503.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/hbase-testing-util-0.98.9-mapr-1503.jar:/opt/mapr/hbase/ hbase-0.98.9/bin/../lib/hbase-thrift-0.98.9-mapr-1503.jar:/opt/mapr/hbase/ hbase-0.98.9/bin/../lib/high-scale-lib-1.1.1.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/htrace-core-2.04.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/httpclient-4.2.5.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/httpcore-4.1.3.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/jackson-core-asl-1.8.8.jar:/opt/mapr/hbase/hbase- 0.98.9/bin/../lib/jackson-jaxrs-1.8.8.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/jackson-mapper-asl-1.8.8.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/jackson-xc-1.8.8.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/jamon-runtime-2.3.1.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/jasper-compiler-5.5.23.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/jasper-runtime-5.5.23.jar:/opt/mapr/hbase/hbase- 0.98.9/bin/../lib/javax.inject-1.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/java-xmlbuilder-0.4.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/jaxb-api-2.2.2.jar:/opt/mapr/hbase/hbase -0.98.9/bin/../lib/jaxb-impl-2.2.3-1.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/jcodings-1.0.8.jar:/opt/mapr/hbase/hbase-0.98.9/bin/.. /lib/jersey-client-1.9.jar:/opt/mapr/hbase/hbase-0.98.9/ bin/../lib/jersey-core-1.8.jar:/opt/mapr/hbase/hbase-0. 98.9/bin/../lib/jersey-guice-1.9.jar:/opt/mapr/hbase/hbase- 0.98.9/bin/
Re: hbase bulk load with map reduce error
Which version of hbase / Hadoop are you using ? (So that line number matching can be more accurate) It would be good if you can show your code snippet. Thanks > On Jun 11, 2016, at 12:57 AM, Jilani Shaik wrote: > > Hi, > > I am trying to do hbase table bulk load from data file using map reduce. in > the process I am getting the Null pointer exception during the shuffle and > sort. I have both mapper and reducer in the flow. where as mapper is ending > with success and reducer not being invoked even setup. > > It is showing the below error in terminal where the job executed. > > Error: java.lang.NullPointerException > at > com.google.common.base.Preconditions.checkNotNull(Preconditions.java:191) > at com.google.common.base.Splitter.split(Splitter.java:375) > at > org.apache.hadoop.hbase.mapreduce.ImportTsv$TsvParser.(ImportTsv.java:151) > at > org.apache.hadoop.hbase.mapreduce.TextSortReducer.setup(TextSortReducer.java:105) > at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:168) > at > org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:627) > at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:389) > at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:168) > at java.security.AccessController.doPrivileged(Native Method) > at javax.security.auth.Subject.doAs(Subject.java:415) > at > org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1566) > at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:163) > > > Please let me know what could be the issue. > > > Thanks, > Jilani