Dear All, Please help us on this. We need to bulk import data into Hbase using pig (or any alternate way).
Any help on this will be appreciated. Thanks and Regards Nishant Aggarwal, PMP Cell No:- +91 99588 94305 http://in.linkedin.com/pub/nishant-aggarwal/53/698/11b On Tue, Nov 17, 2015 at 4:10 AM, Laurent H <laurent.hat...@gmail.com> wrote: > I remember that Pig lib with HBaseStorage (0.13 or 0.14) doesn't accept > bulk loading, (if you look at the java class, you could see that there is > only put method and no bulk function...) Hope it's available righ now ! > > -- > Laurent HATIER - Consultant Big Data & Business Intelligence chez CapGemini > fr.linkedin.com/pub/laurent-hatier/25/36b/a86/ > <http://fr.linkedin.com/pub/laurent-h/25/36b/a86/> > > 2015-11-05 13:58 GMT+01:00 Naresh Reddy <naresh.re...@aletheconsulting.com > >: > > > Hi > > > > I have already replaced the hbase version with "*hbase95.version=1.1.2*" > in > > libraries.properties file and compiled it, but I am getting the same > error. > > > > Regards > > Naresh > > > > On Wed, Nov 4, 2015 at 11:29 PM, Daniel Dai <dai...@gmail.com> wrote: > > > > > Will need to change ivy/libraries.properties, specify the right hbase > > > version and compile again. > > > > > > On Wed, Nov 4, 2015 at 6:31 AM, Ted Yu <yuzhih...@gmail.com> wrote: > > > > > > > ... 22 moreCaused by: java.lang.NoSuchMethodError: > > > > org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V at > > > > > > > > Looks like the version of Pig you use is not compiled against hbase > > 1.1.2 > > > > > > > > This is related: > > > > Author: Enis Soztutar <e...@apache.org> > > > > Date: Fri Sep 5 18:48:38 2014 -0700 > > > > > > > > HBASE-10841 Scan,Get,Put,Delete,etc setters should consistently > > > return > > > > this > > > > > > > > FYI > > > > > > > > On Tue, Nov 3, 2015 at 10:36 PM, Naresh Reddy < > > > > naresh.re...@aletheconsulting.com> wrote: > > > > > > > >> Hi > > > >> Thanks for the reply.Below is the full error log. > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> > > > >> *Pig Stack Trace---------------ERROR 1200: Pig script failed to > parse: > > > >> <line 2, column 0> pig script failed to validate: > > > >> java.lang.RuntimeException: could not instantiate > > > >> 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments > > > >> '[info:fname info:lname]'Failed to parse: Pig script failed to > parse: > > > >> <line > > > >> 2, column 0> pig script failed to validate: > > java.lang.RuntimeException: > > > >> could not instantiate > > 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' > > > >> with arguments '[info:fname info:lname]' at > > > >> > > > > org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:199) > > > >> at org.apache.pig.PigServer$Graph.validateQuery(PigServer.java:1707) > > > at > > > >> org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1680) > > at > > > >> org.apache.pig.PigServer.registerQuery(PigServer.java:623) at > > > >> > > org.apache.pig.tools.grunt.GruntParser.processPig(GruntParser.java:1063) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.tools.pigscript.parser.PigScriptParser.parse(PigScriptParser.java:501) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:230) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.tools.grunt.GruntParser.parseStopOnError(GruntParser.java:205) > > > >> at org.apache.pig.tools.grunt.Grunt.run(Grunt.java:66) at > > > >> org.apache.pig.Main.run(Main.java:558) at > > > >> org.apache.pig.Main.main(Main.java:170) at > > > >> sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at > > > >> > > > >> > > > > > > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > > > >> at > > > >> > > > >> > > > > > > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > > > >> at java.lang.reflect.Method.invoke(Method.java:606) at > > > >> org.apache.hadoop.util.RunJar.main(RunJar.java:212)Caused by: <line > 2, > > > >> column 0> pig script failed to validate: java.lang.RuntimeException: > > > could > > > >> not instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' > > with > > > >> arguments '[info:fname info:lname]' at > > > >> > > > >> > > > > > > org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:1009) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.parser.LogicalPlanGenerator.store_clause(LogicalPlanGenerator.java:7775) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1669) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:1102) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:560) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:421) > > > >> at > > > >> > > > > org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191) > > > >> ... 15 moreCaused by: java.lang.RuntimeException: could not > > instantiate > > > >> 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments > > > >> '[info:fname info:lname]' at > > > >> > > > >> > > > > > > org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:772) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.parser.LogicalPlanBuilder.buildStoreOp(LogicalPlanBuilder.java:988) > > > >> ... 21 moreCaused by: java.lang.reflect.InvocationTargetException > > at > > > >> sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native > Method) > > > >> at > > > >> > > > >> > > > > > > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > > > >> at > > > >> > > > >> > > > > > > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > > > >> at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > > at > > > >> > > > >> > > > > > > org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:740) > > > >> ... 22 moreCaused by: java.lang.NoSuchMethodError: > > > >> org.apache.hadoop.hbase.client.Scan.setCacheBlocks(Z)V at > > > >> > > > >> > > > > > > org.apache.pig.backend.hadoop.hbase.HBaseStorage.initScan(HBaseStorage.java:405) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:346) > > > >> at > > > >> > > > >> > > > > > > org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:235) > > > >> ... 27 > > > >> > > > >> > > > > > > more================================================================================* > > > >> > > > >> Regards > > > >> Naresh > > > >> > > > >> On Wed, Nov 4, 2015 at 11:46 AM, Naresh Reddy < > > > >> naresh.re...@aletheconsulting.com> wrote: > > > >> > > > >> > Hi > > > >> > Thanks for the reply.PFA for the full error. > > > >> > > > > >> > Regard > > > >> > Naresh > > > >> > > > > >> > > > > >> > On Wed, Nov 4, 2015 at 11:29 AM, Ted Yu <yuzhih...@gmail.com> > > wrote: > > > >> > > > > >> >> Naresh: > > > >> >> Can you pastebin the full error ? > > > >> >> It should be in pig_XXXX.log > > > >> >> > > > >> >> Cheers > > > >> >> > > > >> >> > On Nov 3, 2015, at 9:07 PM, Naresh Reddy < > > > >> >> naresh.re...@aletheconsulting.com> wrote: > > > >> >> > > > > >> >> > Hi > > > >> >> > > > > >> >> > I am getting the below error while loading bulk data from pig > to > > > >> hbase > > > >> >> > through HBaseStorage.Please help me to resolve this > issue.Thanks > > in > > > >> >> advance. > > > >> >> > > > > >> >> > > > > >> >> > *ERROR:pig script failed to validate: > java.lang.RuntimeException: > > > >> could > > > >> >> not > > > >> >> > instantiate 'org.apache.pig.backend.hadoop.hbase.HBaseStorage' > > with > > > >> >> > arguments.* > > > >> >> > > > > >> >> > Below is my pig script. > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > > > > >> >> > *register > > > '/home/hduser/hbase-1.1.2/lib/zookeeper-3.4.6.jar';register > > > >> >> > '/home/hduser/hbase-1.1.2/lib/guava-12.0.1.jar';register > > > >> >> > '/home/hduser/hbase-1.1.2/lib/hbase-common-1.1.2.jar';register > > > >> >> > '/home/hduser/hbase-1.1.2/lib/hbase-client-1.1.2.jar';register > > > >> >> > > '/home/hduser/hbase-1.1.2/lib/protobuf-java-2.5.0.jar';raw_data = > > > >> LOAD > > > >> >> > '/user/hduser/input.csv' USING PigStorage(',') AS > (id:chararray, > > > >> >> > fname:chararray, lname:chararray);STORE raw_data INTO > > > >> >> > 'hbase://sample_names' USING > > > >> >> > org.apache.pig.backend.hadoop.hbase.HBaseStorage ('info:fname > > > >> >> info:lname');* > > > >> >> > > > > >> >> > > > > >> >> > Regards > > > >> >> > Naresh > > > >> >> > > > >> > > > > >> > > > > >> > > > > > > > > > > > > > >
