Hi,
I am trying to instantiate several embedded pig scripts (that uses
pigserver) in Java using a thread pool, so that all of them can be
executed in parallel. Each of these scripts load the same table from HBase
using HBaseStorage. However some times few scripts fail with a Runtime
Exception such as,
*
org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1000: Error
during parsing. Pig script failed to parse:
<line 1, column 22> pig script failed to validate:
java.lang.RuntimeException: could not instantiate
'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[v:date
v:meid v:spid v:coid v:ip, -caching 50000 -lt 1368316800&sbo -gt
1365120000&sbo ]'
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1607)
at org.apache.pig.PigServer$Graph.registerQuery(PigServer.java:1546)
at org.apache.pig.PigServer.registerQuery(PigServer.java:516)
at org.apache.pig.PigServer.registerQuery(PigServer.java:529)
at
com.jayeson.hantu.pig.MultipleAccountsPigFeature.compute(MultipleAccountsPigFeature.java:112)
at
com.jayeson.hantu.pig.MultipleAccountsPigFeature.call(MultipleAccountsPigFeature.java:40)
at
com.jayeson.hantu.pig.MultipleAccountsPigFeature.call(MultipleAccountsPigFeature.java:27)
at
java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:722)
Caused by: Failed to parse: Pig script failed to parse:
<line 1, column 22> pig script failed to validate:
java.lang.RuntimeException: could not instantiate
'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[v:date
v:meid v:spid v:coid v:ip, -caching 50000 -lt 1368316800&sbo -gt
1365120000&sbo ]'
at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:191)
at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1599)
... 11 more
Caused by:
<line 1, column 22> pig script failed to validate:
java.lang.RuntimeException: could not instantiate
'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[v:date
v:meid v:spid v:coid v:ip, -caching 50000 -lt 1368316800&sbo -gt
1365120000&sbo ]'
at
org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:835)
at
org.apache.pig.parser.LogicalPlanGenerator.load_clause(LogicalPlanGenerator.java:3236)
at
org.apache.pig.parser.LogicalPlanGenerator.op_clause(LogicalPlanGenerator.java:1315)
at
org.apache.pig.parser.LogicalPlanGenerator.general_statement(LogicalPlanGenerator.java:799)
at
org.apache.pig.parser.LogicalPlanGenerator.statement(LogicalPlanGenerator.java:517)
at
org.apache.pig.parser.LogicalPlanGenerator.query(LogicalPlanGenerator.java:392)
at
org.apache.pig.parser.QueryParserDriver.parse(QueryParserDriver.java:184)
... 12 more
Caused by: java.lang.RuntimeException: could not instantiate
'org.apache.pig.backend.hadoop.hbase.HBaseStorage' with arguments '[v:date
v:meid v:spid v:coid v:ip, -caching 50000 -lt 1368316800&sbo -gt
1365120000&sbo ]'
at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:618)
at
org.apache.pig.parser.LogicalPlanBuilder.buildLoadOp(LogicalPlanBuilder.java:823)
... 18 more
Caused by: java.lang.reflect.InvocationTargetException
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
Method)
at
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
at
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
at
org.apache.pig.impl.PigContext.instantiateFuncFromSpec(PigContext.java:586)
... 19 more
Caused by: java.util.ConcurrentModificationException
at
java.util.ArrayList$Itr.checkForComodification(ArrayList.java:819)
at java.util.ArrayList$Itr.next(ArrayList.java:791)
at
org.apache.commons.cli.CommandLine.resolveOption(CommandLine.java:196)
at org.apache.commons.cli.CommandLine.hasOption(CommandLine.java:69)
at
org.apache.pig.backend.hadoop.hbase.HBaseStorage.<init>(HBaseStorage.java:262)
*Are there any synchronization issues that need to be taken care of in this
case ?*
*
--
Regards,
Praveen Bysani
http://www.praveenbysani.com
