Ah thanks everyone for the help on this. Yes Ashutosh was correct I was also using multiple threads so I will try this patch after I get into work.
zjffdu wrote: > Yes, Pig is not thread safe now. There's a jira item for this issue. > https://issues.apache.org/jira/browse/PIG-240 > > I attach a patch for this item. I am not sure whether this patch resolve > this issue completely. > But at least it works for me. I have ready used this patch for several > months. Maybe you can try this patch. > > > Jeff Zhang > > > > -----Original Message----- > From: Ashutosh Chauhan [mailto:ashutosh.chau...@gmail.com] > Sent: 2009年11月3日 10:05 > To: pig-user@hadoop.apache.org > Subject: Re: Unable to find clone for op Const 16-169 > > Hi Vincent, > > AFAIK PigServer is not thread-safe. So, it can't support queries running > concurrently in multiple threads. As a result, you may end up in race > conditions as the one Bennie encountered. I guess, there is a jira open to > make PigServer thread-safe. You may want to comment there with your > use-case. > > Thanks, > Ashutosh > > On Tue, Nov 3, 2009 at 12:57, Vincent Barat <vincent.ba...@ubikod.com>wrote: > > >> I have the exact same problem, and yes, I run my queries from Java >> concurrently. The issue is easier to reproduce in local mode than in MR >> > mode > >> (I never saw it in MR mode actually). >> >> I've tried to add calls to pigServer.shutdown() to see if this can help, >> but the issue remains. >> >> Ashutosh Chauhan a écrit : >> >> Hi Bennie, >> >>> Are you using Pig Java API to run your queries? If so, are you trying to >>> run >>> queries concurrently in multiple threads ? >>> >>> Ashutosh >>> >>> On Tue, Nov 3, 2009 at 11:00, Bennie Schut <bsc...@ebuddy.com> wrote: >>> >>> From time to time I receive this error: >>> >>>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable >>>> to store alias uqusers11 >>>> at org.apache.pig.PigServer.store(PigServer.java:536) >>>> at org.apache.pig.PigServer.store(PigServer.java:493) >>>> at >>>> >>>> >>>> >>>> > com.ebuddy.dwhmapreduce.pig.chatsessions.UniqueUsers.run(UniqueUsers.java:76 > ) > >>>> at >>>> >>>> >>>> >>>> > java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja > va:886) > >>>> at >>>> >>>> >>>> >>>> > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9 > 08) > >>>> at java.lang.Thread.run(Thread.java:619) >>>> Caused by: java.lang.RuntimeException: Unable to find clone for op Const >>>> 16-169( 1 ) >>>> at >>>> org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:139) >>>> at >>>> >>>> >>>> >>>> > org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCl > oneHelper.java:63) > >>>> at >>>> >>>> >>>> >>>> > org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlan > Cloner.java:45) > >>>> at >>>> >>>> >>>> >>>> > org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParse > r.java:3044) > >>>> at >>>> >>>> >>>> >>>> > org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.jav > a:1328) > >>>> at >>>> >>>> >>>> >>>> > org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:90 > 7) > >>>> at >>>> >>>> >>>> >>>> > org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:6 > 96) > >>>> at >>>> >>>> >>>> >>>> > org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder > .java:63) > >>>> at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1034) >>>> at org.apache.pig.PigServer$Graph.clone(PigServer.java:1077) >>>> at org.apache.pig.PigServer.clonePlan(PigServer.java:389) >>>> at org.apache.pig.PigServer.compileLp(PigServer.java:804) >>>> at org.apache.pig.PigServer.compileLp(PigServer.java:791) >>>> at org.apache.pig.PigServer.store(PigServer.java:509) >>>> ... 5 more >>>> >>>> However running it a 2nd time it runs just fine. It's a little hard to >>>> reproduce. I received this on a line like this: >>>> uqusers11 = FOREACH uqusers10 GENERATE user_id, protocol, logincldr_id, >>>> logintime_id; >>>> >>>> but also sometimes on this: >>>> uqusers6 = FOREACH uqusers5 GENERATE flatten($0), MIN(uqusers4.login) as >>>> mindate; >>>> >>>> so for now it mostly seems to happen on foreach statements (I'm >>>> currently using pig trunk and hadoop 20.1) >>>> >>>> Can anyone point me in the right direction on what to look at when >>>> looking at errors like this. >>>> Thanks, >>>> Bennie. >>>> >>>> >>>> > >
