Re: Unable to find clone for op Const 16-169

[Jeff Zhang]

Yes, even when you create a new PigServer each thread, they are still not
thread safe. Because they are in one JVM, and some static variable is
shared.


Jeff Zhang


2009/11/4 Vincent Barat <vincent.ba...@ubikod.com>

> I will try this patch too...
>
> zjffdu a écrit :
> > Yes, Pig is not thread safe now. There's a jira item for this issue.
> > https://issues.apache.org/jira/browse/PIG-240
> >
> > I attach a patch for this item. I am not sure whether this patch resolve
> > this issue completely.
> > But at least it works for me. I have ready used this patch for several
> > months. Maybe you can try this patch.
> >
> >
> > Jeff Zhang
> >
> >
> >
> > -----Original Message-----
> > From: Ashutosh Chauhan [mailto:ashutosh.chau...@gmail.com]
> > Sent: 2009年11月3日 10:05
> > To: pig-user@hadoop.apache.org
> > Subject: Re: Unable to find clone for op Const 16-169
> >
> > Hi Vincent,
> >
> > AFAIK PigServer is not thread-safe. So, it can't support queries running
> > concurrently in multiple threads. As a result, you may end up in race
> > conditions as the one Bennie encountered. I guess, there is a jira open
> to
> > make PigServer thread-safe. You may want to comment there with your
> > use-case.
> >
> > Thanks,
> > Ashutosh
> >
> > On Tue, Nov 3, 2009 at 12:57, Vincent Barat <vincent.ba...@ubikod.com
> >wrote:
> >
> >> I have the exact same problem, and yes, I run my queries from Java
> >> concurrently. The issue is easier to reproduce in local mode than in MR
> > mode
> >> (I never saw it in MR mode actually).
> >>
> >> I've tried to add calls to pigServer.shutdown() to see if this can help,
> >> but the issue remains.
> >>
> >> Ashutosh Chauhan a écrit :
> >>
> >>  Hi Bennie,
> >>> Are you using Pig Java API to run your queries? If so, are you trying
> to
> >>> run
> >>> queries concurrently in multiple threads ?
> >>>
> >>> Ashutosh
> >>>
> >>> On Tue, Nov 3, 2009 at 11:00, Bennie Schut <bsc...@ebuddy.com> wrote:
> >>>
> >>>  From time to time I receive this error:
> >>>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable
> >>>> to store alias uqusers11
> >>>>       at org.apache.pig.PigServer.store(PigServer.java:536)
> >>>>       at org.apache.pig.PigServer.store(PigServer.java:493)
> >>>>       at
> >>>>
> >>>>
> >>>>
> >
> com.ebuddy.dwhmapreduce.pig.chatsessions.UniqueUsers.run(UniqueUsers.java:76
> > )
> >>>>       at
> >>>>
> >>>>
> >>>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.ja
> > va:886)
> >>>>       at
> >>>>
> >>>>
> >>>>
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:9
> > 08)
> >>>>       at java.lang.Thread.run(Thread.java:619)
> >>>> Caused by: java.lang.RuntimeException: Unable to find clone for op
> Const
> >>>> 16-169( 1 )
> >>>>       at
> >>>>
> org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:139)
> >>>>       at
> >>>>
> >>>>
> >>>>
> >
> org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCl
> > oneHelper.java:63)
> >>>>       at
> >>>>
> >>>>
> >>>>
> >
> org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlan
> > Cloner.java:45)
> >>>>       at
> >>>>
> >>>>
> >>>>
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParse
> > r.java:3044)
> >>>>       at
> >>>>
> >>>>
> >>>>
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.jav
> > a:1328)
> >>>>       at
> >>>>
> >>>>
> >>>>
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:90
> > 7)
> >>>>       at
> >>>>
> >>>>
> >>>>
> >
> org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:6
> > 96)
> >>>>       at
> >>>>
> >>>>
> >>>>
> >
> org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder
> > .java:63)
> >>>>       at
> org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1034)
> >>>>       at org.apache.pig.PigServer$Graph.clone(PigServer.java:1077)
> >>>>       at org.apache.pig.PigServer.clonePlan(PigServer.java:389)
> >>>>       at org.apache.pig.PigServer.compileLp(PigServer.java:804)
> >>>>       at org.apache.pig.PigServer.compileLp(PigServer.java:791)
> >>>>       at org.apache.pig.PigServer.store(PigServer.java:509)
> >>>>       ... 5 more
> >>>>
> >>>> However running it a 2nd time it runs just fine. It's a little hard to
> >>>> reproduce. I received this on a line like this:
> >>>> uqusers11 = FOREACH uqusers10 GENERATE user_id, protocol,
> logincldr_id,
> >>>> logintime_id;
> >>>>
> >>>> but also sometimes on this:
> >>>> uqusers6 = FOREACH uqusers5 GENERATE flatten($0), MIN(uqusers4.login)
> as
> >>>> mindate;
> >>>>
> >>>> so for now it mostly seems to happen on foreach statements (I'm
> >>>> currently using pig trunk and hadoop 20.1)
> >>>>
> >>>> Can anyone point me in the right direction on what to look at when
> >>>> looking at errors like this.
> >>>> Thanks,
> >>>> Bennie.
> >>>>
> >>>>
> >
> >
> >
>