Re: Unable to find clone for op Const 16-169

[Ashutosh Chauhan]

Hi Vincent,

AFAIK PigServer is not thread-safe. So, it can't support queries running
concurrently in multiple threads. As a result, you may end up in race
conditions as the one Bennie encountered. I guess, there is a jira open to
make PigServer thread-safe. You may want to comment there with your
use-case.

Thanks,
Ashutosh

On Tue, Nov 3, 2009 at 12:57, Vincent Barat <vincent.ba...@ubikod.com>wrote:

> I have the exact same problem, and yes, I run my queries from Java
> concurrently. The issue is easier to reproduce in local mode than in MR mode
> (I never saw it in MR mode actually).
>
> I've tried to add calls to pigServer.shutdown() to see if this can help,
> but the issue remains.
>
> Ashutosh Chauhan a écrit :
>
>  Hi Bennie,
>>
>> Are you using Pig Java API to run your queries? If so, are you trying to
>> run
>> queries concurrently in multiple threads ?
>>
>> Ashutosh
>>
>> On Tue, Nov 3, 2009 at 11:00, Bennie Schut <bsc...@ebuddy.com> wrote:
>>
>>  From time to time I receive this error:
>>>
>>> org.apache.pig.impl.logicalLayer.FrontendException: ERROR 1002: Unable
>>> to store alias uqusers11
>>>       at org.apache.pig.PigServer.store(PigServer.java:536)
>>>       at org.apache.pig.PigServer.store(PigServer.java:493)
>>>       at
>>>
>>>
>>> com.ebuddy.dwhmapreduce.pig.chatsessions.UniqueUsers.run(UniqueUsers.java:76)
>>>       at
>>>
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
>>>       at
>>>
>>>
>>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
>>>       at java.lang.Thread.run(Thread.java:619)
>>> Caused by: java.lang.RuntimeException: Unable to find clone for op Const
>>> 16-169( 1 )
>>>       at
>>> org.apache.pig.impl.logicalLayer.LogicalPlan.clone(LogicalPlan.java:139)
>>>       at
>>>
>>>
>>> org.apache.pig.impl.logicalLayer.LogicalPlanCloneHelper.<init>(LogicalPlanCloneHelper.java:63)
>>>       at
>>>
>>>
>>> org.apache.pig.impl.logicalLayer.LogicalPlanCloner.getClonedPlan(LogicalPlanCloner.java:45)
>>>       at
>>>
>>>
>>> org.apache.pig.impl.logicalLayer.parser.QueryParser.ForEachClause(QueryParser.java:3044)
>>>       at
>>>
>>>
>>> org.apache.pig.impl.logicalLayer.parser.QueryParser.BaseExpr(QueryParser.java:1328)
>>>       at
>>>
>>>
>>> org.apache.pig.impl.logicalLayer.parser.QueryParser.Expr(QueryParser.java:907)
>>>       at
>>>
>>>
>>> org.apache.pig.impl.logicalLayer.parser.QueryParser.Parse(QueryParser.java:696)
>>>       at
>>>
>>>
>>> org.apache.pig.impl.logicalLayer.LogicalPlanBuilder.parse(LogicalPlanBuilder.java:63)
>>>       at org.apache.pig.PigServer$Graph.parseQuery(PigServer.java:1034)
>>>       at org.apache.pig.PigServer$Graph.clone(PigServer.java:1077)
>>>       at org.apache.pig.PigServer.clonePlan(PigServer.java:389)
>>>       at org.apache.pig.PigServer.compileLp(PigServer.java:804)
>>>       at org.apache.pig.PigServer.compileLp(PigServer.java:791)
>>>       at org.apache.pig.PigServer.store(PigServer.java:509)
>>>       ... 5 more
>>>
>>> However running it a 2nd time it runs just fine. It's a little hard to
>>> reproduce. I received this on a line like this:
>>> uqusers11 = FOREACH uqusers10 GENERATE user_id, protocol, logincldr_id,
>>> logintime_id;
>>>
>>> but also sometimes on this:
>>> uqusers6 = FOREACH uqusers5 GENERATE flatten($0), MIN(uqusers4.login) as
>>> mindate;
>>>
>>> so for now it mostly seems to happen on foreach statements (I'm
>>> currently using pig trunk and hadoop 20.1)
>>>
>>> Can anyone point me in the right direction on what to look at when
>>> looking at errors like this.
>>> Thanks,
>>> Bennie.
>>>
>>>
>>