[
https://issues.apache.org/jira/browse/PIG-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12546335
]
Alan Gates commented on PIG-31:
-------------------------------
I am fine with BigDataBag.finalize() not calling DataBag.clear(). The question
I still have is should DataBag.clear() call content.clear(). Shouldn't it
instead allocate a new content object, so that if someone still has a reference
to the old content object they aren't left to die with a comodification error?
If we think that once someone calls DataBag.clear() then any other instances of
content should be declared invalid, we at least need to give a way to check
that, so that we don't encounter this error again and again have no idea how we
got it.
> concurrent modification error
> -----------------------------
>
> Key: PIG-31
> URL: https://issues.apache.org/jira/browse/PIG-31
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Olga Natkovich
> Assignee: Olga Natkovich
> Priority: Critical
> Attachments: concurrentmod.patch
>
>
> Many users encountered this problem. The stack looks like:
> Error:java.lang.RuntimeException
> at
> java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
> at java.util.AbstractList$Itr.next(AbstractList.java:343)
> at com.yahoo.pig.builtin.SUM.sum(SUM.java:58)
> at com.yahoo.pig.builtin.SUM.exec(SUM.java:25)
> at com.yahoo.pig.builtin.SUM.exec(SUM.java:21)
> at com.yahoo.pig.impl.eval.FuncEvalSpec$1.add(FuncEvalSpec.java:101)
> at
> com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.(GenerateSpec.java:140)
> at com.yahoo.pig.impl.eval.GenerateSpec$1.add(GenerateSpec.java:52)
> at
> com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.add(GenerateSpec.java:217)
> at
> com.yahoo.pig.impl.eval.collector.UnflattenCollector.add(UnflattenCollector.java:42)
> at
> com.yahoo.pig.impl.eval.collector.DataCollector.addToSuccessor(DataCollector.java:79)
> at com.yahoo.pig.impl.eval.SimpleEvalSpec$1.add(SimpleEvalSpec.java:21)
> at
> com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.exec(GenerateSpec.java:248)
> at com.yahoo.pig.impl.eval.GenerateSpec$1.add(GenerateSpec.java:61)
> at
> com.yahoo.pig.impl.mapreduceExec.PigMapReduce.reduce(PigMapReduce.java:144)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:355)
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1707)
> We traced the issue to the same bag being shared between multiple operations
> and clearing it in one place caused problem in another. Will attach Ben's fix
> shortly. All unit and end-to-end tests passed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
