[
https://issues.apache.org/jira/browse/PIG-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12546061
]
Alan Gates commented on PIG-31:
-------------------------------
As I understand the problem, the issue is that DataBag.clear() (which is
eventually called by BigDataBag.finalize()) calls content.clear(). But we have
no guarantee that we haven't handed out a reference to DataBag.content via
DataBag.content(). This seems like a bigger issue than just
BigDataBag.finalize(), as there are other places that DataBag.clear() is
called. Isn't the correct solution to have DataBag.clear() not call
content.clear()? Do we ever need to reuse the content of the bag?
> concurrent modification error
> -----------------------------
>
> Key: PIG-31
> URL: https://issues.apache.org/jira/browse/PIG-31
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Olga Natkovich
> Assignee: Olga Natkovich
> Priority: Critical
> Attachments: concurrentmod.patch
>
>
> Many users encountered this problem. The stack looks like:
> Error:java.lang.RuntimeException
> at
> java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
> at java.util.AbstractList$Itr.next(AbstractList.java:343)
> at com.yahoo.pig.builtin.SUM.sum(SUM.java:58)
> at com.yahoo.pig.builtin.SUM.exec(SUM.java:25)
> at com.yahoo.pig.builtin.SUM.exec(SUM.java:21)
> at com.yahoo.pig.impl.eval.FuncEvalSpec$1.add(FuncEvalSpec.java:101)
> at
> com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.(GenerateSpec.java:140)
> at com.yahoo.pig.impl.eval.GenerateSpec$1.add(GenerateSpec.java:52)
> at
> com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.add(GenerateSpec.java:217)
> at
> com.yahoo.pig.impl.eval.collector.UnflattenCollector.add(UnflattenCollector.java:42)
> at
> com.yahoo.pig.impl.eval.collector.DataCollector.addToSuccessor(DataCollector.java:79)
> at com.yahoo.pig.impl.eval.SimpleEvalSpec$1.add(SimpleEvalSpec.java:21)
> at
> com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.exec(GenerateSpec.java:248)
> at com.yahoo.pig.impl.eval.GenerateSpec$1.add(GenerateSpec.java:61)
> at
> com.yahoo.pig.impl.mapreduceExec.PigMapReduce.reduce(PigMapReduce.java:144)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:355)
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1707)
> We traced the issue to the same bag being shared between multiple operations
> and clearing it in one place caused problem in another. Will attach Ben's fix
> shortly. All unit and end-to-end tests passed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
