[
https://issues.apache.org/jira/browse/PIG-31?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12546278
]
Benjamin Reed commented on PIG-31:
----------------------------------
The point of the CoModificationException is to catch places where the bag is
modified while an iterator is outstanding. By reuse the bag I assume you mean
clear the bag and readd tuples. You certainly can do such a thing using the
API. It may be useful to reexamine the DataBag API. Perhaps it would be good to
do in the context of PIG-30.
DataBag.clear() should call content.clear() since that is the semantics of
clear(). The exception will also arise if someone calls DataBag.add().
There is no reason to call clear() in finalize. The GC will collect everything
when it is not being used. The only thing we need finalize() for is to clean up
any temporary files we have created.
> concurrent modification error
> -----------------------------
>
> Key: PIG-31
> URL: https://issues.apache.org/jira/browse/PIG-31
> Project: Pig
> Issue Type: Bug
> Components: impl
> Reporter: Olga Natkovich
> Assignee: Olga Natkovich
> Priority: Critical
> Attachments: concurrentmod.patch
>
>
> Many users encountered this problem. The stack looks like:
> Error:java.lang.RuntimeException
> at
> java.util.AbstractList$Itr.checkForComodification(AbstractList.java:372)
> at java.util.AbstractList$Itr.next(AbstractList.java:343)
> at com.yahoo.pig.builtin.SUM.sum(SUM.java:58)
> at com.yahoo.pig.builtin.SUM.exec(SUM.java:25)
> at com.yahoo.pig.builtin.SUM.exec(SUM.java:21)
> at com.yahoo.pig.impl.eval.FuncEvalSpec$1.add(FuncEvalSpec.java:101)
> at
> com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.(GenerateSpec.java:140)
> at com.yahoo.pig.impl.eval.GenerateSpec$1.add(GenerateSpec.java:52)
> at
> com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.add(GenerateSpec.java:217)
> at
> com.yahoo.pig.impl.eval.collector.UnflattenCollector.add(UnflattenCollector.java:42)
> at
> com.yahoo.pig.impl.eval.collector.DataCollector.addToSuccessor(DataCollector.java:79)
> at com.yahoo.pig.impl.eval.SimpleEvalSpec$1.add(SimpleEvalSpec.java:21)
> at
> com.yahoo.pig.impl.eval.GenerateSpec$CrossProductItem.exec(GenerateSpec.java:248)
> at com.yahoo.pig.impl.eval.GenerateSpec$1.add(GenerateSpec.java:61)
> at
> com.yahoo.pig.impl.mapreduceExec.PigMapReduce.reduce(PigMapReduce.java:144)
> at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:355)
> at
> org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:1707)
> We traced the issue to the same bag being shared between multiple operations
> and clearing it in one place caused problem in another. Will attach Ben's fix
> shortly. All unit and end-to-end tests passed.
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
