GitHub user viirya opened a pull request:
https://github.com/apache/spark/pull/13211
[SPARK-15430][SQL] Fix potential ConcurrentModificationException for
ListAccumulator
## What changes were proposed in this pull request?
In `ListAccumulator` we create an unmodifiable view for underlying list.
However, it doesn't prevent the underlying to be modified further. So as we
access the unmodifiable list, the underlying list can be modified in the same
time. It could cause `java.util.ConcurrentModificationException`. We can
observe such exception in recent tests.
To fix it, we can copy a list of the underlying list and then create the
unmodifiable view of this list instead.
## How was this patch tested?
The exception might be difficult to test.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/viirya/spark-1 fix-concurrentmodify
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/13211.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #13211
----
commit 4d97bf093f4f4d41cf530a4c7464532635c2b3fe
Author: Liang-Chi Hsieh <[email protected]>
Date: 2016-05-20T04:15:49Z
Fix potential ConcurrentModificationException for ListAccumulator.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
