[
https://issues.apache.org/jira/browse/HADOOP-13500?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17441534#comment-17441534
]
Dhananjay Badaya edited comment on HADOOP-13500 at 11/10/21, 7:16 AM:
----------------------------------------------------------------------
Encountered this with Hive
{code:java}
java.util.ConcurrentModificationException at
java.util.Hashtable$Enumerator.next(Hashtable.java:1387) at
org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2625) at
org.apache.hadoop.hive.conf.HiveConf.getProperties(HiveConf.java:4047) at
org.apache.hadoop.hive.conf.HiveConf.getAllProperties(HiveConf.java:4043) at
org.apache.hadoop.hive.conf.HiveConf.getChangedProperties(HiveConf.java:4362)
at
org.apache.hadoop.hive.ql.exec.Utilities.isDefaultNameNode(Utilities.java:3572)
at
org.apache.hadoop.hive.ql.plan.CreateTableDesc.toTable(CreateTableDesc.java:795)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4321) at
org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:354) at
org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100){code}
As [~jlowe] mentioned earlier, the fix could be to wrap
[this|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L2964]
for loop in a *synchronized (properties)* block.
If this solution looks acceptable then I will be happy to pick this up.
was (Author: JIRAUSER279802):
Encountered this with Hive
{code:java}
java.util.ConcurrentModificationException at
java.util.Hashtable$Enumerator.next(Hashtable.java:1387) at
org.apache.hadoop.conf.Configuration.iterator(Configuration.java:2625) at
org.apache.hadoop.hive.conf.HiveConf.getProperties(HiveConf.java:4047) at
org.apache.hadoop.hive.conf.HiveConf.getAllProperties(HiveConf.java:4043) at
org.apache.hadoop.hive.conf.HiveConf.getChangedProperties(HiveConf.java:4362)
at
org.apache.hadoop.hive.ql.exec.Utilities.isDefaultNameNode(Utilities.java:3572)
at
org.apache.hadoop.hive.ql.plan.CreateTableDesc.toTable(CreateTableDesc.java:795)
at org.apache.hadoop.hive.ql.exec.DDLTask.createTable(DDLTask.java:4321) at
org.apache.hadoop.hive.ql.exec.DDLTask.execute(DDLTask.java:354) at
org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:199) at
org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100){code}
As [~jlowe] mentioned earlier the fix could be to wrap this
[this|https://github.com/apache/hadoop/blob/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/conf/Configuration.java#L2964]
for loop in a *synchronized (properties)* block.
If this solution looks acceptable then I will be happy to pick this up.
> Concurrency issues when using Configuration iterator
> ----------------------------------------------------
>
> Key: HADOOP-13500
> URL: https://issues.apache.org/jira/browse/HADOOP-13500
> Project: Hadoop Common
> Issue Type: Bug
> Components: conf
> Reporter: Jason Darrell Lowe
> Priority: Major
>
> It is possible to encounter a ConcurrentModificationException while trying to
> iterate a Configuration object. The iterator method tries to walk the
> underlying Property object without proper synchronization, so another thread
> simultaneously calling the set method can trigger it.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
