[
https://issues.apache.org/jira/browse/SPARK-21078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16047860#comment-16047860
]
Sean Owen commented on SPARK-21078:
-----------------------------------
I agree there's a problem here, which might or might not actually cause a
problem in practice, but should be touched up.
First, {{applications}} doesn't need to be a mutable List, because it's not
modified, and this is the justification for not synchronizing any other access
to it. It's only ever swapped out for a new value. But that happens in two
places and only one of them is protected in any way.
Is it acceptable to just return immediately from {{mergeApplicationListing}} or
{{cleanLogs}} if it is already being run by another thread? that would solve
it. It may be that the assumption is that these are only ever run by one thread
anyway, in which case no synchronization is necessary
CC [~parth.brahmbhatt] [~tgraves] for
https://issues.apache.org/jira/browse/SPARK-13988
> JobHistory applications synchronized is invalid
> -----------------------------------------------
>
> Key: SPARK-21078
> URL: https://issues.apache.org/jira/browse/SPARK-21078
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 2.1.1
> Reporter: fangfengbin
>
> FsHistoryProvider has some threads to mergeApplicationListing.
> mergeApplicationListing function has code like this:
> applications.synchronized {
> ...
> applications = mergedApps
> ...
> }
> applications object is changed, so the synchronized do not work.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
