Alessandro Bellina created SPARK-29780:
------------------------------------------
Summary: The UI can access into the ResourceAllocator, whose data
structures are being updated from scheduler threads
Key: SPARK-29780
URL: https://issues.apache.org/jira/browse/SPARK-29780
Project: Spark
Issue Type: Bug
Components: Spark Core
Affects Versions: 3.0.0
Reporter: Alessandro Bellina
A class extending ResourceAllocator (WorkerResourceInfo), has some potential
issues (raised here:
https://github.com/apache/spark/pull/26078#discussion_r340342820)
The WorkerInfo class is calling into availableAddrs and assignedAddrs but those
calls appear to be coming from the UI (looking at the resourcesInfo*
functions), e.g. JsonProtocol and MasterPage call this. Since the
datastructures in ResourceAllocator are not concurrent, we could end up with
bad data or potentially crashes, depending on when the calls are made.
Note that there are other calls to the resourceInfo* functions, but those are
from the event loop.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
