[
https://issues.apache.org/jira/browse/SPARK-40902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17623270#comment-17623270
]
Mridul Muralidharan commented on SPARK-40902:
---------------------------------------------
+CC [~dongjoon], [~yumwang]
> Quick submission of drivers in tests to mesos scheduler results in dropping
> drivers
> -----------------------------------------------------------------------------------
>
> Key: SPARK-40902
> URL: https://issues.apache.org/jira/browse/SPARK-40902
> Project: Spark
> Issue Type: Bug
> Components: Mesos
> Affects Versions: 2.4.8, 3.0.3, 3.4.0
> Reporter: Mridul Muralidharan
> Priority: Major
>
> Queued drivers in MesosClusterScheduler are ordered based on
> MesosDriverDescription - and the default ordering checks for priority,
> followed by submission time. For two driver submissions with same priority
> and if made in quick succession (such that submission time is same due to
> millisecond granularity of Date), this results in dropping the second
> MesosDriverDescription from the queuedDrivers - as driverOrdering returns 0
> when comparing the descriptions. This jira fixes the more immediate issue
> with tests, but we do need to relook at this for mess scheduler in general
> later.
> Currently, this affects tests - for example, in the latest VOTE for 3.3.1 [1]
> - and is not consistently reproducible unless on a fast machine.
> [1] https://lists.apache.org/thread/jof098qxp0s6qqmt9qwv52f9665b1pjg
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
