Kousuke Saruta created SPARK-32000:
--------------------------------------
Summary: Fix the flaky testcase for partially launched task in
barrier-mode.
Key: SPARK-32000
URL: https://issues.apache.org/jira/browse/SPARK-32000
Project: Spark
Issue Type: Improvement
Components: Spark Core, Tests
Affects Versions: 3.1.0
Reporter: Kousuke Saruta
Assignee: Kousuke Saruta
I noticed sometimes the testcase for SPARK-31485 fails.
The reason should be related to the locality wait.
If the scheduler waits for a resource offer which meets the preferred location
for a task until the time-limit of process-local but no resource can be offered
for the locality level, the scheduler will give up the preferred location. In
this case, such task can be assigned to off-preferred location.
The testcase for SPARK-31485, there are two tasks and only one task is supposed
to be assigned at one schedule round but both two tasks can be assigned in that
situation mentioned above and the testcase will fail.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
