[
https://issues.apache.org/jira/browse/SPARK-52505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-52505:
-----------------------------------
Labels: pull-request-available (was: )
> Connection timeout on Kubernetes to decommissioned executor
> -----------------------------------------------------------
>
> Key: SPARK-52505
> URL: https://issues.apache.org/jira/browse/SPARK-52505
> Project: Spark
> Issue Type: Sub-task
> Components: Kubernetes
> Affects Versions: 4.1.0
> Reporter: Enrico Minack
> Priority: Major
> Labels: pull-request-available
>
> Running Spark on Kubernetes with storage decommissioning frequently runs into
> the situation where a task is started that has to read from another executor.
> By the time a connection is established, that other executor has been
> decommissioned, the data migrated, and the executor terminated. Connecting to
> that executor times out, which has a 2 minutes default timeout. The task
> blocks for that amount of time.
> Reducing the timeout has only limited success as all connections are created
> one-by-one. Multiple such timeouts occur until the task hits a fetch failure.
> This delays the task execution significantly, impacting performance of the
> Spark job.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
