[
https://issues.apache.org/jira/browse/SPARK-60804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ASF GitHub Bot updated SPARK-60804:
-----------------------------------
Labels: pull-request-available (was: )
> Fix off-by-one error in Kubernetes executor pod allocation maximum
> ------------------------------------------------------------------
>
> Key: SPARK-60804
> URL: https://issues.apache.org/jira/browse/SPARK-60804
> Project: Spark
> Issue Type: Bug
> Components: Kubernetes
> Affects Versions: 5.0.0
> Reporter: Tongwei
> Priority: Major
> Labels: pull-request-available
>
> spark.kubernetes.allocation.maximum specifies the maximum number of executor
> pod creation attempts during an application lifecycle. However,
> ExecutorPodsAllocator increments the executor ID before checking
> newExecutorId >= podAllocationMaximum, allowing only N-1 attempts for a limit
> of N.
> With the direct allocator, spark.kubernetes.allocation.maximum=1,
> spark.executor.instances=1 and spark.dynamicAllocation.enabled=false, the
> first allocation throws "Exceed the pod creation limit: 1" before creating
> any executor pod. It should allow one creation attempt.
> The fix checks the limit before incrementing the counter, allowing exactly N
> attempts and preventing integer overflow. Regression tests cover limits of 1,
> 3 and Int.MaxValue, including repeated attempts after reaching the limit. All
> 32 tests in ExecutorPodsAllocatorSuite pass.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
