[
https://issues.apache.org/jira/browse/SPARK-58362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
GANHONGNAN updated SPARK-58362:
-------------------------------
Description:
Spark's default resource profile id is 0. In stage level resource profile, the
first returned resource id is also 0 which would lead to a collision.
val UNKNOWN_RESOURCE_PROFILE_ID = -1
val DEFAULT_RESOURCE_PROFILE_ID = 0
private lazy val nextProfileId = new AtomicInteger(0) \{{ }}
[spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L365]
Line 365
|private lazy val nextProfileId = new AtomicInteger(0)|
[spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L363]
Line 363
|val DEFAULT_RESOURCE_PROFILE_ID = 0|
was:
Spark's default resource profile id is 0. In stage level resource profile, the
first returned resource id is also 0 which would lead to a collision.
val UNKNOWN_RESOURCE_PROFILE_ID = -1
val DEFAULT_RESOURCE_PROFILE_ID = 0
private lazy val nextProfileId = new AtomicInteger(0) {{ }}
[spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L365]
Line 365 in
[f883693|https://github.com/apache/spark/commit/f8836939c6a89bd788ad648ec30df757cde204ac]
|private lazy val nextProfileId = new AtomicInteger(0)|
[spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L363]
Line 363 in
[f883693|https://github.com/apache/spark/commit/f8836939c6a89bd788ad648ec30df757cde204ac]
|val DEFAULT_RESOURCE_PROFILE_ID = 0|
> [CORE] bug report: collision of resource profile id
> ---------------------------------------------------
>
> Key: SPARK-58362
> URL: https://issues.apache.org/jira/browse/SPARK-58362
> Project: Spark
> Issue Type: Bug
> Components: Bug
> Affects Versions: 3.2.1
> Reporter: GANHONGNAN
> Priority: Critical
>
> Spark's default resource profile id is 0. In stage level resource profile,
> the first returned resource id is also 0 which would lead to a collision.
> val UNKNOWN_RESOURCE_PROFILE_ID = -1
> val DEFAULT_RESOURCE_PROFILE_ID = 0
> private lazy val nextProfileId = new AtomicInteger(0) \{{ }}
>
> [spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L365]
> Line 365
> |private lazy val nextProfileId = new AtomicInteger(0)|
>
>
> [spark/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala|https://github.com/apache/spark/blob/f8836939c6a89bd788ad648ec30df757cde204ac/core/src/main/scala/org/apache/spark/resource/ResourceProfile.scala#L363]
> Line 363
> |val DEFAULT_RESOURCE_PROFILE_ID = 0|
>
>
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
