[
https://issues.apache.org/jira/browse/SPARK-45508?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Josh Rosen updated SPARK-45508:
-------------------------------
Description:
In JDK >= 9.b110, the code at
[https://github.com/apache/spark/blob/v3.5.0/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L213]
hits a fallback path because we are using the wrong cleaner class name:
`jdk.internal.ref.Cleaner` was removed in JDK-8149925
[https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8149925]
This can be verified via
```
val f =
classOf[org.apache.spark.unsafe.Platform].getDeclaredField("CLEANER_CREATE_METHOD")
f.setAccessible(true)
f.get(null)
```
returning `null` instead of a method.
was:
In JDK 11+, the code at
[https://github.com/apache/spark/blob/v3.5.0/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L213]
hits a fallback path because we are using the wrong cleaner class name.
This can be verified via
```
val f =
classOf[org.apache.spark.unsafe.Platform].getDeclaredField("CLEANER_CREATE_METHOD")
f.setAccessible(true)
f.get(null)
```
returning `null` instead of a method.
Summary: org.apache.spark.unsafe.Platform uses wrong cleaner class name
in JDK 9.b110+ (was: org.apache.spark.unsafe.Platform uses wrong cleaner class
name in JDK 11+)
> org.apache.spark.unsafe.Platform uses wrong cleaner class name in JDK 9.b110+
> -----------------------------------------------------------------------------
>
> Key: SPARK-45508
> URL: https://issues.apache.org/jira/browse/SPARK-45508
> Project: Spark
> Issue Type: Bug
> Components: Spark Core
> Affects Versions: 3.5.0
> Reporter: Josh Rosen
> Priority: Major
>
> In JDK >= 9.b110, the code at
> [https://github.com/apache/spark/blob/v3.5.0/common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java#L213]
> hits a fallback path because we are using the wrong cleaner class name:
> `jdk.internal.ref.Cleaner` was removed in JDK-8149925
> [https://bugs.java.com/bugdatabase/view_bug.do?bug_id=8149925]
> This can be verified via
>
> ```
> val f =
> classOf[org.apache.spark.unsafe.Platform].getDeclaredField("CLEANER_CREATE_METHOD")
> f.setAccessible(true)
> f.get(null)
> ```
> returning `null` instead of a method.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
