[
https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16680436#comment-16680436
]
Sean Owen commented on SPARK-24421:
-----------------------------------
EDIT: I replaced most of this comment because Kris got me past one problem and
onto another. See revised:
sun.misc.Cleaner is the tough case here, and there are two sites where we touch
sun.misc.Cleaner. One is in Platform.java, where we 'hack' allocation of a
DirectByteBuffer and piece it together manually in order to avoid
MaxDirectMemorySize limits in the JVM. The other is in StorageUtils where we
get a DirectBuffer and manually trigger its Cleaner's clean() method.
The latter is actually not a problem, because from JDK 9, sun.misc.Unsafe
provides an invokeCleaner() method, and we can still use that, it seems.
The former is the issue. We can still reflectively allocate a DirectByteBuffer
and set its Cleaner, but, we can't instantiate the Cleaner anymore.
sun.misc.Cleaner is gone. While java.lang.ref.Cleaner as was added in Java 9,
that isn't the type of DirectByteBuffer's Cleaner. It's
jdk.internal.ref.Cleaner.
That class is not instantiable even with reflection, unless a flag like
{{--add-opens java.base/java.lang=ALL-UNNAMED}} is set on the JVM. We'd like to
avoid that, if possible, but it may not be possible.
> sun.misc.Unsafe in JDK11
> ------------------------
>
> Key: SPARK-24421
> URL: https://issues.apache.org/jira/browse/SPARK-24421
> Project: Spark
> Issue Type: Sub-task
> Components: Build
> Affects Versions: 2.3.0
> Reporter: DB Tsai
> Priority: Major
>
> Many internal APIs such as unsafe are encapsulated in JDK9+, see
> http://openjdk.java.net/jeps/260 for detail.
> To use Unsafe, we need to add *jdk.unsupported* to our code’s module
> declaration:
> {code:java}
> module java9unsafe {
> requires jdk.unsupported;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
