[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK9+
[
https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16679897#comment-16679897
]
Sean Owen commented on SPARK-24421:
---
Using reflection to set the Cleaner works. We also have to use reflection to
call Cleaner.clean() in StorageUtils. However this causes an
IllegalAccessException, not because it's private, but because the methods are
of course not exported from the java.base module. This can be disabled with
command line flags but that's not a great solution. I'll look at workarounds
but ideas welcome!
Good news is that this appears to be the only change needed to get compilation
to work, and tests are running pretty well other than this so far.
> sun.misc.Unsafe in JDK9+
>
>
> Key: SPARK-24421
> URL: https://issues.apache.org/jira/browse/SPARK-24421
> Project: Spark
> Issue Type: Sub-task
> Components: Build
>Affects Versions: 2.3.0
>Reporter: DB Tsai
>Priority: Major
>
> Many internal APIs such as unsafe are encapsulated in JDK9+, see
> http://openjdk.java.net/jeps/260 for detail.
> To use Unsafe, we need to add *jdk.unsupported* to our code’s module
> declaration:
> {code:java}
> module java9unsafe {
> requires jdk.unsupported;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK9+
[
https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16677280#comment-16677280
]
Sean Owen commented on SPARK-24421:
---
BTW thanks for the tip [~akorzhuev] ; I checked and unfortunately
DirectByteBuffer in Java 9 holds an instance of jdk.internal.ref.Cleaner, not
java.lang.ref.Cleaner. The latter isn't available in Java 8 anyway. So we'd
have to hack this with reflection regardless.
> sun.misc.Unsafe in JDK9+
>
>
> Key: SPARK-24421
> URL: https://issues.apache.org/jira/browse/SPARK-24421
> Project: Spark
> Issue Type: Sub-task
> Components: Build
>Affects Versions: 2.3.0
>Reporter: DB Tsai
>Priority: Major
>
> Many internal APIs such as unsafe are encapsulated in JDK9+, see
> http://openjdk.java.net/jeps/260 for detail.
> To use Unsafe, we need to add *jdk.unsupported* to our code’s module
> declaration:
> {code:java}
> module java9unsafe {
> requires jdk.unsupported;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK9+
[
https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16532262#comment-16532262
]
vaquar khan commented on SPARK-24421:
-
1) I would suggest to use "Java Dependency Analysis Tool" to identify all code
for dependencies on the rt.jar and tools.jar files as both of them were removed
in Java 9
[https://wiki.openjdk.java.net/display/JDK8/Java+Dependency+Analysis+Tool]
2) Need to run code on Java 7 , 8 and 9 to identify backward-compatibility ,
Java 9 not given full backward-compatibility.
> sun.misc.Unsafe in JDK9+
>
>
> Key: SPARK-24421
> URL: https://issues.apache.org/jira/browse/SPARK-24421
> Project: Spark
> Issue Type: Sub-task
> Components: Build
>Affects Versions: 2.3.0
>Reporter: DB Tsai
>Priority: Major
>
> Many internal APIs such as unsafe are encapsulated in JDK9+, see
> http://openjdk.java.net/jeps/260 for detail.
> To use Unsafe, we need to add *jdk.unsupported* to our code’s module
> declaration:
> {code:java}
> module java9unsafe {
> requires jdk.unsupported;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK9+
[
https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16531705#comment-16531705
]
DB Tsai commented on SPARK-24421:
-
In JDK9+, `sun.misc.Unsafe` is private. We can either access it through
refection or add java flag to make it accessible.
> sun.misc.Unsafe in JDK9+
>
>
> Key: SPARK-24421
> URL: https://issues.apache.org/jira/browse/SPARK-24421
> Project: Spark
> Issue Type: Sub-task
> Components: Build
>Affects Versions: 2.3.0
>Reporter: DB Tsai
>Priority: Major
>
> Many internal APIs such as unsafe are encapsulated in JDK9+, see
> http://openjdk.java.net/jeps/260 for detail.
> To use Unsafe, we need to add *jdk.unsupported* to our code’s module
> declaration:
> {code:java}
> module java9unsafe {
> requires jdk.unsupported;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-24421) sun.misc.Unsafe in JDK9+
[
https://issues.apache.org/jira/browse/SPARK-24421?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16531235#comment-16531235
]
Andrew Korzhuev commented on SPARK-24421:
-
If I understand this correctly, then the only deprecated JDK9+ API Spark is
using is `sun.misc.Cleaner` (while `sun.misc.Unsafe` is still accessible) in
`[common/unsafe/src/main/java/org/apache/spark/unsafe/Platform.java|https://github.com/andrusha/spark/commit/7da06d3c725169f9764225f5a29886eb56bee191#diff-c7483c7efce631c783676f014ba2b0ed]`,
which is fixable in the following way:
{code:java}
@@ -22,7 +22,7 @@
import java.lang.reflect.Method;
import java.nio.ByteBuffer;
-import sun.misc.Cleaner;
+import java.lang.ref.Cleaner;
import sun.misc.Unsafe;
public final class Platform {
@@ -169,7 +169,8 @@ public static ByteBuffer allocateDirectBuffer(int size) {
cleanerField.setAccessible(true);
long memory = allocateMemory(size);
ByteBuffer buffer = (ByteBuffer) constructor.newInstance(memory, size);
- Cleaner cleaner = Cleaner.create(buffer, () -> freeMemory(memory));
+ Cleaner cleaner = Cleaner.create();
+ cleaner.register(buffer, () -> freeMemory(memory));
cleanerField.set(buffer, cleaner);
return buffer;
{code}
[https://github.com/andrusha/spark/commit/7da06d3c725169f9764225f5a29886eb56bee191#diff-c7483c7efce631c783676f014ba2b0ed]
> sun.misc.Unsafe in JDK9+
>
>
> Key: SPARK-24421
> URL: https://issues.apache.org/jira/browse/SPARK-24421
> Project: Spark
> Issue Type: Sub-task
> Components: Build
>Affects Versions: 2.3.0
>Reporter: DB Tsai
>Priority: Major
>
> Many internal APIs such as unsafe are encapsulated in JDK9+, see
> http://openjdk.java.net/jeps/260 for detail.
> To use Unsafe, we need to add *jdk.unsupported* to our code’s module
> declaration:
> {code:java}
> module java9unsafe {
> requires jdk.unsupported;
> }
> {code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
