Re: Spark on JDK 14
Java 16 will also includes Vector API (incubator), which is a part of Project Panama, as shown in https://mail.openjdk.java.net/pipermail/panama-dev/2020-October/011149.html When the next LTS will be available, we could exploit it in Spark. Kazuaki Ishizaki From: Dongjoon Hyun To: Sean Owen Cc: dev Date: 2020/10/29 11:34 Subject:Re: Spark on JDK 14 Thank you for the sharing, Sean. Although Java 14 is already EOL (Sep. 2020), that is important information because we are tracking the Java upstream. Bests, Dongjoon. On Wed, Oct 28, 2020 at 1:44 PM Sean Owen wrote: For kicks, I tried Spark on JDK 14. 11 -> 14 doesn't change much, not as much as 8 -> 9 (-> 11), and indeed, virtually all tests pass. For the interested, these two seem to fail: - ZooKeeperPersistenceEngine *** FAILED *** org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /spark/master_status - parsing hour with various patterns *** FAILED *** java.time.format.DateTimeParseException: Text '2009-12-12 12 am' could not be parsed: Invalid value for HourOfAmPm (valid values 0 - 11): 12 I'd expect that most applications would just work now on Spark 3 + Java 14. I'd guess the same is true for Java 16 even, but, we're probably focused on the LTS releases. Kris Mok pointed out that Project Panama (in Java 17 maybe?) might have implications as it changes off-heap memory access.
Re: Spark on JDK 14
Thank you for the sharing, Sean. Although Java 14 is already EOL (Sep. 2020), that is important information because we are tracking the Java upstream. Bests, Dongjoon. On Wed, Oct 28, 2020 at 1:44 PM Sean Owen wrote: > For kicks, I tried Spark on JDK 14. 11 -> 14 doesn't change much, not as > much as 8 -> 9 (-> 11), and indeed, virtually all tests pass. For the > interested, these two seem to fail: > > - ZooKeeperPersistenceEngine *** FAILED *** > org.apache.zookeeper.KeeperException$ConnectionLossException: > KeeperErrorCode = ConnectionLoss for /spark/master_status > > - parsing hour with various patterns *** FAILED *** > java.time.format.DateTimeParseException: Text '2009-12-12 12 am' could > not be parsed: Invalid value for HourOfAmPm (valid values 0 - 11): 12 > > I'd expect that most applications would just work now on Spark 3 + Java > 14. I'd guess the same is true for Java 16 even, but, we're probably > focused on the LTS releases. > > Kris Mok pointed out that Project Panama (in Java 17 maybe?) might have > implications as it changes off-heap memory access. >
Spark on JDK 14
For kicks, I tried Spark on JDK 14. 11 -> 14 doesn't change much, not as much as 8 -> 9 (-> 11), and indeed, virtually all tests pass. For the interested, these two seem to fail: - ZooKeeperPersistenceEngine *** FAILED *** org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /spark/master_status - parsing hour with various patterns *** FAILED *** java.time.format.DateTimeParseException: Text '2009-12-12 12 am' could not be parsed: Invalid value for HourOfAmPm (valid values 0 - 11): 12 I'd expect that most applications would just work now on Spark 3 + Java 14. I'd guess the same is true for Java 16 even, but, we're probably focused on the LTS releases. Kris Mok pointed out that Project Panama (in Java 17 maybe?) might have implications as it changes off-heap memory access.
