wangyum opened a new pull request #25426: [SPARK-28703][SQL][TEST] Skip test HiveExternalCatalogVersionsSuite and skip 3 tests in HiveSparkSubmitSuite when testing with JAVA_9 or later URL: https://github.com/apache/spark/pull/25426 ## What changes were proposed in this pull request? This PR skip more test when testing with `JAVA_9` or later: 1. Skip test `HiveExternalCatalogVersionsSuite` when testing with `JAVA_9` or later because our previous version does not support `JAVA_9` or later. 2. Skip 3 tests in `HiveSparkSubmitSuite` because the `spark.sql.hive.metastore.version` of these tests is lower than `2.0`, however Datanucleus 3.x seem does not support `JAVA_9` or later. Hive upgrade Datanucleus to 4.x from Hive 2.0([HIVE-6113](https://issues.apache.org/jira/browse/HIVE-6113)): ``` [info] Cause: org.datanucleus.exceptions.NucleusException: The java type java.lang.Long (jdbc-type="", sql-type="") cant be mapped for this datastore. No mapping is available. [info] at org.datanucleus.store.rdbms.mapping.RDBMSMappingManager.getDatastoreMappingClass(RDBMSMappingManager.java:1215) [info] at org.datanucleus.store.rdbms.mapping.RDBMSMappingManager.createDatastoreMapping(RDBMSMappingManager.java:1378) [info] at org.datanucleus.store.rdbms.table.AbstractClassTable.addDatastoreId(AbstractClassTable.java:392) [info] at org.datanucleus.store.rdbms.table.ClassTable.initializePK(ClassTable.java:1087) [info] at org.datanucleus.store.rdbms.table.ClassTable.preInitialize(ClassTable.java:247) ``` ## How was this patch tested? manual tests: Test with JDK 11: ``` [info] HiveExternalCatalogVersionsSuite: [info] - backward compatibility !!! CANCELED !!! (37 milliseconds) [info] HiveSparkSubmitSuite: ... [info] - SPARK-8020: set sql conf in spark conf !!! CANCELED !!! (30 milliseconds) [info] org.apache.commons.lang3.SystemUtils.isJavaVersionAtLeast(JAVA_9) was true (HiveSparkSubmitSuite.scala:130) ... [info] - SPARK-9757 Persist Parquet relation with decimal column !!! CANCELED !!! (1 millisecond) [info] org.apache.commons.lang3.SystemUtils.isJavaVersionAtLeast(JAVA_9) was true (HiveSparkSubmitSuite.scala:168) ... [info] - SPARK-16901: set javax.jdo.option.ConnectionURL !!! CANCELED !!! (1 millisecond) [info] org.apache.commons.lang3.SystemUtils.isJavaVersionAtLeast(JAVA_9) was true (HiveSparkSubmitSuite.scala:260) ... ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] With regards, Apache Git Services --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
