This is an automated email from the ASF dual-hosted git repository.
dongjoon-hyun pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/spark.git
The following commit(s) were added to refs/heads/master by this push:
new c17c63e5a2c9 [SPARK-52822][BUILD] Upgrade Derby to 10.17.1.0
c17c63e5a2c9 is described below
commit c17c63e5a2c9a90a13eea33ec7efd805e4ef3515
Author: Dongjoon Hyun <[email protected]>
AuthorDate: Tue Nov 3 14:40:37 2026 -0800
[SPARK-52822][BUILD] Upgrade Derby to 10.17.1.0
### What changes were proposed in this pull request?
This PR aims to upgrade `Apache Derby` from `10.16.1.1` to `10.17.1.0` for
Apache Spark 5.0.0.
- https://db.apache.org/derby/releases/release-10_17_1_0.cgi (2023-11-10)
### Why are the changes needed?
Since Apache Spark 5.0.0 requires Java 21 or later (SPARK-60669), we can
use `Derby 10.17.x`, which supports only Java 21 and later. Previously, we
stayed on `Derby 10.16.1.1` because `10.17.1.0` requires Java 21+.
- https://github.com/apache/spark/pull/59897
`Derby 10.17.1.0` brings the following fixes.
- [DERBY-7143](https://issues.apache.org/jira/browse/DERBY-7143)
HarmonySerialBlob.getBinaryStream(long, long) makes it impossible to retrieve
the last character of the Blob
- [DERBY-7144](https://issues.apache.org/jira/browse/DERBY-7144) MERGE
INSERT failing when target has GENERATED IDENTITY column
- [DERBY-7147](https://issues.apache.org/jira/browse/DERBY-7147) LDAP
injection vulnerability in LDAPAuthenticationImpl (CVE-2022-46337)
- [DERBY-7149](https://issues.apache.org/jira/browse/DERBY-7149) Make it
possible to build and test Derby cleanly with JDK 20
In addition, `10.17.1.0` is the final release of Apache Derby because the
Derby developers voted to retire the project on 2025-10-10.
### Does this PR introduce _any_ user-facing change?
No behavior change is expected. Only the bundled Derby jars (`derby`,
`derbyshared`, `derbytools`) are upgraded.
### How was this patch tested?
Pass the CIs.
In addition, I manually ran the test suites that use Derby on Java 21.
```
$ build/sbt -Phive -Phive-thriftserver \
"core/testOnly *JdbcRDDSuite" \
"sql/testOnly *DerbyTableCatalogSuite *JDBCSuite" \
"hive/testOnly *HiveSharedStateSuite *HiveExternalCatalogSuite" \
"hive-thriftserver/testOnly *SparkSQLEnvSuite *CliSuite"
```
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Opus 5.5
Closes #60065 from dongjoon-hyun/dongjoon/derby-upgrade-9b7bf2.
Authored-by: Dongjoon Hyun <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>
---
dev/deps/spark-deps-hadoop-3-hive-2.3 | 6 +++---
pom.xml | 3 +--
2 files changed, 4 insertions(+), 5 deletions(-)
diff --git a/dev/deps/spark-deps-hadoop-3-hive-2.3
b/dev/deps/spark-deps-hadoop-3-hive-2.3
index fd1617c83e60..d134d0dda2fe 100644
--- a/dev/deps/spark-deps-hadoop-3-hive-2.3
+++ b/dev/deps/spark-deps-hadoop-3-hive-2.3
@@ -64,9 +64,9 @@ datanucleus-core/4.1.17//datanucleus-core-4.1.17.jar
datanucleus-rdbms/4.1.19//datanucleus-rdbms-4.1.19.jar
datasketches-java/6.2.0//datasketches-java-6.2.0.jar
datasketches-memory/3.0.2//datasketches-memory-3.0.2.jar
-derby/10.16.1.1//derby-10.16.1.1.jar
-derbyshared/10.16.1.1//derbyshared-10.16.1.1.jar
-derbytools/10.16.1.1//derbytools-10.16.1.1.jar
+derby/10.17.1.0//derby-10.17.1.0.jar
+derbyshared/10.17.1.0//derbyshared-10.17.1.0.jar
+derbytools/10.17.1.0//derbytools-10.17.1.0.jar
dom4j/2.1.4//dom4j-2.1.4.jar
dropwizard-metrics-hadoop-metrics2-reporter/0.1.2//dropwizard-metrics-hadoop-metrics2-reporter-0.1.2.jar
failureaccess/1.0.3//failureaccess-1.0.3.jar
diff --git a/pom.xml b/pom.xml
index 9b16ddf3fba9..74a82a3b24d9 100644
--- a/pom.xml
+++ b/pom.xml
@@ -143,8 +143,7 @@
<hive.version>2.3.10</hive.version>
<!-- note that this should be compatible with Kafka brokers version 0.10
and up -->
<kafka.version>3.9.2</kafka.version>
- <!-- After 10.17.1.0, the minimum required version is JDK19 -->
- <derby.version>10.16.1.1</derby.version>
+ <derby.version>10.17.1.0</derby.version>
<parquet.version>1.18.1</parquet.version>
<orc.version>2.3.1</orc.version>
<orc.classifier>shaded-protobuf</orc.classifier>
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]