dongjoon-hyun opened a new pull request, #60114:
URL: https://github.com/apache/spark/pull/60114
### What changes were proposed in this pull request?
This PR replaces the APIs that were deprecated in Java 19 with their
non-deprecated counterparts, which are available because Spark now requires
Java 21:
- `Thread#getId()` -> `Thread#threadId()`: 37 call sites in 20 files
(`core`, `yarn`, `catalyst`, `sql/core`, `connect`, `pipelines`,
`hive-thriftserver`, and their tests). `getId()` is implemented as `return
threadId()` in Java 21, so the value is the same.
- `new Locale(...)` -> `Locale.of(...)`: 4 call sites in 3 files
(`StopWordsRemover`, `ExpressionImplUtils#getSentences`,
`QueryTest#withLocale`). The two forms were compared on 11,891 combinations of
language, country and variant, including malformed ones such as `en_US`, an
empty string and non-ASCII letters: the resulting locales were identical, and
both throw `NullPointerException` for `null`.
The following calls are intentionally left as they are, because there is no
replacement with the same behavior:
- `new URL(URL, String)` in `JettyUtils` (redirect handler) and `new
URL(String)` in `UISuite`. `URI.create(...).toURL()` and `URI#resolve` reject
characters that `URL` accepts (e.g., `|` or a space, and the `<` in the
`UISuite` request that checks the security filter), and `URI#resolve("")`
returns a different URL from `new URL(context, "")`.
- `ThreadDeath` in `BroadcastValueProjector` (deprecated for removal in Java
20, no replacement; the class can still be thrown by user code).
- `Object#finalize` and `Subject#doAs` in `hive-thriftserver` (deprecated
for removal in Java 18). Their replacements, `Cleaner` and `Subject#callAs`,
change the lifecycle and the exception wrapping, so they need a separate design.
- `sun.misc.Unsafe#objectFieldOffset`, `#staticFieldOffset` and
`#staticFieldBase` in `Platform.java` (deprecated in Java 18, no drop-in
replacement).
### Why are the changes needed?
SPARK-60669 raised the `-release` of `scalac` and `javac` to 21, so the
calls above now produce deprecation warnings at compile time. Replacing them
keeps the build free of warnings that have a simple fix.
### Does this PR introduce _any_ user-facing change?
No.
### How was this patch tested?
- Compiled all modules, including the optional ones (`-Pyarn -Pkubernetes
-Pvolcano -Phive -Phive-thriftserver -Phadoop-cloud -Pjvm-profiler
-Pspark-ganglia-lgpl -Pkinesis-asl -Pcredential-aws -Poidc-e2e
-Pkubernetes-integration-tests -Pdocker-integration-tests`), with `build/sbt
Test/compile` on JDK 21 and `-Xlint:deprecation` enabled for `javac` (not
committed). No warning remains for `Thread#getId` or the `Locale` constructors.
- Ran the suites that cover the changed code on JDK 21, all passed:
- `core`: `ExecutorSuite`, `ExternalSorterSuite`,
`ExternalAppendOnlyMapSuite`, `UnsafeExternalSorterSuite`,
`UnsafeShuffleWriterSuite`, `SparkHadoopUtilSuite`, `InputOutputMetricsSuite`,
and `RDDSuite -z SPARK-27666`
- `catalyst`: `AnalysisSuite`
- `sql`: `XmlSuite`, `SparkPlanSuite`, `SQLConfSuite`, `RocksDBSuite`,
`RocksDBStateStoreSuite`, `RocksDBStateStoreLockHardeningSuite`,
`RocksDBCheckpointFailureInjectionSuite`, `StringFunctionsSuite -z "string
sentences function"`, `InMemoryCatalogedDDLSuite -z "locale tr"`,
`DataSourceV2DataFrameSuite -z "Turkish locale fold collision"`
- `mllib`: `StopWordsRemoverSuite`
- `connect`: `MLSuite`
- `pipelines`: `GraphExecutionSuite`, `TriggeredGraphExecutionSuite`
- `yarn`: `ApplicationMasterSuite`
- `hive-thriftserver`: `ThriftServerWithSparkContextInBinarySuite`
- Ran `dev/scalastyle` and `dev/lint-java`.
### Was this patch authored or co-authored using generative AI tooling?
Generated-by: Claude Sonnet 5.5
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
