[
https://issues.apache.org/jira/browse/SPARK-45144?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-45144.
-----------------------------------
Fix Version/s: 4.0.0
Resolution: Fixed
Issue resolved by pull request 42899
[https://github.com/apache/spark/pull/42899]
> Downgrade scala-maven-plugin to 4.7.1
> -------------------------------------
>
> Key: SPARK-45144
> URL: https://issues.apache.org/jira/browse/SPARK-45144
> Project: Spark
> Issue Type: Improvement
> Components: Build
> Affects Versions: 4.0.0
> Reporter: Yang Jie
> Assignee: Yang Jie
> Priority: Major
> Labels: pull-request-available
> Fix For: 4.0.0
>
>
> The `scala-maven-plugin` versions 4.7.2 and later will try to automatically
> append the `-release` option as a Scala compilation argument when it is not
> specified by the user:
> 1. 4.7.2 and 4.8.0: Try to add the `-release` option for Scala versions
> 2.13.9 and higher.
> 2. 4.8.1: Try to append the `-release` option for Scala versions
> 2.12.x/2.13.x/3.1.1, and append `-java-output-version` for Scala 3.1.2.
> The addition of the `-release` option has caused issues mentioned in
> SPARK-44376 | https://github.com/apache/spark/pull/41943 and
> https://github.com/apache/spark/pull/40442#issuecomment-1474161688. This is
> because the `-release` option has stronger compilation restrictions than
> `-target`, ensuring not only bytecode format, but also that the API used in
> the code is compatible with the specified version of Java. However, many APIs
> in the `sun.*` package are not `exports` in Java 11, 17, and 21, such as
> `sun.nio.ch.DirectBuffer`, `sun.util.calendar.ZoneInfo`, and
> `sun.nio.cs.StreamDecoder`, making them invisible when compiling across
> different versions.
> For discussions within the Scala community, see
> https://github.com/scala/bug/issues/12643,
> https://github.com/scala/bug/issues/12824,
> https://github.com/scala/bug/issues/12866.
> I have also submitted an issue to the `scala-maven-plugin` community to
> discuss the possibility of adding additional settings to control the addition
> of the `-release` option:
> https://github.com/davidB/scala-maven-plugin/issues/722.
> For Apache Spark 4.0, in the short term, I suggest downgrading
> `scala-maven-plugin` to version 4.7.1 to avoid the automatic adding the
> `-release` option as a Scala compilation argument. In the long term, we
> should reduce our use of APIs that are not `exports` for compatibility with
> the `-release` compilation option.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
