GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/12334
[SPARK-14574][BUILD] Stop cross-building pure Java modules; simplify build
target dirs
Spark has a few modules which do not depend on Scala, including `launcher`,
`unsafe`, `sketch`, and the `network-*` libraries. However, we currently cross
build and publish these artifacts for different Scala versions.
This patch refactors Spark's build so that pure-Java modules can be
published once and so that their artifact ids do not include `_2.xx` suffixes.
A consequence of this change is that pure Java modules' build output
directories will be `/target/classes` rather than `/target/scala-*/classes`.
For uniformity, I updated _all_ projects, including Scala ones, to use this
output directory convention. I think that the only reason to prefer SBT's
`crossPaths` convention is if you're using SBT's cross-build support (which,
AFAIK, doesn't currently work for Spark and isn't something that we're aiming
to support).
The `target` directory changes, in turn, allowed me to significantly
simplify the launcher code and removed the need to detect and set
`SPARK_SCALA_VERSION` in the launcher scripts. This simplification will have a
huge payoff when if we eventually have to simultaneously support Scala 2.10.x,
2.11.x, and 2.12.x, since there's now much less version detection logic to
maintain.
/cc @rxin @marmbrus @ahirreddy @srowen @vanzin
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark
remove-unnecessary-scala-exposure-in-poms
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/12334.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #12334
----
commit 42947032e565f6eccfa2ccceb26b00a3da7d9ad6
Author: Josh Rosen <[email protected]>
Date: 2016-04-12T19:02:46Z
Remove Scala version from parent POM artifact ID
commit 34d52d3b66cc9f50e0d5620cba37580716848bed
Author: Josh Rosen <[email protected]>
Date: 2016-04-12T20:11:46Z
WIP towards fixing artifact names and build output directories.
commit c7db9acd49179859cd8b1083923150e63bff5676
Author: Josh Rosen <[email protected]>
Date: 2016-04-12T20:22:01Z
Remove SPARK_SCALA_VERSION env. var. and clean up launcher code.
commit f4de68930339227bf414d4a27cbff9222fee322e
Author: Josh Rosen <[email protected]>
Date: 2016-04-12T20:37:33Z
Remove more references to target/scala-
commit 04161558a1097b634d459117dac66bd661b7cb79
Author: Josh Rosen <[email protected]>
Date: 2016-04-12T20:42:56Z
More updates in examples/
commit 449e93464b16b97c8af3806b73d96059cba5b45c
Author: Josh Rosen <[email protected]>
Date: 2016-04-12T20:49:11Z
Remove Scala download from build/mvn
commit acd738f33c1210bfae44d631c86fbd830a95d7d1
Author: Josh Rosen <[email protected]>
Date: 2016-04-12T20:53:11Z
Update spark-sketch artifactid and pom.
commit 944e8ab3da77a406fec9a8c1da118299f228a3e0
Author: Josh Rosen <[email protected]>
Date: 2016-04-12T20:53:53Z
More cleanup in make-distribution.sh
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
