GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/7219
[SPARK-8819] Fix build for maven 3.3.x
The goal of this patch is to remove the dependency on spark-core tests
across all modules. This is a workaround for MSHADE-148, which leads to the
following infinite loop when building Spark with maven 3.3.x:
```
[INFO] Replacing
/Users/andrew/Documents/dev/spark/andrew-spark/bagel/target/spark-bagel_2.10-1.5.0-SNAPSHOT.jar
with
/Users/andrew/Documents/dev/spark/andrew-spark/bagel/target/spark-bagel_2.10-1.5.0-SNAPSHOT-shaded.jar
[INFO] Dependency-reduced POM written at:
/Users/andrew/Documents/dev/spark/andrew-spark/bagel/dependency-reduced-pom.xml
[INFO] Dependency-reduced POM written at:
/Users/andrew/Documents/dev/spark/andrew-spark/bagel/dependency-reduced-pom.xml
...
```
To do this, we need to do the following three things:
- Move `SparkFunSuite` from the test module to the main module
- Revert #5983, which relied on this dependency that we're removing
- Remove `HiveSparkSubmitSuite`'s dependency on spark-core tests
This was originally caused by #6441, which added a bunch of test
dependencies on the spark-core test module. Recently, it was revealed by #7193.
**Lastly, note that this adds the scalatest dependency to the spark-core
main module for `SparkFunSuite`. In the future we should remove this dependency
once we figure out how to structure the test dependencies without hitting
MSHADE-148.**
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark fix-maven-build
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/7219.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #7219
----
commit 28170d8df04710e92ab99f039d6476ff8f0d64a7
Author: Andrew Or <[email protected]>
Date: 2015-07-04T04:03:19Z
Move SparkFunSuite to the main module
This is a workaround for MSHADE-148. This commit essentially
reverts the build changes introduced in SPARK-7558.
commit 0225324833f9e9dd88a8bbb7abd0923b3344fb0e
Author: Andrew Or <[email protected]>
Date: 2015-07-04T04:18:56Z
Revert "[SPARK-7444] [TESTS] Eliminate noisy css warn/error logs for
UISeleniumSuite"
This reverts commit 95690a17d328f205c3398b9b477b4072b6fe908f.
commit 77b43ece337b3fb735ebb91f237ae404dd370d0d
Author: Andrew Or <[email protected]>
Date: 2015-07-04T04:28:38Z
Remove sql/hive test dependency on spark-core
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
