GitHub user andrewor14 opened a pull request:
https://github.com/apache/spark/pull/6657
[SPARK-7205] [SPARK-7224] [SPARK-7306] Backport packages fixes
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/andrewor14/spark backport-pr-5892-1.3
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/6657.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #6657
----
commit e696c2133f15fa84734f4ac29ac44ad77983f8d8
Author: Burak Yavuz <[email protected]>
Date: 2015-04-29T06:05:02Z
[SPARK-7205] Support `.ivy2/local` and `.m2/repositories/` in --packages
In addition, I made a small change that will allow users to import 2
different artifacts with the same name. That change is made in
`[organization]_[artifact]-[revision].[ext]`. This used to be only
`[artifact].[ext]` which might have caused collisions between artifacts with
the same artifactId, but different groupId's.
cc pwendell
Author: Burak Yavuz <[email protected]>
Closes #5755 from brkyvz/local-caches and squashes the following commits:
c47c9c5 [Burak Yavuz] Small fixes to --packages
Conflicts:
core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
commit f4f7fa8488d6d7927ce10e5a267906360730a7c8
Author: Burak Yavuz <[email protected]>
Date: 2015-05-23T00:48:09Z
[SPARK-7224] [SPARK-7306] mock repository generator for --packages tests
without nio.Path
The previous PR for SPARK-7224 (#5790) broke JDK 6, because it used
java.nio.Path, which was in jdk 7, and not in 6. This PR uses Guava's `Files`
to handle directory creation, and etc...
The description from the previous PR:
> This patch contains an `IvyTestUtils` file, which dynamically generates
jars and pom files to test the `--packages` feature without having to rely on
the internet, and Maven Central.
cc pwendell
I also rand the flaky test about 20 times locally, it didn't fail a single
time, but I think it may fail like once every 100 builds? I still haven't
figured the cause yet, but the test before it, `--jars` was also failing after
we turned off the `--packages` test in `SparkSubmitSuite`. It may be related to
the launch of SparkSubmit.
Author: Burak Yavuz <[email protected]>
Closes #5892 from brkyvz/maven-utils and squashes the following commits:
e9b1903 [Burak Yavuz] fix merge conflict
68214e0 [Burak Yavuz] remove ignore for test(neglect spark dependencies)
e632381 [Burak Yavuz] fix ignore
9ef1408 [Burak Yavuz] re-enable --packages test
22eea62 [Burak Yavuz] Merge branch 'master' of github.com:apache/spark into
maven-utils
05cd0de [Burak Yavuz] added mock repository generator
Conflicts:
core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala
core/src/test/scala/org/apache/spark/deploy/SparkSubmitUtilsSuite.scala
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
