GitHub user JoshRosen opened a pull request:
https://github.com/apache/spark/pull/10679
[SPARK-12612][PROJECT-INFRA] Backport dev/test-dependencies script to
branch-1.5
This patch backports the `dev/test-dependencies` script (from #10461) to
branch-1.5.
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/JoshRosen/spark test-deps-15-backport
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/spark/pull/10679.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #10679
----
commit 34e0b8aeafbcee1adcdf868b02a01d1df7d627f8
Author: Josh Rosen <[email protected]>
Date: 2015-12-25T23:07:05Z
[SPARK-12477][HOTIFX] Fix test compilation in branch-1.5
commit f06c6c1f47849852e317e463de79cc82f561877e
Author: Josh Rosen <[email protected]>
Date: 2015-12-30T20:47:42Z
[SPARK-10359] Enumerate dependencies in a file and diff against it for new
pull requests
This patch adds a new build check which enumerates Spark's resolved runtime
classpath and saves it to a file, then diffs against that file to detect
whether pull requests have introduced dependency changes. The aim of this check
is to make it simpler to reason about whether pull request which modify the
build have introduced new dependencies or changed transitive dependencies in a
way that affects the final classpath.
This supplants the checks added in SPARK-4123 / #5093, which are currently
disabled due to bugs.
This patch is based on pwendell's work in #8531.
Closes #8531.
Author: Josh Rosen <[email protected]>
Author: Patrick Wendell <[email protected]>
Closes #10461 from JoshRosen/SPARK-10359.
commit e74675e859deb9fa839362620931b37ba4840b36
Author: Josh Rosen <[email protected]>
Date: 2016-01-01T04:23:19Z
[SPARK-10359][PROJECT-INFRA] Multiple fixes to dev/test-dependencies.sh
script
This patch includes multiple fixes for the `dev/test-dependencies.sh`
script (which was introduced in #10461):
- Use `build/mvn --force` instead of `mvn` in one additional place.
- Explicitly set a zero exit code on success.
- Set `LC_ALL=C` to make `sort` results agree across machines (see
https://stackoverflow.com/questions/28881/).
- Set `should_run_build_tests=True` for `build` module (this somehow got
lost).
Author: Josh Rosen <[email protected]>
Closes #10543 from JoshRosen/dep-script-fixes.
commit f43d1bdaabd101580e1ca7da59ce2e3d46392360
Author: Josh Rosen <[email protected]>
Date: 2016-01-04T09:04:29Z
[SPARK-10359][PROJECT-INFRA] Use more random number in
dev/test-dependencies.sh; fix version switching
This patch aims to fix another potential source of flakiness in the
`dev/test-dependencies.sh` script.
pwendell's original patch and my version used `$(date +%s | tail -c6)` to
generate a suffix to use when installing temporary Spark versions into the
local Maven cache, but this value only changes once per second and thus is
highly collision-prone when concurrent builds launch on AMPLab Jenkins. In
order to reduce the potential for conflicts, this patch updates the script to
call Python's random number generator instead.
I also fixed a bug in how we captured the original project version; the bug
was causing the exit handler code to fail.
Author: Josh Rosen <[email protected]>
Closes #10558 from JoshRosen/build-dep-tests-round-3.
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
