Github user nchammas commented on a diff in the pull request:
https://github.com/apache/spark/pull/2420#discussion_r17692117
--- Diff: dev/run-tests ---
@@ -70,42 +99,69 @@ echo ""
echo
"========================================================================="
echo "Running Apache RAT checks"
echo
"========================================================================="
-dev/check-license
+./dev/check-license
echo ""
echo
"========================================================================="
echo "Running Scala style checks"
echo
"========================================================================="
-dev/lint-scala
+./dev/lint-scala
echo ""
echo
"========================================================================="
echo "Running Python style checks"
echo
"========================================================================="
-dev/lint-python
+./dev/lint-python
+
+echo ""
+echo
"========================================================================="
+echo "Building Spark"
+echo
"========================================================================="
+
+{
+ # We always build with Hive because the PySpark Spark SQL tests need it.
+ BUILD_MVN_PROFILE_ARGS="$SBT_MAVEN_PROFILES_ARGS -Phive"
+
+ echo "[info] Building Spark with these arguments:
$BUILD_MVN_PROFILE_ARGS"
+
+ # NOTE: echo "q" is needed because sbt on encountering a build file with
failure
+ #+ (either resolution or compilation) prompts the user for input either
q, r, etc
+ #+ to quit or retry. This echo is there to make it not block.
+ # QUESTION: Why doesn't 'yes "q"' work?
+ # QUESTION: Why doesn't 'grep -v -e "^\[info\] Resolving"' work?
+ echo -e "q\n" \
+ | sbt/sbt $BUILD_MVN_PROFILE_ARGS clean package assembly/assembly \
+ | grep -v -e "info.*Resolving" -e "warn.*Merging" -e "info.*Including"
+}
echo ""
echo
"========================================================================="
echo "Running Spark unit tests"
echo
"========================================================================="
-# Build Spark; we always build with Hive because the PySpark Spark SQL
tests need it.
-# echo "q" is needed because sbt on encountering a build file with failure
-# (either resolution or compilation) prompts the user for input either q,
r,
-# etc to quit or retry. This echo is there to make it not block.
-BUILD_MVN_PROFILE_ARGS="$SBT_MAVEN_PROFILES_ARGS -Phive "
-echo -e "q\n" | sbt/sbt $BUILD_MVN_PROFILE_ARGS clean package
assembly/assembly | \
- grep -v -e "info.*Resolving" -e "warn.*Merging" -e "info.*Including"
-
-# If the Spark SQL tests are enabled, run the tests with the Hive profiles
enabled:
-if [ -n "$_RUN_SQL_TESTS" ]; then
- SBT_MAVEN_PROFILES_ARGS="$SBT_MAVEN_PROFILES_ARGS -Phive"
-fi
-# echo "q" is needed because sbt on encountering a build file with failure
-# (either resolution or compilation) prompts the user for input either q,
r,
-# etc to quit or retry. This echo is there to make it not block.
-echo -e "q\n" | sbt/sbt $SBT_MAVEN_PROFILES_ARGS test | \
- grep -v -e "info.*Resolving" -e "warn.*Merging" -e "info.*Including"
+{
+ # If the Spark SQL tests are enabled, run the tests with the Hive
profiles enabled.
+ if [ -n "$_RUN_SQL_TESTS" ]; then
+ SBT_MAVEN_PROFILES_ARGS="$SBT_MAVEN_PROFILES_ARGS -Phive"
+ fi
+
+ if [ -n "$_SQL_TESTS_ONLY" ]; then
+ SBT_MAVEN_TEST_ARGS="catalyst/test sql/test hive/test"
+ else
+ SBT_MAVEN_TEST_ARGS="test"
+ fi
+
+ echo "[info] Running Spark tests with these arguments:
$SBT_MAVEN_PROFILES_ARGS $SBT_MAVEN_TEST_ARGS"
+
+ # NOTE: echo "q" is needed because sbt on encountering a build file with
failure
+ #+ (either resolution or compilation) prompts the user for input either
q, r, etc
+ #+ to quit or retry. This echo is there to make it not block.
+ # QUESTION: Why doesn't 'yes "q"' work?
+ # QUESTION: Why doesn't 'grep -v -e "^\[info\] Resolving"' work?
+ echo -e "q\n" \
+ | sbt/sbt "$SBT_MAVEN_PROFILES_ARGS" "$SBT_MAVEN_TEST_ARGS" \
--- End diff --
`$SBT_MAVEN_TEST_ARGS` needs to be an array, otherwise this won't work as
expected.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
