[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-06-09 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/20909 @HyukjinKwon This PR is mostly obsolete. I will close it and re-open something smaller... maybe a one-line documentation change to handle the missing UDF case for those who build with sbt.

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20909 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91612/ Test FAILed. ---

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-06-09 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20909 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20909 **[Test build #91612 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91612/testReport)** for PR 20909 at commit

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-06-09 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20909 **[Test build #91612 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91612/testReport)** for PR 20909 at commit

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-06-09 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20909 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-06-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20909 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-12 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20909 cc @viirya too. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20909 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20909 **[Test build #88863 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88863/testReport)** for PR 20909 at commit

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20909 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88863/ Test PASSed. ---

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20909 **[Test build #88863 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88863/testReport)** for PR 20909 at commit

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-02 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/20909 > can you investigate and try to explicitly skip some doctests conditionally? @HyukjinKwon I will take a look to see how that can be done. ---

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20909 Yea, I know the hidden output in the console and I believe that's a known issue. In my case, I made such change before - https://github.com/apache/spark/pull/20487. Also see the discussion in

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-02 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/20909 @HyukjinKwon That makes sense. Note that when the tests are run using python/run-tests, run-tests.py steals stdout and stderr. I would need to make a small change to run-tests.py to

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20909 @holdenk, I just saw https://issues.apache.org/jira/browse/SPARK-23853. I think this PR could fix it together if I understood correctly :-). ---

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-02 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20909 Yes, I feel sure that's more consistent and correct. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-02 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/20909 > actually have been thinking about skipping and proceeding the tests in pyspark/streaming/tests.py with an explicit message as well. Can we skip and continue the tests? Hi

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-04-01 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20909 > I modeled this after pyspark/streaming/tests.py, which checks for prereqs and raises exceptions with a useful message so one can get past the error (although pyspark/streaming/tests.py only

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20909 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88736/ Test PASSed. ---

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-29 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20909 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20909 **[Test build #88736 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88736/testReport)** for PR 20909 at commit

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-29 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20909 **[Test build #88736 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88736/testReport)** for PR 20909 at commit

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-28 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/20909 @HyukjinKwon The HiveSparkSubmitTests error message is [here](https://gist.github.com/bersprockets/746ab6f2353bd5b54404d5cbb1df7403) I propose the following: - Fix

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-28 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20909 maybe the approach in hiveContextSQLTests is better for HiveSparkSubmitTests?

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20909 @bersprockets, do you have the error messages? I could (will) check it by myself in the following week but want to take a quick look if you already have them. ---

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-27 Thread bersprockets
Github user bersprockets commented on the issue: https://github.com/apache/spark/pull/20909 Thanks @felixcheung . Turns out HiveSparkSubmitTests will fail if Spark is not built with the hive profile (AssertionError: 0 != 1). In addition, at least one pyspark.sql.readwriter

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20909 **[Test build #88606 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88606/testReport)** for PR 20909 at commit

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20909 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88606/ Test PASSed. ---

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20909 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20909 **[Test build #88606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88606/testReport)** for PR 20909 at commit

[GitHub] spark issue #20909: [SPARK-23776][python][test] Check for needed components/...

2018-03-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20909 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional