[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19370 Seems there is a conflict while backporting to branch-2.2. @jsnowacki, mind opening a backporting PR to branch-2.2 please? I think this is important for many Windows users and I guess re

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19370 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-22 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19370 Build started: [SparkR] `ALL` [![PR-19370](https://ci.appveyor.com/api/projects/status/github/spark-test/spark?branch=A5AAFE0C-D09F-4FBE-B834-811E8AC9FF06&svg=true)](https://ci.appveyor.com/proje

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-22 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84119/ Test PASSed. ---

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #84119 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84119/testReport)** for PR 19370 at commit [`b58f740`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-22 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #84119 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84119/testReport)** for PR 19370 at commit [`b58f740`](https://github.com/apache/spark/commit/b5

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-22 Thread jsnowacki
Github user jsnowacki commented on the issue: https://github.com/apache/spark/pull/19370 Thanks for looking into it again. I've followed your suggestions and updated PR. It also seems to work for me now. --- - To un

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19370 In `if not exist "%FIND_SPARK_HOME_PYTHON_SCRIPT%" (`, I switched the condition to be matched with https://github.com/apache/spark/blob/a36a76ac43c36a3b897a748bd9f138b629dbc684/bin/find-spark-ho

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-21 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19370 I suggest this: ```cmd rem Path to Python script finding SPARK_HOME set FIND_SPARK_HOME_PYTHON_SCRIPT=%~dp0find_spark_home.py rem Default to standard python interpreter u

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84077/ Test PASSed. ---

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #84077 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84077/testReport)** for PR 19370 at commit [`a4d516f`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #84077 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84077/testReport)** for PR 19370 at commit [`a4d516f`](https://github.com/apache/spark/commit/a4

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #84068 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84068/testReport)** for PR 19370 at commit [`84d468b`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84068/ Test PASSed. ---

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #84068 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84068/testReport)** for PR 19370 at commit [`84d468b`](https://github.com/apache/spark/commit/84

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84038/ Test PASSed. ---

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #84038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84038/testReport)** for PR 19370 at commit [`ed379b1`](https://github.com/apache/spark/commit/e

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #84038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84038/testReport)** for PR 19370 at commit [`ed379b1`](https://github.com/apache/spark/commit/ed

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-20 Thread jsnowacki
Github user jsnowacki commented on the issue: https://github.com/apache/spark/pull/19370 I've added this check using `where`; it is at least some protection against missing python with some more meaningful information returned. --- ---

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/19370 I think the concern is we are adding python dependency for even non python use cases. We could track handling missing python better separately. My concerns is python is not a standard c

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19370 Yup, +1 for going ahead with branch-2.2. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional co

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/84004/ Test PASSed. ---

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #84004 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84004/testReport)** for PR 19370 at commit [`c0138a9`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #84004 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/84004/testReport)** for PR 19370 at commit [`c0138a9`](https://github.com/apache/spark/commit/c0

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-18 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19370 Will take a final look tomorrow. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83809/ Test PASSed. ---

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #83809 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83809/testReport)** for PR 19370 at commit [`f25b330`](https://github.com/apache/spark/commit/f

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-13 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19370 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83806/ Test PASSed. ---

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #83806 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83806/testReport)** for PR 19370 at commit [`95037e5`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #83809 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83809/testReport)** for PR 19370 at commit [`f25b330`](https://github.com/apache/spark/commit/f2

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-13 Thread jsnowacki
Github user jsnowacki commented on the issue: https://github.com/apache/spark/pull/19370 I've corrected the code with the above comments and linked this PR to [SPARK-22495](https://issues.apache.org/jira/browse/SPARK-22495). The only thing I could think of regarding solving the varian

[GitHub] spark issue #19370: [SPARK-22495] Fix setup of SPARK_HOME variable on Window...

2017-11-13 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19370 **[Test build #83806 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83806/testReport)** for PR 19370 at commit [`95037e5`](https://github.com/apache/spark/commit/95