[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-11 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20799 @mridulm , would you please take a review. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For addit

[GitHub] spark pull request #20774: [SPARK-23549][SQL] Cast to timestamp when compari...

2018-03-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20774#discussion_r173701347 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercion.scala --- @@ -135,11 +135,14 @@ object TypeCoercion {

[GitHub] spark pull request #20757: [SPARK-23595][SQL] ValidateExternalType should su...

2018-03-11 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20757#discussion_r173700914 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -1421,13 +1421,36 @@ case class ValidateExterna

[GitHub] spark issue #20757: [SPARK-23595][SQL] ValidateExternalType should support i...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20757 **[Test build #88169 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88169/testReport)** for PR 20757 at commit [`6dc1fc2`](https://github.com/apache/spark/commit/6d

[GitHub] spark issue #20757: [SPARK-23595][SQL] ValidateExternalType should support i...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20757 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1460/ Tes

[GitHub] spark issue #20757: [SPARK-23595][SQL] ValidateExternalType should support i...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20757 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88167/ Test PASSed. ---

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88167 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88167/testReport)** for PR 20705 at commit [`159489c`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #20649: [SPARK-23462][SQL] improve missing field error message i...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20649 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88166/ Test PASSed. ---

[GitHub] spark issue #20649: [SPARK-23462][SQL] improve missing field error message i...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20649 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20649: [SPARK-23462][SQL] improve missing field error message i...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20649 **[Test build #88166 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88166/testReport)** for PR 20649 at commit [`78e037d`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #20795: [SPARK-23486]cache the function name from the cat...

2018-03-11 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/20795#discussion_r173692751 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1192,10 +1193,18 @@ class Analyzer( * @see ht

[GitHub] spark issue #20782: [SPARK-23627][SQL] Provide isEmpty in DataSet

2018-03-11 Thread goungoun
Github user goungoun commented on the issue: https://github.com/apache/spark/pull/20782 As unnecessary information is included, I closed this pull request. Please refer request #20800 instead of #20782. I am sorry for your inconvenience. --- -

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20799 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20799 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88168/ Test PASSed. ---

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20799 **[Test build #88168 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88168/testReport)** for PR 20799 at commit [`afd88e1`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20800: [SPARK-23627][SQL] Provide isEmpty in DataSet

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20800 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20800: [SPARK-23627][SQL] Provide isEmpty in DataSet

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20800 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20800: isEmpty in Dataset and its testSuite

2018-03-11 Thread goungoun
GitHub user goungoun opened a pull request: https://github.com/apache/spark/pull/20800 isEmpty in Dataset and its testSuite ## What changes were proposed in this pull request? This PR adds isEmpty() in DataSet ## How was this patch tested? Unit tests added

[GitHub] spark issue #20796: [SPARK-23649][SQL] Prevent crashes on schema inferring o...

2018-03-11 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20796 Could you add more tests for all the invalid range `253-255` in `UTF8StringSuite`? @HyukjinKwon Could you trigger this test? Also, kindly pinging cuz this is probably your domain? --- ---

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20799 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1459/ Tes

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20799 **[Test build #88168 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88168/testReport)** for PR 20799 at commit [`afd88e1`](https://github.com/apache/spark/commit/af

[GitHub] spark issue #20799: [SPARK-23635][YARN] AM env variable should not overwrite...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20799 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20799: [SPARK-23635][YARN] AM env variable should not ov...

2018-03-11 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/20799 [SPARK-23635][YARN] AM env variable should not overwrite same name env variable set through spark.executorEnv. ## What changes were proposed in this pull request? In the current Spark on

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1458/ Tes

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88167 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88167/testReport)** for PR 20705 at commit [`159489c`](https://github.com/apache/spark/commit/15

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20705 Retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark pull request #20757: [SPARK-23595][SQL] ValidateExternalType should su...

2018-03-11 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/20757#discussion_r173683323 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -1440,7 +1463,7 @@ case class ValidateExternalT

[GitHub] spark issue #20649: [SPARK-23462][SQL] improve missing field error message i...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20649 **[Test build #88166 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88166/testReport)** for PR 20649 at commit [`78e037d`](https://github.com/apache/spark/commit/78

[GitHub] spark issue #20649: [SPARK-23462][SQL] improve missing field error message i...

2018-03-11 Thread xysun
Github user xysun commented on the issue: https://github.com/apache/spark/pull/20649 Latest code review fixes pushed. cc @HyukjinKwon --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20788: [SPARK-23647][PYTHON][SQL] Adds more types for hi...

2018-03-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20788#discussion_r173682841 --- Diff: python/pyspark/sql/dataframe.py --- @@ -437,10 +437,11 @@ def hint(self, name, *parameters): if not isinstance(name, str):

[GitHub] spark pull request #20649: [SPARK-23462][SQL] improve missing field error me...

2018-03-11 Thread xysun
Github user xysun commented on a diff in the pull request: https://github.com/apache/spark/pull/20649#discussion_r173682838 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala --- @@ -271,7 +271,9 @@ case class StructType(fields: Array[StructField]) e

[GitHub] spark pull request #20797: [SPARK-23583][SQL] Invoke should support interpre...

2018-03-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20797#discussion_r173682156 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -266,8 +266,26 @@ case class Invoke( ove

[GitHub] spark pull request #20797: [SPARK-23583][SQL] Invoke should support interpre...

2018-03-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20797#discussion_r173682087 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/objects/objects.scala --- @@ -266,8 +266,26 @@ case class Invoke( ove

[GitHub] spark issue #20797: [SPARK-23583][SQL] Invoke should support interpreted exe...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20797 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20797: [SPARK-23583][SQL] Invoke should support interpreted exe...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20797 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88164/ Test PASSed. ---

[GitHub] spark issue #20797: [SPARK-23583][SQL] Invoke should support interpreted exe...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20797 **[Test build #88164 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88164/testReport)** for PR 20797 at commit [`3dbea12`](https://github.com/apache/spark/commit/3

[GitHub] spark issue #20798: [SPARK-23645][PYTHON] Allow python udfs to be called wit...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20798 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20798: [SPARK-23645][PYTHON] Allow python udfs to be called wit...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20798 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88165/ Test FAILed. ---

[GitHub] spark issue #20798: [SPARK-23645][PYTHON] Allow python udfs to be called wit...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20798 **[Test build #88165 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88165/testReport)** for PR 20798 at commit [`5ec810a`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #20798: [SPARK-23645][PYTHON] Allow python udfs to be called wit...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20798 **[Test build #88165 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88165/testReport)** for PR 20798 at commit [`5ec810a`](https://github.com/apache/spark/commit/5e

[GitHub] spark issue #20798: [SPARK-23645][PYTHON] Allow python udfs to be called wit...

2018-03-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20798 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20763 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20763 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88163/ Test FAILed. ---

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20763 **[Test build #88163 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88163/testReport)** for PR 20763 at commit [`c0ac5ef`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #20798: [SPARK-23645][PYTHON] Allow python udfs to be called wit...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20798 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20798: [SPARK-23645][PYTHON] Allow python udfs to be called wit...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20798 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20798: [SPARK-23645][PYTHON] Allow python udfs to be called wit...

2018-03-11 Thread mstewart141
Github user mstewart141 commented on the issue: https://github.com/apache/spark/pull/20798 [WIP] cc @HyukjinKwon 👍 i'd love to run tests here to make sure i haven't broken something. i will update pr with new tests once i set up testing better on my local box. ---

[GitHub] spark pull request #20798: [SPARK-23645][PYTHON] Allow python udfs to be cal...

2018-03-11 Thread mstewart141
GitHub user mstewart141 opened a pull request: https://github.com/apache/spark/pull/20798 [SPARK-23645][PYTHON] Allow python udfs to be called with keyword arguments ## [WIP] ## What changes were proposed in this pull request? Currently one can not pass keyword argu

[GitHub] spark issue #20331: [SPARK-23158] [SQL] Move HadoopFsRelationTest test suite...

2018-03-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20331 Please don't format [this comment](https://github.com/apache/spark/pull/20331#pullrequestreview-90358792) during next update, @gatorsmile . Thanks. --- -

[GitHub] spark issue #20797: [SPARK-23583][SQL] Invoke should support interpreted exe...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20797 **[Test build #88164 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88164/testReport)** for PR 20797 at commit [`3dbea12`](https://github.com/apache/spark/commit/3d

[GitHub] spark issue #20797: [SPARK-23583][SQL] Invoke should support interpreted exe...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20797 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20208: [SPARK-23007][SQL][TEST] Add schema evolution test suite...

2018-03-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20208 Thank you so much for review and approval, @HyukjinKwon ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.or

[GitHub] spark issue #20797: [SPARK-23583][SQL] Invoke should support interpreted exe...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20797 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1457/ Tes

[GitHub] spark pull request #20687: [SPARK-23500][SQL] Fix complex type simplificatio...

2018-03-11 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20687#discussion_r173667671 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -22,54 +22,34 @@ import org.apache.spark.sql

[GitHub] spark pull request #20797: [SPARK-23583][SQL] Invoke should support interpre...

2018-03-11 Thread kiszk
GitHub user kiszk opened a pull request: https://github.com/apache/spark/pull/20797 [SPARK-23583][SQL] Invoke should support interpreted execution ## What changes were proposed in this pull request? This pr added interpreted execution for `Invoke`. ## How was this p

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20763 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1456/ Tes

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20763 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20763 **[Test build #88163 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88163/testReport)** for PR 20763 at commit [`c0ac5ef`](https://github.com/apache/spark/commit/c0

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20763 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-11 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r173665102 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DriverConfigPropertiesStep.scala --- @@ -0,0 +1,85 @

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20763 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20763 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88162/ Test FAILed. ---

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20763 **[Test build #88162 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88162/testReport)** for PR 20763 at commit [`c0ac5ef`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #20683: [SPARK-8605] Exclude files in StreamingContext. textFile...

2018-03-11 Thread ConcurrencyPractitioner
Github user ConcurrencyPractitioner commented on the issue: https://github.com/apache/spark/pull/20683 @jerryshao In Spark Streaming, I think ```.tmp``` is used as a suffix to indicate that the object was a file, although I do not know if this is universal. --- ---

[GitHub] spark pull request #20669: [SPARK-22839][K8S] Remove the use of init-contain...

2018-03-11 Thread ifilonenko
Github user ifilonenko commented on a diff in the pull request: https://github.com/apache/spark/pull/20669#discussion_r173663339 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/submit/steps/DriverConfigPropertiesStep.scala --- @@ -0,0 +1,85 @

[GitHub] spark issue #20796: [SPARK-23649][SQL] Prevent crashes on schema inferring o...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20796 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20796: [SPARK-23649][SQL] Prevent crashes on schema inferring o...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20796 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20795: [SPARK-23486]cache the function name from the cat...

2018-03-11 Thread kevinyu98
Github user kevinyu98 commented on a diff in the pull request: https://github.com/apache/spark/pull/20795#discussion_r173662663 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1192,10 +1193,18 @@ class Analyzer( * @see ht

[GitHub] spark pull request #20796: [SPARK-23649][SQL] Prevent crashes on schema infe...

2018-03-11 Thread MaxGekk
GitHub user MaxGekk opened a pull request: https://github.com/apache/spark/pull/20796 [SPARK-23649][SQL] Prevent crashes on schema inferring of CSV containing wrong UTF-8 chars ## What changes were proposed in this pull request? The mapping of UTF-8 char's first byte to cha

[GitHub] spark pull request #20795: [SPARK-23486]cache the function name from the cat...

2018-03-11 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20795#discussion_r173660285 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1192,10 +1193,18 @@ class Analyzer( * @see h

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread kevinyu98
Github user kevinyu98 commented on the issue: https://github.com/apache/spark/pull/20795 @viirya Thanks a lot. I will create a new test file LookupFunctionsSuite under sql/catalyst/analysis. --- - To unsubscribe, e

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20763 **[Test build #88162 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88162/testReport)** for PR 20763 at commit [`c0ac5ef`](https://github.com/apache/spark/commit/c0

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20763 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1455/ Tes

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20763 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20763: [SPARK-23523] [SQL] [BACKPORT-2.3] Fix the incorrect res...

2018-03-11 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20763 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up t...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20793 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88160/ Test FAILed. ---

[GitHub] spark issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up t...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20793 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20793: [WIP][SPARK-23643] Shrinking the buffer in hashSeed up t...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20793 **[Test build #88160 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88160/testReport)** for PR 20793 at commit [`177afcc`](https://github.com/apache/spark/commit/1

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20795 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88161/ Test FAILed. ---

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20795 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20795 **[Test build #88161 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88161/testReport)** for PR 20795 at commit [`701100c`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20795 I've added a test like this: ```scala class LookupFunctionsSuite extends PlanTest { test("SPARK-23486: LookupFunctions should not check the same function name more than once")

[GitHub] spark pull request #20795: [SPARK-23486]cache the function name from the cat...

2018-03-11 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20795#discussion_r173655309 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1192,10 +1193,18 @@ class Analyzer( * @see https

[GitHub] spark issue #20788: [SPARK-23647][PYTHON][SQL] Adds more types for hint in p...

2018-03-11 Thread DylanGuedes
Github user DylanGuedes commented on the issue: https://github.com/apache/spark/pull/20788 Hi, I added two subtasks (one for Python and one for R) [here](https://issues.apache.org/jira/browse/SPARK-21030) Should I close this PR and open a new one or it is ok to just rename it

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20795 **[Test build #88161 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88161/testReport)** for PR 20795 at commit [`701100c`](https://github.com/apache/spark/commit/70

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20795 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20795 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20795 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20795 ok to test (seems uppercase not working somehow with Jenkins .. ) --- - To unsubscribe, e-mail: reviews-unsubsc

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-11 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20793 **[Test build #88160 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88160/testReport)** for PR 20793 at commit [`177afcc`](https://github.com/apache/spark/commit/17

[GitHub] spark issue #20795: [SPARK-23486]cache the function name from the catalog fo...

2018-03-11 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/20795 Ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-11 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20793 The question is that existing output of pseudo random/sample is guaranteed by public API. It seems it doesn't. --- - To unsubsc

[GitHub] spark issue #20793: [SPARK-23643] Shrinking the buffer in hashSeed up to siz...

2018-03-11 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20793 At least some tests expect that particular values would be result of sample/random: https://github.com/apache/spark/blob/master/sql/core/src/test/scala/org/apache/spark/sql/DatasetSuite.scala#L550-L