[GitHub] spark pull request #20896: [SPARK-23788][SS] Fix race in StreamingQuerySuite

2018-03-23 Thread jose-torres
GitHub user jose-torres opened a pull request: https://github.com/apache/spark/pull/20896 [SPARK-23788][SS] Fix race in StreamingQuerySuite ## What changes were proposed in this pull request? The serializability test uses the same MemoryStream instance for 3 different

[GitHub] spark issue #20896: [SPARK-23788][SS] Fix race in StreamingQuerySuite

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20896 **[Test build #88555 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88555/testReport)** for PR 20896 at commit

[GitHub] spark issue #20896: [SPARK-23788][SS] Fix race in StreamingQuerySuite

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20896 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20896: [SPARK-23788][SS] Fix race in StreamingQuerySuite

2018-03-23 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20896 @zsxwing --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20884: [SPARK-23773][SQL] JacksonGenerator does not incl...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20884#discussion_r176897268 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonGeneratorSuite.scala --- @@ -56,7 +56,7 @@ class JacksonGeneratorSuite

[GitHub] spark issue #20895: [SPARK-23787][tests] Fix file download test in SparkSubm...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20895 **[Test build #88553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88553/testReport)** for PR 20895 at commit

[GitHub] spark issue #20895: [SPARK-23787][tests] Fix file download test in SparkSubm...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20895 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20895: [SPARK-23787][tests] Fix file download test in SparkSubm...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20895 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88553/ Test PASSed. ---

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r176897485 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -353,6 +353,13 @@ object SQLConf { .booleanConf

[GitHub] spark pull request #20884: [SPARK-23773][SQL] JacksonGenerator does not incl...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20884#discussion_r176897510 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -1229,7 +1229,7 @@ class JsonSuite extends

[GitHub] spark pull request #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark....

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20892#discussion_r176897639 --- Diff: python/pyspark/sql/functions.py --- @@ -28,10 +27,10 @@ from pyspark import since, SparkContext from pyspark.rdd import

[GitHub] spark issue #20879: [MINOR][R] Fix R lint failure

2018-03-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20879 thanks! those are on me then. are we not running lintr? or these checks are on the newer version only? --- - To

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20889 can you check if we have other typo in the md file? or similar type of typo in other md files? --- - To unsubscribe,

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20864 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20885 **[Test build #88539 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88539/testReport)** for PR 20885 at commit

[GitHub] spark issue #20888: [SPARK-23775][TEST] DataFrameRangeSuite should wait for ...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20888 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88536/ Test PASSed. ---

[GitHub] spark pull request #20873: [SPARK-22744][CORE] Cannot get the submit hostnam...

2018-03-23 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20873#discussion_r176646160 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -776,6 +776,9 @@ object SparkSubmit extends CommandLineUtils with Logging

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20864 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88537/ Test FAILed. ---

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20864 **[Test build #88537 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88537/testReport)** for PR 20864 at commit

[GitHub] spark pull request #20836: SPARK-23685 : Fix for the Spark Structured Stream...

2018-03-23 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/20836#discussion_r176656905 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaDataConsumer.scala --- @@ -279,9 +279,8 @@ private[kafka010]

[GitHub] spark pull request #20873: [SPARK-22744][CORE] Cannot get the submit hostnam...

2018-03-23 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/20873#discussion_r176647523 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -776,6 +776,9 @@ object SparkSubmit extends CommandLineUtils with Logging

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20885 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread Lemonjing
GitHub user Lemonjing opened a pull request: https://github.com/apache/spark/pull/20889 [MINOR][DOC] Fix ml-guide markdown typos ## What changes were proposed in this pull request? Easy fix in the markdown. ## How was this patch tested? Tested manually

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20889 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20873: [SPARK-22744][CORE] Cannot get the submit hostnam...

2018-03-23 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/20873#discussion_r176647022 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -776,6 +776,9 @@ object SparkSubmit extends CommandLineUtils with Logging

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20864 **[Test build #88537 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88537/testReport)** for PR 20864 at commit

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20860 **[Test build #88538 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88538/testReport)** for PR 20860 at commit

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1719/

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20836: SPARK-23685 : Fix for the Spark Structured Streaming Kaf...

2018-03-23 Thread gaborgsomogyi
Github user gaborgsomogyi commented on the issue: https://github.com/apache/spark/pull/20836 I don't see the problem. You see an exception which tells exactly what can be done: > If you don't want your streaming query to fail on such cases, set the source option

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread zuotingbing
Github user zuotingbing commented on the issue: https://github.com/apache/spark/pull/20864 @felixcheung Thanks. The test result is 0 failed, what is meaning of > This patch fails due to an unknown error code, -9 ? what should i do to fix the test fails? ---

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20889 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20657: [SPARK-23361][yarn] Allow AM to restart after ini...

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20657 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20873: [SPARK-22744][CORE] Cannot get the submit hostnam...

2018-03-23 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20873#discussion_r176647902 --- Diff: core/src/main/scala/org/apache/spark/deploy/SparkSubmit.scala --- @@ -776,6 +776,9 @@ object SparkSubmit extends CommandLineUtils with Logging

[GitHub] spark issue #20771: [SPARK-23587][SQL] Add interpreted execution for MapObje...

2018-03-23 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20771 ping @hvanhovell --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-23 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r176660685 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -353,6 +353,13 @@ object SQLConf { .booleanConf

[GitHub] spark issue #20888: [SPARK-23775][TEST] DataFrameRangeSuite should wait for ...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20888 **[Test build #88536 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88536/testReport)** for PR 20888 at commit

[GitHub] spark issue #20888: [SPARK-23775][TEST] DataFrameRangeSuite should wait for ...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20888 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20873: [SPARK-22744][CORE] Cannot get the submit hostname of ap...

2018-03-23 Thread LantaoJin
Github user LantaoJin commented on the issue: https://github.com/apache/spark/pull/20873 Close it while found a work around way. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20873: [SPARK-22744][CORE] Cannot get the submit hostnam...

2018-03-23 Thread LantaoJin
Github user LantaoJin closed the pull request at: https://github.com/apache/spark/pull/20873 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread Lemonjing
Github user Lemonjing commented on the issue: https://github.com/apache/spark/pull/20889 @felixcheung Yes, I read the entire docs of Spark Mlib. There is no problem with other mds (I may not have seen it or other contributors solved it). This error is obvious, and it affects the link

[GitHub] spark pull request #20860: [SPARK-23743][SQL] Changed a comparison logic fro...

2018-03-23 Thread jongyoul
Github user jongyoul commented on a diff in the pull request: https://github.com/apache/spark/pull/20860#discussion_r176652812 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala --- @@ -179,7 +179,7 @@ private[hive] class

[GitHub] spark pull request #20860: [SPARK-23743][SQL] Changed a comparison logic fro...

2018-03-23 Thread jongyoul
Github user jongyoul commented on a diff in the pull request: https://github.com/apache/spark/pull/20860#discussion_r176654007 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala --- @@ -179,7 +179,7 @@ private[hive] class

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20885 **[Test build #88539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88539/testReport)** for PR 20885 at commit

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20885 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88539/ Test FAILed. ---

[GitHub] spark pull request #20860: [SPARK-23743][SQL] Changed a comparison logic fro...

2018-03-23 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/20860#discussion_r176646535 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/IsolatedClientLoader.scala --- @@ -179,7 +179,7 @@ private[hive] class

[GitHub] spark issue #20864: [SPARK-23745][SQL]Remove the directories of the “hive....

2018-03-23 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20864 Jenkins, ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20851: [SPARK-23727][SQL] Support for pushing down filte...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20851#discussion_r176676405 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -353,6 +353,13 @@ object SQLConf { .booleanConf

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20811 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1720/

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20811 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20811 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1701/ ---

[GitHub] spark issue #20811: [SPARK-23668][K8S] Add config option for passing through...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20811 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-spark-integration/1701/ ---

[GitHub] spark issue #20880: [SPARK-23769][Core]Remove comments that unnecessarily di...

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20880 Merged to master and branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20880: [SPARK-23769][Core]Remove comments that unnecessa...

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20880 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20879: [MINOR][R] Fix R lint failure

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20879 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88538/ Test PASSed. ---

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20860 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20811: [SPARK-23668][K8S] Add config option for passing ...

2018-03-23 Thread andrusha
Github user andrusha commented on a diff in the pull request: https://github.com/apache/spark/pull/20811#discussion_r176687092 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodFactory.scala --- @@ -108,6 +109,8 @@

[GitHub] spark issue #20541: [SPARK-23356][SQL]Pushes Project to both sides of Union ...

2018-03-23 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/20541 oh, I see, I fallback to the modification of the non-deterministic expression, and to keep the newly added test cases for a+1 and a+b, can you? ---

[GitHub] spark issue #20889: [MINOR][DOC] Fix ml-guide markdown typos

2018-03-23 Thread Lemonjing
Github user Lemonjing commented on the issue: https://github.com/apache/spark/pull/20889 @felixcheung hi,In my reading process,I found another md file typos and fix it. Do I need to close this pr and merge it into one commit and give a new pr? Thanks. ---

[GitHub] spark issue #20885: [SPARK-23724][SPARK-23765][SQL] Line separator for the j...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20885 **[Test build #88540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88540/testReport)** for PR 20885 at commit

[GitHub] spark issue #20879: [MINOR][R] Fix R lint failure

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20879 Merged to master and branch-2.3. Thanks for reviewing this @shaneknapp and @felixcheung. --- - To unsubscribe,

[GitHub] spark issue #20879: [MINOR][R] Fix R lint failure

2018-03-23 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20879 Yup, we are running old lint in PR builders and those are on the newer one. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20860: [SPARK-23743][SQL] Changed a comparison logic from conta...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20860 **[Test build #88538 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88538/testReport)** for PR 20860 at commit

[GitHub] spark issue #20877: [SPARK-23765][SQL] Supports custom line separator for js...

2018-03-23 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/20877 > Does that fix actual usecases? I see the following use cases: 1. Jsons coming usually from embedded systems have not-standard separators (invisible in some cases). It is very

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176817667 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite extends

[GitHub] spark pull request #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet st...

2018-03-23 Thread BryanCutler
Github user BryanCutler commented on a diff in the pull request: https://github.com/apache/spark/pull/18982#discussion_r176825209 --- Diff: python/pyspark/ml/wrapper.py --- @@ -118,11 +118,18 @@ def _transfer_params_to_java(self): """ Transforms the

[GitHub] spark issue #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to Spark ...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19876 **[Test build #88546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88546/testReport)** for PR 19876 at commit

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176827527 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite extends

[GitHub] spark issue #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to Spark ...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark....

2018-03-23 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/20892 [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql ## What changes were proposed in this pull request? This cleans up unused imports, mainly from pyspark.sql module. Added a note

[GitHub] spark issue #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to Spark ...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19876 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1727/

[GitHub] spark issue #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20892 **[Test build #88547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88547/testReport)** for PR 20892 at commit

[GitHub] spark issue #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20892 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1728/

[GitHub] spark issue #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20892 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20892 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20892 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88547/ Test FAILed. ---

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2018-03-23 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/18982 Thanks @holdenk @HyukjinKwon and @viirya ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20717: [SPARK-23564][SQL] Add isNotNull check for left anti and...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20717 **[Test build #88542 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88542/testReport)** for PR 20717 at commit

[GitHub] spark pull request #20885: [SPARK-23724][SPARK-23765][SQL] Line separator fo...

2018-03-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20885#discussion_r176820312 --- Diff: python/pyspark/sql/readwriter.py --- @@ -770,12 +773,15 @@ def json(self, path, mode=None, compression=None, dateFormat=None, timestampForm

[GitHub] spark pull request #20885: [SPARK-23724][SPARK-23765][SQL] Line separator fo...

2018-03-23 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20885#discussion_r176821110 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JSONOptions.scala --- @@ -85,6 +85,38 @@ private[sql] class JSONOptions(

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-03-23 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r176821060 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -86,7 +88,80 @@ private[util] sealed trait BaseReadWrite { }

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread gerashegalov
Github user gerashegalov commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176822894 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20891 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20891 **[Test build #88545 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88545/testReport)** for PR 20891 at commit

[GitHub] spark issue #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql

2018-03-23 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20892 @HyukjinKwon and @ueshin I tried to be pretty conservative and only remove imports that were obviously not being used. ---

[GitHub] spark pull request #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet st...

2018-03-23 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/18982#discussion_r176828963 --- Diff: python/pyspark/ml/wrapper.py --- @@ -118,11 +118,18 @@ def _transfer_params_to_java(self): """ Transforms the embedded

[GitHub] spark issue #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet state sho...

2018-03-23 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/18982 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18982: [SPARK-21685][PYTHON][ML] PySpark Params isSet st...

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/18982 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20839: [SPARK-23699][PYTHON][SQL] Raise same type of error caug...

2018-03-23 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20839 Thanks @HyukjinKwon for reviewing! I agree with what you said about the rephrasing the warning message, I'll try to make that sound better. ---

[GitHub] spark pull request #19876: [ML][SPARK-23783][SPARK-11239] Add PMML export to...

2018-03-23 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19876 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20892: [SPARK-23700][PYTHON] Cleanup imports in pyspark.sql

2018-03-23 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20892 Jenkins retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20858: [SPARK-23736][SQL] Implementation of the concat_a...

2018-03-23 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/20858#discussion_r176841337 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/FunctionRegistry.scala --- @@ -408,6 +408,7 @@ object FunctionRegistry {

[GitHub] spark pull request #20884: [SPARK-23773][SQL] JacksonGenerator does not incl...

2018-03-23 Thread makagonov
Github user makagonov commented on a diff in the pull request: https://github.com/apache/spark/pull/20884#discussion_r176842732 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonGeneratorSuite.scala --- @@ -56,7 +56,7 @@ class JacksonGeneratorSuite

[GitHub] spark pull request #20327: [SPARK-12963][CORE] NM host for driver end points

2018-03-23 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20327#discussion_r176823881 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/YarnClusterSuite.scala --- @@ -136,6 +135,39 @@ class YarnClusterSuite extends

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20891 **[Test build #88545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88545/testReport)** for PR 20891 at commit

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20891 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20891: [SPARK-23782][CORE][UI] SHS should list only application...

2018-03-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20891 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1726/

[GitHub] spark issue #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to Spark ...

2018-03-23 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/19876 LGTM pending Jenkins will merge. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #19876: [ML][SPARK-11171][SPARK-11239] Add PMML export to...

2018-03-23 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/19876#discussion_r176826923 --- Diff: mllib/src/main/scala/org/apache/spark/ml/util/ReadWrite.scala --- @@ -86,7 +88,80 @@ private[util] sealed trait BaseReadWrite { }

  1   2   3   >