[GitHub] spark pull request #21369: [SPARK-22713][CORE] ExternalAppendOnlyMap leaks w...

2018-06-03 Thread eyalfa
Github user eyalfa commented on a diff in the pull request: https://github.com/apache/spark/pull/21369#discussion_r192631230 --- Diff: core/src/test/scala/org/apache/spark/util/collection/ExternalAppendOnlyMapSuite.scala --- @@ -414,7 +415,106 @@ class ExternalAppendOnlyMapSuite

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3789/

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #91437 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91437/testReport)** for PR 13599 at commit

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21156 **[Test build #91436 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91436/testReport)** for PR 21156 at commit

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21156 **[Test build #91435 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91435/testReport)** for PR 21156 at commit

[GitHub] spark issue #15297: [SPARK-9862]Handling data skew

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15297 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-06-03 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21313#discussion_r192627641 --- Diff: R/pkg/R/functions.R --- @@ -3006,6 +3008,27 @@ setMethod("array_contains", column(jc) }) +#'

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-06-03 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/21313#discussion_r192627948 --- Diff: R/pkg/R/functions.R --- @@ -3006,6 +3008,27 @@ setMethod("array_contains", column(jc) }) +#'

[GitHub] spark pull request #20894: [SPARK-23786][SQL] Checking column names of csv h...

2018-06-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20894 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20894: [SPARK-23786][SQL] Checking column names of csv headers

2018-06-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20894 LGTM Thanks! Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21487: [SPARK-24369][SQL] Correct handling for multiple ...

2018-06-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21487 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21487 Thanks! Merged to master and 2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21156: [SPARK-24087][SQL] Avoid shuffle when join keys are a su...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21156 **[Test build #91434 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91434/testReport)** for PR 21156 at commit

[GitHub] spark pull request #21486: [SPARK-24387][Core] Heartbeat-timeout executor is...

2018-06-03 Thread lirui-apache
Github user lirui-apache commented on a diff in the pull request: https://github.com/apache/spark/pull/21486#discussion_r192614888 --- Diff: core/src/main/scala/org/apache/spark/HeartbeatReceiver.scala --- @@ -197,14 +197,14 @@ private[spark] class HeartbeatReceiver(sc:

[GitHub] spark pull request #21486: [SPARK-24387][Core] Heartbeat-timeout executor is...

2018-06-03 Thread lirui-apache
Github user lirui-apache commented on a diff in the pull request: https://github.com/apache/spark/pull/21486#discussion_r192614874 --- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala --- @@ -207,6 +210,55 @@ class HeartbeatReceiverSuite

[GitHub] spark pull request #21486: [SPARK-24387][Core] Heartbeat-timeout executor is...

2018-06-03 Thread lirui-apache
Github user lirui-apache commented on a diff in the pull request: https://github.com/apache/spark/pull/21486#discussion_r192614878 --- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala --- @@ -207,6 +210,55 @@ class HeartbeatReceiverSuite

[GitHub] spark issue #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration ...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21467 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91433/ Test FAILed. ---

[GitHub] spark issue #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration ...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21467 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration ...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21467 **[Test build #91433 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91433/testReport)** for PR 21467 at commit

[GitHub] spark issue #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration ...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21467 **[Test build #91433 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91433/testReport)** for PR 21467 at commit

[GitHub] spark issue #21467: [SPARK-23754][PYTHON][FOLLOWUP] Move UDF stop iteration ...

2018-06-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21467 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-06-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192610559 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +354,70 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-06-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192610390 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +354,70 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-06-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192610512 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +354,70 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-06-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192610308 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +354,70 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-06-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192610839 --- Diff: python/pyspark/sql/tests.py --- @@ -3040,6 +3040,36 @@ def test_csv_sampling_ratio(self): .csv(rdd, samplingRatio=0.5).schema

[GitHub] spark pull request #21370: [SPARK-24215][PySpark] Implement _repr_html_ for ...

2018-06-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21370#discussion_r192610620 --- Diff: python/pyspark/sql/dataframe.py --- @@ -351,8 +354,70 @@ def show(self, n=20, truncate=True, vertical=False): else:

[GitHub] spark pull request #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl...

2018-06-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21485 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21485 Merged to master and branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21487 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3788/

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21488 **[Test build #91432 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91432/testReport)** for PR 21488 at commit

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91432/ Test FAILed. ---

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21488 **[Test build #91432 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91432/testReport)** for PR 21488 at commit

[GitHub] spark issue #21313: [SPARK-24187][R][SQL]Add array_join function to SparkR

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21313 **[Test build #91430 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91430/testReport)** for PR 21313 at commit

[GitHub] spark issue #21313: [SPARK-24187][R][SQL]Add array_join function to SparkR

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21313 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21313: [SPARK-24187][R][SQL]Add array_join function to SparkR

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21313 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91430/ Test PASSed. ---

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3787/

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21488 **[Test build #91431 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91431/testReport)** for PR 21488 at commit

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91431/ Test FAILed. ---

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21488 **[Test build #91431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91431/testReport)** for PR 21488 at commit

[GitHub] spark issue #21313: [SPARK-24187][R][SQL]Add array_join function to SparkR

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21313 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3786/

[GitHub] spark issue #21313: [SPARK-24187][R][SQL]Add array_join function to SparkR

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21313 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21485 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91428/ Test PASSed. ---

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21485 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21313: [SPARK-24187][R][SQL]Add array_join function to SparkR

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21313 **[Test build #91430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91430/testReport)** for PR 21313 at commit

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21485 **[Test build #91428 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91428/testReport)** for PR 21485 at commit

[GitHub] spark pull request #21488: SPARK-18057 Update structured streaming kafka fro...

2018-06-03 Thread ijuma
Github user ijuma commented on a diff in the pull request: https://github.com/apache/spark/pull/21488#discussion_r192602632 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala --- @@ -96,10 +101,13 @@ class

[GitHub] spark pull request #21488: SPARK-18057 Update structured streaming kafka fro...

2018-06-03 Thread tedyu
Github user tedyu commented on a diff in the pull request: https://github.com/apache/spark/pull/21488#discussion_r192601997 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala --- @@ -96,10 +101,13 @@ class

[GitHub] spark pull request #21488: SPARK-18057 Update structured streaming kafka fro...

2018-06-03 Thread ijuma
Github user ijuma commented on a diff in the pull request: https://github.com/apache/spark/pull/21488#discussion_r192601219 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaTestUtils.scala --- @@ -96,10 +101,13 @@ class

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3785/

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21488 **[Test build #91429 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91429/testReport)** for PR 21488 at commit

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21488 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91429/ Test FAILed. ---

[GitHub] spark issue #21488: SPARK-18057 Update structured streaming kafka from 0.10....

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21488 **[Test build #91429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91429/testReport)** for PR 21488 at commit

[GitHub] spark pull request #21488: SPARK-18057 Update structured streaming kafka fro...

2018-06-03 Thread tedyu
GitHub user tedyu opened a pull request: https://github.com/apache/spark/pull/21488 SPARK-18057 Update structured streaming kafka from 0.10.0.1 to 2.0.0 ## What changes were proposed in this pull request? This PR upgrades to the Kafka 2.0.0 release where KIP-266 is

[GitHub] spark issue #21398: [SPARK-24338][SQL] Fixed Hive CREATETABLE error in Sentr...

2018-06-03 Thread skonto
Github user skonto commented on the issue: https://github.com/apache/spark/pull/21398 @vanzin should we merge this or add a comment in the docs? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21485 **[Test build #91428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91428/testReport)** for PR 21485 at commit

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21485 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21485: [SPARK-24455][CORE] fix typo in TaskSchedulerImpl commen...

2018-06-03 Thread xueyumusic
Github user xueyumusic commented on the issue: https://github.com/apache/spark/pull/21485 I took another look and find some typos, please review them, @HyukjinKwon , thank you for reminding --- - To unsubscribe,

[GitHub] spark pull request #21486: [SPARK-24387][Core] Heartbeat-timeout executor is...

2018-06-03 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/21486#discussion_r192592142 --- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala --- @@ -207,6 +210,55 @@ class HeartbeatReceiverSuite

[GitHub] spark pull request #21486: [SPARK-24387][Core] Heartbeat-timeout executor is...

2018-06-03 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/21486#discussion_r192592170 --- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala --- @@ -207,6 +210,55 @@ class HeartbeatReceiverSuite

[GitHub] spark pull request #21486: [SPARK-24387][Core] Heartbeat-timeout executor is...

2018-06-03 Thread Ngone51
Github user Ngone51 commented on a diff in the pull request: https://github.com/apache/spark/pull/21486#discussion_r192591845 --- Diff: core/src/test/scala/org/apache/spark/HeartbeatReceiverSuite.scala --- @@ -207,6 +210,55 @@ class HeartbeatReceiverSuite

[GitHub] spark issue #21449: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-03 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21449 This will definitely not go into 2.3.1, so we have plenty of time. I'll think deeper into it after the spark summit. IMO `df.join(df, df("id") >= df("id"))` is ambiguous, especially when

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21487 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91427/ Test PASSed. ---

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21487 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21487 **[Test build #91427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91427/testReport)** for PR 21487 at commit

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-06-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21231 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21487 LGTM (I checked the related tests passed in my local). --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21472: [SPARK-24445][SQL] Schema in json format for from...

2018-06-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21472#discussion_r192584018 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -747,8 +748,13 @@ case class StructsToJson(

[GitHub] spark issue #21481: [SPARK-24452][SQL][Core] Avoid possible overflow in int ...

2018-06-03 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21481 Good questions. For 2, at first I found one of these issues when I looked at a file. Then, I ran `grep` command with `long .*=.*\*` and `long .*=.*\+` in `.java` file. Then, I picked them up

[GitHub] spark pull request #21472: [SPARK-24445][SQL] Schema in json format for from...

2018-06-03 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21472#discussion_r192583925 --- Diff: sql/core/src/test/resources/sql-tests/inputs/json-functions.sql --- @@ -25,6 +25,10 @@ select from_json('{"a":1}', 'a InvalidType'); select

[GitHub] spark issue #21439: [SPARK-24391][SQL] Support arrays of any types by from_j...

2018-06-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21439 > What kind of tests would you expect in json-functions.sql. Probably you would expect tests > that are different from added to JsonExpressionsSuite.scala. IIUC there is no strict rule there

[GitHub] spark issue #21449: [SPARK-24385][SQL] Resolve self-join condition ambiguity...

2018-06-03 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/21449 I see what you mean. Honestly I have not thought of a full design for this problem (so I can't state what we should support and what not), but focusing on this specific case I think that:

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21487 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3784/

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21487 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21487 **[Test build #91427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91427/testReport)** for PR 21487 at commit

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21487 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21487 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21487 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91425/ Test FAILed. ---

[GitHub] spark issue #21487: [SPARK-24369][SQL] Correct handling for multiple distinc...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21487 **[Test build #91425 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91425/testReport)** for PR 21487 at commit

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-06-03 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21313#discussion_r192578796 --- Diff: R/pkg/R/functions.R --- @@ -3006,6 +3008,27 @@ setMethod("array_contains", column(jc) }) +#' @details

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-06-03 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21313#discussion_r192578774 --- Diff: R/pkg/R/functions.R --- @@ -3006,6 +3008,27 @@ setMethod("array_contains", column(jc) }) +#' @details

[GitHub] spark pull request #21313: [SPARK-24187][R][SQL]Add array_join function to S...

2018-06-03 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/21313#discussion_r192578750 --- Diff: R/pkg/R/functions.R --- @@ -3006,6 +3008,27 @@ setMethod("array_contains", column(jc) }) +#' @details

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/91426/ Test FAILed. ---

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-06-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19602 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19602: [SPARK-22384][SQL] Refine partition pruning when attribu...

2018-06-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19602 **[Test build #91426 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/91426/testReport)** for PR 19602 at commit