[GitHub] spark issue #23124: [SPARK-25829][SQL] remove duplicated map keys with last ...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23124 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23124: [SPARK-25829][SQL] remove duplicated map keys with last ...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23128: [SPARK-26142][SQL] Implement shuffle read metrics in SQL

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23128 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23128: [SPARK-26142][SQL] Implement shuffle read metrics in SQL

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23128 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22575: [SPARK-24630][SS] Support SQLStreaming in Spark

2018-11-27 Thread stczwd
Github user stczwd commented on the issue: https://github.com/apache/spark/pull/22575 I hive send an email to Ryan Blue. > > > Can you send a mail to Ryan blue for adding this SPIP topic in tomorrow meeting. Meeting will be conducted tomorrow 05:00 pm PST. If you confirm

[GitHub] spark issue #22163: [SPARK-25166][CORE]Reduce the number of write operations...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22163 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22163: [SPARK-25166][CORE]Reduce the number of write operations...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22163 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99348/ Test PASSed. ---

[GitHub] spark pull request #21957: [SPARK-24994][SQL] When the data type of the fiel...

2018-11-27 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/21957#discussion_r236965962 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/DataSourceScanExec.scala --- @@ -269,7 +269,8 @@ case class FileSourceScanExec( }

[GitHub] spark pull request #23025: [SPARK-26024][SQL]: Update documentation for repa...

2018-11-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/23025#discussion_r236970732 --- Diff: R/pkg/R/DataFrame.R --- @@ -767,6 +767,14 @@ setMethod("repartition", #' using \code{spark.sql.shuffle.partitions}

[GitHub] spark issue #23124: [SPARK-25829][SQL] remove duplicated map keys with last ...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23124 **[Test build #99357 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99357/testReport)** for PR 23124 at commit

[GitHub] spark issue #23145: [MINOR][Docs][WIP] Fix Typos

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23145 **[Test build #99331 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99331/testReport)** for PR 23145 at commit

[GitHub] spark issue #22979: [SPARK-25977][SQL] Parsing decimals from CSV using local...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22979 **[Test build #99323 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99323/testReport)** for PR 22979 at commit

[GitHub] spark issue #23145: [MINOR][Docs][WIP] Fix Typos

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23145 **[Test build #99331 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99331/testReport)** for PR 23145 at commit

[GitHub] spark issue #23145: [MINOR][Docs][WIP] Fix Typos

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23145 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23124: [SPARK-25829][SQL] remove duplicated map keys with last ...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23124 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99325/ Test PASSed. ---

[GitHub] spark issue #23102: [SPARK-26137][CORE] Use Java system property "file.separ...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23102 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23102: [SPARK-26137][CORE] Use Java system property "file.separ...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23102 **[Test build #99333 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99333/testReport)** for PR 23102 at commit

[GitHub] spark pull request #22995: [SPARK-25998] [CORE] Change TorrentBroadcast to h...

2018-11-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22995#discussion_r236721318 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -92,8 +95,15 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request #23153: [SPARK-26147][SQL] only pull out unevaluable pyth...

2018-11-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/23153#discussion_r236743539 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/joins.scala --- @@ -155,19 +155,20 @@ object EliminateOuterJoin extends

[GitHub] spark pull request #23153: [SPARK-26147][SQL] only pull out unevaluable pyth...

2018-11-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/23153#discussion_r236647128 --- Diff: python/pyspark/sql/tests/test_udf.py --- @@ -209,6 +209,18 @@ def test_udf_in_join_condition(self): with

[GitHub] spark issue #23150: [SPARK-26178][SQL] Use java.time API for parsing timesta...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23150 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99327/ Test PASSed. ---

[GitHub] spark issue #22979: [SPARK-25977][SQL] Parsing decimals from CSV using local...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22979 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99323/ Test PASSed. ---

[GitHub] spark issue #22979: [SPARK-25977][SQL] Parsing decimals from CSV using local...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22979 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage ...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23117 **[Test build #99332 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99332/testReport)** for PR 23117 at commit

[GitHub] spark pull request #23130: [SPARK-26161][SQL] Ignore empty files in load

2018-11-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23130#discussion_r236705916 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/SaveLoadSuite.scala --- @@ -142,4 +143,15 @@ class SaveLoadSuite extends DataSourceTest with

[GitHub] spark issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage ...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23117 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #23098: [WIP][SPARK-26132][BUILD][CORE] Remove support fo...

2018-11-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23098#discussion_r236714704 --- Diff: R/pkg/R/sparkR.R --- @@ -269,7 +269,7 @@ sparkR.sparkContext <- function( #' sparkR.session("yarn-client", "SparkR", "/home/spark", #'

[GitHub] spark pull request #23130: [SPARK-26161][SQL] Ignore empty files in load

2018-11-27 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/23130#discussion_r236719477 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/SaveLoadSuite.scala --- @@ -142,4 +143,15 @@ class SaveLoadSuite extends DataSourceTest with

[GitHub] spark issue #23128: [SPARK-26142][SQL] Support passing shuffle metrics to ex...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23128 **[Test build #99334 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99334/testReport)** for PR 23128 at commit

[GitHub] spark issue #23136: [SPARK-25515][K8s] Adds a config option to keep executor...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23136 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23154: [SQL] Correct two exception message in UnresolvedGenerat...

2018-11-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/23154 Good catch. I believe other files (e.g. `VectorizedRleValuesReader.java`, `Expression.scala`, and `generators.scala` also have the similar problem. Can this PR address them? ---

[GitHub] spark issue #23124: [SPARK-25829][SQL] remove duplicated map keys with last ...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23124 **[Test build #99325 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99325/testReport)** for PR 23124 at commit

[GitHub] spark issue #23102: [SPARK-26137][CORE] Use Java system property "file.separ...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23102 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23128: [SPARK-26142][SQL] Support passing shuffle metric...

2018-11-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/23128#discussion_r236720141 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/ShuffledRowRDD.scala --- @@ -154,7 +156,14 @@ class ShuffledRowRDD(

[GitHub] spark pull request #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pys...

2018-11-27 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23055#discussion_r236737983 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -74,8 +74,13 @@ private[spark] abstract class BasePythonRunner[IN, OUT](

[GitHub] spark issue #23136: [SPARK-25515][K8s] Adds a config option to keep executor...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23136 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5414/ ---

[GitHub] spark issue #23150: [SPARK-26178][SQL] Use java.time API for parsing timesta...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23150 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23127: [SPARK-26159] Codegen for LocalTableScanExec and RDDScan...

2018-11-27 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/23127 > jenkins retest this please @juliuszsompolski It won't help, you need to fix python tests it seems ``` Failed example: df.explain() Differences (ndiff with

[GitHub] spark pull request #23131: [SPARK-25908][SQL][FOLLOW-UP] Add back unionAll

2018-11-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/23131#discussion_r236760822 --- Diff: R/pkg/R/DataFrame.R --- @@ -2732,6 +2732,20 @@ setMethod("union", dataFrame(unioned) }) +#' Return a

[GitHub] spark issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage ...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23117 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23083: [SPARK-26114][CORE] ExternalSorter's readingIterator fie...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23083 **[Test build #99326 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99326/testReport)** for PR 23083 at commit

[GitHub] spark pull request #23130: [SPARK-26161][SQL] Ignore empty files in load

2018-11-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23130#discussion_r236721863 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/SaveLoadSuite.scala --- @@ -142,4 +143,15 @@ class SaveLoadSuite extends DataSourceTest with

[GitHub] spark issue #23128: [SPARK-26142][SQL] Support passing shuffle metrics to ex...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23128 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23128: [SPARK-26142][SQL] Support passing shuffle metrics to ex...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23128 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23058: [SPARK-25905][CORE] When getting a remote block, avoid f...

2018-11-27 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/23058 @attilapiros yes, something like that would be possible. I was thinking you'd just use the existing serializer methods to do it, soemthing like: ```scala val buffer =

[GitHub] spark pull request #23098: [WIP][SPARK-26132][BUILD][CORE] Remove support fo...

2018-11-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23098#discussion_r236726267 --- Diff: dev/create-release/release-build.sh --- @@ -110,16 +110,18 @@ fi # Depending on the version being built, certain extra profiles need to be

[GitHub] spark issue #23068: [SPARK-26098][WebUI] Show associated SQL query in Job pa...

2018-11-27 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/23068 ping @vanzin . Please review this one :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #23136: [SPARK-25515][K8s] Adds a config option to keep executor...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23136 **[Test build #99336 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99336/testReport)** for PR 23136 at commit

[GitHub] spark issue #23145: [MINOR][Docs][WIP] Fix Typos

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23145 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99331/ Test PASSed. ---

[GitHub] spark pull request #23102: [SPARK-26137][CORE] Use Java system property "fil...

2018-11-27 Thread markpavey
Github user markpavey commented on a diff in the pull request: https://github.com/apache/spark/pull/23102#discussion_r236705942 --- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala --- @@ -19,6 +19,7 @@ package org.apache.spark.deploy import

[GitHub] spark pull request #23130: [SPARK-26161][SQL] Ignore empty files in load

2018-11-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23130#discussion_r236705666 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/sources/SaveLoadSuite.scala --- @@ -142,4 +143,15 @@ class SaveLoadSuite extends DataSourceTest with

[GitHub] spark issue #23083: [SPARK-26114][CORE] ExternalSorter's readingIterator fie...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23083 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99326/ Test FAILed. ---

[GitHub] spark issue #23083: [SPARK-26114][CORE] ExternalSorter's readingIterator fie...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23083 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22995: [SPARK-25998] [CORE] Change TorrentBroadcast to h...

2018-11-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22995#discussion_r236726776 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -92,8 +95,15 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark issue #23150: [SPARK-26178][SQL] Use java.time API for parsing timesta...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23150 **[Test build #99327 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99327/testReport)** for PR 23150 at commit

[GitHub] spark issue #23132: [SPARK-26163][SQL] Parsing decimals from JSON using loca...

2018-11-27 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/23132 @dongjoon-hyun @cloud-fan Please, take a look at the PR. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pys...

2018-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23055#discussion_r236692833 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -74,8 +74,13 @@ private[spark] abstract class BasePythonRunner[IN,

[GitHub] spark issue #23145: [MINOR][Docs][WIP] Fix Typos

2018-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23145 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23145: [MINOR][Docs][WIP] Fix Typos

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23145 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23145: [MINOR][Docs][WIP] Fix Typos

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23145 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23130: [SPARK-26161][SQL] Ignore empty files in load

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23130 **[Test build #99335 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99335/testReport)** for PR 23130 at commit

[GitHub] spark issue #23130: [SPARK-26161][SQL] Ignore empty files in load

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23130 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #23098: [WIP][SPARK-26132][BUILD][CORE] Remove support fo...

2018-11-27 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23098#discussion_r236736341 --- Diff: dev/create-release/release-build.sh --- @@ -110,16 +110,18 @@ fi # Depending on the version being built, certain extra profiles need to be

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-11-27 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r236738629 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaTokenUtil.scala --- @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #22887: [SPARK-25880][CORE] user set's hadoop conf should not ov...

2018-11-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22887 > I think this is what this PR tries to fix? To be fair I'm not sure I fully understand the PR description. But I know that the previous patch (which I commented on) broke the functionality

[GitHub] spark issue #23153: [SPARK-26147][SQL] only pull out unevaluable python udf ...

2018-11-27 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/23153 Thanks for the fix from Wenchen, ``` the suites should also construct the dummy python udf from both side. ``` I fix the suite locally, they can be simply modified like: ```

[GitHub] spark issue #23127: [SPARK-26159] Codegen for LocalTableScanExec and RDDScan...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23127 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99329/ Test FAILed. ---

[GitHub] spark issue #23127: [SPARK-26159] Codegen for LocalTableScanExec and RDDScan...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23127 **[Test build #99329 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99329/testReport)** for PR 23127 at commit

[GitHub] spark issue #23127: [SPARK-26159] Codegen for LocalTableScanExec and RDDScan...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23127 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23145: [MINOR][Docs][WIP] Fix Typos

2018-11-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23145 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #23102: [SPARK-26137][CORE] Use Java system property "fil...

2018-11-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23102#discussion_r236703765 --- Diff: core/src/main/scala/org/apache/spark/deploy/DependencyUtils.scala --- @@ -19,6 +19,7 @@ package org.apache.spark.deploy import

[GitHub] spark issue #22683: [SPARK-25696] The storage memory displayed on spark Appl...

2018-11-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22683 Ping @httfighter --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23124: [SPARK-25829][SQL] remove duplicated map keys with last ...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23124 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22995: [SPARK-25998] [CORE] Change TorrentBroadcast to h...

2018-11-27 Thread bkrieger
Github user bkrieger commented on a diff in the pull request: https://github.com/apache/spark/pull/22995#discussion_r23671 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -92,8 +95,15 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request #22995: [SPARK-25998] [CORE] Change TorrentBroadcast to h...

2018-11-27 Thread bkrieger
Github user bkrieger commented on a diff in the pull request: https://github.com/apache/spark/pull/22995#discussion_r236724749 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -92,8 +95,15 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark issue #23130: [SPARK-26161][SQL] Ignore empty files in load

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23130 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23136: [SPARK-25515][K8s] Adds a config option to keep executor...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23136 **[Test build #99336 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99336/testReport)** for PR 23136 at commit

[GitHub] spark issue #23136: [SPARK-25515][K8s] Adds a config option to keep executor...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23136 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99336/ Test FAILed. ---

[GitHub] spark issue #23153: [SPARK-26147][SQL] only pull out unevaluable python udf ...

2018-11-27 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/23153 the change itself seems fine to me, as @xuanyuanking mentioned, though, we should update the existing tests. What about adding a test in the new suite checking the plans instead of a end-to-end

[GitHub] spark issue #23136: [SPARK-25515][K8s] Adds a config option to keep executor...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23136 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/5414/ ---

[GitHub] spark issue #23136: [SPARK-25515][K8s] Adds a config option to keep executor...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23136 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23136: [SPARK-25515][K8s] Adds a config option to keep executor...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23136 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23019: [SPARK-26025][k8s] Speed up docker image build on dev re...

2018-11-27 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23019 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #23025: [SPARK-26024][SQL]: Update documentation for repa...

2018-11-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/23025#discussion_r236762465 --- Diff: R/pkg/R/DataFrame.R --- @@ -767,6 +767,14 @@ setMethod("repartition", #' using \code{spark.sql.shuffle.partitions}

[GitHub] spark pull request #23019: [SPARK-26025][k8s] Speed up docker image build on...

2018-11-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23019 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23157: [SPARK-26185][PYTHON]add weightCol in python MulticlassC...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23157 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23157: [SPARK-26185][PYTHON]add weightCol in python MulticlassC...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23157 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99337/ Test PASSed. ---

[GitHub] spark issue #23157: [SPARK-26185][PYTHON]add weightCol in python MulticlassC...

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23157 **[Test build #99337 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99337/testReport)** for PR 23157 at commit

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-11-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #99330 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99330/testReport)** for PR 22598 at commit

[GitHub] spark pull request #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-27 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/23148#discussion_r236813956 --- Diff: dev/.scalafmt.conf --- @@ -0,0 +1,24 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license

[GitHub] spark pull request #23086: [SPARK-25528][SQL] data source v2 API refactor (b...

2018-11-27 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/23086#discussion_r236816739 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/Table.java --- @@ -0,0 +1,51 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #23145: [MINOR][Docs][WIP] Fix Typos

2018-11-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/23145#discussion_r236765511 --- Diff: docs/index.md --- @@ -67,7 +67,7 @@ Example applications are also provided in Python. For example, ./bin/spark-submit

[GitHub] spark pull request #23158: [SPARK-26186][SPARK-26184] Last updated time is n...

2018-11-27 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/23158 [SPARK-26186][SPARK-26184] Last updated time is not getting updated for the Inprogress application ## What changes were proposed in this pull request? When the

[GitHub] spark issue #22959: [SPARK-25876][k8s] Simplify kubernetes configuration typ...

2018-11-27 Thread liyinan926
Github user liyinan926 commented on the issue: https://github.com/apache/spark/pull/22959 @mccheah do you have any comment? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99330/ Test PASSed. ---

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23108: [Spark-25993][SQL][TEST]Add test cases for resolu...

2018-11-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23108#discussion_r236797263 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -111,6 +111,8 @@ displayTitle: Spark SQL Upgrading Guide - Since Spark 2.0, Spark

[GitHub] spark pull request #23131: [SPARK-25908][SQL][FOLLOW-UP] Add back unionAll

2018-11-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/23131#discussion_r236803943 --- Diff: R/pkg/R/DataFrame.R --- @@ -2732,6 +2732,20 @@ setMethod("union", dataFrame(unioned) }) +#' Return a

[GitHub] spark issue #23102: [SPARK-26137][CORE] Use Java system property "file.separ...

2018-11-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23102 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23131: [SPARK-25908][SQL][FOLLOW-UP] Add back unionAll

2018-11-27 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/23131#discussion_r236763355 --- Diff: R/pkg/R/DataFrame.R --- @@ -2732,6 +2732,20 @@ setMethod("union", dataFrame(unioned) }) +#' Return a

  1   2   3   4   5   >