[GitHub] spark issue #23017: [WIP][SPARK-26015][K8S] Set a default UID for Spark on K...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23017 **[Test build #98882 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98882/testReport)** for PR 23017 at commit [`a9e59b3`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-11-15 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r234037129 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaTokenUtil.scala --- @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark issue #23017: [WIP][SPARK-26015][K8S] Set a default UID for Spark on K...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23017 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98882/ Test PASSed. ---

[GitHub] spark issue #23017: [WIP][SPARK-26015][K8S] Set a default UID for Spark on K...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23017 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #23045: [SPARK-26071][SQL] disallow map as map key

2018-11-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/23045#discussion_r234037904 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/TypeUtils.scala --- @@ -46,12 +46,20 @@ object TypeUtils { if (Type

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-15 Thread ifilonenko
Github user ifilonenko commented on the issue: https://github.com/apache/spark/pull/23037 > This is fine, but please file a bug. Okay, as such, @AzureQ could you add an integration test to `ClientModeTestsSuite` --- --

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #98890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98890/testReport)** for PR 22598 at commit [`2a0cdb7`](https://github.com/apache/spark/commit/2a

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-11-15 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r234040048 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaTokenUtil.scala --- @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Software Fou

[GitHub] spark issue #23039: [SPARK-26066][SQL] Using new SQL config spark.sql.debug....

2018-11-15 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/23039 How about the following? ``` Move `truncatedString` to `sql/catalyst` and add `spark.sql.debug.maxToStringFields` conf ``` --- ---

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-11-15 Thread gaborgsomogyi
Github user gaborgsomogyi commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r234043117 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaTokenUtil.scala --- @@ -0,0 +1,146 @@ +/* + * Licensed to the Apache Softw

[GitHub] spark issue #23050: [SPARK-26079][sql] Ensure listener event delivery in Str...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23050 **[Test build #98886 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98886/testReport)** for PR 23050 at commit [`b9c2dc8`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #23050: [SPARK-26079][sql] Ensure listener event delivery in Str...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23050 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23050: [SPARK-26079][sql] Ensure listener event delivery in Str...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23050 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98886/ Test PASSed. ---

[GitHub] spark issue #22911: [SPARK-25815][k8s] Support kerberos in client mode, keyt...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22911 **[Test build #98885 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98885/testReport)** for PR 22911 at commit [`78b76a8`](https://github.com/apache/spark/commit/7

[GitHub] spark issue #22911: [SPARK-25815][k8s] Support kerberos in client mode, keyt...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22911 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22911: [SPARK-25815][k8s] Support kerberos in client mode, keyt...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22911 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98885/ Test PASSed. ---

[GitHub] spark issue #23034: [SPARK-26035][PYTHON] Break large streaming/tests.py fil...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23034 @BryanCutler, should be ready to work on ML and MLlib as well. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache

[GitHub] spark issue #23034: [SPARK-26035][PYTHON] Break large streaming/tests.py fil...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23034 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews

[GitHub] spark pull request #23034: [SPARK-26035][PYTHON] Break large streaming/tests...

2018-11-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23034 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22999: [SPARK-20319][SQL] Already quoted identifiers are gettin...

2018-11-15 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22999 cc @gatorsmile @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #23034: [SPARK-26035][PYTHON] Break large streaming/tests.py fil...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23034 Also, @BryanCutler, I think we can talk about locations of `testing/...util.py` later when we finished to split the tests. Moving utils would probably cause less conflicts and should be good eno

[GitHub] spark issue #23047: [BACKPORT][SPARK-25883][SQL][MINOR] Override method `pre...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23047 Merged to branch-2.4. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #23052: [SPARK-26081][SQL] Prevent empty files for empty partiti...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23052 **[Test build #98887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98887/testReport)** for PR 23052 at commit [`6f3cb18`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #23052: [SPARK-26081][SQL] Prevent empty files for empty partiti...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23052 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98887/ Test PASSed. ---

[GitHub] spark issue #23052: [SPARK-26081][SQL] Prevent empty files for empty partiti...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23052 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23053: [SPARK-25957][K8S] Add ability to skip building optional...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23053 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23053: [SPARK-25957][K8S] Add ability to skip building o...

2018-11-15 Thread ramaddepally
GitHub user ramaddepally opened a pull request: https://github.com/apache/spark/pull/23053 [SPARK-25957][K8S] Add ability to skip building optional k8s docker i… …mages ## What changes were proposed in this pull request? bin/docker-image-tool.sh tries to build all docker

[GitHub] spark issue #23052: [SPARK-26081][SQL] Prevent empty files for empty partiti...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23052 @MaxGekk, actually this is kind of important behaviour change. This basically means we're unable to read the empty files back. Similar changes were proposed in Parquet few years ago (by me) and

[GitHub] spark issue #23053: [SPARK-25957][K8S] Add ability to skip building optional...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23053 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23053: [SPARK-25957][K8S] Add ability to skip building optional...

2018-11-15 Thread ramaddepally
Github user ramaddepally commented on the issue: https://github.com/apache/spark/pull/23053 @vanzin .. can you please review this change? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additio

[GitHub] spark issue #23053: [SPARK-25957][K8S] Add ability to skip building optional...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23053 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23052: [SPARK-26081][SQL] Prevent empty files for empty ...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23052#discussion_r234062564 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVFileFormat.scala --- @@ -174,13 +174,18 @@ private[csv] class CsvOu

[GitHub] spark issue #23034: [SPARK-26035][PYTHON] Break large streaming/tests.py fil...

2018-11-15 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/23034 > Also, @BryanCutler, I think we can talk about locations of testing/...util.py later when we finished to split the tests. Moving utils would probably cause less conflicts and should be good eno

[GitHub] spark pull request #23046: [SPARK-23207][SQL][FOLLOW-UP] Use `SQLConf.get.en...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23046#discussion_r234063905 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala --- @@ -280,7 +280,7 @@ object ShuffleExchangeExec

[GitHub] spark issue #23034: [SPARK-26035][PYTHON] Break large streaming/tests.py fil...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23034 Thank you @BryanCutler. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-15 Thread AzureQ
Github user AzureQ commented on the issue: https://github.com/apache/spark/pull/23037 > > This is fine, but please file a bug. > > Okay, as such, @AzureQ could you add an integration test to `ClientModeTestsSuite` Sure --- --

[GitHub] spark issue #20303: [SPARK-23128][SQL] A new approach to do adaptive executi...

2018-11-15 Thread carsonwang
Github user carsonwang commented on the issue: https://github.com/apache/spark/pull/20303 @cloud-fan @gatorsmile , are you ready to start reviewing this? I can bring this update to date. --- - To unsubscribe, e-mail

[GitHub] spark pull request #23054: [SPARK-26085][SQL] Key attribute of primitive typ...

2018-11-15 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/23054 [SPARK-26085][SQL] Key attribute of primitive type under typed aggregation should be named as "key" too ## What changes were proposed in this pull request? When doing typed aggregation on a

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of primitive type under...

2018-11-15 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/23054 cc @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@sp

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of primitive type under...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23054 **[Test build #98891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98891/testReport)** for PR 23054 at commit [`c7bbe91`](https://github.com/apache/spark/commit/c7

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of primitive type under...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23054: [SPARK-26085][SQL] Key attribute of primitive type under...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23054 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5064/

[GitHub] spark pull request #23055: [SPARK-26080][SQL] Disable 'spark.executor.pyspar...

2018-11-15 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/23055 [SPARK-26080][SQL] Disable 'spark.executor.pyspark.memory' always on Windows ## What changes were proposed in this pull request? `resource` package is a Unit specific package. See http

[GitHub] spark issue #23055: [SPARK-26080][SQL] Disable 'spark.executor.pyspark.memor...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23055 cc @rdblue, @vanzin and @haydenjeune --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comman

[GitHub] spark pull request #23025: [SPARK-26024][SQL]: Update documentation for repa...

2018-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23025#discussion_r234071213 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2813,6 +2819,11 @@ class Dataset[T] private[sql]( * When no explicit sort

[GitHub] spark issue #23055: [SPARK-26080][SQL] Disable 'spark.executor.pyspark.memor...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23055 **[Test build #98892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98892/testReport)** for PR 23055 at commit [`2d3315a`](https://github.com/apache/spark/commit/2d

[GitHub] spark issue #23055: [SPARK-26080][SQL] Disable 'spark.executor.pyspark.memor...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23055 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23055: [SPARK-26080][SQL] Disable 'spark.executor.pyspark.memor...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23055 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5065/

[GitHub] spark pull request #23025: [SPARK-26024][SQL]: Update documentation for repa...

2018-11-15 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/23025#discussion_r234071565 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -2813,6 +2819,11 @@ class Dataset[T] private[sql]( * When no explicit sort

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22598 **[Test build #98890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98890/testReport)** for PR 22598 at commit [`2a0cdb7`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98890/ Test FAILed. ---

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22598 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #23038: [SPARK-25451][CORE][WEBUI]Aggregated metrics tabl...

2018-11-15 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23038#discussion_r234072070 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/api.scala --- @@ -63,6 +63,7 @@ case class ApplicationAttemptInfo private[spark](

[GitHub] spark issue #23038: [SPARK-25451][CORE][WEBUI]Aggregated metrics table doesn...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23038 **[Test build #98893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98893/testReport)** for PR 23038 at commit [`0d92185`](https://github.com/apache/spark/commit/0d

[GitHub] spark pull request #23046: [SPARK-23207][SQL][FOLLOW-UP] Use `SQLConf.get.en...

2018-11-15 Thread ueshin
Github user ueshin commented on a diff in the pull request: https://github.com/apache/spark/pull/23046#discussion_r234073072 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala --- @@ -280,7 +280,7 @@ object ShuffleExchangeExec {

[GitHub] spark issue #23038: [SPARK-25451][CORE][WEBUI]Aggregated metrics table doesn...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23038 **[Test build #98894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98894/testReport)** for PR 23038 at commit [`805ebb8`](https://github.com/apache/spark/commit/80

[GitHub] spark pull request #23046: [SPARK-23207][SQL][FOLLOW-UP] Use `SQLConf.get.en...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23046#discussion_r234073703 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala --- @@ -280,7 +280,7 @@ object ShuffleExchangeExec

[GitHub] spark issue #23038: [SPARK-25451][CORE][WEBUI]Aggregated metrics table doesn...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23038 **[Test build #98895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98895/testReport)** for PR 23038 at commit [`7c3a80b`](https://github.com/apache/spark/commit/7c

[GitHub] spark issue #23031: [SPARK-26060][SQL] Track SparkConf entries and make SET ...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23031 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23031: [SPARK-26060][SQL] Track SparkConf entries and make SET ...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23031 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5066/

[GitHub] spark issue #23031: [SPARK-26060][SQL] Track SparkConf entries and make SET ...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23031 **[Test build #98896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98896/testReport)** for PR 23031 at commit [`336a331`](https://github.com/apache/spark/commit/33

[GitHub] spark issue #23026: [SPARK-25960][k8s] Support subpath mounting with Kuberne...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23026 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23026: [SPARK-25960][k8s] Support subpath mounting with Kuberne...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23026 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5062/

[GitHub] spark issue #23041: [SPARK-26069][TESTS]Fix flaky test: RpcIntegrationSuite....

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23041 **[Test build #4427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4427/testReport)** for PR 23041 at commit [`6bebcb5`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/te...

2018-11-15 Thread BryanCutler
GitHub user BryanCutler opened a pull request: https://github.com/apache/spark/pull/23056 [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py file into smaller files ## What changes were proposed in this pull request? This PR breaks down the large mllib/tests.py file th

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/23056 Dist by line count: ``` 313 ./test_algorithms.py 201 ./test_feature.py 642 ./test_linalg.py 197 ./test_stat.py 523 ./test_streaming_algorithms.py 115 ./test_util.py

[GitHub] spark issue #23026: [SPARK-25960][k8s] Support subpath mounting with Kuberne...

2018-11-15 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/23026 > > if such a list exists it should be the same list that triggers regular tests. > > I defer that to @shaneknapp no, @vanzin is right. i'll update that tomorrow. @vanz

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5067/

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23056 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/23056 cc @HyukjinKwon @squito --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23056 **[Test build #98897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98897/testReport)** for PR 23056 at commit [`2759521`](https://github.com/apache/spark/commit/27

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5063/

[GitHub] spark issue #23037: [SPARK-26083][k8s] Add Copy pyspark into corresponding d...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23037 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23056 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23056 **[Test build #98897 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98897/testReport)** for PR 23056 at commit [`2759521`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23056 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98897/ Test FAILed. ---

[GitHub] spark pull request #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pys...

2018-11-15 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/23055#discussion_r234080290 --- Diff: python/pyspark/worker.py --- @@ -268,9 +272,11 @@ def main(infile, outfile): # set up memory limits memory_limit_mb

[GitHub] spark pull request #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/te...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23056#discussion_r234080249 --- Diff: python/pyspark/testing/mllibutils.py --- @@ -0,0 +1,44 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more

[GitHub] spark pull request #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/te...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23056#discussion_r234080468 --- Diff: python/pyspark/mllib/tests/test_linalg.py --- @@ -0,0 +1,642 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or mor

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/23056 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark pull request #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pys...

2018-11-15 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/23055#discussion_r234080578 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -74,8 +74,13 @@ private[spark] abstract class BasePythonRunner[IN, OUT](

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23056 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5068/

[GitHub] spark issue #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pyspark.me...

2018-11-15 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/23055 Thanks for fixing this so quickly, @HyukjinKwon! I'd like a couple of changes, but overall it is going in the right direction. We should also plan on porting this to the 2.4 branch when it is

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23056 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #23056: [SPARK-26034][PYTHON][TESTS] Break large mllib/tests.py ...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23056 **[Test build #98898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98898/testReport)** for PR 23056 at commit [`2759521`](https://github.com/apache/spark/commit/27

[GitHub] spark issue #23049: [SPARK-26076][Build][Minor] Revise ambiguous error messa...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23049 **[Test build #98899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98899/testReport)** for PR 23049 at commit [`daf5e33`](https://github.com/apache/spark/commit/da

[GitHub] spark issue #23049: [SPARK-26076][Build][Minor] Revise ambiguous error messa...

2018-11-15 Thread gengliangwang
Github user gengliangwang commented on the issue: https://github.com/apache/spark/pull/23049 Hi @vanzin , thanks for pointing it out! I have updated the script and PR description. --- - To unsubscribe, e-mail: re

[GitHub] spark pull request #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pys...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23055#discussion_r234081475 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -74,8 +74,13 @@ private[spark] abstract class BasePythonRunner[IN, O

[GitHub] spark issue #23049: [SPARK-26076][Build][Minor] Revise ambiguous error messa...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23049 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5069/

[GitHub] spark issue #23049: [SPARK-26076][Build][Minor] Revise ambiguous error messa...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23049 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pys...

2018-11-15 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/23055#discussion_r234084002 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -74,8 +74,13 @@ private[spark] abstract class BasePythonRunner[IN, OUT](

[GitHub] spark issue #23049: [SPARK-26076][Build][Minor] Revise ambiguous error messa...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23049 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/5070/

[GitHub] spark issue #23049: [SPARK-26076][Build][Minor] Revise ambiguous error messa...

2018-11-15 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23049 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #22309: [SPARK-20384][SQL] Support value class in schema ...

2018-11-15 Thread mt40
Github user mt40 commented on a diff in the pull request: https://github.com/apache/spark/pull/22309#discussion_r234085471 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala --- @@ -373,6 +383,32 @@ object ScalaReflection extends ScalaReflect

[GitHub] spark issue #23049: [SPARK-26076][Build][Minor] Revise ambiguous error messa...

2018-11-15 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23049 **[Test build #98900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98900/testReport)** for PR 23049 at commit [`3269862`](https://github.com/apache/spark/commit/32

[GitHub] spark pull request #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pys...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/23055#discussion_r234086569 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -74,8 +74,13 @@ private[spark] abstract class BasePythonRunner[IN, O

[GitHub] spark issue #22309: [SPARK-20384][SQL] Support value class in schema of Data...

2018-11-15 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22309 adding @liancheng BTW. IIRC, he took a look for this one before and abandoned the change (fix me if I'm wrongly remembering this). --- -

[GitHub] spark pull request #23046: [SPARK-23207][SQL][FOLLOW-UP] Use `SQLConf.get.en...

2018-11-15 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/23046#discussion_r234088968 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/exchange/ShuffleExchangeExec.scala --- @@ -280,7 +280,7 @@ object ShuffleExchangeExec {

[GitHub] spark issue #23046: [SPARK-23207][SQL][FOLLOW-UP] Use `SQLConf.get.enableRad...

2018-11-15 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/23046 thanks, merging to master/2.4! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark pull request #23046: [SPARK-23207][SQL][FOLLOW-UP] Use `SQLConf.get.en...

2018-11-15 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23046 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

<    1   2   3   4   5   >