[GitHub] spark pull request #23098: [WIP][SPARK-26132][BUILD][CORE] Remove support fo...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23098#discussion_r236452713 --- Diff: dev/create-release/release-build.sh --- @@ -110,16 +110,18 @@ fi # Depending on the version being built, certain extra profiles need to be

[GitHub] spark issue #23139: [SPARK-25860][SPARK-26107] [FOLLOW-UP] Rule ReplaceNullW...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23139 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99283/ Test PASSed. ---

[GitHub] spark issue #23139: [SPARK-25860][SPARK-26107] [FOLLOW-UP] Rule ReplaceNullW...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23139 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23139: [SPARK-25860][SPARK-26107] [FOLLOW-UP] Rule ReplaceNullW...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23139 **[Test build #99283 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99283/testReport)** for PR 23139 at commit

[GitHub] spark issue #23149: [SPARK-25451][HOTFIX] Call stage.attemptNumber instead o...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23149 **[Test build #99292 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99292/testReport)** for PR 23149 at commit

[GitHub] spark issue #23149: [SPARK-25451][HOTFIX] Call stage.attemptNumber instead o...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23149 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23149: [SPARK-25451][HOTFIX] Call stage.attemptNumber instead o...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23149 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #23139: [SPARK-25860][SPARK-26107] [FOLLOW-UP] Rule Repla...

2018-11-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/23139#discussion_r236450239 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala --- @@ -79,29 +80,31 @@ object

[GitHub] spark pull request #23149: [SPARK-25451][HOTFIX] Call stage.attemptNumber in...

2018-11-26 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/23149 [SPARK-25451][HOTFIX] Call stage.attemptNumber instead of attemptId. You can merge this pull request into a Git repository by running: $ git pull https://github.com/vanzin/spark

[GitHub] spark issue #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metrics table...

2018-11-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23038 This seems to have broken the master build, probably some other change that happened since this was last tested. Will send a follow up. ---

[GitHub] spark issue #23127: [SPARK-26159] Codegen for LocalTableScanExec and RDDScan...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23127 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99286/ Test FAILed. ---

[GitHub] spark issue #23127: [SPARK-26159] Codegen for LocalTableScanExec and RDDScan...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23127 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23127: [SPARK-26159] Codegen for LocalTableScanExec and RDDScan...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23127 **[Test build #99286 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99286/testReport)** for PR 23127 at commit

[GitHub] spark pull request #23098: [WIP][SPARK-26132][BUILD][CORE] Remove support fo...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23098#discussion_r236446713 --- Diff: R/pkg/R/sparkR.R --- @@ -269,7 +269,7 @@ sparkR.sparkContext <- function( #' sparkR.session("yarn-client", "SparkR", "/home/spark", #'

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r236441225 --- Diff: pom.xml --- @@ -128,6 +128,7 @@ 1.2.1.spark2 1.2.1 +2.1.0 --- End diff -- Since you're adding this

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r236438713 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaTokenUtil.scala --- @@ -0,0 +1,168 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r236440232 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSecurityHelperSuite.scala --- @@ -0,0 +1,94 @@ +/* + * Licensed

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r236439851 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaSecurityHelperSuite.scala --- @@ -0,0 +1,94 @@ +/* + * Licensed

[GitHub] spark pull request #23117: [WIP][SPARK-7721][INFRA] Run and generate test co...

2018-11-26 Thread shaneknapp
Github user shaneknapp commented on a diff in the pull request: https://github.com/apache/spark/pull/23117#discussion_r236440252 --- Diff: dev/run-tests.py --- @@ -434,6 +434,63 @@ def run_python_tests(test_modules, parallelism): run_cmd(command) +def

[GitHub] spark pull request #23117: [WIP][SPARK-7721][INFRA] Run and generate test co...

2018-11-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/23117#discussion_r236431048 --- Diff: dev/run-tests.py --- @@ -434,6 +434,63 @@ def run_python_tests(test_modules, parallelism): run_cmd(command) +def

[GitHub] spark pull request #23117: [WIP][SPARK-7721][INFRA] Run and generate test co...

2018-11-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/23117#discussion_r236431733 --- Diff: dev/run-tests.py --- @@ -434,6 +434,63 @@ def run_python_tests(test_modules, parallelism): run_cmd(command) +def

[GitHub] spark pull request #23106: [SPARK-26141] Enable custom metrics implementatio...

2018-11-26 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/23106#discussion_r236432889 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -242,8 +243,13 @@ private void writeSortedFile(boolean

[GitHub] spark pull request #22612: [SPARK-24958] Add executors' process tree total m...

2018-11-26 Thread rezasafi
Github user rezasafi commented on a diff in the pull request: https://github.com/apache/spark/pull/22612#discussion_r236432819 --- Diff: core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala --- @@ -84,6 +122,8 @@ case object MappedPoolMemory extends

[GitHub] spark pull request #23136: [SPARK-25515][K8s] Adds a config option to keep e...

2018-11-26 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/23136#discussion_r236432458 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManagerSuite.scala --- @@ -100,6

[GitHub] spark pull request #23136: [SPARK-25515][K8s] Adds a config option to keep e...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23136#discussion_r236431997 --- Diff: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsLifecycleManagerSuite.scala --- @@ -100,6

[GitHub] spark issue #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metrics table...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/23038 Thank you @vanzin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metrics table...

2018-11-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23038 (Also merged to 2.4.) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23102: [SPARK-26137][CORE] Use Java system property "file.separ...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23102 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metric...

2018-11-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23038 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23102: [SPARK-26137][CORE] Use Java system property "file.separ...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23102 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99281/ Test PASSed. ---

[GitHub] spark issue #23102: [SPARK-26137][CORE] Use Java system property "file.separ...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23102 **[Test build #99281 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99281/testReport)** for PR 23102 at commit

[GitHub] spark pull request #23038: [SPARK-25451][SPARK-26100][CORE]Aggregated metric...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23038#discussion_r236426530 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala --- @@ -565,7 +571,16 @@ private[spark] class AppStatusListener( if

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23148 **[Test build #99291 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99291/testReport)** for PR 23148 at commit

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23148 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23148 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/23148#discussion_r236423672 --- Diff: .scalafmt.conf --- @@ -0,0 +1,24 @@ +# --- End diff -- Sure, moved ---

[GitHub] spark pull request #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread koeninger
Github user koeninger commented on a diff in the pull request: https://github.com/apache/spark/pull/23148#discussion_r236423438 --- Diff: pom.xml --- @@ -156,6 +156,10 @@ 3.2.2 2.12.7 2.12 +1.5.1 --- End diff -- I moved the scalafmt

[GitHub] spark pull request #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23148#discussion_r236421304 --- Diff: .scalafmt.conf --- @@ -0,0 +1,24 @@ +# --- End diff -- Can this file live in dev/? I tried to stuff as many things in there as I

[GitHub] spark pull request #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23148#discussion_r236421583 --- Diff: pom.xml --- @@ -156,6 +156,10 @@ 3.2.2 2.12.7 2.12 +1.5.1 --- End diff -- If these are all just

[GitHub] spark pull request #23106: [SPARK-26141] Enable custom metrics implementatio...

2018-11-26 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/23106#discussion_r236418843 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -242,8 +243,13 @@ private void writeSortedFile(boolean

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23148 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23148 **[Test build #99290 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99290/testReport)** for PR 23148 at commit

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23148 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23141: [SPARK-26021][SQL][followup] add test for special floati...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23141 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23141: [SPARK-26021][SQL][followup] add test for special floati...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23141 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99279/ Test PASSed. ---

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23148 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99288/ Test FAILed. ---

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23148 **[Test build #99288 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99288/testReport)** for PR 23148 at commit

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23148 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23141: [SPARK-26021][SQL][followup] add test for special floati...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23141 **[Test build #99279 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99279/testReport)** for PR 23141 at commit

[GitHub] spark pull request #23100: [SPARK-26133][ML] Remove deprecated OneHotEncoder...

2018-11-26 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/23100#discussion_r236411677 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoder.scala --- @@ -17,126 +17,512 @@ package org.apache.spark.ml.feature

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23148 **[Test build #99288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99288/testReport)** for PR 23148 at commit

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23148 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23148 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23088 **[Test build #99289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99289/testReport)** for PR 23088 at commit

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23088 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23088 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23148: [SPARK-26177] Automated formatting for Scala code

2018-11-26 Thread koeninger
GitHub user koeninger opened a pull request: https://github.com/apache/spark/pull/23148 [SPARK-26177] Automated formatting for Scala code ## What changes were proposed in this pull request? Add a maven plugin and wrapper script at ./dev/scalafmt to use scalafmt to format

[GitHub] spark pull request #23100: [SPARK-26133][ML] Remove deprecated OneHotEncoder...

2018-11-26 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/23100#discussion_r236410750 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoder.scala --- @@ -17,126 +17,512 @@ package org.apache.spark.ml.feature

[GitHub] spark pull request #23100: [SPARK-26133][ML] Remove deprecated OneHotEncoder...

2018-11-26 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/23100#discussion_r236410306 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/OneHotEncoder.scala --- @@ -17,126 +17,512 @@ package org.apache.spark.ml.feature

[GitHub] spark issue #23111: [SPARK-26148][PYTHON][TESTS] Increases default paralleli...

2018-11-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/23111 we might need to be careful that this doesn't un-intentionally overload the jenkins workers so that we end up hitting more timeouts from too many things running concurrently (I dunno how isolated

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23088 **[Test build #99287 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99287/testReport)** for PR 23088 at commit

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23088 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236409746 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusStoreSuite.scala --- @@ -95,10 +123,18 @@ class AppStatusStoreSuite extends SparkFunSuite

[GitHub] spark issue #23088: [SPARK-26119][CORE][WEBUI]Task summary table should cont...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23088 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236409661 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark issue #23111: [SPARK-26148][PYTHON][TESTS] Increases default paralleli...

2018-11-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/23111 wow, thats great! glad there is a big speedup. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread shahidki31
Github user shahidki31 commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236409557 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-11-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21688 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23109: [SPARK-26069][TESTS][FOLLOWUP]Add another possible error...

2018-11-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/23109 lte review, but lgtm anyway --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23058: [SPARK-25905][CORE] When getting a remote block, avoid f...

2018-11-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/23058 lgtm I looked more into the lifecycle of the buffers and when they get `disposed`, and it looks fine to me. (In fact I think there is no need for the `dispose` in the first place, as

[GitHub] spark issue #21688: [SPARK-21809] : Change Stage Page to use datatables to s...

2018-11-26 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21688 +1 , going to merge to master There are a few followup jiras on this. 1) make the timeline visualization better: https://issues.apache.org/jira/browse/SPARK-26130 2) improve

[GitHub] spark issue #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pyspark.me...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23055 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/99278/ Test PASSed. ---

[GitHub] spark issue #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pyspark.me...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23055 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236405732 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark issue #23055: [SPARK-26080][PYTHON] Disable 'spark.executor.pyspark.me...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23055 **[Test build #99278 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99278/testReport)** for PR 23055 at commit

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236405278 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark issue #23127: [SPARK-26159] Codegen for LocalTableScanExec and RDDScan...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23127 **[Test build #99286 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99286/testReport)** for PR 23127 at commit

[GitHub] spark issue #23127: [SPARK-26159] Codegen for LocalTableScanExec and RDDScan...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23127 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23127: [SPARK-26159] Codegen for LocalTableScanExec and RDDScan...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23127 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236398604 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusStoreSuite.scala --- @@ -95,10 +123,18 @@ class AppStatusStoreSuite extends SparkFunSuite {

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236398482 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark pull request #23088: [SPARK-26119][CORE][WEBUI]Task summary table shou...

2018-11-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/23088#discussion_r236398316 --- Diff: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala --- @@ -222,29 +223,20 @@ private[spark] class AppStatusStore( val

[GitHub] spark issue #23106: [SPARK-26141] Enable custom metrics implementation in sh...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23106 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23106: [SPARK-26141] Enable custom metrics implementation in sh...

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23106 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #23127: [SPARK-26159] Codegen for LocalTableScanExec and ...

2018-11-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/23127#discussion_r236395764 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -406,14 +415,62 @@ trait

[GitHub] spark issue #23106: [SPARK-26141] Enable custom metrics implementation in sh...

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23106 **[Test build #99285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99285/testReport)** for PR 23106 at commit

[GitHub] spark pull request #23139: [SPARK-25860][SPARK-26107] [FOLLOW-UP] Rule Repla...

2018-11-26 Thread dbtsai
Github user dbtsai commented on a diff in the pull request: https://github.com/apache/spark/pull/23139#discussion_r236394865 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ReplaceNullWithFalseInPredicate.scala --- @@ -79,29 +80,31 @@ object

[GitHub] spark pull request #22995: [SPARK-25998] [CORE] Change TorrentBroadcast to h...

2018-11-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22995#discussion_r236394496 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -93,7 +96,14 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark pull request #23127: [SPARK-26159] Codegen for LocalTableScanExec and ...

2018-11-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/23127#discussion_r236393985 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -406,14 +415,62 @@ trait

[GitHub] spark issue #23139: [SPARK-25860][SPARK-26107] [FOLLOW-UP] Rule ReplaceNullW...

2018-11-26 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/23139 Although we are trying to make sure in the caller side to only call `replaceNullWithFalse` when the expression is boolean type, I agree that for safety, we should check it and throw exception for

[GitHub] spark pull request #23127: [SPARK-26159] Codegen for LocalTableScanExec and ...

2018-11-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/23127#discussion_r236391673 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -406,14 +415,62 @@ trait

[GitHub] spark pull request #22995: [SPARK-25998] [CORE] Change TorrentBroadcast to h...

2018-11-26 Thread bkrieger
Github user bkrieger commented on a diff in the pull request: https://github.com/apache/spark/pull/22995#discussion_r236391043 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -93,7 +96,14 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark issue #23117: [WIP][SPARK-7721][INFRA] Run and generate test coverage ...

2018-11-26 Thread shaneknapp
Github user shaneknapp commented on the issue: https://github.com/apache/spark/pull/23117 not yet, but i will carve out some time today and wednesday to look closer. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #23026: [SPARK-25960][k8s] Support subpath mounting with ...

2018-11-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/23026 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #23026: [SPARK-25960][k8s] Support subpath mounting with Kuberne...

2018-11-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/23026 Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #23147: [SPARK-26140] followup: rename ShuffleMetricsReporter

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23147 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #23147: [SPARK-26140] followup: rename ShuffleMetricsReporter

2018-11-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/23147 **[Test build #99284 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/99284/testReport)** for PR 23147 at commit

[GitHub] spark issue #23147: [SPARK-26140] followup: rename ShuffleMetricsReporter

2018-11-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/23147 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #23147: [SPARK-26140] followup: rename ShuffleMetricsReporter

2018-11-26 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/23147 cc @gatorsmile @xuanyuanking @cloud-fan I misunderstood your comment. Finally saw it today when I was looking at my other PR. ---

[GitHub] spark pull request #23147: [SPARK-26140] followup: rename ShuffleMetricsRepo...

2018-11-26 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/23147 [SPARK-26140] followup: rename ShuffleMetricsReporter ## What changes were proposed in this pull request? In https://github.com/apache/spark/pull/23105, due to working on two parallel PRs at

<    1   2   3   4   5   >