[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for ContextBarrierSta...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22165 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98516/ Test PASSed. ---

[GitHub] spark issue #22957: [SPARK-25951][SQL] Remove Alias when canonicalize

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98521/ Test FAILed. ---

[GitHub] spark issue #22880: [SPARK-25407][SQL] Ensure we pass a compatible pruned sc...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22880 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22880: [SPARK-25407][SQL] Ensure we pass a compatible pruned sc...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22880 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22943: [SPARK-25098][SQL] Trim the string when cast stri...

2018-11-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22943#discussion_r231207627 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -441,7 +441,7 @@ object DateTimeUtils { }

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231213369 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,17 @@ import

[GitHub] spark issue #22598: [SPARK-25501][SS] Add kafka delegation token support.

2018-11-06 Thread arunmahadevan
Github user arunmahadevan commented on the issue: https://github.com/apache/spark/pull/22598 +1, LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22937: [SPARK-25934] [Mesos] Don't propagate SPARK_CONF_DIR fro...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22937 **[Test build #98525 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98525/testReport)** for PR 22937 at commit

[GitHub] spark issue #22937: [SPARK-25934] [Mesos] Don't propagate SPARK_CONF_DIR fro...

2018-11-06 Thread mpmolek
Github user mpmolek commented on the issue: https://github.com/apache/spark/pull/22937 Added test cases, and also refactored RestSubmissionClient to put the blacklisted spark vars in a Set so its easier to read ---

[GitHub] spark pull request #22880: [SPARK-25407][SQL] Ensure we pass a compatible pr...

2018-11-06 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/22880#discussion_r231243760 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala --- @@ -202,11 +204,15 @@ private[parquet]

[GitHub] spark issue #22855: [SPARK-25839] [Core] Implement use of KryoPool in KryoSe...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22855 **[Test build #4415 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4415/testReport)** for PR 22855 at commit

[GitHub] spark pull request #22880: [SPARK-25407][SQL] Ensure we pass a compatible pr...

2018-11-06 Thread mallman
Github user mallman commented on a diff in the pull request: https://github.com/apache/spark/pull/22880#discussion_r231249401 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala --- @@ -202,11 +204,15 @@ private[parquet]

[GitHub] spark pull request #22938: [SPARK-25935][SQL] Prevent null rows from JSON pa...

2018-11-06 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22938#discussion_r231251408 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -550,15 +550,23 @@ case class JsonToStructs(

[GitHub] spark issue #22087: [SPARK-25097][ML] Support prediction on single instance ...

2018-11-06 Thread erikerlandson
Github user erikerlandson commented on the issue: https://github.com/apache/spark/pull/22087 This LGTM, but it raises a more general question about the lack of single-sample prediction over the entire hierarchy. For example (IMO) there should be some kind of single-sample method

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231214136 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -39,21 +46,28 @@ class MulticlassMetrics

[GitHub] spark pull request #22855: [SPARK-25839] [Core] Implement use of KryoPool in...

2018-11-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22855#discussion_r231213606 --- Diff: core/src/main/scala/org/apache/spark/serializer/KryoSerializer.scala --- @@ -298,30 +309,40 @@ class KryoDeserializationStream( } }

[GitHub] spark issue #22958: [SPARK-25952][SQL] Passing actual schema to JacksonParse...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22958 **[Test build #98523 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98523/testReport)** for PR 22958 at commit

[GitHub] spark issue #22818: [SPARK-25904][CORE] Allocate arrays smaller than Int.Max...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22818 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22818: [SPARK-25904][CORE] Allocate arrays smaller than Int.Max...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22818 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98520/ Test PASSed. ---

[GitHub] spark issue #22957: [SPARK-25951][SQL] Remove Alias when canonicalize

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22957 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22957: [SPARK-25951][SQL] Remove Alias when canonicalize

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22957 **[Test build #98521 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98521/testReport)** for PR 22957 at commit

[GitHub] spark issue #22959: [SPARK-25876][k8s] Simplify kubernetes configuration typ...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22959 **[Test build #98526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98526/testReport)** for PR 22959 at commit

[GitHub] spark pull request #22882: [SPARK-25871][STREAMING][WIP] Don't use EC for st...

2018-11-06 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22882 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22823: [SPARK-25676][SQL][TEST] Rename and refactor BenchmarkWi...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22823 **[Test build #98522 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98522/testReport)** for PR 22823 at commit

[GitHub] spark issue #22823: [SPARK-25676][SQL][TEST] Rename and refactor BenchmarkWi...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22823 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22947: [SPARK-24913][SQL] Make AssertNotNull and AssertT...

2018-11-06 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22947#discussion_r231191913 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -66,6 +66,8 @@ case class AssertTrue(child: Expression)

[GitHub] spark pull request #22823: [SPARK-25676][SQL][TEST] Rename and refactor Benc...

2018-11-06 Thread yucai
Github user yucai commented on a diff in the pull request: https://github.com/apache/spark/pull/22823#discussion_r231198094 --- Diff: sql/core/benchmarks/WideTableBenchmark-results.txt --- @@ -0,0 +1,17 @@

[GitHub] spark pull request #22823: [SPARK-25676][SQL][TEST] Rename and refactor Benc...

2018-11-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22823#discussion_r231201293 --- Diff: sql/core/benchmarks/WideTableBenchmark-results.txt --- @@ -0,0 +1,17 @@

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for ContextBarrierSta...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22165 **[Test build #98516 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98516/testReport)** for PR 22165 at commit

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231215393 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,17 @@ import org.apache.spark.sql.DataFrame

[GitHub] spark issue #22959: [SPARK-25876][k8s] Simplify kubernetes configuration typ...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22959 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22959: [SPARK-25876][k8s] Simplify kubernetes configuration typ...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22959 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22959: [SPARK-25876][k8s] Simplify kubernetes configuration typ...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22959 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4799/ ---

[GitHub] spark issue #22882: [SPARK-25871][STREAMING][WIP] Don't use EC for streaming...

2018-11-06 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/22882 Merging to master. I'll remove the "WIP" during merge. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #22911: [SPARK-25815][k8s] Support kerberos in client mod...

2018-11-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22911#discussion_r231256397 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/KubernetesClusterSchedulerBackend.scala --- @@ -123,7

[GitHub] spark pull request #22944: [SPARK-25942][SQL] Fix Dataset.groupByKey to make...

2018-11-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22944#discussion_r231201502 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala --- @@ -262,25 +262,39 @@ object AppendColumns {

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for ContextBarrierSta...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22165 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22956: [SPARK-25950][SQL] from_csv should respect to spark.sql....

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22956 **[Test build #98518 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98518/testReport)** for PR 22956 at commit

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231227854 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,17 @@ import

[GitHub] spark issue #22959: [SPARK-25876][k8s] Simplify kubernetes configuration typ...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22959 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98526/ Test PASSed. ---

[GitHub] spark issue #22959: [SPARK-25876][k8s] Simplify kubernetes configuration typ...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22959 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22880: [SPARK-25407][SQL] Ensure we pass a compatible pruned sc...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22880 **[Test build #98528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98528/testReport)** for PR 22880 at commit

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-11-06 Thread abellina
Github user abellina commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r231147892 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,905 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-11-06 Thread abellina
Github user abellina commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r231150325 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,965 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-11-06 Thread abellina
Github user abellina commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r231148125 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,905 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-11-06 Thread abellina
Github user abellina commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r231163762 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,965 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #21688: [SPARK-21809] : Change Stage Page to use datatabl...

2018-11-06 Thread abellina
Github user abellina commented on a diff in the pull request: https://github.com/apache/spark/pull/21688#discussion_r230923570 --- Diff: core/src/main/resources/org/apache/spark/ui/static/stagepage.js --- @@ -0,0 +1,965 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #22958: [SPARK-25952][SQL] Passing actual schema to JacksonParse...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22958 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22937: [SPARK-25934] [Mesos] Don't propagate SPARK_CONF_...

2018-11-06 Thread mpmolek
Github user mpmolek commented on a diff in the pull request: https://github.com/apache/spark/pull/22937#discussion_r231226354 --- Diff: core/src/main/scala/org/apache/spark/deploy/rest/RestSubmissionClient.scala --- @@ -418,8 +418,8 @@ private[spark] object RestSubmissionClient {

[GitHub] spark pull request #22959: [SPARK-25876][k8s] Simplify kubernetes configurat...

2018-11-06 Thread vanzin
GitHub user vanzin opened a pull request: https://github.com/apache/spark/pull/22959 [SPARK-25876][k8s] Simplify kubernetes configuration types. There are a few issues with the current configuration types used in the kubernetes backend: - they use type parameters for

[GitHub] spark issue #22959: [SPARK-25876][k8s] Simplify kubernetes configuration typ...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22959 **[Test build #98526 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98526/testReport)** for PR 22959 at commit

[GitHub] spark issue #22823: [SPARK-25676][SQL][TEST] Rename and refactor BenchmarkWi...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22823 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98522/ Test PASSed. ---

[GitHub] spark issue #22855: [SPARK-25839] [Core] Implement use of KryoPool in KryoSe...

2018-11-06 Thread patrickbrownsync
Github user patrickbrownsync commented on the issue: https://github.com/apache/spark/pull/22855 @srowen should be fixed now, I rebased off master and changed KryoSerializerBenchmark to conform to how BenchmarkBase had changed on master ---

[GitHub] spark issue #22956: [SPARK-25950][SQL] from_csv should respect to spark.sql....

2018-11-06 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/22956 @HyukjinKwon @cloud-fan May I ask you to take a look at the PR. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22943: [SPARK-25098][SQL] Trim the string when cast stri...

2018-11-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/22943#discussion_r231207128 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala --- @@ -274,7 +274,7 @@ object DateTimeUtils { }

[GitHub] spark issue #22956: [SPARK-25950][SQL] from_csv should respect to spark.sql....

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22956 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98518/ Test PASSed. ---

[GitHub] spark issue #22956: [SPARK-25950][SQL] from_csv should respect to spark.sql....

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22956 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231214331 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -39,21 +46,28 @@ class MulticlassMetrics

[GitHub] spark pull request #22855: [SPARK-25839] [Core] Implement use of KryoPool in...

2018-11-06 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22855#discussion_r231213322 --- Diff: core/src/test/scala/org/apache/spark/serializer/KryoSerializerSuite.scala --- @@ -431,9 +434,11 @@ class KryoSerializerSuite extends SparkFunSuite

[GitHub] spark issue #22958: [SPARK-25952][SQL] Passing actual schema to JacksonParse...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22958 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22598: [SPARK-25501][SS] Add kafka delegation token supp...

2018-11-06 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/22598#discussion_r231252803 --- Diff: core/src/main/scala/org/apache/spark/deploy/security/KafkaDelegationTokenProvider.scala --- @@ -0,0 +1,65 @@ +/* + * Licensed to the

[GitHub] spark issue #22614: [SPARK-25561][SQL] Implement a new config to control par...

2018-11-06 Thread rezasafi
Github user rezasafi commented on the issue: https://github.com/apache/spark/pull/22614 @tejasapatil @kmanamcheri any update? thank you very much in advance. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22904: [SPARK-25887][K8S] Configurable K8S context support

2018-11-06 Thread rvesse
Github user rvesse commented on the issue: https://github.com/apache/spark/pull/22904 @mccheah I have made the requested changes, can I get another review please? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22911: [SPARK-25815][k8s] Support kerberos in client mod...

2018-11-06 Thread rvesse
Github user rvesse commented on a diff in the pull request: https://github.com/apache/spark/pull/22911#discussion_r231195413 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/KerberosConfDriverFeatureStep.scala --- @@ -126,20 +134,53

[GitHub] spark issue #22937: [SPARK-25934] [Mesos] Don't propagate SPARK_CONF_DIR fro...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22937 **[Test build #98524 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98524/testReport)** for PR 22937 at commit

[GitHub] spark pull request #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use...

2018-11-06 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/17086#discussion_r231227296 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/evaluation/MulticlassMetrics.scala --- @@ -27,10 +27,17 @@ import

[GitHub] spark pull request #22947: [SPARK-24913][SQL] Make AssertNotNull and AssertT...

2018-11-06 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22947#discussion_r231230211 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/misc.scala --- @@ -66,6 +66,8 @@ case class AssertTrue(child:

[GitHub] spark issue #22855: [SPARK-25839] [Core] Implement use of KryoPool in KryoSe...

2018-11-06 Thread patrickbrownsync
Github user patrickbrownsync commented on the issue: https://github.com/apache/spark/pull/22855 @srowen I went ahead and fixed those issues where I could find them, mostly for my own learning experience. Thanks for taking the time to review this many times, really appreciate it ---

[GitHub] spark issue #22959: [SPARK-25876][k8s] Simplify kubernetes configuration typ...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22959 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/testing-k8s-prb-make-spark-distribution-unified/4799/ ---

[GitHub] spark issue #22894: [SPARK-25885][Core][Minor] HighlyCompressedMapStatus des...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22894 **[Test build #4414 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4414/testReport)** for PR 22894 at commit

[GitHub] spark issue #22818: [SPARK-25904][CORE] Allocate arrays smaller than Int.Max...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22818 **[Test build #98520 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98520/testReport)** for PR 22818 at commit

[GitHub] spark issue #22938: [SPARK-25935][SQL] Prevent null rows from JSON parser

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22938 **[Test build #98527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98527/testReport)** for PR 22938 at commit

[GitHub] spark issue #22909: [SPARK-25897][k8s] Hook up k8s integration tests to sbt ...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22909 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98532/ Test FAILed. ---

[GitHub] spark issue #22909: [SPARK-25897][k8s] Hook up k8s integration tests to sbt ...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22909 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22943: [SPARK-25098][SQL] Trim the string when cast stringToTim...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22943 **[Test build #98540 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98540/testReport)** for PR 22943 at commit

[GitHub] spark pull request #22960: [SPARK-25955][TEST] Porting JSON tests for CSV fu...

2018-11-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22960#discussion_r231344120 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/CsvFunctionsSuite.scala --- @@ -86,4 +86,82 @@ class CsvFunctionsSuite extends QueryTest with

[GitHub] spark pull request #22944: [SPARK-25942][SQL] Fix Dataset.groupByKey to make...

2018-11-06 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22944#discussion_r231350156 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala --- @@ -262,25 +262,39 @@ object AppendColumns { def

[GitHub] spark issue #22937: [SPARK-25934] [Mesos] Don't propagate SPARK_CONF_DIR fro...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22937 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98535/ Test PASSed. ---

[GitHub] spark issue #22937: [SPARK-25934] [Mesos] Don't propagate SPARK_CONF_DIR fro...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22937 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22909: [SPARK-25897][k8s] Hook up k8s integration tests to sbt ...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22909 **[Test build #98532 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98532/testReport)** for PR 22909 at commit

[GitHub] spark issue #22943: [SPARK-25098][SQL] Trim the string when cast stringToTim...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22943 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22943: [SPARK-25098][SQL] Trim the string when cast stringToTim...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22943 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22954: [DO-NOT-MERGE][POC] Enables Arrow optimization from R Da...

2018-11-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22954 So far, the regressions tests are passed and newly added test for R optimization is verified locally. Let me fix CRAN test and some nits. ---

[GitHub] spark pull request #22961: [SPARK-25947][SQL] Reduce memory usage in Shuffle...

2018-11-06 Thread mu5358271
GitHub user mu5358271 opened a pull request: https://github.com/apache/spark/pull/22961 [SPARK-25947][SQL] Reduce memory usage in ShuffleExchangeExec by selecting only the sort columns ## What changes were proposed in this pull request? When sorting rows,

[GitHub] spark issue #22961: [SPARK-25947][SQL] Reduce memory usage in ShuffleExchang...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22961 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22855: [SPARK-25839] [Core] Implement use of KryoPool in KryoSe...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22855 **[Test build #4417 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4417/testReport)** for PR 22855 at commit

[GitHub] spark issue #22165: [SPARK-25017][Core] Add test suite for ContextBarrierSta...

2018-11-06 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22165 gental ping @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #22944: [SPARK-25942][SQL] Fix Dataset.groupByKey to make...

2018-11-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22944#discussion_r231358690 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala --- @@ -262,25 +262,39 @@ object AppendColumns {

[GitHub] spark pull request #22944: [SPARK-25942][SQL] Fix Dataset.groupByKey to make...

2018-11-06 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22944#discussion_r231358749 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/object.scala --- @@ -262,25 +262,39 @@ object AppendColumns {

[GitHub] spark issue #22617: [SPARK-25484][SQL][TEST] Refactor ExternalAppendOnlyUnsa...

2018-11-06 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22617 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22937: [SPARK-25934] [Mesos] Don't propagate SPARK_CONF_DIR fro...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22937 **[Test build #98535 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98535/testReport)** for PR 22937 at commit

[GitHub] spark pull request #22956: [SPARK-25950][SQL] from_csv should respect to spa...

2018-11-06 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22956#discussion_r231370599 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/csvExpressions.scala --- @@ -92,8 +93,14 @@ case class CsvToStructs(

[GitHub] spark issue #22921: [SPARK-25908][CORE][SQL] Remove old deprecated items in ...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22921 **[Test build #98536 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98536/testReport)** for PR 22921 at commit

[GitHub] spark issue #22921: [SPARK-25908][CORE][SQL] Remove old deprecated items in ...

2018-11-06 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22921 **[Test build #98537 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98537/testReport)** for PR 22921 at commit

[GitHub] spark issue #22921: [SPARK-25908][CORE][SQL] Remove old deprecated items in ...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22921 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22921: [SPARK-25908][CORE][SQL] Remove old deprecated items in ...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22921 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98537/ Test FAILed. ---

[GitHub] spark issue #22867: [SPARK-25778] WriteAheadLogBackedBlockRDD in YARN Cluste...

2018-11-06 Thread gss2002
Github user gss2002 commented on the issue: https://github.com/apache/spark/pull/22867 @vanzin you are right! I appreciate the help with this one. I will cut a patch in the AM after testing on a large scale cluster job that is taking from IBM MQ and ETLing data and shipping off to

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17086 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #17086: [SPARK-24101][ML][MLLIB] ML Evaluators should use weight...

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17086 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22960: [SPARK-25955][TEST] Porting JSON tests for CSV functions

2018-11-06 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22960 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   3   4   >