[GitHub] spark pull request #22219: [SPARK-25224][SQL] Improvement of Spark SQL Thrif...

2018-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22219#discussion_r213006794 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -3237,6 +3237,28 @@ class Dataset[T] private[sql](

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21976 **[Test build #95290 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95290/testReport)** for PR 21976 at commit

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21976 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21976 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-27 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21976 test this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-27 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21976 Ok the issue is not related to this patch will be filing a separate jira for it. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22238: [SPARK-25245][DOCS][SS] Explain regarding limitin...

2018-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22238#discussion_r213005430 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -2812,6 +2812,12 @@ See [Input Sources](#input-sources) and [Output

[GitHub] spark pull request #22221: [SPARK-25231] : Fix synchronization of executor h...

2018-08-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r213005226 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -465,7 +465,7 @@ private[spark] class TaskSchedulerImpl( var

[GitHub] spark issue #22226: [SPARK-24391][SQL] Support arrays of any types by to_jso...

2018-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/6 +1 for separate JIRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22234: [SPARK-25241][SQL] Configurable empty values when readin...

2018-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22234 Seems okay but I or someone else should take a closer look before getting this in. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22237 I think one thing we could do this for now is, only to support both FAILFAST and PERMISSIVE mode (and throws an exception otherwise, to match the current behaviour to PERMISSIVE mode, explain

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r213003614 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -469,4 +470,23 @@ class JsonFunctionsSuite extends QueryTest

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-08-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r213003599 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -130,6 +130,16 @@ private[spark] class Executor( private val

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-08-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r213003400 --- Diff: core/src/test/java/org/apache/spark/ExecutorPluginSuite.java --- @@ -0,0 +1,104 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark pull request #21330: [SPARK-22234] Support distinct window functions

2018-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21330#discussion_r213003383 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1883,7 +1883,19 @@ class Analyzer( //

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/3 **[Test build #4295 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4295/testReport)** for PR 3 at commit

[GitHub] spark issue #22243: [MINOR] Avoid code duplication for nullable in Higher Or...

2018-08-27 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/22243 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21330: [SPARK-22234] Support distinct window functions

2018-08-27 Thread jinxing64
Github user jinxing64 commented on a diff in the pull request: https://github.com/apache/spark/pull/21330#discussion_r213000357 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -1883,7 +1883,19 @@ class Analyzer( //

[GitHub] spark issue #22243: [MINOR] Avoid code duplication for nullable in Higher Or...

2018-08-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22243 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22241 Adding tests looks good. Probably beyond the purpose of this PR, is the following difference intentionally designed? ``` // If the data type is in @specialized annotation, and

[GitHub] spark issue #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22236 @srowen then what about recomputing when reading saved models? This seems a good compromise to me as it saves the writing of the data, it allows having lift for old models, but it doesn't

[GitHub] spark pull request #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22236#discussion_r212997377 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -326,6 +327,20 @@ object FPGrowthModel extends MLReadable[FPGrowthModel] {

[GitHub] spark pull request #22226: [SPARK-24391][SQL] Support arrays of any types by...

2018-08-27 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/6#discussion_r212996418 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -32,29 +32,29 @@ object JacksonUtils { }

[GitHub] spark pull request #22226: [SPARK-24391][SQL] Support arrays of any types by...

2018-08-27 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/6#discussion_r212995984 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala --- @@ -65,6 +66,8 @@ private[sql] class JacksonGenerator(

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #95289 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95289/testReport)** for PR 22173 at commit

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22173 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95289/ Test FAILed. ---

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22173 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22243: [MINOR] Avoid code duplication for nullable in Higher Or...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22243 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22243: [MINOR] Avoid code duplication for nullable in Higher Or...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22243 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22236 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22236 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #95289 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95289/testReport)** for PR 22173 at commit

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22241 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22236 **[Test build #95287 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95287/testReport)** for PR 22236 at commit

[GitHub] spark issue #22243: [MINOR] Avoid code duplication for nullable in Higher Or...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22243 **[Test build #95288 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95288/testReport)** for PR 22243 at commit

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22241 **[Test build #95286 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95286/testReport)** for PR 22241 at commit

[GitHub] spark issue #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory operat...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21968 **[Test build #4294 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4294/testReport)** for PR 21968 at commit

[GitHub] spark pull request #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22236#discussion_r212987480 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/FPGrowth.scala --- @@ -326,6 +327,20 @@ object FPGrowthModel extends MLReadable[FPGrowthModel] {

[GitHub] spark pull request #22227: [SPARK-25202] [Core] Implements split with limit ...

2018-08-27 Thread phegstrom
Github user phegstrom commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r212986332 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -232,30 +232,41 @@ case class RLike(left:

[GitHub] spark issue #22029: [SPARK-24395][SQL] IN operator should return NULL when c...

2018-08-27 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22029 kindly ping @cloud-fan @gatorsmile --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #22221: [SPARK-25231] : Fix synchronization of executor h...

2018-08-27 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/1#discussion_r212978188 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala --- @@ -465,7 +465,7 @@ private[spark] class TaskSchedulerImpl(

[GitHub] spark pull request #22227: [SPARK-25202] [Core] Implements split with limit ...

2018-08-27 Thread phegstrom
Github user phegstrom commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r212977986 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -232,30 +232,41 @@ case class RLike(left:

[GitHub] spark pull request #22227: [SPARK-25202] [Core] Implements split with limit ...

2018-08-27 Thread phegstrom
Github user phegstrom commented on a diff in the pull request: https://github.com/apache/spark/pull/7#discussion_r212977742 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/regexpExpressions.scala --- @@ -232,30 +232,41 @@ case class RLike(left:

[GitHub] spark issue #22243: [MINOR] Avoid code duplication for nullable in Higher Or...

2018-08-27 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22243 cc @ueshin @mn-mikke --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22243: [MINOR] Avoid code duplication for nullable in Hi...

2018-08-27 Thread mgaido91
GitHub user mgaido91 opened a pull request: https://github.com/apache/spark/pull/22243 [MINOR] Avoid code duplication for nullable in Higher Order function ## What changes were proposed in this pull request? All `SimpleHigherOrderFunction `s have the same `nullable`

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-27 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21976 still working on it, getting close, takes a bit to reproduce the case so taking a bit of time. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22241 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95285/ Test PASSed. ---

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22241 **[Test build #95285 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95285/testReport)** for PR 22241 at commit

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-08-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r212964193 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -17,11 +17,13 @@ package org.apache.spark.executor

[GitHub] spark pull request #22224: [SPARK-24090][K8S] Update running-on-kubernetes.m...

2018-08-27 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/4#discussion_r212963256 --- Diff: docs/running-on-kubernetes.md --- @@ -299,21 +328,15 @@ RBAC authorization and how to configure Kubernetes service accounts for pods, pl

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-27 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r212960763 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,9 @@ trait

[GitHub] spark issue #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory operat...

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21968 The change looks reasonable to me, so can you trigger tests? @gatorsmile @cloud-fan @hvanhovell --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r212959634 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala --- @@ -141,9 +141,6 @@ class

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r212956571 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala --- @@ -48,6 +48,8 @@ class

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22241 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95282/ Test PASSed. ---

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22241 **[Test build #95282 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95282/testReport)** for PR 22241 at commit

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22198 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22198 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95284/ Test PASSed. ---

[GitHub] spark issue #22198: [SPARK-25121][SQL] Supports multi-part table names for b...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22198 **[Test build #95284 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95284/testReport)** for PR 22198 at commit

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21860 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21860 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95283/ Test PASSed. ---

[GitHub] spark issue #21860: [SPARK-24901][SQL]Merge the codegen of RegularHashMap an...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21860 **[Test build #95283 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95283/testReport)** for PR 21860 at commit

[GitHub] spark pull request #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22236#discussion_r212941261 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/fpm/AssociationRules.scala --- @@ -107,7 +124,8 @@ object AssociationRules {

[GitHub] spark pull request #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22236#discussion_r212940988 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/fpm/AssociationRules.scala --- @@ -61,6 +61,18 @@ class AssociationRules private[fpm] ( */

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-08-27 Thread LucaCanali
Github user LucaCanali commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r212939411 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -17,11 +17,13 @@ package org.apache.spark.executor

[GitHub] spark pull request #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22236#discussion_r212936085 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/fpm/AssociationRules.scala --- @@ -61,6 +61,18 @@ class AssociationRules private[fpm] ( */

[GitHub] spark pull request #22218: [SPARK-25228][CORE]Add executor CPU time metric.

2018-08-27 Thread LucaCanali
Github user LucaCanali commented on a diff in the pull request: https://github.com/apache/spark/pull/22218#discussion_r212933292 --- Diff: core/src/main/scala/org/apache/spark/executor/ExecutorSource.scala --- @@ -73,6 +75,13 @@ class ExecutorSource(threadPool: ThreadPoolExecutor,

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-27 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r212928747 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -554,18 +554,22 @@ case class JsonToStructs(

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-27 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/22197 @dongjoon-hyun In the **schema matched case** as you listed, it is expected behavior in current master. ``` spark.sparkContext.hadoopConfiguration.setInt("parquet.block.size", 8 * 1024 *

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-27 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r212926645 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,9 @@ trait

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-27 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r212925256 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -469,4 +470,23 @@ class JsonFunctionsSuite extends QueryTest with

[GitHub] spark issue #18099: [SPARK-18406][CORE][Backport-2.1] Race between end-of-ta...

2018-08-27 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/18099 same issue in spark 2.2.1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-27 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r212924389 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/JsonFunctionsSuite.scala --- @@ -469,4 +470,23 @@ class JsonFunctionsSuite extends QueryTest with

[GitHub] spark pull request #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_...

2018-08-27 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22237#discussion_r212922787 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -554,18 +554,22 @@ case class JsonToStructs(

[GitHub] spark issue #22226: [SPARK-24391][SQL] Support arrays of any types by to_jso...

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/6 Probably, you'd be better to file separate jira for each function. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22226: [SPARK-24391][SQL] Support arrays of any types by...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/6#discussion_r212918991 --- Diff: sql/core/src/test/resources/sql-tests/inputs/json-functions.sql --- @@ -51,3 +51,8 @@ select from_json('[null, {"a":2}]', 'array>');

[GitHub] spark pull request #22226: [SPARK-24391][SQL] Support arrays of any types by...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/6#discussion_r212918588 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonUtils.scala --- @@ -32,29 +32,29 @@ object JacksonUtils { }

[GitHub] spark pull request #22205: [SPARK-25212][SQL] Support Filter in ConvertToLoc...

2018-08-27 Thread bogdanrdc
Github user bogdanrdc commented on a diff in the pull request: https://github.com/apache/spark/pull/22205#discussion_r212918379 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -130,6 +130,10 @@ abstract class

[GitHub] spark pull request #22226: [SPARK-24391][SQL] Support arrays of any types by...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/6#discussion_r212917164 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonGenerator.scala --- @@ -65,6 +66,8 @@ private[sql] class JacksonGenerator(

[GitHub] spark issue #22226: [SPARK-24391][SQL] Support arrays of any types by to_jso...

2018-08-27 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/6 @maropu The JIRA ticket was about both `to_json` and `from_json` originally. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22242: Branch 2.3

2018-08-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22242 @ArunkumarRamanan would it be possible to close this? probably something is wrong. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22226: [SPARK-24391][SQL] Support arrays of any types by to_jso...

2018-08-27 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/6 @MaxGekk btw, why did you attach this pr to the resolved jira? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r212909243 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -164,9 +164,8 @@ private[joins] class

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22241 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22241 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22241 **[Test build #95285 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95285/testReport)** for PR 22241 at commit

[GitHub] spark issue #22242: Branch 2.3

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22242 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22242: Branch 2.3

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22242 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22242: Branch 2.3

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22242 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22242: Branch 2.3

2018-08-27 Thread ArunkumarRamanan
GitHub user ArunkumarRamanan opened a pull request: https://github.com/apache/spark/pull/22242 Branch 2.3 ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix) ## How was this patch tested? (Please explain how

[GitHub] spark pull request #22241: [SPARK-25249][CORE][TEST]add a unit test for Open...

2018-08-27 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/22241#discussion_r212902991 --- Diff: core/src/test/scala/org/apache/spark/util/collection/OpenHashMapSuite.scala --- @@ -194,4 +194,42 @@ class OpenHashMapSuite extends

[GitHub] spark pull request #22241: [SPARK-25249][CORE][TEST]add a unit test for Open...

2018-08-27 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/22241#discussion_r212901257 --- Diff: core/src/test/scala/org/apache/spark/util/collection/OpenHashMapSuite.scala --- @@ -194,4 +194,42 @@ class OpenHashMapSuite extends SparkFunSuite

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-27 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r212898251 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala --- @@ -44,6 +44,12 @@ class

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/22241 thanks @maropu --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22241: [SPARK-25249][TEST]add a unit test for OpenHashMa...

2018-08-27 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/22241#discussion_r212897158 --- Diff: core/src/test/scala/org/apache/spark/util/collection/OpenHashMapSuite.scala --- @@ -194,4 +194,42 @@ class OpenHashMapSuite extends

[GitHub] spark pull request #21968: [SPARK-24999][SQL]Reduce unnecessary 'new' memory...

2018-08-27 Thread heary-cao
Github user heary-cao commented on a diff in the pull request: https://github.com/apache/spark/pull/21968#discussion_r212896640 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/HashedRelation.scala --- @@ -164,9 +164,8 @@ private[joins] class

[GitHub] spark pull request #22184: [SPARK-25132][SQL][DOC] Add migration doc for cas...

2018-08-27 Thread seancxmao
Github user seancxmao commented on a diff in the pull request: https://github.com/apache/spark/pull/22184#discussion_r212894532 --- Diff: docs/sql-programming-guide.md --- @@ -1895,6 +1895,10 @@ working with timestamps in `pandas_udf`s to get the best performance, see -

<    1   2   3   4   5   6   >