[GitHub] spark issue #22246: [SPARK-25235] [SHELL] Merge the REPL code in Scala 2.11 ...

2018-08-27 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/22246 Ping @srowen --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22246: [SPARK-25235] [SHELL] Merge the REPL code in Scal...

2018-08-27 Thread dbtsai
GitHub user dbtsai opened a pull request: https://github.com/apache/spark/pull/22246 [SPARK-25235] [SHELL] Merge the REPL code in Scala 2.11 and 2.12 branches ## What changes were proposed in this pull request? Using some reflection tricks to merge Scala 2.11 and 2.12

[GitHub] spark issue #22244: [WIP][SPARK-24721][SPARK-25213][SQL] extract python UDF ...

2018-08-27 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/22244 @cloud-fan Thanks! I will take a look later today and incorporate this with my patch. --- - To unsubscribe, e-mail:

[GitHub] spark issue #22208: [SPARK-25216][SQL] Improve error message when a column c...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22208 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22208: [SPARK-25216][SQL] Improve error message when a column c...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22208 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r213057049 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -671,7 +674,7 @@ case class

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22173 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95300/ Test FAILed. ---

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #95300 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95300/testReport)** for PR 22173 at commit

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22173 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21976: [SPARK-24909][core] Always unregister pending par...

2018-08-27 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/21976#discussion_r213056190 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2474,19 +2478,21 @@ class DAGSchedulerSuite extends

[GitHub] spark issue #22208: [SPARK-25216][SQL] Improve error message when a column c...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22208 **[Test build #95301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95301/testReport)** for PR 22208 at commit

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22173 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95299/ Test FAILed. ---

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22173 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22245: [SPARK-24882][FOLLOWUP] Fix flaky synchronization in Kaf...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22245 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #95299 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95299/testReport)** for PR 22173 at commit

[GitHub] spark issue #22245: [SPARK-24882][FOLLOWUP] Fix flaky synchronization in Kaf...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22245 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95298/ Test PASSed. ---

[GitHub] spark issue #22245: [SPARK-24882][FOLLOWUP] Fix flaky synchronization in Kaf...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22245 **[Test build #95298 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95298/testReport)** for PR 22245 at commit

[GitHub] spark issue #22226: [SPARK-25252][SQL] Support arrays of any types by to_jso...

2018-08-27 Thread MaxGekk
Github user MaxGekk commented on the issue: https://github.com/apache/spark/pull/6 > Probably, you'd be better to file separate jira for each function. > +1 for separate JIRA. I created the JIRA ticket: https://issues.apache.org/jira/browse/SPARK-25252 ---

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22042 **[Test build #95297 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95297/testReport)** for PR 22042 at commit

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22042 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22042 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95297/ Test PASSed. ---

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #95300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95300/testReport)** for PR 22173 at commit

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-27 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r213050406 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -671,7 +674,7 @@ case class AlterTableRecoverPartitionsCommand(

[GitHub] spark pull request #22085: [SPARK-25095][PySpark] Python support for Barrier...

2018-08-27 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/22085#discussion_r213050049 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -180,7 +188,73 @@ private[spark] abstract class

[GitHub] spark pull request #22238: [SPARK-25245][DOCS][SS] Explain regarding limitin...

2018-08-27 Thread arunmahadevan
Github user arunmahadevan commented on a diff in the pull request: https://github.com/apache/spark/pull/22238#discussion_r213049895 --- Diff: docs/structured-streaming-programming-guide.md --- @@ -2812,6 +2812,12 @@ See [Input Sources](#input-sources) and [Output

[GitHub] spark pull request #22213: [SPARK-25221][DEPLOY] Consistent trailing whitesp...

2018-08-27 Thread gerashegalov
Github user gerashegalov commented on a diff in the pull request: https://github.com/apache/spark/pull/22213#discussion_r213049701 --- Diff: core/src/main/scala/org/apache/spark/util/Utils.scala --- @@ -2062,8 +2062,10 @@ private[spark] object Utils extends Logging { try {

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #95299 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95299/testReport)** for PR 22173 at commit

[GitHub] spark pull request #22146: [SPARK-24434][K8S] pod template files

2018-08-27 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/22146#discussion_r213047538 --- Diff: docs/running-on-kubernetes.md --- @@ -185,6 +185,21 @@ To use a secret through an environment variable use the following options to the

[GitHub] spark issue #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in SQL fun...

2018-08-27 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/18447 Yea I'd probably reject this for now, until we see bigger needs for it. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22241: [SPARK-25249][CORE][TEST]add a unit test for Open...

2018-08-27 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22241 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22245: [SPARK-24882][FOLLOWUP] Fix flaky synchronization in Kaf...

2018-08-27 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/22245 LGTM pending tests --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22192: [SPARK-24918][Core] Executor Plugin API

2018-08-27 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22192#discussion_r213045752 --- Diff: core/src/main/scala/org/apache/spark/executor/Executor.scala --- @@ -130,6 +130,16 @@ private[spark] class Executor( private val

[GitHub] spark issue #22245: [SPARK-24882][FOLLOWUP] Fix flaky synchronization in Kaf...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22245 **[Test build #95298 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95298/testReport)** for PR 22245 at commit

[GitHub] spark issue #22245: [SPARK-24882][FOLLOWUP] Fix flaky synchronization in Kaf...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22245 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/3 **[Test build #4296 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4296/testReport)** for PR 3 at commit

[GitHub] spark issue #22245: [SPARK-24882][FOLLOWUP] Fix flaky synchronization in Kaf...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22245 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22245: [SPARK-24882][FOLLOWUP] Fix flaky synchronization...

2018-08-27 Thread jose-torres
GitHub user jose-torres opened a pull request: https://github.com/apache/spark/pull/22245 [SPARK-24882][FOLLOWUP] Fix flaky synchronization in Kafka tests. ## What changes were proposed in this pull request? Fix flaky synchronization in Kafka tests - we need to use the scan

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22241 Merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22085: [SPARK-25095][PySpark] Python support for Barrier...

2018-08-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22085#discussion_r213043068 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -180,7 +188,73 @@ private[spark] abstract class BasePythonRunner[IN,

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22042 **[Test build #95297 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95297/testReport)** for PR 22042 at commit

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22042 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22042 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21330: [SPARK-22234] Support distinct window functions

2018-08-27 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/21330 cc @jiangxb1987 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21976: [SPARK-24909][core] Always unregister pending par...

2018-08-27 Thread jiangxb1987
Github user jiangxb1987 commented on a diff in the pull request: https://github.com/apache/spark/pull/21976#discussion_r213042176 --- Diff: core/src/test/scala/org/apache/spark/scheduler/DAGSchedulerSuite.scala --- @@ -2474,19 +2478,21 @@ class DAGSchedulerSuite extends

[GitHub] spark issue #21546: [SPARK-23030][SQL][PYTHON] Use Arrow stream format for c...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21546 **[Test build #95296 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95296/testReport)** for PR 21546 at commit

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-27 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r213041684 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -671,7 +674,7 @@ case class

[GitHub] spark issue #22042: [SPARK-25005][SS]Support non-consecutive offsets for Kaf...

2018-08-27 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/22042 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21546: [SPARK-23030][SQL][PYTHON] Use Arrow stream format for c...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21546: [SPARK-23030][SQL][PYTHON] Use Arrow stream format for c...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21546 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/3 **[Test build #4296 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4296/testReport)** for PR 3 at commit

[GitHub] spark issue #22197: [SPARK-25207][SQL] Case-insensitve field resolution for ...

2018-08-27 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/22197 Thanks. I got it. Definitely, it's irrelevant to this and an intentional regression due to that reverting. --- - To

[GitHub] spark pull request #22024: [SPARK-25034][CORE] Remove allocations in onBlock...

2018-08-27 Thread vincent-grosbois
Github user vincent-grosbois commented on a diff in the pull request: https://github.com/apache/spark/pull/22024#discussion_r213037747 --- Diff: core/src/main/scala/org/apache/spark/network/BlockTransferService.scala --- @@ -101,15 +101,7 @@ abstract class BlockTransferService

[GitHub] spark issue #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22236 **[Test build #95294 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95294/testReport)** for PR 22236 at commit

[GitHub] spark issue #21638: [SPARK-22357][CORE] SparkContext.binaryFiles ignore minP...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21638 **[Test build #95295 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95295/testReport)** for PR 21638 at commit

[GitHub] spark issue #21638: [SPARK-22357][CORE] SparkContext.binaryFiles ignore minP...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21638 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21638: [SPARK-22357][CORE] SparkContext.binaryFiles ignore minP...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21638 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22236 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22236 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22173 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95293/ Test FAILed. ---

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22173 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #95293 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95293/testReport)** for PR 22173 at commit

[GitHub] spark pull request #21977: [SPARK-25004][CORE] Add spark.executor.pyspark.me...

2018-08-27 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r213035238 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -91,6 +91,13 @@ private[spark] class Client(

[GitHub] spark issue #22173: [SPARK-24335] Spark external shuffle server improvement ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22173 **[Test build #95293 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95293/testReport)** for PR 22173 at commit

[GitHub] spark issue #22236: [SPARK-10697][ML] Add lift to Association rules

2018-08-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22236 Yeah, I like that idea. Just compute it on initializing the model. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #22085: [SPARK-25095][PySpark] Python support for Barrier...

2018-08-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22085#discussion_r213032992 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -180,7 +188,73 @@ private[spark] abstract class BasePythonRunner[IN,

[GitHub] spark pull request #22243: [MINOR] Avoid code duplication for nullable in Hi...

2018-08-27 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22243#discussion_r213029487 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -155,6 +155,8 @@ trait

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-27 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r213026874 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -815,6 +815,24 @@ class Dataset[T] private[sql](

[GitHub] spark issue #22244: [WIP][SPARK-24721][SPARK-25213][SQL] extract python UDF ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22244 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22244: [WIP][SPARK-24721][SPARK-25213][SQL] extract python UDF ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22244 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95292/ Test FAILed. ---

[GitHub] spark issue #22244: [WIP][SPARK-24721][SPARK-25213][SQL] extract python UDF ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22244 **[Test build #95292 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95292/testReport)** for PR 22244 at commit

[GitHub] spark pull request #22243: [MINOR] Avoid code duplication for nullable in Hi...

2018-08-27 Thread mn-mikke
Github user mn-mikke commented on a diff in the pull request: https://github.com/apache/spark/pull/22243#discussion_r213022884 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/higherOrderFunctions.scala --- @@ -155,6 +155,8 @@ trait

[GitHub] spark issue #22244: [WIP][SPARK-24721][SPARK-25213][SQL] extract python UDF ...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22244 **[Test build #95292 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95292/testReport)** for PR 22244 at commit

[GitHub] spark issue #22244: [WIP][SPARK-24721][SPARK-25213][SQL] extract python UDF ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22244 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22244: [WIP][SPARK-24721][SPARK-25213][SQL] extract python UDF ...

2018-08-27 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22244 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22244: [WIP][SPARK-24721][SPARK-25213][SQL] extract python UDF ...

2018-08-27 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22244 cc @icexelloss @HyukjinKwon @rdblue @icexelloss feel free to take this over and verify if it can pass the tests you added in #22104 , thanks! ---

[GitHub] spark pull request #22244: [WIP][SPARK-24721][SPARK-25213][SQL] extract pyth...

2018-08-27 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/22244 [WIP][SPARK-24721][SPARK-25213][SQL] extract python UDF at the end of optimizer ## What changes were proposed in this pull request? In https://github.com/apache/spark/pull/12127 , we

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-27 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21976 just an fyi, the other jira is https://issues.apache.org/jira/browse/SPARK-25250, its related to a race with SPARK-23433 ---

[GitHub] spark issue #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in Pyspark

2018-08-27 Thread RussellSpitzer
Github user RussellSpitzer commented on the issue: https://github.com/apache/spark/pull/21990 What I wanted was to just call the Scala Methods, instead of having half the code and half in python, but we create the JVM in the SparkContext creation code so this ends up not being a good

[GitHub] spark pull request #22184: [SPARK-25132][SQL][DOC] Add migration doc for cas...

2018-08-27 Thread seancxmao
Github user seancxmao commented on a diff in the pull request: https://github.com/apache/spark/pull/22184#discussion_r213020789 --- Diff: docs/sql-programming-guide.md --- @@ -1895,6 +1895,10 @@ working with timestamps in `pandas_udf`s to get the best performance, see -

[GitHub] spark issue #22241: [SPARK-25249][CORE][TEST]add a unit test for OpenHashMap

2018-08-27 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22241 @kiszk I guess it's because in this case the underlying value type is a primitive like int or long, so null can't be returned? ---

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-27 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22162 sure, no worries @kiszk, I can take it if needed. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #22112: [SPARK-23243][Core] Fix RDD.repartition() data co...

2018-08-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22112#discussion_r213009399 --- Diff: core/src/main/scala/org/apache/spark/Partitioner.scala --- @@ -33,6 +33,9 @@ import org.apache.spark.util.random.SamplingUtils /** * An

[GitHub] spark pull request #22112: [SPARK-23243][Core] Fix RDD.repartition() data co...

2018-08-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22112#discussion_r213017779 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1865,6 +1871,62 @@ abstract class RDD[T: ClassTag]( // RDD chain.

[GitHub] spark pull request #22112: [SPARK-23243][Core] Fix RDD.repartition() data co...

2018-08-27 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/22112#discussion_r213010846 --- Diff: core/src/main/scala/org/apache/spark/rdd/RDD.scala --- @@ -1918,3 +1980,19 @@ object RDD { new DoubleRDDFunctions(rdd.map(x =>

[GitHub] spark issue #22024: [SPARK-25034][CORE] Remove allocations in onBlockFetchSu...

2018-08-27 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/22024 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22024: [SPARK-25034][CORE] Remove allocations in onBlock...

2018-08-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22024#discussion_r213015113 --- Diff: core/src/main/scala/org/apache/spark/network/BlockTransferService.scala --- @@ -101,15 +101,7 @@ abstract class BlockTransferService extends

[GitHub] spark pull request #22024: [SPARK-25034][CORE] Remove allocations in onBlock...

2018-08-27 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/22024#discussion_r213015245 --- Diff: core/src/main/scala/org/apache/spark/broadcast/TorrentBroadcast.scala --- @@ -160,7 +160,13 @@ private[spark] class TorrentBroadcast[T:

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-27 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/22162 Would someone please take it? I have less bandwidth next two days since I will be in a training session at my office. --- -

[GitHub] spark pull request #20637: [SPARK-23466][SQL] Remove redundant null checks i...

2018-08-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/20637#discussion_r213013507 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ExpressionEvalHelper.scala --- @@ -223,8 +223,9 @@ trait

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-27 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r213010807 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -815,6 +815,24 @@ class Dataset[T] private[sql](

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-27 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r213010706 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -815,6 +815,24 @@ class Dataset[T] private[sql](

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-27 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r213010879 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -815,6 +815,24 @@ class Dataset[T] private[sql](

[GitHub] spark pull request #22162: [spark-24442][SQL] Added parameters to control th...

2018-08-27 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22162#discussion_r213010672 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -815,6 +815,24 @@ class Dataset[T] private[sql](

[GitHub] spark pull request #21990: [SPARK-25003][PYSPARK] Use SessionExtensions in P...

2018-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21990#discussion_r213010408 --- Diff: python/pyspark/sql/session.py --- @@ -218,7 +218,9 @@ def __init__(self, sparkContext, jsparkSession=None):

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-27 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22162 sure @HyukjinKwon, thanks for pinging me anyway. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #22223: [SPARK-25233][Streaming] Give the user the option of spe...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/3 **[Test build #4295 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4295/testReport)** for PR 3 at commit

[GitHub] spark pull request #22233: [SPARK-25240][SQL] Fix for a deadlock in RECOVER ...

2018-08-27 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/22233#discussion_r213009058 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala --- @@ -671,7 +674,7 @@ case class AlterTableRecoverPartitionsCommand(

[GitHub] spark issue #22226: [SPARK-24391][SQL] Support arrays of any types by to_jso...

2018-08-27 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/6 **[Test build #95291 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95291/testReport)** for PR 6 at commit

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22162 @viirya, @kiszk, @mgaido91 and @maropu, would you be interested in taking this over if this gets inactive for few more days? ---

[GitHub] spark issue #22162: [spark-24442][SQL] Added parameters to control the defau...

2018-08-27 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22162 ping @AndrewKL --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

<    1   2   3   4   5   6   >