[GitHub] spark issue #19338: [SPARK-22123][CORE] Add latest failure reason for task s...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19338 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82190/ Test FAILed. ---

[GitHub] spark issue #19338: [SPARK-22123][CORE] Add latest failure reason for task s...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19338 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19338: [SPARK-22123][CORE] Add latest failure reason for task s...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19338 **[Test build #82190 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82190/testReport)** for PR 19338 at commit

[GitHub] spark issue #19263: [SPARK-22050][CORE] Allow BlockUpdated events to be opti...

2017-09-26 Thread michaelmior
Github user michaelmior commented on the issue: https://github.com/apache/spark/pull/19263 @jerryshao I agree that the history server itself doesn't provide useful information but for detailed analysis I've found it helpful to persist the logs and write my own scripts to analyze

[GitHub] spark issue #19274: [SPARK-22056][Streaming] Add subconcurrency for KafkaRDD...

2017-09-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19274 Yes, I understand your scenario, but my concern is that your proposal is quite scenario specific, it may well serve your scenario, but somehow it breaks the design purpose of KafkaRDD. From my

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19287 **[Test build #82192 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82192/testReport)** for PR 19287 at commit

[GitHub] spark issue #19324: [SPARK-22103] Move HashAggregateExec parent consume to a...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19324 a late LGTM :) --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141039774 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala --- @@ -186,8 +186,7 @@ case class

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141039264 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala --- @@ -186,8 +186,7 @@ case class

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141039074 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -201,11 +201,14 @@ case class

[GitHub] spark issue #19340: [SPARK-22119] Add cosine distance to KMeans

2017-09-26 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/19340 @yanboliang may you please take a look at this when you have time? Thanks. --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141038684 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -201,11 +201,14 @@ case class FilterExec(condition:

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141038175 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala --- @@ -235,10 +228,12 @@ case class

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141037818 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -462,18 +464,36 @@ case class

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141037588 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -462,18 +464,36 @@ case class

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141037611 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/joins/BroadcastHashJoinExec.scala --- @@ -186,8 +186,7 @@ case class

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141037227 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/basicPhysicalOperators.scala --- @@ -201,11 +201,14 @@ case class FilterExec(condition:

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141035515 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -462,18 +464,36 @@ case class

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141035313 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -654,18 +680,23 @@ case class

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141035150 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -462,18 +464,36 @@ case class

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141034217 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -654,18 +680,23 @@ case class

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread goldmedal
Github user goldmedal commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141033740 --- Diff: python/pyspark/sql/readwriter.py --- @@ -420,7 +425,22 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread juliuszsompolski
Github user juliuszsompolski commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141033249 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -197,11 +197,14 @@ trait CodegenSupport

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/19337 Ok, thanks. we don't need to change the code here. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19350: [SPARK-22126][ML] Fix model-specific optimization suppor...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19350 **[Test build #82191 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82191/testReport)** for PR 19350 at commit

[GitHub] spark issue #19350: [SPARK-22126][ML] Fix model-specific optimization suppor...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19350 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19350: [SPARK-22126][ML] Fix model-specific optimization suppor...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82191/ Test PASSed. ---

[GitHub] spark pull request #19351: [SPARK-22127][CORE]The Master Register Applicatio...

2017-09-26 Thread srowen
Github user srowen commented on a diff in the pull request: https://github.com/apache/spark/pull/19351#discussion_r141032457 --- Diff: core/src/main/scala/org/apache/spark/deploy/master/Master.scala --- @@ -265,6 +265,9 @@ private[deploy] class Master( val app =

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141032306 --- Diff: python/pyspark/sql/readwriter.py --- @@ -420,7 +425,22 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141031904 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala --- @@ -599,10 +621,14 @@ case class

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r141031861 --- Diff: python/pyspark/sql/readwriter.py --- @@ -420,7 +425,22 @@ def csv(self, path, schema=None, sep=None, encoding=None, quote=None, escape=Non

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19337 `require` is better i think --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19351: [SPARK-22127][CORE]The Master Register Application Funct...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19351 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141031193 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -197,11 +197,14 @@ trait CodegenSupport extends

[GitHub] spark pull request #19351: [SPARK-22127][CORE]The Master Register Applicatio...

2017-09-26 Thread guoxiaolongzte
GitHub user guoxiaolongzte opened a pull request: https://github.com/apache/spark/pull/19351 [SPARK-22127][CORE]The Master Register Application Function requires an warn log to increase the waiting status ## What changes were proposed in this pull request? The Master

[GitHub] spark issue #19343: [SPARK-22121][SQL] Correct database location for namenod...

2017-09-26 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/19343 whoops, sorry I wrote [CORE] out of habit! > Spark SQL might not be deployed in the HDFS system. Conceptually, this HDFS-specific codes should not be part of our HiveExternalCatalog .

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/19337 Sorry, I got wrong. So you think assert is better here? now we use require. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-09-26 Thread jinxing64
Github user jinxing64 commented on the issue: https://github.com/apache/spark/pull/19330 Jenkins, retest this plesase. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19337 Why not possible ? `meanGammaChange` is always positive. The loop will become `while (true) {...}` and should use `require` instead of `assert` ---

[GitHub] spark issue #19208: [SPARK-21087] [ML] CrossValidator, TrainValidationSplit ...

2017-09-26 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19208 I will update this PR after #19350 get merged. We need to address another issue first. Thanks! --- - To unsubscribe,

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18747 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82187/ Test PASSed. ---

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/19337 Because epsilon is Double, negative value should not cause the code run into dead loop. All other setting in LDA using require for check or no check. Should we use assert only for this change? ---

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82187 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82187/testReport)** for PR 18747 at commit

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/19337 OK, I will change it to assert. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19350: [SPARK-22126][ML] Fix model-specific optimization suppor...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19350 **[Test build #82191 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82191/testReport)** for PR 19350 at commit

[GitHub] spark issue #19274: [SPARK-22056][Streaming] Add subconcurrency for KafkaRDD...

2017-09-26 Thread lonelytrooper
Github user lonelytrooper commented on the issue: https://github.com/apache/spark/pull/19274 lonelytrooper... : Pwill more executors be used in RDD#mapPartitions way ? I'll try that later to see if it works. I think if Spark provides a convenient way for this , it would help

[GitHub] spark issue #19350: [SPARK-22126][ML] Fix model-specific optimization suppor...

2017-09-26 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19350 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19350: [SPARK-22126][ML] Fix model-specific optimization suppor...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19350 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19350: [SPARK-22126][ML] Fix model-specific optimization suppor...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19350 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82189/ Test FAILed. ---

[GitHub] spark issue #19350: [SPARK-22126][ML] Fix model-specific optimization suppor...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19350 **[Test build #82189 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82189/testReport)** for PR 19350 at commit

[GitHub] spark issue #19168: [SPARK-21956][CORE] Fetch up to max bytes when buf reall...

2017-09-26 Thread caneGuy
Github user caneGuy commented on the issue: https://github.com/apache/spark/pull/19168 @cloud-fan Sorry for bothering you.Could you help review this?Thanks --- - To unsubscribe, e-mail:

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19337 OK. But I'm afraid accidently negative value cause the code run into dead loop. Maybe adding `assert` is better. --- - To

[GitHub] spark pull request #18034: [SPARK-20797][MLLIB]fix LocalLDAModel.save() bug.

2017-09-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/18034#discussion_r141016974 --- Diff: mllib/src/main/scala/org/apache/spark/mllib/clustering/LDAModel.scala --- @@ -468,7 +469,16 @@ object LocalLDAModel extends

[GitHub] spark issue #19338: [SPARK-22123][CORE] Add latest failure reason for task s...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19338 **[Test build #82190 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82190/testReport)** for PR 19338 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82184/ Test PASSed. ---

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82184 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82184/testReport)** for PR 19349 at commit

[GitHub] spark issue #19337: [SPARK-22114][ML][MLLIB]add epsilon for LDA

2017-09-26 Thread mpjlu
Github user mpjlu commented on the issue: https://github.com/apache/spark/pull/19337 Not check is also ok, user should know epsilon > 0 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #19350: [SPARK-22126][ML] Fix model-specific optimization suppor...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19350 **[Test build #82189 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82189/testReport)** for PR 19350 at commit

[GitHub] spark issue #19175: [SPARK-21964][SQL]Enable splitting the Aggregate (on Exp...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19175 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19175: [SPARK-21964][SQL]Enable splitting the Aggregate (on Exp...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19175 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82185/ Test PASSed. ---

[GitHub] spark pull request #19350: [SPARK-22126][ML] Fix model-specific optimization...

2017-09-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19350#discussion_r141015509 --- Diff: mllib/src/main/scala/org/apache/spark/ml/Estimator.scala --- @@ -82,5 +85,32 @@ abstract class Estimator[M <: Model[M]] extends

[GitHub] spark issue #19175: [SPARK-21964][SQL]Enable splitting the Aggregate (on Exp...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19175 **[Test build #82185 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82185/testReport)** for PR 19175 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82183/ Test PASSed. ---

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19349 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82183 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82183/testReport)** for PR 19349 at commit

[GitHub] spark pull request #19350: [SPARK-22126][ML] Fix model-specific optimization...

2017-09-26 Thread WeichenXu123
GitHub user WeichenXu123 opened a pull request: https://github.com/apache/spark/pull/19350 [SPARK-22126][ML] Fix model-specific optimization support for ML tuning ## What changes were proposed in this pull request? Push down fitting parallelization code from

[GitHub] spark pull request #19338: [SPARK-22123][CORE] Add latest failure reason for...

2017-09-26 Thread caneGuy
Github user caneGuy commented on a diff in the pull request: https://github.com/apache/spark/pull/19338#discussion_r141014410 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetBlacklist.scala --- @@ -61,6 +61,8 @@ private[scheduler] class TaskSetBlacklist(val conf:

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19186 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82186/ Test FAILed. ---

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19186 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19186 **[Test build #82186 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82186/testReport)** for PR 19186 at commit

[GitHub] spark issue #18760: [SPARK-21560][Core] Add hold mode for the LiveListenerBu...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18760 **[Test build #82188 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82188/testReport)** for PR 18760 at commit

[GitHub] spark issue #19263: [SPARK-22050][CORE] Allow BlockUpdated events to be opti...

2017-09-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19263 Hi @michaelmalak , history server only shows the last state of application before finished, and cached blocks can be evicted/unpersisted during the middle of application. So you probably cannot

[GitHub] spark issue #19274: [SPARK-22056][Streaming] Add subconcurrency for KafkaRDD...

2017-09-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19274 Hi @loneknightpy , think a bit on your PR, I think this can also be done in the user side. User could create several threads in one task (RDD#mapPartitions) to consume the records concurrently,

[GitHub] spark pull request #19324: [SPARK-22103] Move HashAggregateExec parent consu...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19324#discussion_r141003450 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -328,6 +331,22 @@ class

[GitHub] spark pull request #19338: [SPARK-22123][CORE] Add latest failure reason for...

2017-09-26 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19338#discussion_r141002261 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetBlacklist.scala --- @@ -61,6 +61,8 @@ private[scheduler] class TaskSetBlacklist(val conf:

[GitHub] spark pull request #19338: [SPARK-22123][CORE] Add latest failure reason for...

2017-09-26 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19338#discussion_r141001713 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala --- @@ -671,8 +671,9 @@ private[spark] class TaskSetManager(

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-26 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r140996395 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala --- @@ -74,6 +81,10 @@ class TaskInfo( gettingResultTime = time }

[GitHub] spark pull request #19287: [SPARK-22074][Core] Task killed by other attempt ...

2017-09-26 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/19287#discussion_r140995846 --- Diff: core/src/main/scala/org/apache/spark/scheduler/TaskInfo.scala --- @@ -66,6 +66,13 @@ class TaskInfo( */ var finishTime: Long = 0

[GitHub] spark issue #19348: [BUILD] Close stale PRs

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19348 Added. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19348: [BUILD] Close stale PRs

2017-09-26 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19348 I'd add https://github.com/apache/spark/pull/19236 https://github.com/apache/spark/pull/19244 https://github.com/apache/spark/pull/19300

[GitHub] spark pull request #19339: [SPARK-22112][PYSPARK] Add an API to create a Dat...

2017-09-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19339#discussion_r140987769 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala --- @@ -456,6 +456,40 @@ class DataFrameReader private[sql](sparkSession:

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18747 **[Test build #82187 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82187/testReport)** for PR 18747 at commit

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-09-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18747 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19186 **[Test build #82186 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82186/testReport)** for PR 19186 at commit

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-26 Thread zhengruifeng
Github user zhengruifeng commented on the issue: https://github.com/apache/spark/pull/19186 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #18747: [WIP][SPARK-20822][SQL] Generate code to directly get va...

2017-09-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/18747 Jenkins, retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19175: [SPARK-21964][SQL]Enable splitting the Aggregate (on Exp...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19175 **[Test build #82185 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82185/testReport)** for PR 19175 at commit

[GitHub] spark pull request #19345: [SPARK-22124][SQL] Sample and Limit should also d...

2017-09-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19345 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19345: [SPARK-22124][SQL] Sample and Limit should also defer in...

2017-09-26 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/19345 LGTM, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82184 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82184/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19349 **[Test build #82183 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/82183/testReport)** for PR 19349 at commit

[GitHub] spark issue #19349: [SPARK-22125][PYSPARK][SQL] Enable Arrow Stream format f...

2017-09-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19349 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19330 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82174/ Test FAILed. ---

[GitHub] spark issue #19287: [SPARK-22074][Core] Task killed by other attempt task sh...

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19287 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19330 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #19186: [SPARK-21972][ML] Add param handlePersistence

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19186 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/82179/ Test FAILed. ---

[GitHub] spark issue #19348: [BUILD] Close stale PRs

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19348 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19330: [SPARK-18134][SQL] Orderable MapType

2017-09-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19330 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

<    1   2   3   4   5   >