[GitHub] spark issue #19579: [SPARK-22356][SQL] data source table should support over...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19579 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83073/ Test PASSed. ---

[GitHub] spark issue #19579: [SPARK-22356][SQL] data source table should support over...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19579 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19559 **[Test build #83076 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83076/testReport)** for PR 19559 at commit

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19559 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83076/ Test FAILed. ---

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19559 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-10-26 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/19459 I guess this pr is almost ready to be merged. I'd cc @gatorsmile @cloud-fan for another look. --- - To unsubscribe, e-mail:

[GitHub] spark issue #19529: [SPARK-22308] Support alternative unit testing styles in...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19529 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19529: [SPARK-22308] Support alternative unit testing styles in...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19529 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83074/ Test PASSed. ---

[GitHub] spark issue #19579: [SPARK-22356][SQL] data source table should support over...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19579 **[Test build #83073 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83073/testReport)** for PR 19579 at commit

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19559 **[Test build #83077 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83077/testReport)** for PR 19559 at commit

[GitHub] spark issue #19529: [SPARK-22308] Support alternative unit testing styles in...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19529 **[Test build #83074 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83074/testReport)** for PR 19529 at commit

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19559 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19559 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83077/ Test FAILed. ---

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19559 **[Test build #83077 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83077/testReport)** for PR 19559 at commit

[GitHub] spark pull request #19511: [SPARK-22293][SQL] Avoid unnecessary traversal in...

2017-10-26 Thread ConeyLiu
Github user ConeyLiu closed the pull request at: https://github.com/apache/spark/pull/19511 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r147060120 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -78,6 +79,22 @@ case class

[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r147060840 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -78,6 +79,22 @@ case class

[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r147062715 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -801,10 +832,49 @@ class

[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r147061443 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -801,10 +832,49 @@ class

[GitHub] spark pull request #19480: [SPARK-22226][SQL] splitExpression can create too...

2017-10-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19480#discussion_r147061319 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/codegen/CodeGenerator.scala --- @@ -801,10 +832,49 @@ class

[GitHub] spark issue #19529: [SPARK-22308] Support alternative unit testing styles in...

2017-10-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19529 Thanks! Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19529: [SPARK-22308] Support alternative unit testing st...

2017-10-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19529 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19559 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19559 **[Test build #83078 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83078/testReport)** for PR 19559 at commit

[GitHub] spark issue #19519: [SPARK-21840][core] Add trait that allows conf to be dir...

2017-10-26 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/19519 LGTM, merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-26 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r147066762 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -175,17 +176,36 @@ trait

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-26 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r147067641 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -451,15 +465,20 @@ trait

[GitHub] spark pull request #19519: [SPARK-21840][core] Add trait that allows conf to...

2017-10-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19519 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, ...

2017-10-26 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/19559#discussion_r147067888 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -783,6 +783,25 @@ class Analyzer( }

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-26 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r147067951 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -349,13 +349,22

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread DonnyZone
Github user DonnyZone commented on the issue: https://github.com/apache/spark/pull/19559 @gatorsmile @gatorsmile There are still two issues need to be figured out. (1)It will be complicated to determine whether a literal function should be resolved as Expression or

[GitHub] spark pull request #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, ...

2017-10-26 Thread hvanhovell
Github user hvanhovell commented on a diff in the pull request: https://github.com/apache/spark/pull/19559#discussion_r147068227 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -844,7 +863,12 @@ class Analyzer(

[GitHub] spark pull request #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, ...

2017-10-26 Thread DonnyZone
Github user DonnyZone commented on a diff in the pull request: https://github.com/apache/spark/pull/19559#discussion_r147068164 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -783,6 +783,25 @@ class Analyzer( }

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-26 Thread ArtRand
Github user ArtRand commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r147069130 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -427,10 +441,10 @@ trait

[GitHub] spark issue #19573: [SPARK-22350][SQL] select grouping__id from subquery

2017-10-26 Thread DonnyZone
Github user DonnyZone commented on the issue: https://github.com/apache/spark/pull/19573 Is it similar to the below issue? https://github.com/apache/spark/pull/19178 --- - To unsubscribe, e-mail:

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-26 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147075121 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -0,0 +1,258 @@ +/* + * Licensed to the Apache Software

[GitHub] spark issue #16578: [SPARK-4502][SQL] Parquet nested column pruning

2017-10-26 Thread DaimonPl
Github user DaimonPl commented on the issue: https://github.com/apache/spark/pull/16578 @mallman how about finalizing it as is? IMHO performance improvements are worth more than (possibly) redundant workaround - it could be cleaned later ---

[GitHub] spark pull request #19563: [SPARK-22284][SQL] Fix 64KB JVM bytecode limit pr...

2017-10-26 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/19563#discussion_r147084523 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala --- @@ -389,9 +389,15 @@ abstract class HashExpression[E]

[GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...

2017-10-26 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19565 Regarding caching: I think that can be ignored for purposes of this change. All this does is add a filter, and it doesn't cause an RDD to computed more than it was before. The only question

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-26 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r147087721 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -175,17 +176,36 @@ trait

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-26 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r147087477 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosSchedulerUtils.scala --- @@ -451,15 +465,20 @@ trait

[GitHub] spark pull request #19390: [SPARK-18935][MESOS] Fix dynamic reservations on ...

2017-10-26 Thread skonto
Github user skonto commented on a diff in the pull request: https://github.com/apache/spark/pull/19390#discussion_r147087405 --- Diff: resource-managers/mesos/src/main/scala/org/apache/spark/scheduler/cluster/mesos/MesosCoarseGrainedSchedulerBackend.scala --- @@ -349,13 +349,22

[GitHub] spark issue #19480: [SPARK-22226][SQL] splitExpression can create too many m...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19480 **[Test build #83079 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83079/testReport)** for PR 19480 at commit

[GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...

2017-10-26 Thread akopich
Github user akopich commented on the issue: https://github.com/apache/spark/pull/19565 I am sure that caching may by avoided here. Hence, it should not be used. @srowen, maybe I don't get something, but I'm afraid, that currently lineage for a single mini-batch submission

[GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...

2017-10-26 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/19565 @akopich IMO the filter won't cost too much, don't worry about the performance. (Or you can make a test to make sure) ---

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19559 **[Test build #83078 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83078/testReport)** for PR 19559 at commit

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19559 **[Test build #83080 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83080/testReport)** for PR 19559 at commit

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19559 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19559 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83078/ Test FAILed. ---

[GitHub] spark issue #9282: [SPARK-10986][Mesos] Set the context class loader in the ...

2017-10-26 Thread klion26
Github user klion26 commented on the issue: https://github.com/apache/spark/pull/9282 received ClassNotFound error in Yarn-Cluster mode(spark 1.6.2),_doesn't reproduce the problem_ The error message is such as below: ``` [2017-10-26 16:53:18,274] ERROR Error while

[GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...

2017-10-26 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/19565 Filtering after sampling makes more sense. Though sampling isn't deterministic, it doesn't change the probability that any particular sample is produced. ---

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-26 Thread thunterdb
Github user thunterdb commented on the issue: https://github.com/apache/spark/pull/19439 @hhbyyh I recall now the reason for an extra `origin` field, which is to get around the standard issue of many small image files in S3 or other distributed file systems. It is standard to compact

[GitHub] spark issue #19222: [SPARK-10399][CORE][SQL] Introduce multiple MemoryBlocks...

2017-10-26 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/19222 @hvanhovell @tejasapatil would it be possible to review this? Or, do you know who is appropriate for reviewing this? --- - To

[GitHub] spark issue #19559: [SPARK-22333][SQL]timeFunctionCall(CURRENT_DATE, CURRENT...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19559 **[Test build #83081 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83081/testReport)** for PR 19559 at commit

[GitHub] spark pull request #19563: [SPARK-22284][SQL] Fix 64KB JVM bytecode limit pr...

2017-10-26 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/19563#discussion_r147106199 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/hash.scala --- @@ -389,9 +389,15 @@ abstract class HashExpression[E] extends

[GitHub] spark issue #19563: [SPARK-22284][SQL] Fix 64KB JVM bytecode limit problem i...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19563 **[Test build #83082 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83082/testReport)** for PR 19563 at commit

[GitHub] spark issue #19267: [WIP][SPARK-20628][CORE] Blacklist nodes when they trans...

2017-10-26 Thread juanrh
Github user juanrh commented on the issue: https://github.com/apache/spark/pull/19267 Hi @tgravescs, thanks again for your feedback. Regarding concrete uses cases, this change might be used extend the existing graceful decommission mechanism available in AWS EMR from a while ago.

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19439 **[Test build #83098 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83098/testReport)** for PR 19439 at commit

[GitHub] spark pull request #19510: [SPARK-22292][Mesos] Added spark.mem.max support ...

2017-10-26 Thread susanxhuynh
Github user susanxhuynh commented on a diff in the pull request: https://github.com/apache/spark/pull/19510#discussion_r147265329 --- Diff: docs/running-on-mesos.md --- @@ -613,6 +621,39 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark pull request #19510: [SPARK-22292][Mesos] Added spark.mem.max support ...

2017-10-26 Thread susanxhuynh
Github user susanxhuynh commented on a diff in the pull request: https://github.com/apache/spark/pull/19510#discussion_r147265261 --- Diff: docs/running-on-mesos.md --- @@ -613,6 +621,39 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark pull request #19510: [SPARK-22292][Mesos] Added spark.mem.max support ...

2017-10-26 Thread susanxhuynh
Github user susanxhuynh commented on a diff in the pull request: https://github.com/apache/spark/pull/19510#discussion_r147274681 --- Diff: docs/running-on-mesos.md --- @@ -344,6 +345,13 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark pull request #19510: [SPARK-22292][Mesos] Added spark.mem.max support ...

2017-10-26 Thread susanxhuynh
Github user susanxhuynh commented on a diff in the pull request: https://github.com/apache/spark/pull/19510#discussion_r147277408 --- Diff: docs/running-on-mesos.md --- @@ -613,6 +621,39 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark pull request #19510: [SPARK-22292][Mesos] Added spark.mem.max support ...

2017-10-26 Thread susanxhuynh
Github user susanxhuynh commented on a diff in the pull request: https://github.com/apache/spark/pull/19510#discussion_r147265373 --- Diff: docs/running-on-mesos.md --- @@ -613,6 +621,39 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] spark issue #19459: [SPARK-20791][PYSPARK] Use Arrow to create Spark DataFra...

2017-10-26 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/19459 @ueshin if possible I'd like to have #18664 merged first and then I can fix this PR up if needed, thanks! --- - To

[GitHub] spark issue #19567: [SPARK-22291][SQL] Postgresql UUID[] to Cassandra: Conve...

2017-10-26 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/19567 @jmchung There are another StringType types "json", "jsonb" in `PostgresDialect`. Can you also add them into tests? Thanks. ---

[GitHub] spark issue #19567: [SPARK-22291][SQL] Postgresql UUID[] to Cassandra: Conve...

2017-10-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19567 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19582: [SPARK-20644][core] Initial ground work for kvstore UI b...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19582 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83096/ Test PASSed. ---

[GitHub] spark issue #19582: [SPARK-20644][core] Initial ground work for kvstore UI b...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19582 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19567: [SPARK-22291][SQL] Postgresql UUID[] to Cassandra...

2017-10-26 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/19567#discussion_r147309380 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -456,8 +456,17 @@ object JdbcUtils extends

[GitHub] spark issue #19433: [SPARK-3162] [MLlib] Add local tree training for decisio...

2017-10-26 Thread smurching
Github user smurching commented on the issue: https://github.com/apache/spark/pull/19433 Made a few updates, here’s a quick summary/what I’d propose moving forward: Right now: * Shared row indices for all (categorical & continuous) features are stored & updated in

[GitHub] spark pull request #19580: [SPARK-11334][CORE] Fix bug in Executor allocatio...

2017-10-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19580#discussion_r147289166 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -787,7 +791,9 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19468 **[Test build #83100 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83100/testReport)** for PR 19468 at commit

[GitHub] spark issue #19580: [SPARK-11334][CORE] Fix bug in Executor allocation manag...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19580 **[Test build #83099 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83099/testReport)** for PR 19580 at commit

[GitHub] spark issue #19583: [WIP][SPARK-22339] [CORE] [NETWORK-SHUFFLE]

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19583 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-26 Thread MrBago
Github user MrBago commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147298609 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,122 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19567: [SPARK-22291][SQL] Postgresql UUID[] to Cassandra...

2017-10-26 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/19567#discussion_r147305034 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -456,8 +456,17 @@ object JdbcUtils extends Logging

[GitHub] spark issue #19582: [SPARK-20644][core] Initial ground work for kvstore UI b...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19582 **[Test build #83096 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83096/testReport)** for PR 19582 at commit

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19468 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19439: [SPARK-21866][ML][PySpark] Adding spark image rea...

2017-10-26 Thread imatiach-msft
Github user imatiach-msft commented on a diff in the pull request: https://github.com/apache/spark/pull/19439#discussion_r147289498 --- Diff: python/pyspark/ml/image.py --- @@ -0,0 +1,122 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one or more +#

[GitHub] spark pull request #19580: [SPARK-11334][CORE] Fix bug in Executor allocatio...

2017-10-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19580#discussion_r147291760 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -787,7 +791,9 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark pull request #19581: [SPARK-22366] Support ignoring missing files

2017-10-26 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/19581 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19579: [SPARK-22356][SQL] data source table should support over...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19579 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19580: [SPARK-11334][CORE] Fix bug in Executor allocation manag...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19580 **[Test build #83101 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83101/testReport)** for PR 19580 at commit

[GitHub] spark issue #19579: [SPARK-22356][SQL] data source table should support over...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19579 **[Test build #83094 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83094/testReport)** for PR 19579 at commit

[GitHub] spark issue #19579: [SPARK-22356][SQL] data source table should support over...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19579 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83094/ Test PASSed. ---

[GitHub] spark issue #19579: [SPARK-22356][SQL] data source table should support over...

2017-10-26 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/19579 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #19582: [SPARK-20644][core] Initial ground work for kvstore UI b...

2017-10-26 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/19582 For context: * Project link: https://issues.apache.org/jira/browse/SPARK-18085 * Upcoming PRs that build on this code: https://github.com/vanzin/spark/pulls A special note about

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19451 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19451 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83095/ Test PASSed. ---

[GitHub] spark issue #19451: SPARK-22181 Adds ReplaceExceptWithNotFilter rule

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19451 **[Test build #83095 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83095/testReport)** for PR 19451 at commit

[GitHub] spark issue #19567: [SPARK-22291][SQL] Postgresql UUID[] to Cassandra: Conve...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19567 **[Test build #83103 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83103/testReport)** for PR 19567 at commit

[GitHub] spark issue #19567: [SPARK-22291][SQL] Postgresql UUID[] to Cassandra: Conve...

2017-10-26 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/19567 add to whitelist --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #19576: [SPARK-19727][SQL][followup] Fix for round functi...

2017-10-26 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/19576#discussion_r147311315 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala --- @@ -390,7 +390,7 @@ case class Cast(child: Expression,

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-26 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19468 **[Test build #83100 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/83100/testReport)** for PR 19468 at commit

[GitHub] spark issue #19468: [SPARK-18278] [Scheduler] Spark on Kubernetes - Basic Sc...

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19468 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83100/ Test PASSed. ---

[GitHub] spark pull request #19567: [SPARK-22291][SQL] Postgresql UUID[] to Cassandra...

2017-10-26 Thread jmchung
Github user jmchung commented on a diff in the pull request: https://github.com/apache/spark/pull/19567#discussion_r147314628 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JdbcUtils.scala --- @@ -456,8 +456,17 @@ object JdbcUtils extends

[GitHub] spark issue #19439: [SPARK-21866][ML][PySpark] Adding spark image reader

2017-10-26 Thread imatiach-msft
Github user imatiach-msft commented on the issue: https://github.com/apache/spark/pull/19439 @MrBago in regards to the question below -- I discussed with @drdarshan who is familiar with Python, and he said that it wouldn't be standard to create a class just to change the namespace

[GitHub] spark pull request #19580: [SPARK-11334][CORE] Fix bug in Executor allocatio...

2017-10-26 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/19580#discussion_r147288373 --- Diff: core/src/main/scala/org/apache/spark/ExecutorAllocationManager.scala --- @@ -787,7 +791,9 @@ private[spark] class ExecutorAllocationManager(

[GitHub] spark issue #19577: [SPARK-22355][SQL] Dataset.collect is not threadsafe

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19577 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/83091/ Test PASSed. ---

[GitHub] spark issue #19577: [SPARK-22355][SQL] Dataset.collect is not threadsafe

2017-10-26 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19577 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

  1   2   3   4   >