[GitHub] spark issue #16352: [SPARK-18947][SQL] SQLContext.tableNames should not call...

2016-12-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16352 LGTM again pending test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16352: [SPARK-18947][SQL] SQLContext.tableNames should n...

2016-12-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16352#discussion_r93390578 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -276,11 +276,12 @@ private[sql] object SQLUtils extends Logging {

[GitHub] spark issue #16369: [SPARK-18956][SQL][PySpark] Reuse existing SparkSession ...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16369 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70460/ Test PASSed. ---

[GitHub] spark issue #16369: [SPARK-18956][SQL][PySpark] Reuse existing SparkSession ...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16369 **[Test build #70460 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70460/testReport)** for PR 16369 at commit

[GitHub] spark issue #16356: [SPARK-18949] [SQL] Add recoverPartitions API to Catalog

2016-12-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16356 Sure, let me do it now. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16352: [SPARK-18947][SQL] SQLContext.tableNames should not call...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16352 **[Test build #70466 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70466/testReport)** for PR 16352 at commit

[GitHub] spark issue #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTableAsSelec...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15996 **[Test build #70465 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70465/testReport)** for PR 15996 at commit

[GitHub] spark pull request #16352: [SPARK-18947][SQL] SQLContext.tableNames should n...

2016-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16352#discussion_r93388755 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -276,11 +276,12 @@ private[sql] object SQLUtils extends Logging {

[GitHub] spark pull request #16371: [SPARK-18932][SQL] Support partial aggregation fo...

2016-12-20 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16371#discussion_r93388333 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/aggregate/Percentile.scala --- @@ -33,10 +33,9 @@ import

[GitHub] spark issue #16371: [SPARK-18932][SQL] Support partial aggregation for colle...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16371 **[Test build #70464 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70464/testReport)** for PR 16371 at commit

[GitHub] spark pull request #16356: [SPARK-18949] [SQL] Add recoverPartitions API to ...

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16356 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16356: [SPARK-18949] [SQL] Add recoverPartitions API to Catalog

2016-12-20 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16356 Can you send a pr for branch-2.1? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16356: [SPARK-18949] [SQL] Add recoverPartitions API to Catalog

2016-12-20 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/16356 Merging in master/branch-2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #13909: [SPARK-16213][SQL] Reduce runtime overhead of a p...

2016-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/13909#discussion_r93387942 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/complexTypeCreator.scala --- @@ -56,33 +58,89 @@ case class

[GitHub] spark pull request #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTable...

2016-12-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15996#discussion_r93387828 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala --- @@ -364,48 +366,157 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] spark issue #13909: [SPARK-16213][SQL] Reduce runtime overhead of a program ...

2016-12-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/13909 sorry for the delay. Yea looks like we can't reuse the byte array of unsafe data in expressions, which may get cached expectedly and leads to wrong result. I'm a little concerned about

[GitHub] spark issue #16351: [SPARK-18943][SQL] Avoid per-record type dispatch in CSV...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16351 **[Test build #70463 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70463/testReport)** for PR 16351 at commit

[GitHub] spark issue #16351: [SPARK-18943][SQL] Avoid per-record type dispatch in CSV...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16351 **[Test build #70462 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70462/testReport)** for PR 16351 at commit

[GitHub] spark issue #16337: [SPARK-18871][SQL] New test cases for IN/NOT IN subquery

2016-12-20 Thread kevinyu98
Github user kevinyu98 commented on the issue: https://github.com/apache/spark/pull/16337 Nat will run against DB2 and provide result. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16351: [SPARK-18943][SQL] Avoid per-record type dispatch in CSV...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16351 **[Test build #70461 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70461/testReport)** for PR 16351 at commit

[GitHub] spark issue #16369: [SPARK-18956][SQL][PySpark] Reuse existing SparkSession ...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16369 **[Test build #70460 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70460/testReport)** for PR 16369 at commit

[GitHub] spark issue #16337: [SPARK-18871][SQL] New test cases for IN/NOT IN subquery

2016-12-20 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/16337 What are the refernce query results we can compare? For example, from DB2 or Hive? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #16356: [SPARK-18949] [SQL] Add recoverPartitions API to Catalog

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16356 **[Test build #3511 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3511/testReport)** for PR 16356 at commit

[GitHub] spark issue #16371: [SPARK-18932][SQL] Support partial aggregation for colle...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16371 **[Test build #70459 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70459/testReport)** for PR 16371 at commit

[GitHub] spark pull request #16371: [SPARK-18932][SQL] Support partial aggregation fo...

2016-12-20 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/16371 [SPARK-18932][SQL] Support partial aggregation for collect_set/collect_list ## What changes were proposed in this pull request? Currently collect_set/collect_list aggregation expression

[GitHub] spark issue #12775: [SPARK-14958][Core] Failed task not handled when there's...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12775 **[Test build #70458 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70458/testReport)** for PR 12775 at commit

[GitHub] spark issue #16351: [SPARK-18943][SQL] Avoid per-record type dispatch in CSV...

2016-12-20 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/16351 Thank you @cloud-fan, will add some in the PR description soon after cleaning up. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #16352: [SPARK-18947][SQL] SQLContext.tableNames should n...

2016-12-20 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/16352#discussion_r93385438 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/api/r/SQLUtils.scala --- @@ -276,11 +276,12 @@ private[sql] object SQLUtils extends Logging {

[GitHub] spark issue #12775: [SPARK-14958][Core] Failed task not handled when there's...

2016-12-20 Thread kayousterhout
Github user kayousterhout commented on the issue: https://github.com/apache/spark/pull/12775 Jenkins, retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16337: [SPARK-18871][SQL] New test cases for IN/NOT IN subquery

2016-12-20 Thread kevinyu98
Github user kevinyu98 commented on the issue: https://github.com/apache/spark/pull/16337 Hello All: I have divided the test case to small groups based the discussion, and this pr will be the first pr for the IN subquery, it covers the simple and group-by cases. This the

[GitHub] spark pull request #16367: [SPARK-18903][SPARKR] Add API to get SparkUI URL

2016-12-20 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/16367#discussion_r93384251 --- Diff: R/pkg/R/sparkR.R --- @@ -410,6 +410,30 @@ sparkR.session <- function( sparkSession } +#' Get the URL of the SparkUI

[GitHub] spark issue #16337: [SPARK-18871][SQL] New test cases for IN/NOT IN subquery

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16337 **[Test build #70457 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70457/testReport)** for PR 16337 at commit

[GitHub] spark issue #16355: [SPARK-16473][MLLIB] Fix BisectingKMeans Algorithm faili...

2016-12-20 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/16355 @imatiach-msft Can you add a test case? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16355: [SPARK-16473][MLLIB] Fix BisectingKMeans Algorithm faili...

2016-12-20 Thread wangmiao1981
Github user wangmiao1981 commented on the issue: https://github.com/apache/spark/pull/16355 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16370 **[Test build #70456 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70456/testReport)** for PR 16370 at commit

[GitHub] spark pull request #16367: [SPARK-18903][SPARKR] Add API to get SparkUI URL

2016-12-20 Thread wangmiao1981
Github user wangmiao1981 commented on a diff in the pull request: https://github.com/apache/spark/pull/16367#discussion_r93381426 --- Diff: R/pkg/R/sparkR.R --- @@ -410,6 +410,30 @@ sparkR.session <- function( sparkSession } +#' Get the URL of the SparkUI

[GitHub] spark issue #16370: [SPARK-18960][SQL][SS] Avoid double reading file which i...

2016-12-20 Thread uncleGen
Github user uncleGen commented on the issue: https://github.com/apache/spark/pull/16370 cc @zsxwing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16370: [SPARK-18960][SQL][SS] Avoid double reading file ...

2016-12-20 Thread uncleGen
GitHub user uncleGen opened a pull request: https://github.com/apache/spark/pull/16370 [SPARK-18960][SQL][SS] Avoid double reading file which is being copied. ## What changes were proposed in this pull request? In HDFS, when we copy a file into target directory, there will

[GitHub] spark issue #16351: [SPARK-18943][SQL] Avoid per-record type dispatch in CSV...

2016-12-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16351 mostly LGTM, do you have some performance numbers about this optimization? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #16351: [SPARK-18943][SQL] Avoid per-record type dispatch...

2016-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16351#discussion_r93381128 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVInferSchema.scala --- @@ -215,84 +215,133 @@ private[csv] object

[GitHub] spark pull request #16351: [SPARK-18943][SQL] Avoid per-record type dispatch...

2016-12-20 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/16351#discussion_r93381049 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/csv/CSVInferSchema.scala --- @@ -215,84 +215,133 @@ private[csv] object

[GitHub] spark issue #16240: [SPARK-16792][SQL] Dataset containing a Case Class with ...

2016-12-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16240 How about we assign priority to implicit rules like http://stackoverflow.com/questions/1886953/is-there-a-way-to-control-which-implicit-conversion-will-be-the-default-used ? I think we

[GitHub] spark issue #12775: [SPARK-14958][Core] Failed task not handled when there's...

2016-12-20 Thread lirui-intel
Github user lirui-intel commented on the issue: https://github.com/apache/spark/pull/12775 I don't think the failure is related, and it can't be reproduced locally. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15212: [SPARK-17645][MLLIB][ML]add feature selector method base...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15212 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15212: [SPARK-17645][MLLIB][ML]add feature selector method base...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15212 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70454/ Test PASSed. ---

[GitHub] spark issue #15212: [SPARK-17645][MLLIB][ML]add feature selector method base...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15212 **[Test build #70454 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70454/testReport)** for PR 15212 at commit

[GitHub] spark issue #16323: [SPARK-18911] [SQL] Define CatalogStatistics to interact...

2016-12-20 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/16323 SGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15996: [SPARK-18567][SQL] Simplify CreateDataSourceTableAsSelec...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15996 **[Test build #70455 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70455/testReport)** for PR 15996 at commit

[GitHub] spark issue #16282: [SPARK-18588][SS][Kafka]Create a new KafkaConsumer when ...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16282 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16282: [SPARK-18588][SS][Kafka]Create a new KafkaConsumer when ...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16282 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70444/ Test FAILed. ---

[GitHub] spark issue #16282: [SPARK-18588][SS][Kafka]Create a new KafkaConsumer when ...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16282 **[Test build #70444 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70444/testReport)** for PR 16282 at commit

[GitHub] spark issue #16350: [SPARK-18700][SQL][BACKPORT-2.0] Add StripedLock for eac...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16350 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70452/ Test PASSed. ---

[GitHub] spark issue #16350: [SPARK-18700][SQL][BACKPORT-2.0] Add StripedLock for eac...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16350 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16350: [SPARK-18700][SQL][BACKPORT-2.0] Add StripedLock for eac...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16350 **[Test build #70452 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70452/consoleFull)** for PR 16350 at commit

[GitHub] spark issue #16360: [SPARK-18234][SS] Made update mode public

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16360 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70450/ Test PASSed. ---

[GitHub] spark issue #16360: [SPARK-18234][SS] Made update mode public

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16360 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16360: [SPARK-18234][SS] Made update mode public

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16360 **[Test build #70450 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70450/testReport)** for PR 16360 at commit

[GitHub] spark issue #16369: [SPARK-18956][SQL][PySpark] Reuse existing SparkSession ...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16369 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70453/ Test FAILed. ---

[GitHub] spark issue #16369: [SPARK-18956][SQL][PySpark] Reuse existing SparkSession ...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16369 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16369: [SPARK-18956][SQL][PySpark] Reuse existing SparkSession ...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16369 **[Test build #70453 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70453/testReport)** for PR 16369 at commit

[GitHub] spark issue #16323: [SPARK-18911] [SQL] Define CatalogStatistics to interact...

2016-12-20 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/16323 Since adding a switch for cbo is not a trivial one, I want to do it in a separate pr, and let this one only deal with decoupling Statistics from CatalogTable. Do you agree? @cloud-fan --- If your

[GitHub] spark pull request #16323: [SPARK-18911] [SQL] Define CatalogStatistics to i...

2016-12-20 Thread wzhfy
Github user wzhfy commented on a diff in the pull request: https://github.com/apache/spark/pull/16323#discussion_r93376249 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -198,6 +200,10 @@ case class CatalogTable(

[GitHub] spark issue #15212: [SPARK-17645][MLLIB][ML]add feature selector method base...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15212 **[Test build #70454 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70454/testReport)** for PR 15212 at commit

[GitHub] spark issue #16304: [SPARK-18894][SS] Fix event time watermark delay thresho...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16304 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16304: [SPARK-18894][SS] Fix event time watermark delay thresho...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16304 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70449/ Test PASSed. ---

[GitHub] spark issue #16304: [SPARK-18894][SS] Fix event time watermark delay thresho...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16304 **[Test build #70449 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70449/testReport)** for PR 16304 at commit

[GitHub] spark issue #16356: [SPARK-18949] [SQL] Add recoverPartitions API to Catalog

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16356 **[Test build #3511 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3511/testReport)** for PR 16356 at commit

[GitHub] spark issue #16296: [SPARK-18885][SQL] unify CREATE TABLE syntax for data so...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16296 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70447/ Test PASSed. ---

[GitHub] spark issue #16296: [SPARK-18885][SQL] unify CREATE TABLE syntax for data so...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16296 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16296: [SPARK-18885][SQL] unify CREATE TABLE syntax for data so...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16296 **[Test build #70447 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70447/testReport)** for PR 16296 at commit

[GitHub] spark issue #16369: [SPARK-18956][SQL][PySpark] Reuse existing SparkSession ...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16369 **[Test build #70453 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70453/testReport)** for PR 16369 at commit

[GitHub] spark pull request #16369: [SPARK-18956][SQL][PySpark] Reuse existing SparkS...

2016-12-20 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/16369 [SPARK-18956][SQL][PySpark] Reuse existing SparkSession while creating new SQLContext instances ## What changes were proposed in this pull request? To reuse existing SparkSession while

[GitHub] spark issue #16359: [SPARK-18951] Upgrade com.thoughtworks.paranamer/paranam...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16359 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16359: [SPARK-18951] Upgrade com.thoughtworks.paranamer/paranam...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16359 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70445/ Test PASSed. ---

[GitHub] spark issue #16359: [SPARK-18951] Upgrade com.thoughtworks.paranamer/paranam...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16359 **[Test build #70445 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70445/testReport)** for PR 16359 at commit

[GitHub] spark issue #16352: [SPARK-18947][SQL] SQLContext.tableNames should not call...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16352 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70446/ Test PASSed. ---

[GitHub] spark issue #16352: [SPARK-18947][SQL] SQLContext.tableNames should not call...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16352 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16352: [SPARK-18947][SQL] SQLContext.tableNames should not call...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16352 **[Test build #70446 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70446/testReport)** for PR 16352 at commit

[GitHub] spark issue #16366: [SPARK-18953][CORE][WEB UI] Do now show the link to a de...

2016-12-20 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16366 We also show worker info in `driverRow`. Although it doesn't show worker state, I am wondering if we can also check worker state, and disable the link and add a suffix like `(DEAD)` if a worker is

[GitHub] spark issue #16350: [SPARK-18700][SQL][BACKPORT-2.0] Add StripedLock for eac...

2016-12-20 Thread xuanyuanking
Github user xuanyuanking commented on the issue: https://github.com/apache/spark/pull/16350 Delete the UT and metrics done. :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16350: [SPARK-18700][SQL][BACKPORT-2.0] Add StripedLock for eac...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16350 **[Test build #70452 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70452/consoleFull)** for PR 16350 at commit

[GitHub] spark issue #16366: [SPARK-18953][CORE][WEB UI] Do now show the link to a de...

2016-12-20 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16366 One question: should we do the same thing for `WorkerState.DECOMMISSIONED` and `WorkerState.UNKNOWN` ? --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #16282: [SPARK-18588][SS][Kafka]Create a new KafkaConsumer when ...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16282 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70439/ Test FAILed. ---

[GitHub] spark issue #16282: [SPARK-18588][SS][Kafka]Create a new KafkaConsumer when ...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16282 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16282: [SPARK-18588][SS][Kafka]Create a new KafkaConsumer when ...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16282 **[Test build #70439 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70439/testReport)** for PR 16282 at commit

[GitHub] spark issue #15018: [SPARK-17455][MLlib] Improve PAVA implementation in Isot...

2016-12-20 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15018 @neggert I am fine for throwing an error. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16368: [SPARK-18958][SPARKR] R API toJSON on DataFrame

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16368 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16368: [SPARK-18958][SPARKR] R API toJSON on DataFrame

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16368 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70451/ Test PASSed. ---

[GitHub] spark issue #16368: [SPARK-18958][SPARKR] R API toJSON on DataFrame

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16368 **[Test build #70451 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70451/testReport)** for PR 16368 at commit

[GitHub] spark pull request #16323: [SPARK-18911] [SQL] Define CatalogStatistics to i...

2016-12-20 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/16323#discussion_r93370436 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -198,6 +200,10 @@ case class CatalogTable(

[GitHub] spark issue #16366: [SPARK-18953][CORE][WEB UI] Do now show the link to a de...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16366 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16366: [SPARK-18953][CORE][WEB UI] Do now show the link to a de...

2016-12-20 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16366 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/70442/ Test PASSed. ---

[GitHub] spark issue #16366: [SPARK-18953][CORE][WEB UI] Do now show the link to a de...

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16366 **[Test build #70442 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70442/testReport)** for PR 16366 at commit

[GitHub] spark issue #16350: [SPARK-18700][SQL][BACKPORT-2.0] Add StripedLock for eac...

2016-12-20 Thread ericl
Github user ericl commented on the issue: https://github.com/apache/spark/pull/16350 yeah, i don't think we need the unit test for 2.0 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #16362: [SPARK-18954][Tests]Fix flaky test: o.a.s.streaming.Basi...

2016-12-20 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/16362 LGTM. Did you run it many times? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16343: [FLAKY-TEST] InputStreamsSuite.socket input stream

2016-12-20 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/16343 Why didnt `eventually` and assert on the size of the collected data work? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #16314: [SPARK-18900][FLAKY-TEST] StateStoreSuite.mainten...

2016-12-20 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/16314 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #16314: [SPARK-18900][FLAKY-TEST] StateStoreSuite.maintenance

2016-12-20 Thread tdas
Github user tdas commented on the issue: https://github.com/apache/spark/pull/16314 Merging this master and 2.1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16368: [SPARK-18958][SPARKR] R API toJSON on DataFrame

2016-12-20 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16368 **[Test build #70451 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/70451/testReport)** for PR 16368 at commit

[GitHub] spark pull request #16368: [SPARK-18958][SPARKR] R API toJSON on DataFrame

2016-12-20 Thread felixcheung
GitHub user felixcheung opened a pull request: https://github.com/apache/spark/pull/16368 [SPARK-18958][SPARKR] R API toJSON on DataFrame ## What changes were proposed in this pull request? It would make it easier to integrate with other component expecting JSON format.

  1   2   3   4   5   >