[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80411490 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15224: [SPARK-17650] malformed url's throw exceptions before br...

2016-09-25 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15224 Thanks. Merging to master and 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80411374 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15238: [SPARK-17653][SQL] Remove unnecessary distincts in multi...

2016-09-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15238 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #15233: [SPARK-17659] [SQL] Partitioned View is Not Suppo...

2016-09-25 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/15233#discussion_r8044 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala --- @@ -376,6 +376,10 @@ private[hive] class HiveClientImpl(

[GitHub] spark issue #15238: [SPARK-17653][SQL] Remove unnecessary distincts in multi...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15238 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15238: [SPARK-17653][SQL] Remove unnecessary distincts in multi...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15238 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65892/ Test PASSed. ---

[GitHub] spark issue #15107: [SPARK-17551][SQL] complete the NULL ordering support in...

2016-09-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15107 @xwu0226 Can you please close this, please? @hvanhovell already added you as the contributor in another PR, which has been merged. Thanks! --- If your project is set up for it, you can reply to

[GitHub] spark issue #15238: [SPARK-17653][SQL] Remove unnecessary distincts in multi...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15238 **[Test build #65892 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65892/consoleFull)** for PR 15238 at commit

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15216 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65895/ Test PASSed. ---

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15216 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15216 **[Test build #65895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65895/consoleFull)** for PR 15216 at commit

[GitHub] spark issue #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14897 **[Test build #65896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65896/consoleFull)** for PR 14897 at commit

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12601 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65891/ Test PASSed. ---

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12601 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12601 **[Test build #65891 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65891/consoleFull)** for PR 12601 at commit

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15168 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15168 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65889/ Test PASSed. ---

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15168 **[Test build #65889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65889/consoleFull)** for PR 15168 at commit

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15216 **[Test build #65895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65895/consoleFull)** for PR 15216 at commit

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65894/ Test PASSed. ---

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65894/consoleFull)** for PR 15231 at commit

[GitHub] spark issue #14912: [SPARK-17357][SQL] Fix current predicate pushdown

2016-09-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14912 ping @cloud-fan @hvanhovell @srinathshankar again, please take look if you have time. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80407704 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -60,20 +90,21 @@ case class CreateViewCommand( child:

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65893/ Test PASSed. ---

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65893/consoleFull)** for PR 15231 at commit

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65894/consoleFull)** for PR 15231 at commit

[GitHub] spark pull request #15189: [SPARK-17549][sql] Coalesce cached relation stats...

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15189#discussion_r80406877 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/columnar/InMemoryColumnarQuerySuite.scala --- @@ -232,4 +232,29 @@ class

[GitHub] spark pull request #15189: [SPARK-17549][sql] Coalesce cached relation stats...

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/15189#discussion_r80406261 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala --- @@ -44,6 +44,70 @@ object InMemoryRelation {

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15231 (I just updated the PR description too) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65893/consoleFull)** for PR 15231 at commit

[GitHub] spark issue #15090: [SPARK-17073] [SQL] generate column-level statistics

2016-09-25 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/15090 To help us choose a better design, we need to first clarify the usage of column stats. A simple example may look like this (e.g. predicate: col < 5): ```java filter.condition match {

[GitHub] spark pull request #15053: [Doc] improve python API docstrings

2016-09-25 Thread mortada
Github user mortada commented on a diff in the pull request: https://github.com/apache/spark/pull/15053#discussion_r80405104 --- Diff: python/pyspark/sql/functions.py --- @@ -411,7 +415,7 @@ def monotonically_increasing_id(): The generated ID is guaranteed to be

[GitHub] spark issue #15238: [SPARK-17653][SQL] Remove unnecessary distincts in multi...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15238 **[Test build #65892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65892/consoleFull)** for PR 15238 at commit

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65890/consoleFull)** for PR 15231 at commit

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65890/ Test FAILed. ---

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15238: [SPARK-17653][SQL] Remove unnecessary distincts i...

2016-09-25 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/15238 [SPARK-17653][SQL] Remove unnecessary distincts in multiple unions ## What changes were proposed in this pull request? Currently for `Union [Distinct]`, a `Distinct` operator is necessary

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-25 Thread JustinPihony
Github user JustinPihony commented on the issue: https://github.com/apache/spark/pull/12601 @srowen The doc changes have been reviewed, so this should be good to go --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65890/consoleFull)** for PR 15231 at commit

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12601 **[Test build #65891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65891/consoleFull)** for PR 12601 at commit

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-25 Thread JustinPihony
Github user JustinPihony commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80404639 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCWriteSuite.scala --- @@ -208,4 +210,84 @@ class JDBCWriteSuite extends SharedSQLContext

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-25 Thread JustinPihony
Github user JustinPihony commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80404577 --- Diff: docs/sql-programming-guide.md --- @@ -1096,13 +1096,17 @@ the Data Sources API. The following options are supported: {% highlight

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80403654 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -393,21 +459,25 @@ class SessionCatalog(

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80403612 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -371,16 +431,24 @@ class SessionCatalog(

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80403509 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -142,8 +149,12 @@ class SessionCatalog( //

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80403416 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -37,6 +37,20 @@ import

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80403386 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -222,8 +265,8 @@ case class AlterViewAsCommand(

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15168 **[Test build #65889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65889/consoleFull)** for PR 15168 at commit

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15168 Definitely! Thank you, @gatorsmile . I added the logic to cover that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80402604 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/GlobalTempViewManager.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to

[GitHub] spark pull request #15195: [SPARK-17632][SQL]make console sink and other sin...

2016-09-25 Thread chuanlei
Github user chuanlei commented on a diff in the pull request: https://github.com/apache/spark/pull/15195#discussion_r80401708 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -290,8 +284,8 @@ final class DataStreamWriter[T]

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15168 This PR needs to cover more negative cases. Below is an example: ```Scala spark.range(10).select('id as 'a, 'id as 'b).createTempView("view1") sql("DESC view1 PARTITION

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-09-25 Thread cjjnjust
Github user cjjnjust commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r80399872 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/SparkAesCipher.java --- @@ -0,0 +1,270 @@ +/* + * Licensed to the

[GitHub] spark pull request #15195: [SPARK-17632][SQL]make console sink and other sin...

2016-09-25 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/15195#discussion_r80396970 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -290,8 +284,8 @@ final class DataStreamWriter[T]

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80396050 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -37,6 +37,20 @@ import org.apache.spark.util.{MutableURLClassLoader,

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395764 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -453,7 +532,11 @@ class SessionCatalog(

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80396017 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala --- @@ -277,7 +275,7 @@ class CatalogImpl(sparkSession: SparkSession)

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395565 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -142,8 +149,12 @@ class SessionCatalog( //

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395798 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -222,8 +265,8 @@ case class AlterViewAsCommand(

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395752 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -393,21 +459,25 @@ class SessionCatalog( */

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395517 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/GlobalTempViewManager.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395993 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala --- @@ -277,7 +275,7 @@ class CatalogImpl(sparkSession: SparkSession)

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395571 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -47,6 +50,8 @@ object SessionCatalog { */

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395605 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -329,33 +343,77 @@ class SessionCatalog( //

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395754 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -393,21 +459,25 @@ class SessionCatalog( */

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80396066 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/GlobalTempViewSuite.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395510 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/GlobalTempViewManager.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395895 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -197,6 +201,45 @@ case class CreateViewCommand( } }

[GitHub] spark pull request #15226: [SPARK-17649][CORE] Log how many Spark events got...

2016-09-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15226#discussion_r80395860 --- Diff: core/src/main/scala/org/apache/spark/util/AsynchronousListenerBus.scala --- @@ -117,6 +124,24 @@ private[spark] abstract class

[GitHub] spark pull request #15123: [SPARK-17551][SQL] Add DataFrame API for null ord...

2016-09-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15123 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #15123: [SPARK-17551][SQL] Add DataFrame API for null ordering

2016-09-25 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15123 LGTM - merging to master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15235 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65888/ Test PASSed. ---

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15235 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15235 **[Test build #65888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65888/consoleFull)** for PR 15235 at commit

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15231 Oh, BTW it seems `read.df` also seems not allowing this? I will try to test and fix here together if so. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15231 @felixcheung , I usually don't like to answer by quote but let me do this just to clarify. > Hmm, should we hold till 12601 is merged then? Seems like we shouldn't allow this unless

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80393533 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -459,7 +459,8 @@ class Analyzer( case u:

[GitHub] spark issue #15237: [SPARK-17663] [CORE] SchedulableBuilder should handle in...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15237 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15237: [SPARK-17663] [CORE] SchedulableBuilder should ha...

2016-09-25 Thread erenavsarogullari
GitHub user erenavsarogullari opened a pull request: https://github.com/apache/spark/pull/15237 [SPARK-17663] [CORE] SchedulableBuilder should handle invalid data access via scheduler.al… ## What changes were proposed in this pull request? If `spark.scheduler.allocation.file`

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread karlhigley
Github user karlhigley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80393070 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread karlhigley
Github user karlhigley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80392464 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread karlhigley
Github user karlhigley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80392692 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15235 **[Test build #65888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65888/consoleFull)** for PR 15235 at commit

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/15235 @brkyvz I think this also impacts the change you just did in https://github.com/apache/spark/pull/15153. This change makes both code path consistent. --- If your project is set up for it,

[GitHub] spark issue #15153: [SPARK-17599] Prevent ListingFileCatalog from failing if...

2016-09-25 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15153 @petermaxlee It is true that the parallel version can fail as well, the same kind of race condition can bite people there --- If your project is set up for it, you can reply to this email and have

[GitHub] spark issue #15153: [SPARK-17599] Prevent ListingFileCatalog from failing if...

2016-09-25 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/15153 @brkyvz the change here only affects the serial version, and not the parallel version, does it? Wouldn't that be a problem? --- If your project is set up for it, you can reply to

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65887/ Test FAILed. ---

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #65887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65887/consoleFull)** for PR 15219 at commit

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15232 Oh I meant 244. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #65887 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65887/consoleFull)** for PR 15219 at commit

[GitHub] spark issue #15097: [SPARK-17540][SparkR][Spark Core] fix SparkR array serde...

2016-09-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15097 @WeichenXu123 do you have the user code and sample data that when run with SparkR will cause this issue? I think that will help us understand how this happens better. --- If your project is

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15232 change LGTM to me. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15232 you mean issue 224 of testthat on github? doesn't seem like it's related? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/15231 Hmm, should we hold till 12601 is merged then? Seems like we shouldn't allow this unless internal datasources are supporting this more broadly. Also, before the path parameter type is

[GitHub] spark pull request #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.add...

2016-09-25 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/15216#discussion_r80388454 --- Diff: R/pkg/R/context.R --- @@ -231,17 +231,21 @@ setCheckpointDir <- function(sc, dirName) { #' filesystems), or an HTTP, HTTPS or FTP URI. To

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15168 Thank you for review @gatorsmile . Hi, @hvanhovell . Could you review this again? --- If your project is set up for it, you can reply to this email and have your reply appear on

  1   2   >