[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15231 @felixcheung , I usually don't like to answer by quote but let me do this just to clarify. > Hmm, should we hold till 12601 is merged then? Seems like we shouldn't allow this unless

[GitHub] spark pull request #15226: [SPARK-17649][CORE] Log how many Spark events got...

2016-09-25 Thread zsxwing
Github user zsxwing commented on a diff in the pull request: https://github.com/apache/spark/pull/15226#discussion_r80395860 --- Diff: core/src/main/scala/org/apache/spark/util/AsynchronousListenerBus.scala --- @@ -117,6 +124,24 @@ private[spark] abstract class

[GitHub] spark pull request #15195: [SPARK-17632][SQL]make console sink and other sin...

2016-09-25 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/15195#discussion_r80396970 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -290,8 +284,8 @@ final class DataStreamWriter[T]

[GitHub] spark pull request #15172: [SPARK-13331] AES support for over-the-wire encry...

2016-09-25 Thread cjjnjust
Github user cjjnjust commented on a diff in the pull request: https://github.com/apache/spark/pull/15172#discussion_r80399872 --- Diff: common/network-common/src/main/java/org/apache/spark/network/sasl/aes/SparkAesCipher.java --- @@ -0,0 +1,270 @@ +/* + * Licensed to the

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80402604 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/GlobalTempViewManager.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12601 **[Test build #65891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65891/consoleFull)** for PR 12601 at commit

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65890/consoleFull)** for PR 15231 at commit

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-25 Thread JustinPihony
Github user JustinPihony commented on the issue: https://github.com/apache/spark/pull/12601 @srowen The doc changes have been reviewed, so this should be good to go --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Check null first for lay...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15232 Oh I meant 244. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80403386 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -222,8 +265,8 @@ case class AlterViewAsCommand(

[GitHub] spark issue #15123: [SPARK-17551][SQL] Add DataFrame API for null ordering

2016-09-25 Thread hvanhovell
Github user hvanhovell commented on the issue: https://github.com/apache/spark/pull/15123 LGTM - merging to master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395993 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala --- @@ -277,7 +275,7 @@ class CatalogImpl(sparkSession: SparkSession)

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395754 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -393,21 +459,25 @@ class SessionCatalog( */

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80396066 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/GlobalTempViewSuite.scala --- @@ -0,0 +1,107 @@ +/* + * Licensed to the Apache

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395510 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/GlobalTempViewManager.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395895 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -197,6 +201,45 @@ case class CreateViewCommand( } }

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395571 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -47,6 +50,8 @@ object SessionCatalog { */

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395605 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -329,33 +343,77 @@ class SessionCatalog( //

[GitHub] spark pull request #15195: [SPARK-17632][SQL]make console sink and other sin...

2016-09-25 Thread chuanlei
Github user chuanlei commented on a diff in the pull request: https://github.com/apache/spark/pull/15195#discussion_r80401708 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -290,8 +284,8 @@ final class DataStreamWriter[T]

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65890/consoleFull)** for PR 15231 at commit

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65890/ Test FAILed. ---

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15237: [SPARK-17663] [CORE] SchedulableBuilder should ha...

2016-09-25 Thread erenavsarogullari
GitHub user erenavsarogullari opened a pull request: https://github.com/apache/spark/pull/15237 [SPARK-17663] [CORE] SchedulableBuilder should handle invalid data access via scheduler.al… ## What changes were proposed in this pull request? If `spark.scheduler.allocation.file`

[GitHub] spark issue #15237: [SPARK-17663] [CORE] SchedulableBuilder should handle in...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15237 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #65887 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65887/consoleFull)** for PR 15219 at commit

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15219 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65887/ Test FAILed. ---

[GitHub] spark issue #15153: [SPARK-17599] Prevent ListingFileCatalog from failing if...

2016-09-25 Thread brkyvz
Github user brkyvz commented on the issue: https://github.com/apache/spark/pull/15153 @petermaxlee It is true that the parallel version can fail as well, the same kind of race condition can bite people there --- If your project is set up for it, you can reply to this email and have

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread karlhigley
Github user karlhigley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80392692 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread karlhigley
Github user karlhigley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80393070 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread karlhigley
Github user karlhigley commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80392464 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] spark pull request #15123: [SPARK-17551][SQL] Add DataFrame API for null ord...

2016-09-25 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/15123 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-25 Thread JustinPihony
Github user JustinPihony commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80404577 --- Diff: docs/sql-programming-guide.md --- @@ -1096,13 +1096,17 @@ the Data Sources API. The following options are supported: {% highlight

[GitHub] spark issue #15090: [SPARK-17073] [SQL] generate column-level statistics

2016-09-25 Thread wzhfy
Github user wzhfy commented on the issue: https://github.com/apache/spark/pull/15090 To help us choose a better design, we need to first clarify the usage of column stats. A simple example may look like this (e.g. predicate: col < 5): ```java filter.condition match {

[GitHub] spark issue #15153: [SPARK-17599] Prevent ListingFileCatalog from failing if...

2016-09-25 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/15153 @brkyvz the change here only affects the serial version, and not the parallel version, does it? Wouldn't that be a problem? --- If your project is set up for it, you can reply to

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15235 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65888/ Test PASSed. ---

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15235 **[Test build #65888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65888/consoleFull)** for PR 15235 at commit

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15235 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15231 Oh, BTW it seems `read.df` also seems not allowing this? I will try to test and fix here together if so. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80403416 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -37,6 +37,20 @@ import

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80403509 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -142,8 +149,12 @@ class SessionCatalog( //

[GitHub] spark pull request #15238: [SPARK-17653][SQL] Remove unnecessary distincts i...

2016-09-25 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/15238 [SPARK-17653][SQL] Remove unnecessary distincts in multiple unions ## What changes were proposed in this pull request? Currently for `Union [Distinct]`, a `Distinct` operator is necessary

[GitHub] spark pull request #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work ...

2016-09-25 Thread JustinPihony
Github user JustinPihony commented on a diff in the pull request: https://github.com/apache/spark/pull/12601#discussion_r80404639 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/jdbc/JDBCWriteSuite.scala --- @@ -208,4 +210,84 @@ class JDBCWriteSuite extends SharedSQLContext

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15235 **[Test build #65888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65888/consoleFull)** for PR 15235 at commit

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80393533 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -459,7 +459,8 @@ class Analyzer( case u:

[GitHub] spark issue #15238: [SPARK-17653][SQL] Remove unnecessary distincts in multi...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15238 **[Test build #65892 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65892/consoleFull)** for PR 15238 at commit

[GitHub] spark pull request #15053: [Doc] improve python API docstrings

2016-09-25 Thread mortada
Github user mortada commented on a diff in the pull request: https://github.com/apache/spark/pull/15053#discussion_r80405104 --- Diff: python/pyspark/sql/functions.py --- @@ -411,7 +415,7 @@ def monotonically_increasing_id(): The generated ID is guaranteed to be

[GitHub] spark issue #15219: [WIP][SPARK-14098][SQL] Generate Java code to build Cach...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15219 **[Test build #65887 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65887/consoleFull)** for PR 15219 at commit

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/15235 @brkyvz I think this also impacts the change you just did in https://github.com/apache/spark/pull/15153. This change makes both code path consistent. --- If your project is set up for it,

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80396017 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala --- @@ -277,7 +275,7 @@ class CatalogImpl(sparkSession: SparkSession)

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395565 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -142,8 +149,12 @@ class SessionCatalog( //

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395798 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -222,8 +265,8 @@ case class AlterViewAsCommand(

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395752 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -393,21 +459,25 @@ class SessionCatalog( */

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395517 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/GlobalTempViewManager.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80396050 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/internal/SharedState.scala --- @@ -37,6 +37,20 @@ import org.apache.spark.util.{MutableURLClassLoader,

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread yhuai
Github user yhuai commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80395764 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -453,7 +532,11 @@ class SessionCatalog(

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/15168 This PR needs to cover more negative cases. Below is an example: ```Scala spark.range(10).select('id as 'a, 'id as 'b).createTempView("view1") sql("DESC view1 PARTITION

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15168 **[Test build #65889 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65889/consoleFull)** for PR 15168 at commit

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15168 Definitely! Thank you, @gatorsmile . I added the logic to cover that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80403612 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -371,16 +431,24 @@ class SessionCatalog(

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80403654 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala --- @@ -393,21 +459,25 @@ class SessionCatalog(

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15168 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15168 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65889/ Test PASSed. ---

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80411490 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15224: [SPARK-17650] malformed url's throw exceptions before br...

2016-09-25 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/15224 Thanks. Merging to master and 2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15231 (I just updated the PR description too) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12601 **[Test build #65891 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65891/consoleFull)** for PR 12601 at commit

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12601 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #12601: [SPARK-14525][SQL] Make DataFrameWrite.save work for jdb...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/12601 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65891/ Test PASSed. ---

[GitHub] spark issue #15238: [SPARK-17653][SQL] Remove unnecessary distincts in multi...

2016-09-25 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/15238 cc @rxin --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #15148: [SPARK-5992][ML] Locality Sensitive Hashing

2016-09-25 Thread Yunni
Github user Yunni commented on a diff in the pull request: https://github.com/apache/spark/pull/15148#discussion_r80411374 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/lsh/LSH.scala --- @@ -0,0 +1,290 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65893/ Test PASSed. ---

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65893/consoleFull)** for PR 15231 at commit

[GitHub] spark issue #15168: [SPARK-17612][SQL] Support `DESCRIBE table PARTITION` SQ...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15168 **[Test build #65889 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65889/consoleFull)** for PR 15168 at commit

[GitHub] spark issue #15238: [SPARK-17653][SQL] Remove unnecessary distincts in multi...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15238 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15238: [SPARK-17653][SQL] Remove unnecessary distincts in multi...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15238 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65892/ Test PASSed. ---

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] read.df/write.df API taking path o...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65893/consoleFull)** for PR 15231 at commit

[GitHub] spark pull request #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14897#discussion_r80407704 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala --- @@ -60,20 +90,21 @@ case class CreateViewCommand( child:

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15216 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65895/ Test PASSed. ---

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15216 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15216 **[Test build #65895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65895/consoleFull)** for PR 15216 at commit

[GitHub] spark issue #14897: [SPARK-17338][SQL] add global temp view

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14897 **[Test build #65896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65896/consoleFull)** for PR 14897 at commit

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/15235 @yhuai I think you wrote most of this. Can you take a look? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #15235: [SPARK-17661][SQL] Consolidate various listLeafFi...

2016-09-25 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/15235#discussion_r80375086 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala --- @@ -82,73 +83,177 @@ class

[GitHub] spark pull request #15235: [SPARK-17661][SQL] Consolidate various listLeafFi...

2016-09-25 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/15235#discussion_r80375072 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala --- @@ -82,73 +83,177 @@ class

[GitHub] spark pull request #15235: [SPARK-17661][SQL] Consolidate various listLeafFi...

2016-09-25 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/15235#discussion_r80375056 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/ListingFileCatalog.scala --- @@ -82,73 +83,177 @@ class

[GitHub] spark issue #15235: [SPARK-17661][SQL] Consolidate various listLeafFiles imp...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15235 **[Test build #65876 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65876/consoleFull)** for PR 15235 at commit

[GitHub] spark pull request #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.add...

2016-09-25 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/15216#discussion_r80377484 --- Diff: R/pkg/R/context.R --- @@ -231,17 +231,21 @@ setCheckpointDir <- function(sc, dirName) { #' filesystems), or an HTTP, HTTPS or FTP URI. To

[GitHub] spark pull request #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.add...

2016-09-25 Thread yanboliang
Github user yanboliang commented on a diff in the pull request: https://github.com/apache/spark/pull/15216#discussion_r80377496 --- Diff: R/pkg/R/context.R --- @@ -231,17 +231,21 @@ setCheckpointDir <- function(sc, dirName) { #' filesystems), or an HTTP, HTTPS or FTP URI. To

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15216 **[Test build #65879 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65879/consoleFull)** for PR 15216 at commit

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65878/ Test PASSed. ---

[GitHub] spark issue #14971: [SPARK-17410] [SPARK-17284] Move Hive-generated Stats In...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14971 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15216 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65879/ Test PASSed. ---

[GitHub] spark issue #15216: [SPARK-17577][Follow-up][SparkR] SparkR spark.addFile su...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15216 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15231 @felixcheung Yeap, I just added. As we don't currently have an internal datasource allowing `DataFrameWriter.save()` without a path, I just added a test with CSV data source to make sure it

[GitHub] spark issue #15232: [SPARK-17499][SPARKR][FOLLOWUP] Suppress warnings from i...

2016-09-25 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/15232 Sure, I will take a look for this. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65880 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65880/consoleFull)** for PR 15231 at commit

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15231 **[Test build #65880 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/65880/consoleFull)** for PR 15231 at commit

[GitHub] spark issue #15231: [SPARK-17658][SPARKR] write.df API taking path optionall...

2016-09-25 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/65880/ Test PASSed. ---

  1   2   >