[GitHub] spark pull request: [SPARK-14014] [SQL] Replace existing catalog w...

2016-03-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11836#discussion_r56745379 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveCatalog.scala --- @@ -182,13 +189,15 @@ private[spark] class HiveCatalog(client: HiveClien

[GitHub] spark pull request: [SPARK-14014] [SQL] Replace existing catalog w...

2016-03-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11836#discussion_r56745391 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveContext.scala --- @@ -81,15 +83,31 @@ class HiveContext private[hive]( sc: SparkC

[GitHub] spark pull request: [SPARK-13897][SQL] RelationalGroupedDataset an...

2016-03-18 Thread liancheng
Github user liancheng commented on the pull request: https://github.com/apache/spark/pull/11841#issuecomment-19864 LGTM except for one MiMA check question. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your proje

[GitHub] spark pull request: [Spark-14019] [SQL] Remove noop SortOrder in S...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11840#issuecomment-198648880 **[Test build #53608 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53608/consoleFull)** for PR 11840 at commit [`0cc3f4f`](https://gi

[GitHub] spark pull request: [SPARK-13897][SQL] RelationalGroupedDataset an...

2016-03-18 Thread liancheng
Github user liancheng commented on a diff in the pull request: https://github.com/apache/spark/pull/11841#discussion_r56745324 --- Diff: project/MimaExcludes.scala --- @@ -315,6 +315,7 @@ object MimaExcludes { ProblemFilters.exclude[MissingClassProblem]("org.apache.spa

[GitHub] spark pull request: [SPARK-14011][CORE][SQL] Enable `LineLength` J...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11831#issuecomment-198648881 **[Test build #53609 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53609/consoleFull)** for PR 11831 at commit [`70b41c8`](https://gi

[GitHub] spark pull request: [Spark-14019] [SQL] Remove noop SortOrder in S...

2016-03-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11840#discussion_r56745275 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -826,6 +827,17 @@ object CombineFilters extends Rule

[GitHub] spark pull request: [Spark-14019] [SQL] Remove noop SortOrder in S...

2016-03-18 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/11840#discussion_r56745280 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -826,6 +827,17 @@ object CombineFilters extends Rule

[GitHub] spark pull request: [SPARK-13897][SQL] RelationalGroupedDataset an...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11841#issuecomment-198648728 **[Test build #53607 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53607/consoleFull)** for PR 11841 at commit [`3620da9`](https://gi

[GitHub] spark pull request: [MINOR][DOCS] Update build descriptions and co...

2016-03-18 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the pull request: https://github.com/apache/spark/pull/11838#issuecomment-198648575 Thank you, @rxin. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have th

[GitHub] spark pull request: [SPARK-12789] [SQL] Support Order By Ordinal i...

2016-03-18 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/11815#issuecomment-198634566 LGTM except 2 minor comments --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-13897][SQL] RelationalGroupedDataset an...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11841#issuecomment-198648512 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13897][SQL] RelationalGroupedDataset an...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11841#issuecomment-198648504 **[Test build #53606 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53606/consoleFull)** for PR 11841 at commit [`24eaf42`](https://g

[GitHub] spark pull request: [SPARK-13897][SQL] RelationalGroupedDataset an...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11841#issuecomment-198648514 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13951] Add nested Pipeline load/save su...

2016-03-18 Thread yinxusen
Github user yinxusen commented on the pull request: https://github.com/apache/spark/pull/11835#issuecomment-198634432 @jkbradley MiMa tests failed for changing to the `StageArrayParam`. But I think we need a new Param like `ArrayParam[T]` with the Java compatible `w` function

[GitHub] spark pull request: [SPARK-13980][WIP] Incrementally serialize blo...

2016-03-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11791#issuecomment-198648221 Still WIP? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enab

[GitHub] spark pull request: [SPARK-13908][SQL] Add a LocalLimit for Collec...

2016-03-18 Thread viirya
Github user viirya closed the pull request at: https://github.com/apache/spark/pull/11817 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-13908][SQL] Add a LocalLimit for Collec...

2016-03-18 Thread viirya
Github user viirya commented on the pull request: https://github.com/apache/spark/pull/11817#issuecomment-198648163 Rethink this issue, I think the issue described in the JIRA should not related to pushdown of limit. Because the latest CollectLimit only takes few rows (here is only 1

[GitHub] spark pull request: [Spark-14019] [SQL] Remove noop SortOrder in S...

2016-03-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11840#discussion_r56745107 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -826,6 +827,17 @@ object CombineFilters extends Rule[Logic

[GitHub] spark pull request: [SPARK-14012][SQL] Extract VectorizedColumnRea...

2016-03-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11834#issuecomment-198634733 Did any code change? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fea

[GitHub] spark pull request: [SPARK-13897][SQL] RelationalGroupedDataset an...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11841#issuecomment-198648078 **[Test build #53606 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53606/consoleFull)** for PR 11841 at commit [`24eaf42`](https://gi

[GitHub] spark pull request: [Spark-14019] [SQL] Remove noop SortOrder in S...

2016-03-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11840#discussion_r56745083 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -826,6 +827,17 @@ object CombineFilters extends Rule[Logic

[GitHub] spark pull request: [SPARK-14018][SQL] Use 64-bit num records in B...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11839#issuecomment-198635870 **[Test build #53603 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53603/consoleFull)** for PR 11839 at commit [`cbe436f`](https://gi

[GitHub] spark pull request: [SPARK-13903][SQL] Modify output nullability w...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11722#issuecomment-198640266 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13897][SQL] RelationalGroupedDataset an...

2016-03-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11841#issuecomment-198647907 cc @liancheng and @sameeragarwal --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-13897][SQL] RelationalGroupedDataset an...

2016-03-18 Thread rxin
GitHub user rxin opened a pull request: https://github.com/apache/spark/pull/11841 [SPARK-13897][SQL] RelationalGroupedDataset and KeyValueGroupedDataset ## What changes were proposed in this pull request? Previously, Dataset.groupBy returns a GroupedData, and Dataset.groupByKey

[GitHub] spark pull request: [SPARK-13919] [SQL] fix column pruning through...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11828#issuecomment-198647760 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13919] [SQL] fix column pruning through...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11828#issuecomment-198647761 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13919] [SQL] fix column pruning through...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11828#issuecomment-198647714 **[Test build #53600 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53600/consoleFull)** for PR 11828 at commit [`b1118e5`](https://g

[GitHub] spark pull request: [Spark-14019] [SQL] Remove noop SortOrder in S...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11840#issuecomment-198646675 **[Test build #53605 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53605/consoleFull)** for PR 11840 at commit [`96cd8ce`](https://gi

[GitHub] spark pull request: [SPARK-14012][SQL] Extract VectorizedColumnRea...

2016-03-18 Thread sameeragarwal
Github user sameeragarwal commented on the pull request: https://github.com/apache/spark/pull/11834#issuecomment-198637517 No (we made all the changes in https://github.com/apache/spark/pull/11799) --- If your project is set up for it, you can reply to this email and have your reply a

[GitHub] spark pull request: [SPARK-13993][PySpark] Add pyspark Rformula/Rf...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11807#issuecomment-198638912 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [Spark-14019] [SQL] Remove noop SortOrder in S...

2016-03-18 Thread gatorsmile
GitHub user gatorsmile opened a pull request: https://github.com/apache/spark/pull/11840 [Spark-14019] [SQL] Remove noop SortOrder in Sort What changes were proposed in this pull request? This PR is to add a new Optimizer rule for pruning Sort if its SortOrder is no-op

[GitHub] spark pull request: [SPARK-13993][PySpark] Add pyspark Rformula/Rf...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11807#issuecomment-198638752 **[Test build #53602 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53602/consoleFull)** for PR 11807 at commit [`d31bb3f`](https://g

[GitHub] spark pull request: [Minor] [ML] When trainingSummary is None, it ...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11784#issuecomment-197916455 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13958]Executor OOM due to unbounded gro...

2016-03-18 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11794#discussion_r56690412 --- Diff: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java --- @@ -320,7 +320,15 @@ private void growPointerArrayIfNecessary() thro

[GitHub] spark pull request: [SPARK-13942][CORE][DOCS] Remove Shark-related...

2016-03-18 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11770#issuecomment-197572471 BTW, if you're going to consider changing `SparkEnv` then I'd remove the deprecated methods from back when it used to be a thread-local. --- If your project is set u

[GitHub] spark pull request: [SPARK-13430][PySpark][ML] Python API for trai...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11621#issuecomment-198019144 **[Test build #53449 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53449/consoleFull)** for PR 11621 at commit [`d7e17ab`](https://gi

[GitHub] spark pull request: [SPARK-12719][HOTFIX] Fix compilation against ...

2016-03-18 Thread yy2016
Github user yy2016 commented on the pull request: https://github.com/apache/spark/pull/11787#issuecomment-198053603 I wonder why OneRowRelation isn't covered by the following import ? ``` import org.apache.spark.sql.catalyst.plans.logical._ ``` --- If your project is set u

[GitHub] spark pull request: [SPARK-13908][SQL] Add a LocalLimit for Collec...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11817#issuecomment-198370581 **[Test build #53539 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53539/consoleFull)** for PR 11817 at commit [`2d983c9`](https://gi

[GitHub] spark pull request: [SPARK-9478] [ml] Add class weights to Random ...

2016-03-18 Thread sethah
Github user sethah commented on the pull request: https://github.com/apache/spark/pull/9008#issuecomment-198428763 cc @MLnick thoughts on the above comments? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [SPARK-13449] Naive Bayes wrapper in SparkR

2016-03-18 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/11486#discussion_r56623549 --- Diff: R/pkg/R/mllib.R --- @@ -71,14 +71,23 @@ setMethod("glm", signature(formula = "formula", family = "ANY", data = "DataFram #' @rdname predic

[GitHub] spark pull request: [SPARK-13923] [SQL] Implement SessionCatalog

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11750#issuecomment-197596671 **[Test build #2646 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/2646/consoleFull)** for PR 11750 at commit [`ad43a5f`](https://g

[GitHub] spark pull request: [SPARK-13997][SQL] Use Hadoop 2.0 default valu...

2016-03-18 Thread tomwhite
Github user tomwhite commented on the pull request: https://github.com/apache/spark/pull/11806#issuecomment-198358199 I agree that BLOCK is always to be preferred over RECORD, so leave it at BLOCK. RECORD is the default in Hadoop 1 and 2 (for backwards compatibility reasons), but that

[GitHub] spark pull request: [SPARK-13427][SQL] Support USING clause in JOI...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11297#issuecomment-197749675 **[Test build #53397 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53397/consoleFull)** for PR 11297 at commit [`29a4f59`](https://g

[GitHub] spark pull request: [SPARK-13812][SPARKR] Fix SparkR lint-r test e...

2016-03-18 Thread shaneknapp
Github user shaneknapp commented on the pull request: https://github.com/apache/spark/pull/11652#issuecomment-197597252 i'm currently installing the latest lintr on all of our jenkins workers. this should finish in ~15 mins --- If your project is set up for it, you can reply to this

[GitHub] spark pull request: Added transitive closure transformation to Cat...

2016-03-18 Thread antonoal
GitHub user antonoal opened a pull request: https://github.com/apache/spark/pull/11777 Added transitive closure transformation to Catalyst ## What changes were proposed in this pull request? A relatively simple transformation is missing from Catalyst's arsenal - generation of tr

[GitHub] spark pull request: [SPARK-13281][CORE] Switch broadcast of RDD to...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11735#issuecomment-197351217 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13320] [SQL] Support Star in CreateStru...

2016-03-18 Thread gatorsmile
Github user gatorsmile commented on the pull request: https://github.com/apache/spark/pull/11208#issuecomment-197635547 cc @yhuai --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request: [SPARK-13898][SQL] Merge DatasetHolder and Dat...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11737#issuecomment-197702816 **[Test build #53394 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53394/consoleFull)** for PR 11737 at commit [`59cae95`](https://gi

[GitHub] spark pull request: [SPARK-13923] [SQL] Implement SessionCatalog

2016-03-18 Thread andrewor14
Github user andrewor14 commented on a diff in the pull request: https://github.com/apache/spark/pull/11750#discussion_r56425133 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/interface.scala --- @@ -211,8 +214,7 @@ case class CatalogTablePartition(

[GitHub] spark pull request: [SPARK-13992][WIP] Add support for off-heap ca...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11805#issuecomment-198142117 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13068][PYSPARK][ML] Type conversion for...

2016-03-18 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/11663#discussion_r56548734 --- Diff: python/pyspark/ml/param/_shared_params_code_gen.py --- @@ -105,64 +104,71 @@ def get$Name(self): if __name__ == "__main__": print(heade

[GitHub] spark pull request: [SPARK-12182][ML] Distributed binning for tree...

2016-03-18 Thread sethah
Github user sethah commented on the pull request: https://github.com/apache/spark/pull/10231#issuecomment-197597688 I can set something up. Do you have a specific dataset size in mind or even a specific dataset? --- If your project is set up for it, you can reply to this email and ha

[GitHub] spark pull request: [SPARK-13883][SQL] Parquet Implementation of F...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11709#issuecomment-198553825 **[Test build #53558 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53558/consoleFull)** for PR 11709 at commit [`c7ca5fe`](https://g

[GitHub] spark pull request: [WIP][SPARK-13809][SQL] State store for stream...

2016-03-18 Thread marmbrus
Github user marmbrus commented on a diff in the pull request: https://github.com/apache/spark/pull/11645#discussion_r56375262 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/StateStoreRDD.scala --- @@ -0,0 +1,60 @@ +/* + * Licensed to the A

[GitHub] spark pull request: SPARK-9926: Parallelize partition logic in Uni...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11242#issuecomment-198572980 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13805][SQL] Generate code that get a va...

2016-03-18 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11636#discussion_r56455795 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/BoundAttribute.scala --- @@ -60,17 +60,28 @@ case class BoundReference(ordina

[GitHub] spark pull request: [SPARK-13068][PYSPARK][ML] Type conversion for...

2016-03-18 Thread sethah
Github user sethah commented on a diff in the pull request: https://github.com/apache/spark/pull/11663#discussion_r56547567 --- Diff: python/pyspark/ml/param/__init__.py --- @@ -65,6 +72,106 @@ def __eq__(self, other): return False +class TypeConvert

[GitHub] spark pull request: [SPARK-12343][YARN] Simplify Yarn client and c...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11603#issuecomment-197836044 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13845][CORE]Using onBlockUpdated to rep...

2016-03-18 Thread jeanlyn
Github user jeanlyn closed the pull request at: https://github.com/apache/spark/pull/11679 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is en

[GitHub] spark pull request: [SPARK-13874][Doc]Remove docs of streaming-akk...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11824#issuecomment-198463972 **[Test build #53545 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53545/consoleFull)** for PR 11824 at commit [`ebf35a3`](https://gi

[GitHub] spark pull request: [SPARK-13808][test-maven] Don't build assembly...

2016-03-18 Thread JoshRosen
Github user JoshRosen commented on the pull request: https://github.com/apache/spark/pull/11701#issuecomment-198002898 Jenkins, retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request: [SPARK-13981][SQL] Defer evaluating variables ...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11792#issuecomment-198090947 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13764][SQL] Parse modes in JSON data so...

2016-03-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/11756#discussion_r56325150 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala --- @@ -963,6 +963,28 @@ class JsonSuite extends QueryTe

[GitHub] spark pull request: [SPARK-13038] [PySpark] Add load/save to pipel...

2016-03-18 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/11683#issuecomment-197552039 Hm, if you want to do it under a new JIRA that's OK too. I'll create one now. --- If your project is set up for it, you can reply to this email and have your reply

[GitHub] spark pull request: [SPARK-13919] [SQL] [WIP] Resolving the Confli...

2016-03-18 Thread davies
Github user davies commented on a diff in the pull request: https://github.com/apache/spark/pull/11745#discussion_r56705349 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala --- @@ -306,14 +311,28 @@ object SetOperationPushDown extends R

[GitHub] spark pull request: [SPARK-13977] [SQL] Brings back Shuffled hash ...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11788#issuecomment-198009917 **[Test build #53444 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53444/consoleFull)** for PR 11788 at commit [`2122986`](https://gi

[GitHub] spark pull request: [SPARK-13926] Automatically use Kryo serialize...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11755#issuecomment-197620388 **[Test build #53365 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53365/consoleFull)** for PR 11755 at commit [`45b0c0b`](https://g

[GitHub] spark pull request: SPARK-13991 - Extend the enforcer plugin Maven...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11803#issuecomment-198153139 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13980][WIP] Incrementally serialize blo...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11791#issuecomment-198185758 **[Test build #53496 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53496/consoleFull)** for PR 11791 at commit [`5489748`](https://gi

[GitHub] spark pull request: [SPARK-13982][SparkR] KMean's predict: Feature...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11793#issuecomment-198081036 **[Test build #53464 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53464/consoleFull)** for PR 11793 at commit [`48061de`](https://g

[GitHub] spark pull request: [SPARK-13930][SQL] Apply fast serialization on...

2016-03-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11759 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-13989][SQL] Remove non-vectorized/unsaf...

2016-03-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11799 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-13958]Executor OOM due to unbounded gro...

2016-03-18 Thread davies
Github user davies commented on the pull request: https://github.com/apache/spark/pull/11794#issuecomment-198517715 Merging this into master and 1.6, thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request: [WIP][SPARK-13809][SQL] State store for stream...

2016-03-18 Thread tdas
Github user tdas commented on a diff in the pull request: https://github.com/apache/spark/pull/11645#discussion_r56387171 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/StateStoreRDDSuite.scala --- @@ -0,0 +1,147 @@ +/* + * Licensed to the

[GitHub] spark pull request: [Spark-13034] PySpark ml.classification suppor...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11707#issuecomment-197553313 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-13926] Automatically use Kryo serialize...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11755#issuecomment-197561059 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-12789] [SQL] Support Order By Ordinal i...

2016-03-18 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/11815#discussion_r56618306 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala --- @@ -618,9 +622,33 @@ class Analyzer( * clause. This ru

[GitHub] spark pull request: [SPARK-13432][SQL] add the source file name an...

2016-03-18 Thread kiszk
Github user kiszk commented on a diff in the pull request: https://github.com/apache/spark/pull/11301#discussion_r56354960 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala --- @@ -57,15 +58,15 @@ object CurrentOrigin { def reset

[GitHub] spark pull request: [SPARK-13903][SQL] Modify output nullability w...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11722#issuecomment-198640269 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [Minor] [ML] When trainingSummary is None, it ...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11784#issuecomment-197916052 **[Test build #53432 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53432/consoleFull)** for PR 11784 at commit [`d263159`](https://g

[GitHub] spark pull request: [SPARK-13713][SQL] Migrate parser from ANTLR3 ...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11557#issuecomment-198136155 **[Test build #53480 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53480/consoleFull)** for PR 11557 at commit [`b87f2b8`](https://g

[GitHub] spark pull request: [SPARK-13898][SQL] Merge DatasetHolder and Dat...

2016-03-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11737#issuecomment-197735040 @jodersky there are still two failures here. I probably won't have time to look at it until next week. If you have some time, please go for it! I think some are legitimate

[GitHub] spark pull request: [SPARK-13903][SQL] Modify output nullability w...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11722#issuecomment-198640066 **[Test build #53599 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53599/consoleFull)** for PR 11722 at commit [`7f68967`](https://g

[GitHub] spark pull request: [SPARK-13976][SQL] do not remove sub-queries a...

2016-03-18 Thread cloud-fan
Github user cloud-fan commented on the pull request: https://github.com/apache/spark/pull/11786#issuecomment-198165733 merging to master! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request: [SPARK-13990] Automatically pick serializer wh...

2016-03-18 Thread JoshRosen
GitHub user JoshRosen opened a pull request: https://github.com/apache/spark/pull/11801 [SPARK-13990] Automatically pick serializer when caching RDDs Building on the `SerializerManager` introduced in SPARK-13926/ #11755, this patch Spark modifies Spark's BlockManager to use RDD's Cl

[GitHub] spark pull request: [SPARK-12789] [SQL] Support Order By Ordinal i...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11815#issuecomment-198639998 **[Test build #53604 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53604/consoleFull)** for PR 11815 at commit [`b04529b`](https://gi

[GitHub] spark pull request: [SPARK-13761] [ML] Deprecate validateParams

2016-03-18 Thread jkbradley
Github user jkbradley commented on a diff in the pull request: https://github.com/apache/spark/pull/11620#discussion_r56418520 --- Diff: mllib/src/main/scala/org/apache/spark/ml/regression/GeneralizedLinearRegression.scala --- @@ -178,7 +192,8 @@ class GeneralizedLinearRegression

[GitHub] spark pull request: [SPARK-13921] Store serialized blocks as multi...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11748#issuecomment-198177509 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/

[GitHub] spark pull request: [SPARK-11888] [ML] Decision tree persistence i...

2016-03-18 Thread jkbradley
Github user jkbradley commented on the pull request: https://github.com/apache/spark/pull/11581#issuecomment-197553266 Got the LGTM from @mengxr offline. I'll merge this with master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHu

[GitHub] spark pull request: [SPARK-13894][SQL] SqlContext.range return typ...

2016-03-18 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/11730 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is ena

[GitHub] spark pull request: [SPARK-13808][test-maven] Don't build assembly...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11701#issuecomment-198004371 **[Test build #53443 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53443/consoleFull)** for PR 11701 at commit [`0900b13`](https://gi

[GitHub] spark pull request: [SPARK-13942][CORE][DOCS] Remove Shark-related...

2016-03-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11770#issuecomment-197591689 I'm merging this in master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does no

[GitHub] spark pull request: [SPARK-11011][SQL] Narrow type of UDT serializ...

2016-03-18 Thread yhuai
Github user yhuai commented on the pull request: https://github.com/apache/spark/pull/11379#issuecomment-197598917 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled an

[GitHub] spark pull request: [SPARK-13764][SQL] Parse modes in JSON data so...

2016-03-18 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/11756#discussion_r56323218 --- Diff: python/pyspark/sql/readwriter.py --- @@ -162,6 +162,14 @@ def json(self, path, schema=None): (e.g. 00012) *

[GitHub] spark pull request: [SPARK-14012][SQL] Extract VectorizedColumnRea...

2016-03-18 Thread rxin
Github user rxin commented on the pull request: https://github.com/apache/spark/pull/11834#issuecomment-198638010 Thanks - merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request: [SPARK-13118][SQL] Expression encoding for opt...

2016-03-18 Thread AmplabJenkins
Github user AmplabJenkins commented on the pull request: https://github.com/apache/spark/pull/11708#issuecomment-197595897 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your projec

[GitHub] spark pull request: [SPARK-13985][SQL] Deterministic batches with ...

2016-03-18 Thread SparkQA
Github user SparkQA commented on the pull request: https://github.com/apache/spark/pull/11804#issuecomment-198484446 **[Test build #53548 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/53548/consoleFull)** for PR 11804 at commit [`97503f1`](https://gi

[GitHub] spark pull request: [SPARK-13930][SQL] Apply fast serialization on...

2016-03-18 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/11759#discussion_r56611911 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SparkPlan.scala --- @@ -218,48 +218,64 @@ abstract class SparkPlan extends QueryPlan[SparkPla

  1   2   3   4   5   6   >