[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r112370906 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ArrowConverters.scala --- @@ -0,0 +1,432 @@ +/* +* Licensed to the Apache Software Foundation

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r112370321 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ArrowConverters.scala --- @@ -0,0 +1,432 @@ +/* +* Licensed to the Apache Software Foundation

[GitHub] spark issue #17684: [SPARK-20341][SQL] Support BigInt's value that does not ...

2017-04-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17684 Ok, could you write a comment in the code to explain it? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r112368956 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ArrowConverters.scala --- @@ -0,0 +1,432 @@ +/* +* Licensed to the Apache Software Foundation

[GitHub] spark issue #17695: [SPARK-20400][DOCS] Remove References to 3rd Party Vendo...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17695 **[Test build #75969 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75969/testReport)** for PR 17695 at commit

[GitHub] spark issue #17695: [SPARK-20400][DOCS] Remove References to 3rd Party Vendo...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17695 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75969/ Test PASSed. ---

[GitHub] spark issue #17695: [SPARK-20400][DOCS] Remove References to 3rd Party Vendo...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17695 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r112368367 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/ArrowConverters.scala --- @@ -0,0 +1,432 @@ +/* +* Licensed to the Apache Software Foundation

[GitHub] spark pull request #17670: [SPARK-20281][SQL] Print the identical Range para...

2017-04-19 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/17670#discussion_r112368162 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2606,4 +2607,15 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark issue #17670: [SPARK-20281][SQL] Print the identical Range parameters ...

2017-04-19 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/17670 okay, I'll fix soon. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17582: [SPARK-20239][Core] Improve HistoryServer's ACL mechanis...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17582 **[Test build #75970 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75970/testReport)** for PR 17582 at commit

[GitHub] spark issue #17631: [SPARK-20319][SQL] Already quoted identifiers are gettin...

2017-04-19 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/17631 this doesn't seem safe in general. spark treats quotes as a valid part of the name, eg: ```scala scala> val df = sc.parallelize(1 to 10).map { x => (x, (x + 1).toLong, (x -

[GitHub] spark issue #17695: [SPARK-20400][DOCS] Remove References to 3rd Party Vendo...

2017-04-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17695 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #17695: [SPARK-20400][DOCS] Remove References to 3rd Party Vendo...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17695 **[Test build #75969 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75969/testReport)** for PR 17695 at commit

[GitHub] spark issue #17695: [SPARK-20400][DOCS] Remove References to 3rd Party Vendo...

2017-04-19 Thread anabranch
Github user anabranch commented on the issue: https://github.com/apache/spark/pull/17695 This should be on hold until a JIRA resolution, I'd like to hear what others say. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark pull request #17695: [SPARK-20400][DOCS] Remove References to 3rd Part...

2017-04-19 Thread anabranch
GitHub user anabranch opened a pull request: https://github.com/apache/spark/pull/17695 [SPARK-20400][DOCS] Remove References to 3rd Party Vendor Tools ## What changes were proposed in this pull request? Simple documentation change to remove explicit vendor references.

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15125 **[Test build #75968 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75968/testReport)** for PR 15125 at commit

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15125 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75968/ Test FAILed. ---

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/15125 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17582: [SPARK-20239][Core] Improve HistoryServer's ACL mechanis...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17582 **[Test build #75967 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75967/testReport)** for PR 17582 at commit

[GitHub] spark issue #15125: [SPARK-5484][GraphX] Periodically do checkpoint in Prege...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/15125 **[Test build #75968 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75968/testReport)** for PR 15125 at commit

[GitHub] spark issue #17670: [SPARK-20281][SQL] Print the identical Range parameters ...

2017-04-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17670 As @jaceklaskowski said, it would be good to fill `numSlices` in the rule `ResolveTableValuedFunctions`, if it is None. --- If your project is set up for it, you can reply to this email and

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17680 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17680 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75964/ Test PASSed. ---

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17680 **[Test build #75964 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75964/testReport)** for PR 17680 at commit

[GitHub] spark pull request #17670: [SPARK-20281][SQL] Print the identical Range para...

2017-04-19 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/17670#discussion_r112366125 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -2606,4 +2607,15 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r112365872 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1635,21 +1636,49 @@ def toDF(self, *cols): return DataFrame(jdf, self.sql_ctx)

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r112365773 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1635,21 +1636,49 @@ def toDF(self, *cols): return DataFrame(jdf, self.sql_ctx)

[GitHub] spark issue #17682: [SPARK-20385][WEB-UI]'Submitted Time' field, the date fo...

2017-04-19 Thread guoxiaolongzte
Github user guoxiaolongzte commented on the issue: https://github.com/apache/spark/pull/17682 Can I use the Jenkins test? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark pull request #15821: [SPARK-13534][PySpark] Using Apache Arrow to incr...

2017-04-19 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/15821#discussion_r112365501 --- Diff: python/pyspark/serializers.py --- @@ -182,6 +182,23 @@ def loads(self, obj): raise NotImplementedError +class

[GitHub] spark issue #17649: [SPARK-20380][SQL] Output table comment for DESC FORMATT...

2017-04-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17649 Any update? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #17636: [SPARK-20334][SQL] Return a better error message when co...

2017-04-19 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/17636 LGTM --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-04-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15821 @BryanCutler Are you going to update this for arrow 0.3? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #15821: [SPARK-13534][PySpark] Using Apache Arrow to increase pe...

2017-04-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/15821 Please move ArrowConverters.scala somewhere else that's not top level, e.g. execution.arrow --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17680 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75963/ Test PASSed. ---

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17680 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17680 **[Test build #75963 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75963/testReport)** for PR 17680 at commit

[GitHub] spark issue #17623: [SPARK-20292][SQL] Clean up string representation of Tre...

2017-04-19 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/17623 @ericl Ok. Thanks for pointing that. I will try to add the test. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #17674: [SPARK-20375][R] R wrappers for array and map

2017-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17674 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17674: [SPARK-20375][R] R wrappers for array and map

2017-04-19 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/17674 merged to master. thanks! one step closer to parity --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17087 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16781: [SPARK-12297][SQL] Hive compatibility for Parquet...

2017-04-19 Thread squito
Github user squito commented on a diff in the pull request: https://github.com/apache/spark/pull/16781#discussion_r112362704 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/ParquetHiveCompatibilitySuite.scala --- @@ -141,4 +160,326 @@ class

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17087 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75961/ Test PASSed. ---

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17087 **[Test build #75961 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75961/testReport)** for PR 17087 at commit

[GitHub] spark issue #16781: [SPARK-12297][SQL] Hive compatibility for Parquet Timest...

2017-04-19 Thread squito
Github user squito commented on the issue: https://github.com/apache/spark/pull/16781 @ueshin I've pushed an update which addresses your comments. I also realized that partitioned tables weren't handled correctly! I fixed that as well. --- If your project is set up for it, you

[GitHub] spark pull request #17672: [SPARK-20371][R] Add wrappers for collect_list an...

2017-04-19 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/17672#discussion_r112362340 --- Diff: R/pkg/R/functions.R --- @@ -3652,3 +3652,43 @@ setMethod("posexplode", jc <- callJStatic("org.apache.spark.sql.functions",

[GitHub] spark issue #16781: [SPARK-12297][SQL] Hive compatibility for Parquet Timest...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16781 **[Test build #75966 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75966/testReport)** for PR 16781 at commit

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17680 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17680 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75962/ Test FAILed. ---

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17680 **[Test build #75962 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75962/testReport)** for PR 17680 at commit

[GitHub] spark issue #17631: [SPARK-20319][SQL] Already quoted identifiers are gettin...

2017-04-19 Thread umesh9794
Github user umesh9794 commented on the issue: https://github.com/apache/spark/pull/17631 @marmbrus @squito @vanzin , may I ask you to review this PR ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #17631: [SPARK-20319][SQL] Already quoted identifiers are gettin...

2017-04-19 Thread umesh9794
Github user umesh9794 commented on the issue: https://github.com/apache/spark/pull/17631 @marmbrus @squito @vanzin , may I ask you to review this PR ? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark issue #17678: [SPARK-20381][SQL] Add SQL metrics of numOutputRows for ...

2017-04-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17678 Jenkins, test this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #17582: [SPARK-20239][Core] Improve HistoryServer's ACL m...

2017-04-19 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/17582#discussion_r112358891 --- Diff: core/src/main/scala/org/apache/spark/status/api/v1/ApiRootResource.scala --- @@ -184,14 +184,27 @@ private[v1] class ApiRootResource extends

[GitHub] spark issue #17636: [SPARK-20334][SQL] Return a better error message when co...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17636 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17636: [SPARK-20334][SQL] Return a better error message when co...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17636 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75960/ Test PASSed. ---

[GitHub] spark issue #17636: [SPARK-20334][SQL] Return a better error message when co...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17636 **[Test build #75960 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75960/testReport)** for PR 17636 at commit

[GitHub] spark pull request #17689: [SPARK-20378][CORE][SQL][SS] StreamSinkProvider s...

2017-04-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17689#discussion_r112358516 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/sources/interfaces.scala --- @@ -151,6 +151,7 @@ trait StreamSourceProvider {

[GitHub] spark issue #17678: [SPARK-20381][SQL] Add SQL metrics of numOutputRows for ...

2017-04-19 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/17678 No, ObjectHashAggregateExec dose not support codegen. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #17495: [SPARK-20172][Core] Add file permission check when listi...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17495 **[Test build #75965 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75965/testReport)** for PR 17495 at commit

[GitHub] spark pull request #17690: Fixed typos in docs

2017-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17690 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17678: [SPARK-20381][SQL] Add SQL metrics of numOutputRows for ...

2017-04-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17678 Is there a codegen version we need to worry about? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17690: Fixed typos in docs

2017-04-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17690 Thanks - merging in master/branch-2.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17680 **[Test build #75964 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75964/testReport)** for PR 17680 at commit

[GitHub] spark pull request #17692: [SPARK-20398] [SQL] range() operator should inclu...

2017-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17692 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17692: [SPARK-20398] [SQL] range() operator should include canc...

2017-04-19 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/17692 Merging in master/branch-2.2. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17680 **[Test build #75963 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75963/testReport)** for PR 17680 at commit

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17680 cc @davies too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/17680 I added a test case so that we make sure it does not push down filters in `FileFormat` when there are filters with nested column access. For me, I could not find related test in the test cases

[GitHub] spark issue #17680: [SPARK-20364][SQL] Support Parquet predicate pushdown on...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17680 **[Test build #75962 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75962/testReport)** for PR 17680 at commit

[GitHub] spark issue #17694: [SPARK-12717][PYSPARK] Resolving race condition with pys...

2017-04-19 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/17694 Please remove the boilerplate message at the end of your PR description. ok to test --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark pull request #17691: [MINOR][SS] Fix a missing space in UnsupportedOpe...

2017-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17691 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17691: [MINOR][SS] Fix a missing space in UnsupportedOperationC...

2017-04-19 Thread zsxwing
Github user zsxwing commented on the issue: https://github.com/apache/spark/pull/17691 Thanks! Merging to master, 2.2 and 2.1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #17650: [SPARK-20350] Add optimization rules to apply Com...

2017-04-19 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/17650 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #17087: [SPARK-19372][SQL] Fix throwing a Java exception at df.f...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17087 **[Test build #75961 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75961/testReport)** for PR 17087 at commit

[GitHub] spark issue #17650: [SPARK-20350] Add optimization rules to apply Complement...

2017-04-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17650 thanks, merging to master/2.2! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17694: [SPARK-12717][PYSPARK] Resolving race condition with pys...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17694 Can one of the admins verify this patch? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark issue #17480: [SPARK-20079][Core][yarn] Re registration of AM hangs sp...

2017-04-19 Thread witgo
Github user witgo commented on the issue: https://github.com/apache/spark/pull/17480 OK, I will do the work at weekends. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17665: [SPARK-16742] Mesos Kerberos Support

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17665 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75956/ Test PASSed. ---

[GitHub] spark pull request #17694: [SPARK-12717][PYSPARK] Resolving race condition w...

2017-04-19 Thread vundela
GitHub user vundela opened a pull request: https://github.com/apache/spark/pull/17694 [SPARK-12717][PYSPARK] Resolving race condition with pyspark broadcas… …ts when using multiple threads ## What changes were proposed in this pull request? In pyspark when

[GitHub] spark issue #17665: [SPARK-16742] Mesos Kerberos Support

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17665 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17620: [SPARK-20305][Spark Core]Master may keep in the state of...

2017-04-19 Thread lvdongr
Github user lvdongr commented on the issue: https://github.com/apache/spark/pull/17620 You can see the main method in Master.scala. def main(argStrings: Array[String]) { Utils.initDaemon(log) val conf = new SparkConf val args = new

[GitHub] spark issue #17665: [SPARK-16742] Mesos Kerberos Support

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17665 **[Test build #75956 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75956/testReport)** for PR 17665 at commit

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-04-19 Thread ouyangxiaochen
Github user ouyangxiaochen commented on the issue: https://github.com/apache/spark/pull/17681 It seems more reasonable. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #17620: [SPARK-20305][Spark Core]Master may keep in the state of...

2017-04-19 Thread lvdongr
Github user lvdongr commented on the issue: https://github.com/apache/spark/pull/17620 This happend at the time the previous master leader remove the died worker ,clear the worker's node on persistEngine(we use zookeeper),But before the worker node was removed from the zookeeper ,the

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-04-19 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/17681 shall we follow `CREATE VIEW` and support both `OR REPLACE` and `IF NOT EXISTS`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well.

[GitHub] spark issue #17692: [SPARK-20398] [SQL] range() operator should include canc...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17692 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75959/ Test PASSed. ---

[GitHub] spark issue #17692: [SPARK-20398] [SQL] range() operator should include canc...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17692 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17692: [SPARK-20398] [SQL] range() operator should include canc...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17692 **[Test build #75959 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75959/testReport)** for PR 17692 at commit

[GitHub] spark pull request #17680: [SPARK-20364][SQL] Support Parquet predicate push...

2017-04-19 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/17680#discussion_r112347873 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetFilterSuite.scala --- @@ -536,4 +537,43 @@ class

[GitHub] spark issue #17691: [MINOR][SS] Fix a missing space in UnsupportedOperationC...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17691 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #17691: [MINOR][SS] Fix a missing space in UnsupportedOperationC...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/17691 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75957/ Test PASSed. ---

[GitHub] spark issue #17691: [MINOR][SS] Fix a missing space in UnsupportedOperationC...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17691 **[Test build #75957 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75957/testReport)** for PR 17691 at commit

[GitHub] spark issue #17681: [SPARK-20383][SQL] Supporting Create [temporary] Functio...

2017-04-19 Thread ouyangxiaochen
Github user ouyangxiaochen commented on the issue: https://github.com/apache/spark/pull/17681 The synax of DBMS like this` CREATE OR REPLACE FUNCTION `, it means that if function exists we will replace it, otherwise create one. But i think we'd better not destroy the original

[GitHub] spark issue #17467: [SPARK-20140][DStream] Remove hardcoded kinesis retry wa...

2017-04-19 Thread yssharma
Github user yssharma commented on the issue: https://github.com/apache/spark/pull/17467 @budde - thanks for taking time to review it. Appreciate it. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does

[GitHub] spark pull request #17467: [SPARK-20140][DStream] Remove hardcoded kinesis r...

2017-04-19 Thread yssharma
Github user yssharma commented on a diff in the pull request: https://github.com/apache/spark/pull/17467#discussion_r112347174 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisInputDStream.scala --- @@ -249,6 +252,17 @@ object

[GitHub] spark issue #17636: [SPARK-20334][SQL] Return a better error message when co...

2017-04-19 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/17636 **[Test build #75960 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/75960/testReport)** for PR 17636 at commit

[GitHub] spark pull request #17467: [SPARK-20140][DStream] Remove hardcoded kinesis r...

2017-04-19 Thread yssharma
Github user yssharma commented on a diff in the pull request: https://github.com/apache/spark/pull/17467#discussion_r112346805 --- Diff: external/kinesis-asl/src/main/scala/org/apache/spark/streaming/kinesis/KinesisBackedBlockRDD.scala --- @@ -147,6 +152,14 @@ class

[GitHub] spark issue #13706: [SPARK-15988] [SQL] Implement DDL commands: Create/Drop ...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13706 Build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #13706: [SPARK-15988] [SQL] Implement DDL commands: Create/Drop ...

2017-04-19 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13706 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/75958/ Test FAILed. ---

  1   2   3   4   5   >