[GitHub] spark issue #14579: [SPARK-16921][PYSPARK] RDD/DataFrame persist()/cache() s...

2016-08-10 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/14579 Right I wouldn't expect it to error with subclassing - just not pipeline successfully - but only in a very long shot corner case. I think the try/finally with persistance is not an uncommon

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14580 Oracle supports it... http://docs.oracle.com/javadb/10.10.1.2/ref/rrefsqljusing.html --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #12004: [SPARK-7481][build] [WIP] Add Hadoop 2.6+ spark-cloud mo...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/12004 **[Test build #63588 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63588/consoleFull)** for PR 12004 at commit

[GitHub] spark issue #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and isolat...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14590 **[Test build #3216 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3216/consoleFull)** for PR 14590 at commit

[GitHub] spark issue #14454: [Minor] [ML] Rename TreeEnsembleModels to TreeEnsembleMo...

2016-08-10 Thread yanboliang
Github user yanboliang commented on the issue: https://github.com/apache/spark/pull/14454 ping @srowen --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or

[GitHub] spark issue #14559: [SPARK-16968]Add additional options in jdbc when creatin...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14559 **[Test build #63587 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63587/consoleFull)** for PR 14559 at commit

[GitHub] spark pull request #14591: [SPARK-17010][MINOR][DOC]Wrong description in mem...

2016-08-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14591 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14559: [SPARK-16968]Add additional options in jdbc when creatin...

2016-08-10 Thread GraceH
Github user GraceH commented on the issue: https://github.com/apache/spark/pull/14559 @HyukjinKwon and @srowen, here is the initial proposal. Please let me know your comment. I will refine that with unit test later. BTW, the readwriter.py calls high level api of jdbc(url,

[GitHub] spark issue #14592: [SPARK-17011][SQL] Support testing exceptions in SQLQuer...

2016-08-10 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14592 LGTM pending Jenkins. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so,

[GitHub] spark issue #14591: [SPARK-17010][MINOR][DOC]Wrong description in memory man...

2016-08-10 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14591 Merging in master/2.0. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark pull request #14592: [SPARK-17011][SQL] Support testing exceptions in ...

2016-08-10 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14592#discussion_r74370013 --- Diff: sql/core/src/test/resources/sql-tests/results/number-format.sql.out --- @@ -19,16 +19,24 @@ struct<2147483648:bigint,(-2147483649):bigint>

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14580 Which NoSQL platforms support `Using Outer Join`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #14593: [MINOR][DOCS] Fix style in examples and inconsistent ind...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14593 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14593: [MINOR][DOCS] Fix style in examples and inconsistent ind...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14593 **[Test build #63586 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63586/consoleFull)** for PR 14593 at commit

[GitHub] spark issue #14593: [MINOR][DOCS] Fix style in examples and inconsistent ind...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14593 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63586/ Test PASSed. ---

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-10 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/13775 @yhuai You mean just using `sql("SELECT * FROM t").count()`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark pull request #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13775#discussion_r74369496 --- Diff: sql/hive/src/main/java/org/apache/hadoop/hive/ql/io/orc/VectorizedSparkOrcNewRecordReader.java --- @@ -0,0 +1,318 @@ +/* + * Licensed to

[GitHub] spark pull request #14592: [SPARK-17011][SQL] Support testing exceptions in ...

2016-08-10 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/14592#discussion_r74369492 --- Diff: sql/core/src/test/resources/sql-tests/results/number-format.sql.out --- @@ -19,16 +19,24 @@ struct<2147483648:bigint,(-2147483649):bigint>

[GitHub] spark pull request #14592: [SPARK-17011][SQL] Support testing exceptions in ...

2016-08-10 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14592#discussion_r74369336 --- Diff: sql/core/src/test/resources/sql-tests/results/number-format.sql.out --- @@ -19,16 +19,24 @@ struct<2147483648:bigint,(-2147483649):bigint>

[GitHub] spark pull request #14592: [SPARK-17011][SQL] Support testing exceptions in ...

2016-08-10 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/14592#discussion_r74369163 --- Diff: sql/core/src/test/resources/sql-tests/results/number-format.sql.out --- @@ -19,16 +19,24 @@ struct<2147483648:bigint,(-2147483649):bigint>

[GitHub] spark issue #14593: [MINOR][DOCS] Fix style in examples and inconsistent ind...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14593 **[Test build #63586 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63586/consoleFull)** for PR 14593 at commit

[GitHub] spark issue #14593: [MINOR][DOCS] Fix style in examples and inconsistent ind...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14593 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63583/ Test PASSed. ---

[GitHub] spark issue #14593: [MINOR][DOCS] Fix style in examples and inconsistent ind...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14593 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #14592: [SPARK-17011][SQL] Support testing exceptions in ...

2016-08-10 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14592#discussion_r74369002 --- Diff: sql/core/src/test/resources/sql-tests/results/number-format.sql.out --- @@ -19,16 +19,24 @@ struct<2147483648:bigint,(-2147483649):bigint>

[GitHub] spark issue #14593: [MINOR][DOCS] Fix style in examples and inconsistent ind...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14593 **[Test build #63583 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63583/consoleFull)** for PR 14593 at commit

[GitHub] spark pull request #14592: [SPARK-17011][SQL] Support testing exceptions in ...

2016-08-10 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14592#discussion_r74368974 --- Diff: sql/core/src/test/resources/sql-tests/results/number-format.sql.out --- @@ -19,16 +19,24 @@ struct<2147483648:bigint,(-2147483649):bigint>

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13775 **[Test build #63585 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63585/consoleFull)** for PR 13775 at commit

[GitHub] spark pull request #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-10 Thread dafrista
Github user dafrista commented on a diff in the pull request: https://github.com/apache/spark/pull/13775#discussion_r74368871 --- Diff: sql/hive/src/main/java/org/apache/hadoop/hive/ql/io/orc/VectorizedSparkOrcNewRecordReader.java --- @@ -0,0 +1,318 @@ +/* + * Licensed to

[GitHub] spark issue #14592: [SPARK-17011][SQL] Support testing exceptions in SQLQuer...

2016-08-10 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/14592 I'd like to do it incrementally, and ideally one SQL testing file(xxx.sql) one PR, but we can have many PRs at the same time, they are not likely to get conflicted. --- If your project is set

[GitHub] spark pull request #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and...

2016-08-10 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14590#discussion_r74368764 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala --- @@ -126,14 +129,18 @@ class SQLQueryTestSuite extends QueryTest with

[GitHub] spark pull request #14592: [SPARK-17011][SQL] Support testing exceptions in ...

2016-08-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14592#discussion_r74368708 --- Diff: sql/core/src/test/resources/sql-tests/results/number-format.sql.out --- @@ -19,16 +19,24 @@ struct<2147483648:bigint,(-2147483649):bigint>

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14580 **[Test build #63584 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63584/consoleFull)** for PR 14580 at commit

[GitHub] spark pull request #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and...

2016-08-10 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/14590#discussion_r74368596 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala --- @@ -126,14 +129,18 @@ class SQLQueryTestSuite extends QueryTest with

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14568 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63576/ Test FAILed. ---

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14580 I think we should think out of the SQL box. We know that Spark is not a subset of DBMS. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63576 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63576/consoleFull)** for PR 14568 at commit

[GitHub] spark issue #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-10 Thread yhuai
Github user yhuai commented on the issue: https://github.com/apache/spark/pull/13775 for the benchmark, how about we just test the scan operation? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14593: [MINOR][DOCS] Fix style in examples and inconsistent ind...

2016-08-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14593 Also just for reviewers, the inconsistent stuffs I listed in the PR description happen randomly across documentation. So, this fixes them to be consistent according to style guide lines and

[GitHub] spark issue #14593: [MINOR][DOCS] Fix style in examples and inconsistent ind...

2016-08-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14593 BTW, this is not fixing some wrong examples and inconsistent indentation codes in `structured-streaming-programming-guide.md` because https://github.com/apache/spark/pull/14564 is handling

[GitHub] spark issue #14593: [MINOR][DOCS] Fix style in examples and inconsistent ind...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14593 **[Test build #63583 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63583/consoleFull)** for PR 14593 at commit

[GitHub] spark pull request #14593: [MINOR][DOCS] Fix style in examples and inconsist...

2016-08-10 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/14593 [MINOR][DOCS] Fix style in examples and inconsistent indentation across documentation ## What changes were proposed in this pull request? This PR fixes the documentation as below:

[GitHub] spark issue #14592: [SPARK-17011][SQL] Support testing exceptions in SQLQuer...

2016-08-10 Thread petermaxlee
Github user petermaxlee commented on the issue: https://github.com/apache/spark/pull/14592 @cloud-fan after adding enough features to the test harness, do you think I should port all tests over in a single pull request, or more incremental? --- If your project is set up for it, you

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14580 Not sure which RDBMS are supporting `Using Outer Join`. `NULL` generated by outer joins are removed. This sounds a little bit strange. After all, `NULL` also has a meaning. In the plan

[GitHub] spark issue #14592: [SPARK-17011][SQL] Support testing exceptions in SQLQuer...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14592 **[Test build #63582 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63582/consoleFull)** for PR 14592 at commit

[GitHub] spark pull request #13775: [SPARK-16060][SQL] Vectorized Orc reader

2016-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/13775#discussion_r74367827 --- Diff: sql/hive/src/main/java/org/apache/hadoop/hive/ql/io/orc/VectorizedSparkOrcNewRecordReader.java --- @@ -0,0 +1,318 @@ +/* + * Licensed to

[GitHub] spark issue #14588: [SPARK-17005][SQL] fix method tpe in trait AnnotationApi...

2016-08-10 Thread keypointt
Github user keypointt commented on the issue: https://github.com/apache/spark/pull/14588 Oh I'm so sorry. It's breaking 2.10 anyway. I'll double check scala version compatibility next time submitting a PR --- If your project is set up for it, you can reply to this email and

[GitHub] spark pull request #14588: [SPARK-17005][SQL] fix method tpe in trait Annota...

2016-08-10 Thread keypointt
Github user keypointt closed the pull request at: https://github.com/apache/spark/pull/14588 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14102: [SPARK-16434][SQL] Avoid per-record type dispatch in JSO...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14102 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63575/ Test PASSed. ---

[GitHub] spark issue #14102: [SPARK-16434][SQL] Avoid per-record type dispatch in JSO...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14102 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14102: [SPARK-16434][SQL] Avoid per-record type dispatch in JSO...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14102 **[Test build #63575 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63575/consoleFull)** for PR 14102 at commit

[GitHub] spark pull request #14589: [SPARK-17007][SQL] Move test data files into a te...

2016-08-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14589 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14592: [SPARK-17011][SQL] Support testing exceptions in SQLQuer...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14592 **[Test build #63580 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63580/consoleFull)** for PR 14592 at commit

[GitHub] spark issue #14583: [SPARK-16994][SQL] PushDownPredicate should not ignore l...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14583 **[Test build #63581 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63581/consoleFull)** for PR 14583 at commit

[GitHub] spark issue #14589: [SPARK-17007][SQL] Move test data files into a test-data...

2016-08-10 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14589 Thanks - merging in master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #14592: [SPARK-17011][SQL] Support testing exceptions in ...

2016-08-10 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/14592#discussion_r74367468 --- Diff: sql/core/src/test/resources/sql-tests/results/number-format.sql.out --- @@ -19,16 +19,24 @@ struct<2147483648:bigint,(-2147483649):bigint>

[GitHub] spark issue #14589: [SPARK-17007][SQL] Move test data files into a test-data...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14589 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14589: [SPARK-17007][SQL] Move test data files into a test-data...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14589 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63573/ Test PASSed. ---

[GitHub] spark pull request #14592: [SPARK-17011][SQL] Support testing exceptions in ...

2016-08-10 Thread petermaxlee
Github user petermaxlee commented on a diff in the pull request: https://github.com/apache/spark/pull/14592#discussion_r74367410 --- Diff: sql/core/src/test/resources/sql-tests/results/number-format.sql.out --- @@ -19,16 +19,24 @@ struct<2147483648:bigint,(-2147483649):bigint>

[GitHub] spark issue #14589: [SPARK-17007][SQL] Move test data files into a test-data...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14589 **[Test build #63573 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63573/consoleFull)** for PR 14589 at commit

[GitHub] spark issue #14546: [SPARK-16955][SQL] Using ordinals in ORDER BY and GROUP ...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14546 **[Test build #63579 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63579/consoleFull)** for PR 14546 at commit

[GitHub] spark pull request #14592: [SPARK-17011][SQL] Support testing exceptions in ...

2016-08-10 Thread petermaxlee
GitHub user petermaxlee opened a pull request: https://github.com/apache/spark/pull/14592 [SPARK-17011][SQL] Support testing exceptions in SQLQueryTestSuite ## What changes were proposed in this pull request? This patch adds exception testing to SQLQueryTestSuite. When there is

[GitHub] spark issue #14397: [SPARK-16771][SQL] WITH clause should not fall into infi...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14397 **[Test build #63578 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63578/consoleFull)** for PR 14397 at commit

[GitHub] spark issue #14546: [SPARK-16955][SQL] Using ordinals in ORDER BY and GROUP ...

2016-08-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14546 Thank you for review, @gatorsmile . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14397: [SPARK-16771][SQL] WITH clause should not fall into infi...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14397 **[Test build #63577 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63577/consoleFull)** for PR 14397 at commit

[GitHub] spark issue #14397: [SPARK-16771][SQL] WITH clause should not fall into infi...

2016-08-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14397 Rebased just to resolve conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14452: [SPARK-16849][SQL] Improve subquery execution in CTE by ...

2016-08-10 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/14452 @gatorsmile Do you have concrete example for that? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and...

2016-08-10 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/14590 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is

[GitHub] spark issue #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and isolat...

2016-08-10 Thread rxin
Github user rxin commented on the issue: https://github.com/apache/spark/pull/14590 The failed Python test is unrelated. I'm going to merge this in master. Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and...

2016-08-10 Thread rxin
Github user rxin commented on a diff in the pull request: https://github.com/apache/spark/pull/14590#discussion_r74366616 --- Diff: sql/core/src/test/resources/sql-tests/results/datetime.sql.out --- @@ -0,0 +1,10 @@ +-- Automatically generated by

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14580 Already pinged the previously involved Committers. Let us see what are their feedbacks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14580 Yep. `EliminateOuterJoin` should be updated properly. Any idea? If you have more general idea, you can make a PR to override this. You made this optimizer. :) --- If your project is set up

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14580 That is a public API. We are unable to remove it. https://github.com/apache/spark/pull/8600 has a serious bug. It has been fixed in another PR: https://github.com/apache/spark/pull/10353.

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14580 Yep. Exactly. That is what I mean. That is not a regular outer join you considered in this optimizer and now both features are Spark. --- If your project is set up for it, you can reply to

[GitHub] spark issue #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and isolat...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14590 **[Test build #3216 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/3216/consoleFull)** for PR 14590 at commit

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/14580 For the regular outer join, the rule works fine. The issue you hit is caused by "using outer join" + "outer join elimination". Thus, your fix does not resolve the root issue. --- If

[GitHub] spark issue #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and isolat...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14590 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and isolat...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14590 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63572/ Test FAILed. ---

[GitHub] spark pull request #14576: [SPARK-16391][SQL] ReduceAggregator and partial a...

2016-08-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/14576#discussion_r74365669 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/expressions/ReduceAggregator.scala --- @@ -0,0 +1,79 @@ +/* + * Licensed to the Apache

[GitHub] spark issue #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and isolat...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14590 **[Test build #63572 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63572/consoleFull)** for PR 14590 at commit

[GitHub] spark issue #14591: [SPARK-17010][MINOR][DOC]Wrong description in memory man...

2016-08-10 Thread WangTaoTheTonic
Github user WangTaoTheTonic commented on the issue: https://github.com/apache/spark/pull/14591 @andrewor14 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #14588: [SPARK-17005][SQL] fix method tpe in trait AnnotationApi...

2016-08-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14588 Please let me cc @srowen to make sure because I believe it is about building. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #14568: [SPARK-10868] monotonicallyIncreasingId() supports offse...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14568 **[Test build #63576 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63576/consoleFull)** for PR 14568 at commit

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14580 In addition, I'm wondering if you really want to remove that feature which was merged into 1.6 branch on Sep. 21 2015 and already released? --- If your project is set up for it, you can

[GitHub] spark issue #14591: [SPARK-17010][MINOR][DOC]Wrong description in memory man...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14591 **[Test build #63574 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63574/consoleFull)** for PR 14591 at commit

[GitHub] spark issue #14591: [SPARK-17010][MINOR][DOC]Wrong description in memory man...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14591 Merged build finished. Test PASSed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14591: [SPARK-17010][MINOR][DOC]Wrong description in memory man...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14591 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63574/ Test PASSed. ---

[GitHub] spark issue #14580: [SPARK-16991][SQL] Fix `EliminateOuterJoin` optimizer to...

2016-08-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14580 Hi, @gatorsmile . Thank you for review. BTW, could you give me a reason why you think like the following? > The fix does not look right to me. What is the root cause

[GitHub] spark pull request #14583: [SPARK-16994][SQL] PushDownPredicate should not i...

2016-08-10 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14583#discussion_r74363622 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala --- @@ -1988,6 +1988,11 @@ class SQLQuerySuite extends QueryTest with

[GitHub] spark issue #14102: [SPARK-16434][SQL] Avoid per-record type dispatch in JSO...

2016-08-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/14102 @cloud-fan Thanks! I think it is ready to be reviewed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #14102: [SPARK-16434][SQL] Avoid per-record type dispatch in JSO...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14102 **[Test build #63575 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63575/consoleFull)** for PR 14102 at commit

[GitHub] spark issue #14591: [SPARK-17010][MINOR][DOC]Wrong description in memory man...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14591 **[Test build #63574 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63574/consoleFull)** for PR 14591 at commit

[GitHub] spark pull request #14591: [SPARK-17010][MINOR][DOC]Wrong description in mem...

2016-08-10 Thread WangTaoTheTonic
GitHub user WangTaoTheTonic opened a pull request: https://github.com/apache/spark/pull/14591 [SPARK-17010][MINOR][DOC]Wrong description in memory management document ## What changes were proposed in this pull request? change the remain percent to right one.

[GitHub] spark issue #14589: [SPARK-17007][SQL] Move test data files into a test-data...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14589 **[Test build #63573 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63573/consoleFull)** for PR 14589 at commit

[GitHub] spark issue #14589: [SPARK-17007][SQL] Move test data files into a test-data...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14589 Merged build finished. Test FAILed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #14589: [SPARK-17007][SQL] Move test data files into a test-data...

2016-08-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/14589 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/63571/ Test FAILed. ---

[GitHub] spark issue #14589: [SPARK-17007][SQL] Move test data files into a test-data...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14589 **[Test build #63571 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63571/consoleFull)** for PR 14589 at commit

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-10 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r74362668 --- Diff: python/pyspark/cloudpickle.py --- @@ -194,7 +194,7 @@ def save_function(self, obj, name=None): # we'll pickle the actual function

[GitHub] spark pull request #14567: [SPARK-16992][PYSPARK] Python Pep8 formatting and...

2016-08-10 Thread holdenk
Github user holdenk commented on a diff in the pull request: https://github.com/apache/spark/pull/14567#discussion_r74362636 --- Diff: python/pep8rc --- @@ -0,0 +1,21 @@ +# Licensed to the Apache Software Foundation (ASF) under one or more +# contributor license agreements.

[GitHub] spark issue #14590: [SPARK-17008][SPARK-17009][SQL] Normalization and isolat...

2016-08-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/14590 **[Test build #63572 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/63572/consoleFull)** for PR 14590 at commit

  1   2   3   4   5   6   7   8   >