[GitHub] spark issue #16751: [SPARK-19409][BUILD] Bump parquet version to 1.8.2

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16751 BTW, is there any condition to reproduce that? At least, `sbt` seems not to complain like that. ``` $ build/sbt "project sql" "test-only *.ParquetAvroCom

[GitHub] spark issue #16751: [SPARK-19409][BUILD] Bump parquet version to 1.8.2

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16751 Ur, I reproduce that with maven test. It's about test dependency on `parquet-avro` which now depends on `org.apache.avro` 1.8.0. --- If your project is set up for it, you can rep

[GitHub] spark issue #16791: [SPARK-19409][SPARK-17213] Cleanup Parquet workarounds/h...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16791 Great, @liancheng . BTW, could you specify the test dependency like the following in this PR? ``` diff --git a/sql/core/pom.xml b/sql/core/pom.xml index b8aa698090..f9acf815e3

[GitHub] spark issue #16791: [SPARK-19409][SPARK-17213] Cleanup Parquet workarounds/h...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16791 Here, https://amplab.cs.berkeley.edu/jenkins/view/Spark%20QA%20Test%20(Dashboard)/job/spark-master-test-maven-hadoop-2.7/2530/consoleFull

[GitHub] spark issue #16791: [SPARK-19409][SPARK-17213] Cleanup Parquet workarounds/h...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16791 Originally, it's reported at #16751 by @robbinspg . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project doe

[GitHub] spark issue #16791: [SPARK-19409][SPARK-17213] Cleanup Parquet workarounds/h...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16791 Yep. The failure is on #16751 . It's my bad. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16791: [SPARK-19409][SPARK-17213] Cleanup Parquet workarounds/h...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16791 I'll create another PR and ping you. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this fe

[GitHub] spark issue #16791: [SPARK-19409][SPARK-17213] Cleanup Parquet workarounds/h...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16791 I see. Sure! Right, it happens only maven build so far. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #16795: [WIP][SPARK-19409][BUILD] Fix ParquetAvroCompatib...

2017-02-03 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/16795 [WIP][SPARK-19409][BUILD] Fix ParquetAvroCompatibilitySuite failure due to test dependency on avro ## What changes were proposed in this pull request? After using Apache Parquet

[GitHub] spark issue #16795: [WIP][SPARK-19409][BUILD] Fix ParquetAvroCompatibilitySu...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Thank you for review, @jaceklaskowski . Yep. I made this first because that means more. Actually, I'm running another maven test for that, `bumping avro to 1.8.1`, too. If that

[GitHub] spark issue #16795: [WIP][SPARK-19409][BUILD] Fix ParquetAvroCompatibilitySu...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16795: [WIP][SPARK-19409][BUILD] Fix ParquetAvroCompatibilitySu...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 It seems to be `[SPARK-19386][SPARKR][FOLLOWUP] fix error in vignettes`. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark issue #16795: [WIP][SPARK-19409][BUILD] Fix ParquetAvroCompatibilitySu...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16795: [SPARK-19409][BUILD] Fix ParquetAvroCompatibilitySuite f...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Hi, @liancheng . Could you review this PR? This is the one we talked at #16791 . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as

[GitHub] spark pull request #15987: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION sho...

2017-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/15987 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16804: [SPARK-19459][SQL] Add Hive datatype (char/varcha...

2017-02-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16804#discussion_r99479616 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/OrcSourceSuite.scala --- @@ -162,6 +162,27 @@ abstract class OrcSuite extends

[GitHub] spark pull request #16802: [SPARK-18872][SQL][TESTS] New test cases for EXIS...

2017-02-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16802#discussion_r99479814 --- Diff: sql/core/src/test/resources/sql-tests/inputs/subquery/exists-subquery/exists-joins-and-set-ops.sql --- @@ -0,0 +1,206 @@ +-- Tests

[GitHub] spark pull request #16798: [SPARK-18873][SQL][TEST] New test cases for scala...

2017-02-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16798#discussion_r99480006 --- Diff: sql/core/src/test/resources/sql-tests/inputs/subquery/scalar-subquery/scalar-subquery-predicate.sql --- @@ -0,0 +1,255 @@ +-- A test

[GitHub] spark pull request #16798: [SPARK-18873][SQL][TEST] New test cases for scala...

2017-02-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16798#discussion_r99480025 --- Diff: sql/core/src/test/resources/sql-tests/inputs/subquery/scalar-subquery/scalar-subquery-predicate.sql --- @@ -0,0 +1,255 @@ +-- A test

[GitHub] spark issue #16795: [SPARK-19409][BUILD] Fix ParquetAvroCompatibilitySuite f...

2017-02-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Hi, @gatorsmile . Could you review this PR for fixing maven build? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16783: [SPARK-19441] [SQL] Remove IN type coercion from Promote...

2017-02-04 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16783 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16810: [SPARK-19464][CORE][YARN][test-hadoop2.6] Remove ...

2017-02-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16810#discussion_r99519864 --- Diff: docs/building-spark.md --- @@ -63,57 +63,30 @@ with Maven profile settings and so on like the direct Maven build. Example: This

[GitHub] spark issue #16810: [SPARK-19464][CORE][YARN][test-hadoop2.6] Remove support...

2017-02-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16810 It's great, +1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and w

[GitHub] spark issue #16783: [SPARK-19441] [SQL] Remove IN type coercion from Promote...

2017-02-05 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16783 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16751: [SPARK-19409][BUILD] Bump parquet version to 1.8.2

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16751 Hi, @robbinspg and @srowen . Could you review the PR #16795 for that? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If

[GitHub] spark pull request #16795: [SPARK-19409][BUILD] Fix ParquetAvroCompatibility...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16795#discussion_r99624416 --- Diff: sql/core/pom.xml --- @@ -130,6 +130,12 @@ test + org.apache.avro --- End diff -- Thank

[GitHub] spark issue #16821: [SPARK-19472][SQL] Parser should not mistake CASE WHEN(....

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16821 +1 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark pull request #16795: [SPARK-19409][BUILD] Fix ParquetAvroCompatibility...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16795#discussion_r99643730 --- Diff: sql/core/pom.xml --- @@ -130,6 +130,12 @@ test + org.apache.avro --- End diff -- I moved

[GitHub] spark issue #16795: [SPARK-19409][BUILD] Fix ParquetAvroCompatibilitySuite f...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Oh, thank you, @liancheng ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroC...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16795#discussion_r99678662 --- Diff: sql/core/pom.xml --- @@ -130,6 +130,12 @@ test + org.apache.avro --- End diff -- @srowen

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 The current failure about `ExtendedYarnTest` came from `mesos` module . It seems to be irrelevant to this PR. Let me check that. ``` [info] Running Spark tests using Maven with these

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark pull request #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroC...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16795#discussion_r99740767 --- Diff: resource-managers/mesos/pom.xml --- @@ -49,6 +49,13 @@ + org.apache.spark + spark-tags_

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-06 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 @srowen and @liancheng Could you review this PR again? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark pull request #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroC...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16795#discussion_r99882782 --- Diff: sql/core/pom.xml --- @@ -130,6 +130,12 @@ test + org.apache.avro + avro + 1.8.0

[GitHub] spark pull request #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroC...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16795#discussion_r99883767 --- Diff: sql/core/pom.xml --- @@ -130,6 +130,12 @@ test + org.apache.avro + avro + 1.8.0

[GitHub] spark pull request #16839: [SPARK-19409][BUILD][test-maven] Bump avro versio...

2017-02-07 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/16839 [SPARK-19409][BUILD][test-maven] Bump avro version to fix ParquetAvroCompatibilitySuite failure ## What changes were proposed in this pull request? After using Apache Parquet 1.8.2

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Hi, @srowen . I made another PR #16839 for the option 1. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project

[GitHub] spark issue #16839: [WIP][SPARK-19409][BUILD][test-maven] Bump avro version ...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16839 I also hesitated to bump Avro here. Basically, I think we don't need to do this. So, I tried #16795 . Could we talk there? --- If your project is set up for it, you can reply to this

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 The situation here is `apche.parquet.avro.AvroParquetWriter` of `parquet-avro` uses 1.8.0 specific API. Since we use `AvroParquetWriter` in our test suite, we need to use different `avro

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Thank you, @srowen and @vanzin . Then, the only to-do is using 1.8.1 instead of 1.8.0 here? --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark pull request #16839: [WIP][SPARK-19409][BUILD][test-maven] Bump avro v...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/16839 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #16839: [WIP][SPARK-19409][BUILD][test-maven] Bump avro version ...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16839 According to the current status of Jenkins test result, it hangs due to `ReplSuite` failure. It seems not a good time to do this. I'll close this PR for now. Also, I found that there

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Sure! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if

[GitHub] spark pull request #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroC...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16795#discussion_r99944889 --- Diff: sql/core/pom.xml --- @@ -129,6 +129,19 @@ parquet-avro test + --- End diff -- @srowen

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 The failure seems to be irrelevant to this PR. We can see the result of the next running Jenkins. ``` [INFO] Spark Project Catalyst . FAILURE [11:29 min

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-07 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Retest this please --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Thank you for merging, @srowen . Thank you for review, @vanzin , @liancheng , @jaceklaskowski , too! I'm watching the build system on mockito hotfix. I believe it will become st

[GitHub] spark pull request #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use...

2017-02-08 Thread dongjoon-hyun
GitHub user dongjoon-hyun opened a pull request: https://github.com/apache/spark/pull/16858 [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop2.6 ## What changes were proposed in this pull request? After SPARK-19464, SparkPullRequestBuilder fails because it still

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16858 @srowen Could you review this? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark pull request #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16858#discussion_r100131333 --- Diff: dev/run-tests-jenkins.py --- @@ -165,12 +165,6 @@ def main(): if "test-maven" in ghprb_pull_title:

[GitHub] spark pull request #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16858#discussion_r100133896 --- Diff: dev/run-tests-jenkins.py --- @@ -165,12 +165,6 @@ def main(): if "test-maven" in ghprb_pull_title:

[GitHub] spark pull request #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16858#discussion_r100134388 --- Diff: dev/run-tests.py --- @@ -505,14 +505,14 @@ def main(): # if we're on the Amplab Jenkins build servers setup vari

[GitHub] spark pull request #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16858#discussion_r100134410 --- Diff: dev/run-tests.py --- @@ -505,14 +505,14 @@ def main(): # if we're on the Amplab Jenkins build servers setup vari

[GitHub] spark pull request #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16858#discussion_r100134776 --- Diff: dev/run-tests-jenkins.py --- @@ -165,12 +165,6 @@ def main(): if "test-maven" in ghprb_pull_title:

[GitHub] spark pull request #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16858#discussion_r100135299 --- Diff: dev/run-tests-jenkins.py --- @@ -165,12 +165,6 @@ def main(): if "test-maven" in ghprb_pull_title:

[GitHub] spark pull request #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/16858#discussion_r100137039 --- Diff: R/pkg/inst/tests/testthat/test_utils.R --- @@ -232,9 +232,9 @@ test_that("basenameSansExtFromUrl", { x <-

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16858 I removed those lines, too. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16858 Thank you for review, @srowen . Also, since it touches R now, cc @felixcheung . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 So far, **ReplSuite** failures are observed. One is OOM and the other is the other is ClosureCleaner. Both happens after the testcase, `define case class and create Dataset together with

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 You mean `avro` or prior `parquet`? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16859: [SPARK-17714][Core][maven]Avoid using ExecutorClassLoade...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16859 Hi, @zsxwing . Currently, SparkPullRequestBuilder is broken and HOTFIX is #16858 . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Oh, sorry! I misread your comment, `doubt`. For the ReplSuite, #16859 looks related. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16858 The only failure is irrelevant to this PR, ad also the very next Jenkins pass the test. ``` [info] KafkaSourceSuite: ... [info] - subscribing topic by pattern with topic

[GitHub] spark issue #16795: [SPARK-19409][BUILD][test-maven] Fix ParquetAvroCompatib...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16795 Oh, indeed. Thank you for informing that, too! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16858 Oh, wait a moment. The Jenkins on last commit will succeed soon. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #16858: [SPARK-19464][BUILD][HOTFIX] run-tests should use hadoop...

2017-02-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16858 Finally, it passed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15987: [SPARK-17732][SPARK-18515][SQL] ALTER TABLE DROP PARTITI...

2016-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15987 Hi, @hvanhovell . Could you review this PR? This PR now is the new fix + the reverted commit. --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #15987: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15987 Hi, @hvanhovell , @gatorsmile , @rxin . Should we remove *Expression* related stuff? Originally, #15302 does not have this kind of expression issue. It's just an extension o

[GitHub] spark issue #16021: [SPARK-18593][SQL] JDBCRDD returns incorrect results for...

2016-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16021 Hi, @rxin , @gatorsmile , @hvanhovell , @srowen . Could you give some opinion? What I hoped here is having these 6 commits in `branch-1.6`. But, if backports are not proper

[GitHub] spark issue #16021: [SPARK-18593][SQL] JDBCRDD returns incorrect results for...

2016-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16021 Oh, thank you for considering. There are a few trivial conflicts. One is the following about `WholeStageCodegenExec` testcase which I omitted here. https://github.com/apache

[GitHub] spark issue #16021: [SPARK-18593][SQL] JDBCRDD returns incorrect results for...

2016-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16021 Sure! I think the authors also agree with that if it's allowed. Hi, @maropu , @HyukjinKwon , @viirya . Could you proceed to backport yours to `branch-1.6`? --- If your proje

[GitHub] spark issue #16021: [SPARK-18593][SQL] JDBCRDD returns incorrect results for...

2016-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16021 That seems to be for master (2.2)? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16021: [SPARK-18593][SQL] JDBCRDD returns incorrect results for...

2016-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16021 I see, @rxin. I will split them and tell you. Also I will make separate PRs keeping the original author. I'll ping you a few hour later! --- If your project is set up for it

[GitHub] spark issue #16021: [SPARK-18593][SQL] JDBCRDD returns incorrect results for...

2016-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16021 Hi, @rxin . All 6 commits are not cherry-pickable. Especially, the last 2 commits has inevitable conflicts due to some wide commits (about `import order` and `syntax` cleanups

[GitHub] spark issue #16069: [WIP][SPARK-18638][BUILD] Upgrade sbt, Zinc, and Maven p...

2016-11-30 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16069 nit. In PR description, 0.3.9 instead of 0.13.9 ? > zinc: 0.13.9 -> 0.3.11, --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub a

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 Thank you for approving, @srowen ! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16069: [WIP][SPARK-18638][BUILD] Upgrade sbt, Zinc, and Maven p...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16069 +1 for 2.1! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so

[GitHub] spark issue #16021: [SPARK-18593][SQL] JDBCRDD returns incorrect results for...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16021 Or, may I create just 6 PRs sequentially? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15863: [SPARK-18419][SQL] Fix `JDBCOptions.asConnectionP...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun closed the pull request at: https://github.com/apache/spark/pull/15863 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the

[GitHub] spark issue #15863: [SPARK-18419][SQL] Fix `JDBCOptions.asConnectionProperti...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15863 I close this PR since this is already fixed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this

[GitHub] spark pull request #15863: [SPARK-18419][SQL] Fix `JDBCOptions.asConnectionP...

2016-12-01 Thread dongjoon-hyun
GitHub user dongjoon-hyun reopened a pull request: https://github.com/apache/spark/pull/15863 [SPARK-18419][SQL] Fix `JDBCOptions.asConnectionProperties` to be case-insensitive ## What changes were proposed in this pull request? `JDBCOptions.asConnectionProperties` is

[GitHub] spark issue #15863: [SPARK-18419][SQL] Fix `JDBCOptions.asConnectionProperti...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15863 Oh, it's still there. ```scala scala> new org.apache.spark.sql.execution.datasources.jdbc.JDBCOptions(Map("url" -> "jdbc:mysql://localhost:33

[GitHub] spark pull request #15863: [SPARK-18419][SQL] Fix `JDBCOptions.asConnectionP...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15863#discussion_r90516738 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -129,7 +129,7 @@ object JDBCOptions

[GitHub] spark issue #15863: [SPARK-18419][SQL] Fix `JDBCOptions.asConnectionProperti...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15863 Also, cc @srowen . Sorry for the confusion. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have

[GitHub] spark pull request #15863: [SPARK-18419][SQL] Fix `JDBCOptions.asConnectionP...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15863#discussion_r90527179 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCRelation.scala --- @@ -130,7 +130,7 @@ private[sql] case class

[GitHub] spark issue #15863: [SPARK-18419][SQL] `JDBCRelation.insert` should not remo...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15863 I updated PR description and focus. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16104 Retest this please. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and

[GitHub] spark issue #15863: [SPARK-18419][SQL] `JDBCRelation.insert` should not remo...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15863 The only failure seems to be irrelevant. ``` Had test failures in pyspark.streaming.tests with pypy; see logs. ``` --- If your project is set up for it, you can reply to this

[GitHub] spark issue #16104: [SPARK-18675][SQL] CTAS for hive serde table should work...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/16104 Last time, ``` [info] - set hive.metastore.warehouse.dir *** FAILED *** (5 minutes, 0 seconds) ``` At this time, ``` [info] - SPARK-18360: default table path of tables

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 @rxin . I didn't say this is only a problem for external table. I simply focused on SPARK-11374 use case here. What use case are you considering when you mentioned **ma

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 Ah, I see. Then, I'll try to update PR descriptions or testcases more. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as wel

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 @rxin . I updated the title and description of this PR. For the test case, I think the current test case covers enough the functionality. --- If your project is set up for it, you

[GitHub] spark pull request #14638: [SPARK-11374][SQL] Support `skip.header.line.coun...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/14638#discussion_r90584660 --- Diff: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala --- @@ -2011,6 +2011,54 @@ class SQLQuerySuite extends

[GitHub] spark pull request #15863: [SPARK-18419][SQL] `JDBCRelation.insert` should n...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15863#discussion_r90585299 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -41,10 +41,16 @@ class JDBCOptions

[GitHub] spark pull request #15863: [SPARK-18419][SQL] `JDBCRelation.insert` should n...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/15863#discussion_r90585324 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/JDBCOptions.scala --- @@ -129,7 +129,7 @@ object JDBCOptions

[GitHub] spark issue #14638: [SPARK-11374][SQL] Support `skip.header.line.count` opti...

2016-12-01 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/14638 I didn't read Hive code part for this. I referenced the Spark csv part code. But, I'll search the corresponding one in Hive master branch tomorrow since you requested. -

[GitHub] spark issue #15863: [SPARK-18419][SQL] `JDBCRelation.insert` should not remo...

2016-12-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15863 Thank you for merging, @cloud-fan . Thank you for review, @gatorsmile , @viirya , @HyukjinKwon ! --- If your project is set up for it, you can reply to this email and have your reply appear

[GitHub] spark issue #15987: [SPARK-17732][SQL] ALTER TABLE DROP PARTITION should sup...

2016-12-02 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/15987 Rebased to resolve conflicts. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature

<    4   5   6   7   8   9   10   11   12   13   >