[Impala-ASF-CR] IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15564 ) Change subject: IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Gerrit-Change-Number: 15564 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Fri, 27 Mar 2020 17:15:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15564 ) Change subject: IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 .. IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 With HIVE-22589 Hive3 switched back to using Julian Calendar for historical dates by default which caused an Impala test failure around Avro DATE values. Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Reviewed-on: http://gerrit.cloudera.org:8080/15564 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M testdata/workloads/functional-query/queries/QueryTest/avro_date.test M tests/query_test/test_date_queries.py 2 files changed, 33 insertions(+), 21 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Gerrit-Change-Number: 15564 Gerrit-PatchSet: 3 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar
[Impala-ASF-CR] IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15564 ) Change subject: IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5554/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Gerrit-Change-Number: 15564 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Fri, 27 Mar 2020 12:41:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15564 ) Change subject: IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Gerrit-Change-Number: 15564 Gerrit-PatchSet: 2 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Fri, 27 Mar 2020 12:41:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/15564 ) Change subject: IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 .. Patch Set 1: > >. the test is skipped for ORC (not sure if this is on purpose or > by accident). > My guess is that updating this test was forgotten in the quite > recent https://gerrit.cloudera.org/#/c/14982/ > > I think that in the ideal case we should test both: Julian to test > that invalid dates are handled properly (this probably has to be > file format specific, as error messages are different) and > Gregorian to have a more extended suite of tests that can run on > more file formats. > > The change itself looks good to me, but I am worried about the back > and forth changes in Hive. Thanks for the review. Let's merge this in now to unblock the core test suite. I agree that DATE testing across different fileformats and Hive versions is pretty messy now and we don't cover all the different scenarios. There's a lot of room for improvement, but we should address that in a separate patch-set. -- To view, visit http://gerrit.cloudera.org:8080/15564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Gerrit-Change-Number: 15564 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Fri, 27 Mar 2020 12:39:37 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15564 ) Change subject: IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 .. Patch Set 1: Code-Review+2 >. the test is skipped for ORC (not sure if this is on purpose or by accident). My guess is that updating this test was forgotten in the quite recent https://gerrit.cloudera.org/#/c/14982/ I think that in the ideal case we should test both: Julian to test that invalid dates are handled properly (this probably has to be file format specific, as error messages are different) and Gregorian to have a more extended suite of tests that can run on more file formats. The change itself looks good to me, but I am worried about the back and forth changes in Hive. -- To view, visit http://gerrit.cloudera.org:8080/15564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Gerrit-Change-Number: 15564 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 26 Mar 2020 22:28:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589
Attila Jeges has posted comments on this change. ( http://gerrit.cloudera.org:8080/15564 ) Change subject: IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 .. Patch Set 1: > I have a basic design question: couldn't we set hive.avro.proleptic.gregorian > to true during dataload instead of changing the tests? As other > formats use gregorian as far as I know, this seems a better to me, > at least to test interop with Impala. Parquet and Orc fileformats have the same issues with the DATE type as Avro. They may also use Gregorian or Julian Calendar depending on which version of Hive they were written by. The failing test is failing only for Avro because: 1. the test is skipped for ORC (not sure if this is on purpose or by accident). 2. the Parquet test table has been written by Impala (instead of Hive) during the data load. We also have tests for ORC and Parquet to demonstrate the issues related to the Julian vs Gregorian Calendars, but they use pre-created ORC/Parquet files (written by Hive2) and are not affected by HIVE-22589. I don't see much value in forcing Gregorian Calendar for writing Avro tables. The rewritten tests show the default behavior users can expect: pre -1582-10-12 DATEs are incorrect, but everything after that is working fine. -- To view, visit http://gerrit.cloudera.org:8080/15564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Gerrit-Change-Number: 15564 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 26 Mar 2020 20:38:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15564 ) Change subject: IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/5611/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Gerrit-Change-Number: 15564 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 26 Mar 2020 18:26:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589
Csaba Ringhofer has posted comments on this change. ( http://gerrit.cloudera.org:8080/15564 ) Change subject: IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 .. Patch Set 1: I have a basic design question: couldn't we set hive.avro.proleptic.gregorian to true during dataload instead of changing the tests? As other formats use gregorian as far as I know, this seems a better to me, at least to test interop with Impala. -- To view, visit http://gerrit.cloudera.org:8080/15564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Gerrit-Change-Number: 15564 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 26 Mar 2020 18:01:38 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589
Attila Jeges has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15564 Change subject: IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 .. IMPALA-9555: [Hive3] Fix test failure introduced by HIVE-22589 With HIVE-22589 Hive3 switched back to using Julian Calendar for historical dates by default which caused an Impala test failure around Avro DATE values. Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e --- M testdata/workloads/functional-query/queries/QueryTest/avro_date.test M tests/query_test/test_date_queries.py 2 files changed, 33 insertions(+), 21 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/64/15564/1 -- To view, visit http://gerrit.cloudera.org:8080/15564 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I51dd933867ea7877235e7f6e1f2b56711dca107e Gerrit-Change-Number: 15564 Gerrit-PatchSet: 1 Gerrit-Owner: Attila Jeges
