[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. IMPALA-9673: Add external warehouse dir variable in E2E test Updated CDP build to 7.2.1.0-57 to include new Hive features such as HIVE-22995. In minicluster, we have default values of hive.create.as.acid and hive.create.as.insert.only which are false. So by default hive creates external type table located in external warehouse directory. Due to HIVE-22995, desc db returns external warehouse directory. With above reasons, we need use external warehouse dir in some tests. Also add a new test for "CREATE DATABASE ... LOCATION". Tested: Re-run failed test in minicluster. Run exhaustive tests. Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Reviewed-on: http://gerrit.cloudera.org:8080/15990 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M bin/impala-config.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/create-database.test M testdata/workloads/functional-query/queries/QueryTest/describe-db.test M testdata/workloads/functional-query/queries/QueryTest/describe-hive-db.test M tests/common/environ.py M tests/common/impala_test_suite.py M tests/query_test/test_compressed_formats.py 8 files changed, 78 insertions(+), 34 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 13 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 12: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 12 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Fri, 05 Jun 2020 23:48:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 12: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 12 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Fri, 05 Jun 2020 18:34:32 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 12: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5950/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 12 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Fri, 05 Jun 2020 18:34:33 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 11: The test failure does not look related to this change, restarting -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 11 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Fri, 05 Jun 2020 18:34:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 11: Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5943/ -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 11 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Fri, 05 Jun 2020 00:28:21 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 11: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5943/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 11 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Thu, 04 Jun 2020 19:25:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 11: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5940/ -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 11 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Thu, 04 Jun 2020 03:30:52 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 11: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6208/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 11 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Wed, 03 Jun 2020 23:21:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 10: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6207/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 10 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Wed, 03 Jun 2020 23:20:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Xiaomeng Zhang has uploaded a new patch set (#10). ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. IMPALA-9673: Add external warehouse dir variable in E2E test Updated CDP build to 7.2.1.0-57 to include new Hive features such as HIVE-22995. In minicluster, we have default values of hive.create.as.acid and hive.create.as.insert.only which are false. So by default hive creates external type table located in external warehouse directory. Due to HIVE-22995, desc db returns external warehouse directory. With above reasons, we need use external warehouse dir in some tests. Also add a new test for "CREATE DATABASE ... LOCATION". Tested: Re-run failed test in minicluster. Run exhaustive tests. Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f --- M bin/impala-config.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/create-database.test M testdata/workloads/functional-query/queries/QueryTest/describe-db.test M testdata/workloads/functional-query/queries/QueryTest/describe-hive-db.test M tests/common/environ.py M tests/common/impala_test_suite.py M tests/query_test/test_compressed_formats.py 8 files changed, 78 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/15990/10 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 10 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Xiaomeng Zhang has uploaded a new patch set (#11). ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. IMPALA-9673: Add external warehouse dir variable in E2E test Updated CDP build to 7.2.1.0-57 to include new Hive features such as HIVE-22995. In minicluster, we have default values of hive.create.as.acid and hive.create.as.insert.only which are false. So by default hive creates external type table located in external warehouse directory. Due to HIVE-22995, desc db returns external warehouse directory. With above reasons, we need use external warehouse dir in some tests. Also add a new test for "CREATE DATABASE ... LOCATION". Tested: Re-run failed test in minicluster. Run exhaustive tests. Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f --- M bin/impala-config.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/create-database.test M testdata/workloads/functional-query/queries/QueryTest/describe-db.test M testdata/workloads/functional-query/queries/QueryTest/describe-hive-db.test M tests/common/environ.py M tests/common/impala_test_suite.py M tests/query_test/test_compressed_formats.py 8 files changed, 78 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/15990/11 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 11 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 9: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/5939/ -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 9 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Wed, 03 Jun 2020 19:58:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 9: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/5939/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 9 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Wed, 03 Jun 2020 18:30:06 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 9: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 9 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Wed, 03 Jun 2020 18:30:05 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 8: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6205/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 8 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Wed, 03 Jun 2020 18:02:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 8: Code-Review+2 Looks good! -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 8 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Wed, 03 Jun 2020 17:44:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Xiaomeng Zhang has uploaded a new patch set (#8). ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. IMPALA-9673: Add external warehouse dir variable in E2E test Updated CDP build to 7.2.1.0-57 to include new Hive features such as HIVE-22995. In minicluster, we have default values of hive.create.as.acid and hive.create.as.insert.only which are false. So by default hive creates external type table located in external warehouse directory. Due to HIVE-22995, desc db returns external warehouse directory. With above reasons, we need use external warehouse dir in some tests. Also add a new test for "CREATE DATABASE ... LOCATION". Tested: Re-run failed test in minicluster. Run exhaustive tests. Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f --- M bin/impala-config.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/create-database.test M testdata/workloads/functional-query/queries/QueryTest/describe-db.test M testdata/workloads/functional-query/queries/QueryTest/describe-hive-db.test M tests/common/environ.py M tests/common/impala_test_suite.py M tests/query_test/test_compressed_formats.py 8 files changed, 78 insertions(+), 34 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/15990/8 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 8 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Joe McDonnell has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 7: Code-Review+1 (3 comments) This looks good to me. Just a couple nits. I will bump to +2 when those are fixed. http://gerrit.cloudera.org:8080/#/c/15990/7/bin/impala-config.sh File bin/impala-config.sh: http://gerrit.cloudera.org:8080/#/c/15990/7/bin/impala-config.sh@181 PS7, Line 181: export CDP_RANGER_VERSION=2.0.0.7.2.1.0-57 : export CDP_TEZ_VERSION=0.9.1.7.2.1.0-57 : export CDP_KNOX_VERSION=1.3.0.7.2.1.0-57 : export CDP_OZONE_VERSION=0.6.0.7.2.1.0-57 Nit: I missed this in previous rounds, but please leave the components in alphabetical order (i.e. same order as before). http://gerrit.cloudera.org:8080/#/c/15990/7/tests/query_test/test_compressed_formats.py File tests/query_test/test_compressed_formats.py: http://gerrit.cloudera.org:8080/#/c/15990/7/tests/query_test/test_compressed_formats.py@74 PS7, Line 74: def test_compressed_formats(self, vector): Just in case you haven't seen this before, we have a variant on the test function signature that adds a "unique_database" parameter. i.e. def test_compressed_format(self, vector, unique_database): That fixture creates a unique database before the test runs and drops it afterward. This allows test writers to put tables inside the unique database and not worry about concurrent modifications or naming conflicts. This test is fine either way, but I wanted to make sure you knew about it. http://gerrit.cloudera.org:8080/#/c/15990/7/tests/query_test/test_compressed_formats.py@102 PS7, Line 102: # Since Hive makes tables managed by default (only matters on Hive 3+), it : # will use the managed warehouse location. Nit: Update this comment -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 7 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Wed, 03 Jun 2020 01:36:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 7: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 7 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Tue, 02 Jun 2020 23:48:01 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 7: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6199/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 7 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Tue, 02 Jun 2020 22:49:00 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Xiaomeng Zhang has uploaded a new patch set (#7). ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. IMPALA-9673: Add external warehouse dir variable in E2E test Updated CDP build to 7.2.1.0-57 to include new Hive features such as HIVE-22995. In minicluster, we have default values of hive.create.as.acid and hive.create.as.insert.only which are false. So by default hive creates external type table located in external warehouse directory. Due to HIVE-22995, desc db returns external warehouse directory. With above reasons, we need use external warehouse dir in some tests. Also add a new test for "CREATE DATABASE ... LOCATION". Tested: Re-run failed test in minicluster. Run exhaustive tests. Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f --- M bin/impala-config.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/create-database.test M testdata/workloads/functional-query/queries/QueryTest/describe-db.test M testdata/workloads/functional-query/queries/QueryTest/describe-hive-db.test M tests/common/environ.py M tests/common/impala_test_suite.py M tests/query_test/test_compressed_formats.py 8 files changed, 76 insertions(+), 32 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/15990/7 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 7 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 6: (2 comments) http://gerrit.cloudera.org:8080/#/c/15990/6/tests/query_test/test_compressed_formats.py File tests/query_test/test_compressed_formats.py: http://gerrit.cloudera.org:8080/#/c/15990/6/tests/query_test/test_compressed_formats.py@110 PS6, Line 110: hive_cmd = drop_cmd + 'CREATE TABLE %s LIKE %s LOCATION \'%s\';' % \ I think it makes more sense to make this a CREATE EXTERNAL TABLE since Hive is translating it an external table anyway, and the query is specifying a location. http://gerrit.cloudera.org:8080/#/c/15990/6/tests/query_test/test_compressed_formats.py@132 PS6, Line 132: self.run_stmt_in_hive(drop_cmd) we probably need a delete the dest_table_dir manually (using filesystem_client) after dropping the table, otherwise the data won't get cleaned up -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 6 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Tue, 02 Jun 2020 21:19:55 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 6: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6195/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 6 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Tue, 02 Jun 2020 18:00:36 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Sahil Takiar has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 5: (3 comments) mostly nits; patch almost looks good to me, pending the updates to tests/query_test/test_compressed_formats.py http://gerrit.cloudera.org:8080/#/c/15990/6/testdata/datasets/functional/functional_schema_template.sql File testdata/datasets/functional/functional_schema_template.sql: http://gerrit.cloudera.org:8080/#/c/15990/6/testdata/datasets/functional/functional_schema_template.sql@2313 PS6, Line 2313: nit whitespace http://gerrit.cloudera.org:8080/#/c/15990/6/testdata/datasets/functional/functional_schema_template.sql@2314 PS6, Line 2314: BASE_TABLE_NAME would be good to mention that this is necessary because of a bug in Hive. http://gerrit.cloudera.org:8080/#/c/15990/6/testdata/workloads/functional-query/queries/QueryTest/create-database.test File testdata/workloads/functional-query/queries/QueryTest/create-database.test: http://gerrit.cloudera.org:8080/#/c/15990/6/testdata/workloads/functional-query/queries/QueryTest/create-database.test@60 PS6, Line 60: create database if not exists $DATABASE_cascade nit: line is too long -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 5 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Tue, 02 Jun 2020 17:23:32 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Xiaomeng Zhang has uploaded a new patch set (#6). ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. IMPALA-9673: Add external warehouse dir variable in E2E test Updated CDP build to 7.2.1.0-57 to include new Hive features such as HIVE-22995. In minicluster, we have default values of hive.create.as.acid and hive.create.as.insert.only which are false. So by default hive creates external type table located in external warehouse directory. Due to HIVE-22995, desc db returns external warehouse directory. With above reasons, we need use external warehouse dir in some tests. Also add a new test for "CREATE DATABASE ... LOCATION". Tested: Re-run failed test in minicluster. Run exhaustive tests. Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f --- M bin/impala-config.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/create-database.test M testdata/workloads/functional-query/queries/QueryTest/describe-db.test M testdata/workloads/functional-query/queries/QueryTest/describe-hive-db.test M tests/common/environ.py M tests/common/impala_test_suite.py M tests/query_test/test_compressed_formats.py 8 files changed, 73 insertions(+), 32 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/15990/6 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 6 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Joe McDonnell has posted comments on this change. (
http://gerrit.cloudera.org:8080/15990 )
Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test
..
Patch Set 5:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/15990/5//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15990/5//COMMIT_MSG@7
PS5, Line 7: Add external warehouse dir variable in E2E test
:
: In minicluster, we have default values of hive.create.as.acid and
: hive.create.as.insert.only which are false. So by default hive
creates
: external type table located in external warehouse directory.
: Due to HIVE-22995, desc db returns external warehouse directory.
I think we should mention in the commit message that we are updating the CDP
build number. Then, the other parts can explain why these other changes are
needed to adapt to changes in Hive.
Larger context, Hive made changes to its "create database" statement splitting
out location and managed location. I think we will want a followup JIRA to make
Impala's functionality match (i.e. being able to take a managed location in
"create database" and also display it back when using "describe database").
Given that "location" from Hive's perspective is the external location, this
fix still makes sense on its own.
There's one semantic that we know we want to maintain. And that is that if we
do "create database ... location X" and then "describe database ...", then the
location is X. I don't see a test like that. If there isn't one, then I think
we should add one.
http://gerrit.cloudera.org:8080/#/c/15990/4/testdata/datasets/functional/functional_schema_template.sql
File testdata/datasets/functional/functional_schema_template.sql:
http://gerrit.cloudera.org:8080/#/c/15990/4/testdata/datasets/functional/functional_schema_template.sql@2805
PS4, Line 2805: CREATE MATERIALIZED VIEW IF NOT EXISTS
{db_name}{db_suffix}.{table_name}
> The command exists previously, I just moved to end of file, which has done
I agree with Sahil. It would be good to have a comment here describing why it
is later in the file than expected.
My understanding is that the issue comes from the materialized view being
created before any managed table is created in that database. My guess is that
it has something to do with creating the database's managed directory. Moving
this statement down pushes it past the create statements for some managed
tables (insert_only_transactional_bucketed_table, bucketed_table) and avoids
the issue.
In general, it would be nice for this to be as close to its original location
as possible. I believe it should work to put this after bucketed_table and
before uncomp_src_alltypes.
--
To view, visit http://gerrit.cloudera.org:8080/15990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f
Gerrit-Change-Number: 15990
Gerrit-PatchSet: 5
Gerrit-Owner: Xiaomeng Zhang
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Xiaomeng Zhang
Gerrit-Comment-Date: Mon, 01 Jun 2020 05:02:01 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6176/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 5 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Reviewer: Xiaomeng Zhang Gerrit-Comment-Date: Sun, 31 May 2020 17:45:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Xiaomeng Zhang has posted comments on this change. (
http://gerrit.cloudera.org:8080/15990 )
Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test
..
Patch Set 5:
(4 comments)
http://gerrit.cloudera.org:8080/#/c/15990/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15990/4//COMMIT_MSG@12
PS4, Line 12: HIVE-2299
> this isn't an external JIRA, so we can't reference it here. you probably wa
Done
http://gerrit.cloudera.org:8080/#/c/15990/4//COMMIT_MSG@18
PS4, Line 18: exhaustive tests.
> this isn't a public Jenkins job. You can just say "Ran exhaustive tests"
Done
http://gerrit.cloudera.org:8080/#/c/15990/4/testdata/datasets/functional/functional_schema_template.sql
File testdata/datasets/functional/functional_schema_template.sql:
http://gerrit.cloudera.org:8080/#/c/15990/4/testdata/datasets/functional/functional_schema_template.sql@2805
PS4, Line 2805: CREATE MATERIALIZED VIEW IF NOT EXISTS
{db_name}{db_suffix}.{table_name}
> we should probably mention why this is necessary - I'm not sure if there is
The command exists previously, I just moved to end of file, which has done by
Joe in downstream. It has to be run after insert_only_transactional_table
created. But it looks the sql command doesn't run in order, it didn't work if I
put this sql in middle of file. So I have to put it in the end.
http://gerrit.cloudera.org:8080/#/c/15990/4/tests/query_test/test_compressed_formats.py
File tests/query_test/test_compressed_formats.py:
http://gerrit.cloudera.org:8080/#/c/15990/4/tests/query_test/test_compressed_formats.py@104
PS4, Line 104: dest_base_dir = '/{0}'.format(EXTERNAL_WAREHOUSE_DIR)
> I think we should fix this a different way. After discussing with Naveen, i
Make sense. I'll update the patch.
--
To view, visit http://gerrit.cloudera.org:8080/15990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f
Gerrit-Change-Number: 15990
Gerrit-PatchSet: 5
Gerrit-Owner: Xiaomeng Zhang
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Sahil Takiar
Gerrit-Reviewer: Xiaomeng Zhang
Gerrit-Comment-Date: Sun, 31 May 2020 16:52:49 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Xiaomeng Zhang has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. IMPALA-9673: Add external warehouse dir variable in E2E test In minicluster, we have default values of hive.create.as.acid and hive.create.as.insert.only which are false. So by default hive creates external type table located in external warehouse directory. Due to HIVE-22995, desc db returns external warehouse directory. With above reasons, we need use external warehouse dir in some tests. Tested: Re-run failed test in minicluster. Run exhaustive tests. Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f --- M bin/impala-config.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/create-database.test M testdata/workloads/functional-query/queries/QueryTest/describe-db.test M testdata/workloads/functional-query/queries/QueryTest/describe-hive-db.test M tests/common/environ.py M tests/common/impala_test_suite.py M tests/query_test/test_compressed_formats.py 8 files changed, 40 insertions(+), 32 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/15990/5 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 5 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Sahil Takiar has posted comments on this change. (
http://gerrit.cloudera.org:8080/15990 )
Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test
..
Patch Set 4:
(4 comments)
Needs to be rebased as well.
http://gerrit.cloudera.org:8080/#/c/15990/4//COMMIT_MSG
Commit Message:
http://gerrit.cloudera.org:8080/#/c/15990/4//COMMIT_MSG@12
PS4, Line 12: CDPD-8248
this isn't an external JIRA, so we can't reference it here. you probably want
to reference the public Apache JIRA instead.
http://gerrit.cloudera.org:8080/#/c/15990/4//COMMIT_MSG@18
PS4, Line 18: impala-private-parameterized
this isn't a public Jenkins job. You can just say "Ran exhaustive tests"
http://gerrit.cloudera.org:8080/#/c/15990/4/testdata/datasets/functional/functional_schema_template.sql
File testdata/datasets/functional/functional_schema_template.sql:
http://gerrit.cloudera.org:8080/#/c/15990/4/testdata/datasets/functional/functional_schema_template.sql@2805
PS4, Line 2805: CREATE MATERIALIZED VIEW IF NOT EXISTS
{db_name}{db_suffix}.{table_name}
we should probably mention why this is necessary - I'm not sure if there is an
Apache JIRA for this yet, if not can you file one?
http://gerrit.cloudera.org:8080/#/c/15990/4/tests/query_test/test_compressed_formats.py
File tests/query_test/test_compressed_formats.py:
http://gerrit.cloudera.org:8080/#/c/15990/4/tests/query_test/test_compressed_formats.py@104
PS4, Line 104: dest_base_dir = '/{0}'.format(EXTERNAL_WAREHOUSE_DIR)
I think we should fix this a different way. After discussing with Naveen, it
seems like the expectation is that should actually be the
MANAGED_WAREHOUSE_DIR. The fact that it is the EXTERNAL_WAREHOUSE_DIR is a bug
in Hive, right?
While this change will make this test pass for now, it will just start failing
again when Hive makes the necessary bug fix.
>From Joe: "Impala's test_compressed_formats() shouldn't depend on a Hive
>generated location. That's not what we are testing. I think that
>test_compressed_formats() can be modified to use a unique database, and then
>we can specify the location when creating the table in Hive. That should
>unblock Impala."
--
To view, visit http://gerrit.cloudera.org:8080/15990
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f
Gerrit-Change-Number: 15990
Gerrit-PatchSet: 4
Gerrit-Owner: Xiaomeng Zhang
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Joe McDonnell
Gerrit-Reviewer: Sahil Takiar
Gerrit-Comment-Date: Fri, 29 May 2020 21:37:19 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6153/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 4 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 28 May 2020 23:02:22 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Xiaomeng Zhang has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. IMPALA-9673: Add external warehouse dir variable in E2E test In minicluster, we have default values of hive.create.as.acid and hive.create.as.insert.only which are false. So by default hive creates external type table located in external warehouse directory. Due to CDPD-8248, desc db returns external warehouse directory. With above reasons, we need use external warehouse dir in some tests. Tested: Re-run failed test in minicluster. Run impala-private-parameterized job exhaustively. Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f --- M bin/impala-config.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/create-database.test M testdata/workloads/functional-query/queries/QueryTest/describe-db.test M testdata/workloads/functional-query/queries/QueryTest/describe-hive-db.test M tests/common/environ.py M tests/common/impala_test_suite.py M tests/query_test/test_compressed_formats.py 8 files changed, 38 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/15990/4 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 4 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6144/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 3 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 28 May 2020 18:33:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 3: (2 comments) http://gerrit.cloudera.org:8080/#/c/15990/3/tests/common/impala_test_suite.py File tests/common/impala_test_suite.py: http://gerrit.cloudera.org:8080/#/c/15990/3/tests/common/impala_test_suite.py@41 PS3, Line 41: flake8: W291 trailing whitespace http://gerrit.cloudera.org:8080/#/c/15990/3/tests/common/impala_test_suite.py@41 PS3, Line 41: HIVE_MAJOR_VERSION, line has trailing whitespace -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 3 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Thu, 28 May 2020 17:45:17 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Xiaomeng Zhang has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. IMPALA-9673: Add external warehouse dir variable in E2E test In minicluster, we have default values of hive.create.as.acid and hive.create.as.insert.only which are false. So by default hive creates external type table located in external warehouse directory. Due to CDPD-8248, desc db returns external warehouse directory. With above reasons, we need use external warehouse dir in some tests. Tested: Re-run failed test in minicluster. Run impala-private-parameterized job exhaustively. Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f --- M bin/impala-config.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/create-database.test M testdata/workloads/functional-query/queries/QueryTest/describe-db.test M testdata/workloads/functional-query/queries/QueryTest/describe-hive-db.test M tests/common/environ.py M tests/common/impala_test_suite.py M tests/query_test/test_compressed_formats.py 8 files changed, 38 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/15990/3 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 3 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/6142/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 2 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Wed, 27 May 2020 19:24:27 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/15990 ) Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. Patch Set 2: (1 comment) http://gerrit.cloudera.org:8080/#/c/15990/2/tests/common/impala_test_suite.py File tests/common/impala_test_suite.py: http://gerrit.cloudera.org:8080/#/c/15990/2/tests/common/impala_test_suite.py@40 PS2, Line 40: O flake8: E501 line too long (98 > 90 characters) -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 2 Gerrit-Owner: Xiaomeng Zhang Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Sahil Takiar Gerrit-Comment-Date: Wed, 27 May 2020 18:32:36 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-9673: Add external warehouse dir variable in E2E test
Xiaomeng Zhang has uploaded this change for review. ( http://gerrit.cloudera.org:8080/15990 Change subject: IMPALA-9673: Add external warehouse dir variable in E2E test .. IMPALA-9673: Add external warehouse dir variable in E2E test In minicluster, we have default values of hive.create.as.acid and hive.create.as.insert.only which are false. So by default hive creates external type table located in external warehouse directory. Due to CDPD-8248, desc db returns external warehouse directory. With above reasons, we need use external warehouse dir in some tests. Tested: Re-run failed test in minicluster. Run impala-private-parameterized job exhaustively. Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f --- M bin/impala-config.sh M testdata/datasets/functional/functional_schema_template.sql M testdata/workloads/functional-query/queries/QueryTest/create-database.test M testdata/workloads/functional-query/queries/QueryTest/describe-db.test M testdata/workloads/functional-query/queries/QueryTest/describe-hive-db.test M tests/common/environ.py M tests/common/impala_test_suite.py M tests/query_test/test_compressed_formats.py 8 files changed, 35 insertions(+), 31 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/90/15990/2 -- To view, visit http://gerrit.cloudera.org:8080/15990 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I57926babf4caebfd365e6be65a399f12ea68687f Gerrit-Change-Number: 15990 Gerrit-PatchSet: 2 Gerrit-Owner: Xiaomeng Zhang
