[Impala-ASF-CR] IMPALA-8436: Prohibit write/alter operations on materialized view
Sudhanshu Arora has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/13503 ) Change subject: IMPALA-8436: Prohibit write/alter operations on materialized view .. IMPALA-8436: Prohibit write/alter operations on materialized view Instead of creating an in memory instance of View, we were creating instance of HdfsTable. Modified the code to create instance of View for materialized view. Testing Done: - Added tests in AnalyzerTest. Change-Id: Idcd619303e19b5a2551876a63d67569c76bd22f0 --- M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java M testdata/datasets/functional/functional_schema_template.sql 4 files changed, 56 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/03/13503/5 -- To view, visit http://gerrit.cloudera.org:8080/13503 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Idcd619303e19b5a2551876a63d67569c76bd22f0 Gerrit-Change-Number: 13503 Gerrit-PatchSet: 5 Gerrit-Owner: Sudhanshu Arora Gerrit-Reviewer: Csaba Ringhofer Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Sudhanshu Arora Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Reviewer: Yongzhi Chen Gerrit-Reviewer: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13561 ) Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Sat, 08 Jun 2019 04:57:29 + Gerrit-HasComments: No
[Impala-ASF-CR] Add option to set driver in ImpalaJdbcDriver
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13514 ) Change subject: Add option to set driver in ImpalaJdbcDriver .. Patch Set 3: Verified-1 Build failed: https://jenkins.impala.io/job/gerrit-verify-dryrun/4433/ -- To view, visit http://gerrit.cloudera.org:8080/13514 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6c8cec2f61e76b7c8321c954eef830778d8a8ee Gerrit-Change-Number: 13514 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 08 Jun 2019 04:53:30 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8603: remove confusing logging from custom cluster tests
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13502 ) Change subject: IMPALA-8603: remove confusing logging from custom cluster tests .. IMPALA-8603: remove confusing logging from custom cluster tests IMPALA-1653 added explicit closing of the HS2 session that is created by ImpalaTestSuite during teardown. For the custom cluster tests, this sometimes fails, either because the impalad was shutdown and is no longer reachable or because it was restarted and the new impalad doesn't know about the session that was created before restart. Currently, when these errors occur we log them. However, this is confusing because several custom cluster tests now log errors even though they succeed. This patch changes this to simply ignore the errors. One concern is that a test may hit a genuine error here and if so we won't have the logging for it. If this happens, though, it should generally be possible for a dev to repro the error with the logging enabled. Change-Id: I96144fdbe6cc9bf0f01a9951420ee6f281fa6649 Reviewed-on: http://gerrit.cloudera.org:8080/13502 Reviewed-by: Impala Public Jenkins Tested-by: Impala Public Jenkins --- M tests/common/impala_connection.py 1 file changed, 1 insertion(+), 1 deletion(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/13502 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I96144fdbe6cc9bf0f01a9951420ee6f281fa6649 Gerrit-Change-Number: 13502 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8603: remove confusing logging from custom cluster tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13502 ) Change subject: IMPALA-8603: remove confusing logging from custom cluster tests .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13502 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I96144fdbe6cc9bf0f01a9951420ee6f281fa6649 Gerrit-Change-Number: 13502 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Sat, 08 Jun 2019 03:56:13 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13561 ) Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. Patch Set 2: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Sat, 08 Jun 2019 02:25:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration
Thomas Marshall has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13555 ) Change subject: IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration .. IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration The patch for IMPALA-8504 (part 2) (6bb404dc359) checks to see if Impala and Kudu are configured against the same metastore to determine if the HMS integration is enabled. However, instead of using its own metastore URI config, it uses the configuration stored on the remote HMS. This is error prone because it's not common for the HMS configuration to store its own URI. Instead, we should use our own config. This patch changes to using the local configuration for this purpose. More robust would be to use the HMS "UUID" support, since it's possible that Kudu and Impala are talking to different HMS instances sharing a backing DB, but that work is deferred to a later commit since it depends on Kudu-side changes. This commit doesn't add any tests, but fixes the existing tests when running against Hive 3, where the HMS server side uses a different configuration key for the metastore URIs. Change-Id: Id7a4c2cc0580f7c4dc5cfceed30b91e87c547612 Reviewed-on: http://gerrit.cloudera.org:8080/13555 Reviewed-by: Hao Hao Reviewed-by: Thomas Marshall Tested-by: Impala Public Jenkins --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java 3 files changed, 9 insertions(+), 17 deletions(-) Approvals: Hao Hao: Looks good to me, but someone else must approve Thomas Marshall: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/13555 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: Id7a4c2cc0580f7c4dc5cfceed30b91e87c547612 Gerrit-Change-Number: 13555 Gerrit-PatchSet: 2 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13555 ) Change subject: IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration .. Patch Set 1: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13555 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id7a4c2cc0580f7c4dc5cfceed30b91e87c547612 Gerrit-Change-Number: 13555 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 23:49:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13561 ) Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4434/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 23:49:19 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8638: Fix flaky TestLineage::test create table timestamp
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13560 ) Change subject: IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp .. Patch Set 3: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13560 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 Gerrit-Change-Number: 13560 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Fri, 07 Jun 2019 23:23:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8638: Fix flaky TestLineage::test create table timestamp
Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/13560 ) Change subject: IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp .. IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp This patch fixes the bug TestLineage::test_create_table_timestamp where it uses the same lineage log directory created by TestLineage::test_start_end_timestamp, which could fail the query ID assertion. The fix is to use a different lineage log directory than the one used by TestLineage::test_start_end_timestamp. Testing: - Ran test_lineage.py multiple times and it still passed. Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 Reviewed-on: http://gerrit.cloudera.org:8080/13560 Reviewed-by: Fredy Wijaya Tested-by: Impala Public Jenkins --- M tests/custom_cluster/test_lineage.py 1 file changed, 11 insertions(+), 15 deletions(-) Approvals: Fredy Wijaya: Looks good to me, approved Impala Public Jenkins: Verified -- To view, visit http://gerrit.cloudera.org:8080/13560 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 Gerrit-Change-Number: 13560 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker
[Impala-ASF-CR] Add option to set driver in ImpalaJdbcDriver
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13514 ) Change subject: Add option to set driver in ImpalaJdbcDriver .. Patch Set 3: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13514 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6c8cec2f61e76b7c8321c954eef830778d8a8ee Gerrit-Change-Number: 13514 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 23:17:51 + Gerrit-HasComments: No
[Impala-ASF-CR] Add option to set driver in ImpalaJdbcDriver
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13514 ) Change subject: Add option to set driver in ImpalaJdbcDriver .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4433/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13514 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6c8cec2f61e76b7c8321c954eef830778d8a8ee Gerrit-Change-Number: 13514 Gerrit-PatchSet: 3 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 23:17:51 + Gerrit-HasComments: No
[Impala-ASF-CR] Add option to set driver in ImpalaJdbcDriver
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13514 ) Change subject: Add option to set driver in ImpalaJdbcDriver .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13514 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6c8cec2f61e76b7c8321c954eef830778d8a8ee Gerrit-Change-Number: 13514 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 23:13:20 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13561 ) Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3548/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 23:12:12 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Hello Thomas Marshall, Hao Hao, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13561 to look at the new patch set (#3). Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. IMPALA-8629: (part 1) Add temp KuduStorageHandler This patch adds a temporary KuduStorageHandler so that the Kudu project can change its handler without breaking the integration. It also disabled any tests that depend on the specific handler. A follow up patch will remove the TEMP_KUDU_STORAGE_HANDLER. adjust the KUDU_STORAGE_HANDLER value to be the final value, and re-enable the tests. Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 4 files changed, 24 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/13561/3 -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13561 ) Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. Patch Set 2: (2 comments) http://gerrit.cloudera.org:8080/#/c/13561/1/fe/src/main/java/org/apache/impala/catalog/KuduTable.java File fe/src/main/java/org/apache/impala/catalog/KuduTable.java: http://gerrit.cloudera.org:8080/#/c/13561/1/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@90 PS1, Line 90: // TODO(IMPALA-8629): Remove this after Kudu adjusts its StorageHandler logic. > We may want to add the Jira number (IMPALA-8629) here? Done http://gerrit.cloudera.org:8080/#/c/13561/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java File fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java: http://gerrit.cloudera.org:8080/#/c/13561/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@a444 PS2, Line 444: > removed by mistake? Done -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 22:32:24 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Update mustache to commit b290952d8eb93d085214d8c8c9eab8559df9f606
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13563 ) Change subject: Update mustache to commit b290952d8eb93d085214d8c8c9eab8559df9f606 .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3546/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13563 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If26f3aaa2a3279a1f6a300c4f4cee7ec899e22ed Gerrit-Change-Number: 13563 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2019 22:24:52 + Gerrit-HasComments: No
[Impala-ASF-CR] Add option to set driver in ImpalaJdbcDriver
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13514 ) Change subject: Add option to set driver in ImpalaJdbcDriver .. Patch Set 2: Yeah, it was a draft, should be published now -- To view, visit http://gerrit.cloudera.org:8080/13514 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6c8cec2f61e76b7c8321c954eef830778d8a8ee Gerrit-Change-Number: 13514 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 22:17:12 + Gerrit-HasComments: No
[Impala-ASF-CR] Add option to set driver in ImpalaJdbcDriver
Thomas Marshall has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/13514 ) Change subject: Add option to set driver in ImpalaJdbcDriver .. Add option to set driver in ImpalaJdbcDriver ImpalaJdbcDriver is a simple class that provides a wrapper around the Hive Jdbc driver for use in testing. This patch adds a '-d' parameter that takes a driver class and executes queries with that driver instead of the Hive driver, if its in the classpath. This makes it easy to use ./bin/run-jdbc-client.sh to test other Jdbc drivers, such as the proprietary Cloudera Impala driver. Change-Id: Ie6c8cec2f61e76b7c8321c954eef830778d8a8ee --- M fe/src/test/java/org/apache/impala/testutil/ImpalaJdbcClient.java 1 file changed, 26 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/14/13514/2 -- To view, visit http://gerrit.cloudera.org:8080/13514 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ie6c8cec2f61e76b7c8321c954eef830778d8a8ee Gerrit-Change-Number: 13514 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8603: remove confusing logging from custom cluster tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13502 ) Change subject: IMPALA-8603: remove confusing logging from custom cluster tests .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4432/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13502 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I96144fdbe6cc9bf0f01a9951420ee6f281fa6649 Gerrit-Change-Number: 13502 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 22:15:59 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8603: remove confusing logging from custom cluster tests
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13502 ) Change subject: IMPALA-8603: remove confusing logging from custom cluster tests .. Patch Set 2: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13502 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I96144fdbe6cc9bf0f01a9951420ee6f281fa6649 Gerrit-Change-Number: 13502 Gerrit-PatchSet: 2 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 22:15:59 + Gerrit-HasComments: No
[Impala-ASF-CR] Add option to set driver in ImpalaJdbcDriver
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13514 ) Change subject: Add option to set driver in ImpalaJdbcDriver .. Patch Set 1: For some reason I can't +2 this - maybe there's a hidden draft patch or something? -- To view, visit http://gerrit.cloudera.org:8080/13514 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ie6c8cec2f61e76b7c8321c954eef830778d8a8ee Gerrit-Change-Number: 13514 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 22:11:17 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8603: remove confusing logging from custom cluster tests
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13502 ) Change subject: IMPALA-8603: remove confusing logging from custom cluster tests .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13502 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I96144fdbe6cc9bf0f01a9951420ee6f281fa6649 Gerrit-Change-Number: 13502 Gerrit-PatchSet: 1 Gerrit-Owner: Thomas Marshall Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 22:08:24 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. Patch Set 5: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3545/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 07 Jun 2019 21:58:48 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. Patch Set 4: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3544/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 07 Jun 2019 21:56:13 + Gerrit-HasComments: No
[Impala-ASF-CR] Update mustache to commit b290952d8eb93d085214d8c8c9eab8559df9f606
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13563 ) Change subject: Update mustache to commit b290952d8eb93d085214d8c8c9eab8559df9f606 .. Patch Set 1: (3 comments) http://gerrit.cloudera.org:8080/#/c/13563/1/be/src/thirdparty/mustache/mustache.cc File be/src/thirdparty/mustache/mustache.cc: http://gerrit.cloudera.org:8080/#/c/13563/1/be/src/thirdparty/mustache/mustache.cc@153 PS1, Line 153: // the entity does not exist (i.e. the path is invalid), 'resolved' will be set to nullptr. line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/13563/1/be/src/thirdparty/mustache/mustache.cc@349 PS1, Line 349: int EvaluateLength(const string& document, const int idx, const ContextStack* context_stack, line too long (92 > 90) http://gerrit.cloudera.org:8080/#/c/13563/1/be/src/thirdparty/mustache/mustache.cc@363 PS1, Line 363: int EvaluateLiteral(const string& document, const int idx, const ContextStack* context_stack, line too long (93 > 90) -- To view, visit http://gerrit.cloudera.org:8080/13563 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If26f3aaa2a3279a1f6a300c4f4cee7ec899e22ed Gerrit-Change-Number: 13563 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2019 21:44:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] Update mustache to commit b290952d8eb93d085214d8c8c9eab8559df9f606
Todd Lipcon has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13563 Change subject: Update mustache to commit b290952d8eb93d085214d8c8c9eab8559df9f606 .. Update mustache to commit b290952d8eb93d085214d8c8c9eab8559df9f606 This fixes some issues with regard to template variable scoping that will be useful for Knox integration. Change-Id: If26f3aaa2a3279a1f6a300c4f4cee7ec899e22ed --- M be/src/thirdparty/mustache/README M be/src/thirdparty/mustache/mustache.cc M be/src/thirdparty/mustache/mustache.h M be/src/util/webserver.cc 4 files changed, 184 insertions(+), 102 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/63/13563/1 -- To view, visit http://gerrit.cloudera.org:8080/13563 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: If26f3aaa2a3279a1f6a300c4f4cee7ec899e22ed Gerrit-Change-Number: 13563 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13561 ) Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. Patch Set 2: Code-Review+2 (1 comment) http://gerrit.cloudera.org:8080/#/c/13561/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java File fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java: http://gerrit.cloudera.org:8080/#/c/13561/2/fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java@a444 PS2, Line 444: removed by mistake? -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 21:30:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13561 ) Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3543/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 21:27:28 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. Patch Set 5: (1 comment) http://gerrit.cloudera.org:8080/#/c/13525/4/tests/authorization/test_ranger.py File tests/authorization/test_ranger.py: http://gerrit.cloudera.org:8080/#/c/13525/4/tests/authorization/test_ranger.py@587 PS4, Line 587: H > flake8: E126 continuation line over-indented for hanging indent Done -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 07 Jun 2019 21:18:37 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Fredy Wijaya has uploaded a new patch set (#5). ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 This patch bumps CDP_BUILD_NUMBER to 1153860 to pull RANGER-2437 fix that provides better error messages when granting/revoking with invalid user/group. Testing: - Added E2E test to test grant/revoke with invalid principal - Ran core tests - Ran the build with USE_CDP_HIVE=true Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 --- M bin/impala-config.sh M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M tests/authorization/test_ranger.py 3 files changed, 63 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/13525/5 -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 5 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/13525/4/tests/authorization/test_ranger.py File tests/authorization/test_ranger.py: http://gerrit.cloudera.org:8080/#/c/13525/4/tests/authorization/test_ranger.py@587 PS4, Line 587: " flake8: E126 continuation line over-indented for hanging indent -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 07 Jun 2019 21:17:09 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Fredy Wijaya has uploaded a new patch set (#4). ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 This patch bumps CDP_BUILD_NUMBER to 1153860 to pull RANGER-2437 fix that provides better error messages when granting/revoking with invalid user/group. Testing: - Added E2E test to test grant/revoke with invalid principal - Ran core tests - Ran the build with USE_CDP_HIVE=true Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 --- M bin/impala-config.sh M fe/src/main/java/org/apache/impala/authorization/ranger/RangerCatalogdAuthorizationManager.java M tests/authorization/test_ranger.py 3 files changed, 63 insertions(+), 7 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/13525/4 -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. Patch Set 4: (1 comment) http://gerrit.cloudera.org:8080/#/c/13525/2/tests/authorization/test_ranger.py File tests/authorization/test_ranger.py: http://gerrit.cloudera.org:8080/#/c/13525/2/tests/authorization/test_ranger.py@578 PS2, Line 578: ent in ["grant > hrm. I agree that parsing the error message isn't super robust, but I think I can easily prefix this with "Error granting a privilege to Ranger. Ranger error message: HTTP 403: ...". This way we don't have to worry if Ranger changes their error message and it's pretty clear that the error message came from Ranger. -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 4 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 07 Jun 2019 21:16:16 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13561 ) Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. Patch Set 2: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4431/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 21:07:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Hello Thomas Marshall, Hao Hao, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13561 to look at the new patch set (#2). Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. IMPALA-8629: (part 1) Add temp KuduStorageHandler This patch adds a temporary KuduStorageHandler so that the Kudu project can change its handler without breaking the integration. It also disabled any tests that depend on the specific handler. A follow up patch will remove the TEMP_KUDU_STORAGE_HANDLER. adjust the KUDU_STORAGE_HANDLER value to be the final value, and re-enable the tests. Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 4 files changed, 24 insertions(+), 14 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/13561/2 -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 2 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8629: (part 2) Adjust new KuduStorageHandler package
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13541 ) Change subject: IMPALA-8629: (part 2) Adjust new KuduStorageHandler package .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3542/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 20:29:07 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13557 ) Change subject: IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load .. Patch Set 1: (1 comment) http://gerrit.cloudera.org:8080/#/c/13557/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13557/1//COMMIT_MSG@17 PS1, Line 17: This re-enables some tests that were previously disabled for > I ran the kudu tests but didn't re-run the full suite. You think it's worth You should be able to pass in custom flags to the e2e test cluster with TEST_START_CLUSTER_ARGS - those are forwarded to start-impala-cluster.py I updated tests for catalog v2 by having them inspect the startup flags from the running impalad, so that should be sufficient for the e2e core tests to work. -- To view, visit http://gerrit.cloudera.org:8080/13557 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia85d6b10669866dcc9f2bf7385a4775e483dd6b0 Gerrit-Change-Number: 13557 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 07 Jun 2019 20:15:23 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13561 ) Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3541/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 20:06:39 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13561 ) Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. Patch Set 1: Code-Review+1 (1 comment) Just one minor nit. http://gerrit.cloudera.org:8080/#/c/13561/1/fe/src/main/java/org/apache/impala/catalog/KuduTable.java File fe/src/main/java/org/apache/impala/catalog/KuduTable.java: http://gerrit.cloudera.org:8080/#/c/13561/1/fe/src/main/java/org/apache/impala/catalog/KuduTable.java@90 PS1, Line 90: // TODO(ghenke): Remove this after Kudu adjusts its KuduStorageHandler logic. We may want to add the Jira number (IMPALA-8629) here? -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 19:54:18 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8629: (part 2) Adjust new KuduStorageHandler package
Grant Henke has posted comments on this change. ( http://gerrit.cloudera.org:8080/13541 ) Change subject: IMPALA-8629: (part 2) Adjust new KuduStorageHandler package .. Patch Set 3: Note: This still needs build changes to pass tests. It needs to pull in an updated Kudu version with the new handler. -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 19:52:04 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8629: (part 2) Adjust new KuduStorageHandler package
Hello Thomas Marshall, Hao Hao, Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13541 to look at the new patch set (#3). Change subject: IMPALA-8629: (part 2) Adjust new KuduStorageHandler package .. IMPALA-8629: (part 2) Adjust new KuduStorageHandler package This patch changes the new KuduStorageHandler package from “org.apache.kudu.hive” to “org.apache.hadoop.hive.kudu”. This is being done to ensure the stand-in storage handler can be a real storage handler when a Hive integration is added in the future. The “org.apache.hadoop.hive” package is the standard package all Hive storage handlers lives under. Additionally this patch updates the stand-in InputFormat, OutputFormat, and SerDe entries for Kudu. This allows a future Hive integration to read HMS tables/entries created by Impala. Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 --- M fe/src/main/java/org/apache/impala/catalog/HdfsFileFormat.java M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/test/java/org/apache/impala/analysis/AnalyzeKuduDDLTest.java M fe/src/test/java/org/apache/impala/analysis/ToSqlTest.java M tests/custom_cluster/test_kudu.py M tests/query_test/test_kudu.py 6 files changed, 18 insertions(+), 22 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/41/13541/3 -- To view, visit http://gerrit.cloudera.org:8080/13541 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I4d0c505643247498383472704a37d27c9e1ce473 Gerrit-Change-Number: 13541 Gerrit-PatchSet: 3 Gerrit-Owner: Grant Henke Gerrit-Reviewer: Grant Henke Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] IMPALA-8629: (part 1) Add temp KuduStorageHandler
Grant Henke has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13561 Change subject: IMPALA-8629: (part 1) Add temp KuduStorageHandler .. IMPALA-8629: (part 1) Add temp KuduStorageHandler This patch adds a temporary KuduStorageHandler so that the Kudu project can change its handler without breaking the integration. A follow up patch will remove the TEMP_KUDU_STORAGE_HANDLER and adjust the KUDU_STORAGE_HANDLER value to be the final value. Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java 1 file changed, 8 insertions(+), 1 deletion(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/61/13561/1 -- To view, visit http://gerrit.cloudera.org:8080/13561 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ic9982466699818390fa28efc5ea1aae75b11c12a Gerrit-Change-Number: 13561 Gerrit-PatchSet: 1 Gerrit-Owner: Grant Henke
[Impala-ASF-CR] IMPALA-8536: Add Scalable Pool Configuration to Admission Controller.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/13307 ) Change subject: IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. .. Patch Set 7: In fact I am removing my ClusterMembershipMgr change entirely. Reviewers please wait for a new patch -- To view, visit http://gerrit.cloudera.org:8080/13307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 Gerrit-Change-Number: 13307 Gerrit-PatchSet: 7 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 19:03:51 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8638: Fix flaky TestLineage::test create table timestamp
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13560 ) Change subject: IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3540/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13560 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 Gerrit-Change-Number: 13560 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Fri, 07 Jun 2019 18:28:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8638: Fix flaky TestLineage::test create table timestamp
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13560 ) Change subject: IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp .. Patch Set 2: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3539/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13560 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 Gerrit-Change-Number: 13560 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Fri, 07 Jun 2019 18:21:34 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13555 ) Change subject: IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration .. Patch Set 1: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4430/ DRY_RUN=true -- To view, visit http://gerrit.cloudera.org:8080/13555 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id7a4c2cc0580f7c4dc5cfceed30b91e87c547612 Gerrit-Change-Number: 13555 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 18:16:03 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8638: Fix flaky TestLineage::test create table timestamp
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13560 ) Change subject: IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp .. Patch Set 3: Build started: https://jenkins.impala.io/job/gerrit-verify-dryrun/4429/ DRY_RUN=false -- To view, visit http://gerrit.cloudera.org:8080/13560 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 Gerrit-Change-Number: 13560 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Fri, 07 Jun 2019 17:48:58 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8638: Fix flaky TestLineage::test create table timestamp
Fredy Wijaya has posted comments on this change. ( http://gerrit.cloudera.org:8080/13560 ) Change subject: IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp .. Patch Set 3: Code-Review+2 (1 comment) Carry Lars' +2. http://gerrit.cloudera.org:8080/#/c/13560/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13560/2//COMMIT_MSG@12 PS2, Line 12: to u > missing word Done -- To view, visit http://gerrit.cloudera.org:8080/13560 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 Gerrit-Change-Number: 13560 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Fri, 07 Jun 2019 17:48:34 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8638: Fix flaky TestLineage::test create table timestamp
Fredy Wijaya has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/13560 ) Change subject: IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp .. IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp This patch fixes the bug TestLineage::test_create_table_timestamp where it uses the same lineage log directory created by TestLineage::test_start_end_timestamp, which could fail the query ID assertion. The fix is to use a different lineage log directory than the one used by TestLineage::test_start_end_timestamp. Testing: - Ran test_lineage.py multiple times and it still passed. Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 --- M tests/custom_cluster/test_lineage.py 1 file changed, 11 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/13560/3 -- To view, visit http://gerrit.cloudera.org:8080/13560 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 Gerrit-Change-Number: 13560 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker
[Impala-ASF-CR] IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration
Thomas Marshall has posted comments on this change. ( http://gerrit.cloudera.org:8080/13555 ) Change subject: IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration .. Patch Set 1: Code-Review+2 -- To view, visit http://gerrit.cloudera.org:8080/13555 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id7a4c2cc0580f7c4dc5cfceed30b91e87c547612 Gerrit-Change-Number: 13555 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 17:45:02 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8638: Fix flaky TestLineage::test create table timestamp
Lars Volker has posted comments on this change. ( http://gerrit.cloudera.org:8080/13560 ) Change subject: IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp .. Patch Set 2: Code-Review+2 (1 comment) Thanks for the quick fix! LGTM, only saw one nit. http://gerrit.cloudera.org:8080/#/c/13560/2//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13560/2//COMMIT_MSG@12 PS2, Line 12: to a missing word -- To view, visit http://gerrit.cloudera.org:8080/13560 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 Gerrit-Change-Number: 13560 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Comment-Date: Fri, 07 Jun 2019 17:43:26 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8638: Fix flaky TestLineage::test create table timestamp
Fredy Wijaya has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13560 Change subject: IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp .. IMPALA-8638: Fix flaky TestLineage::test_create_table_timestamp This patch fixes the bug TestLineage::test_create_table_timestamp where it uses the same lineage log directory created by TestLineage::test_start_end_timestamp, which could fail the query ID assertion. The fix is to a different lineage log directory than the one used by TestLineage::test_start_end_timestamp. Testing: - Ran test_lineage.py multiple times and it still passed. Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 --- M tests/custom_cluster/test_lineage.py 1 file changed, 11 insertions(+), 15 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/60/13560/2 -- To view, visit http://gerrit.cloudera.org:8080/13560 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: I5813e4a570c181ba196b9ddf0210c8a0d92e21e8 Gerrit-Change-Number: 13560 Gerrit-PatchSet: 2 Gerrit-Owner: Fredy Wijaya
[Impala-ASF-CR] IMPALA-8536: Add Scalable Pool Configuration to Admission Controller.
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/13307 ) Change subject: IMPALA-8536: Add Scalable Pool Configuration to Admission Controller. .. Patch Set 7: (1 comment) I'm not happy with the ClusterMembershipMgr change, I'm working on doing things more efficiently. http://gerrit.cloudera.org:8080/#/c/13307/7/be/src/scheduling/cluster-membership-mgr.cc File be/src/scheduling/cluster-membership-mgr.cc: http://gerrit.cloudera.org:8080/#/c/13307/7/be/src/scheduling/cluster-membership-mgr.cc@278 PS7, Line 278: if (!entry.second.is_quiescing) count++; This can be done more efficiently, I'm working on an update -- To view, visit http://gerrit.cloudera.org:8080/13307 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: If47508728124076f3b9200c27cffc989f7a4f188 Gerrit-Change-Number: 13307 Gerrit-PatchSet: 7 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Bikramjeet Vig Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 17:41:20 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration
Hao Hao has posted comments on this change. ( http://gerrit.cloudera.org:8080/13555 ) Change subject: IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration .. Patch Set 1: Code-Review+1 -- To view, visit http://gerrit.cloudera.org:8080/13555 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id7a4c2cc0580f7c4dc5cfceed30b91e87c547612 Gerrit-Change-Number: 13555 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 17:41:15 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8143: Enhance DoRpcWithRetry().
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/12672 ) Change subject: IMPALA-8143: Enhance DoRpcWithRetry(). .. Patch Set 12: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3538/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/12672 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia9693151c35e02235665b3c285a48c585973d390 Gerrit-Change-Number: 12672 Gerrit-PatchSet: 12 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 17:28:40 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13559 ) Change subject: WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables .. Patch Set 2: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/3537/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13559 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad Gerrit-Change-Number: 13559 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2019 17:16:56 + Gerrit-HasComments: No
[Impala-ASF-CR] WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13559 ) Change subject: WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables .. Patch Set 1: Build Failed https://jenkins.impala.io/job/gerrit-code-review-checks/3536/ : Initial code review checks failed. See linked job for details on the failure. -- To view, visit http://gerrit.cloudera.org:8080/13559 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad Gerrit-Change-Number: 13559 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2019 17:15:23 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. Patch Set 3: (1 comment) http://gerrit.cloudera.org:8080/#/c/13525/2/tests/authorization/test_ranger.py File tests/authorization/test_ranger.py: http://gerrit.cloudera.org:8080/#/c/13525/2/tests/authorization/test_ranger.py@578 PS2, Line 578: > We can but we have to parse the error message and that can be somewhat erro hrm. I agree that parsing the error message isn't super robust, but I think we could at least do something like a simple regex substitution to remove ^HTTP \d+ Error: .+' and replace it with something like "Ranger error: ... (HTTP code 403)" so that it's clear that the error came from ranger and not produced within Impala or within the user's connection. I'm concerned that if a user using Hue sees an error pop up like this they may think there's something wrong with their HTTP connection to Hue, for example. -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 07 Jun 2019 17:13:07 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. Patch Set 3: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3535/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon Gerrit-Comment-Date: Fri, 07 Jun 2019 17:08:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8143: Enhance DoRpcWithRetry().
Andrew Sherman has posted comments on this change. ( http://gerrit.cloudera.org:8080/12672 ) Change subject: IMPALA-8143: Enhance DoRpcWithRetry(). .. Patch Set 12: (5 comments) Thanks Michael, I believe I addressed the nits. All tests pass. http://gerrit.cloudera.org:8080/#/c/12672/11/be/src/rpc/impala-service-pool.h File be/src/rpc/impala-service-pool.h: http://gerrit.cloudera.org:8080/#/c/12672/11/be/src/rpc/impala-service-pool.h@72 PS11, Line 72: char* > char* Done http://gerrit.cloudera.org:8080/#/c/12672/11/be/src/rpc/impala-service-pool.cc File be/src/rpc/impala-service-pool.cc: http://gerrit.cloudera.org:8080/#/c/12672/11/be/src/rpc/impala-service-pool.cc@192 PS11, Line 192: UNLIKELY(!debug_st > UNLIKELY(!debug_status.ok()) Done http://gerrit.cloudera.org:8080/#/c/12672/11/be/src/rpc/rpc-mgr-test.h File be/src/rpc/rpc-mgr-test.h: http://gerrit.cloudera.org:8080/#/c/12672/11/be/src/rpc/rpc-mgr-test.h@281 PS11, Line 281: GetNumberOfCalls() > nit: number_of_calls() or GetNumberOfCalls() Done http://gerrit.cloudera.org:8080/#/c/12672/11/be/src/rpc/rpc-mgr-test.cc File be/src/rpc/rpc-mgr-test.cc: http://gerrit.cloudera.org:8080/#/c/12672/11/be/src/rpc/rpc-mgr-test.cc@331 PS11, Line 331: // DoRpcWithRetry will fail with probability (1/2)^num_rpc_retry_calls. : ASSERT_TRUE(status.ok()); : } : // There will be no overflows (i.e. service too busy) with probability : // (1/2)^num_retries. > :-). :-) Thanks for thinking about this! http://gerrit.cloudera.org:8080/#/c/12672/11/be/src/rpc/rpc-mgr.h File be/src/rpc/rpc-mgr.h: http://gerrit.cloudera.org:8080/#/c/12672/11/be/src/rpc/rpc-mgr.h@183 PS11, Line 183: /// Pass 'debug_action' to DebugAction() to potentially inject errors. > Please add a TODO: Done -- To view, visit http://gerrit.cloudera.org:8080/12672 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia9693151c35e02235665b3c285a48c585973d390 Gerrit-Change-Number: 12672 Gerrit-PatchSet: 12 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 16:49:04 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8143: Enhance DoRpcWithRetry().
Andrew Sherman has uploaded a new patch set (#12). ( http://gerrit.cloudera.org:8080/12672 ) Change subject: IMPALA-8143: Enhance DoRpcWithRetry(). .. IMPALA-8143: Enhance DoRpcWithRetry(). Allow callers of RpcMgr::DoRpcWithRetry to specify a time to sleep if the remote service is busy. DoRpcWithRetry now only sleeps if the remote service is busy. TESTING: Ran all end-to-end tests. Add two new tests to rpc-mgr-test.cc which test RpcMgr::DoRpcWithRetry. One test uses a fake Proxy class, the other uses a new DebugAction to cause Krpc calls to be rejected as if the service was too busy. Change-Id: Ia9693151c35e02235665b3c285a48c585973d390 --- M be/src/rpc/impala-service-pool.cc M be/src/rpc/impala-service-pool.h M be/src/rpc/rpc-mgr-test.cc M be/src/rpc/rpc-mgr-test.h M be/src/rpc/rpc-mgr.h M be/src/rpc/rpc-mgr.inline.h M be/src/runtime/coordinator-backend-state.cc M be/src/runtime/query-state.h M be/src/service/client-request-state.cc M be/src/service/control-service.h 10 files changed, 189 insertions(+), 44 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/72/12672/12 -- To view, visit http://gerrit.cloudera.org:8080/12672 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia9693151c35e02235665b3c285a48c585973d390 Gerrit-Change-Number: 12672 Gerrit-PatchSet: 12 Gerrit-Owner: Andrew Sherman Gerrit-Reviewer: Andrew Sherman Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Thomas Marshall
[Impala-ASF-CR] WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables
Hello Impala Public Jenkins, I'd like you to reexamine a change. Please visit http://gerrit.cloudera.org:8080/13559 to look at the new patch set (#2). Change subject: WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables .. WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables Initial prototype of INSERT into ACID tables. Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad --- M be/src/exec/hdfs-table-sink.cc M be/src/exec/hdfs-table-sink.h M be/src/runtime/coordinator.cc M be/src/service/client-request-state.cc M be/src/service/frontend.cc M be/src/service/frontend.h M be/src/util/jni-util.h M common/thrift/DataSinks.thrift M common/thrift/Frontend.thrift M common/thrift/ImpalaInternalService.thrift M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java A fe/src/main/java/org/apache/impala/common/TransactionException.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M fe/src/main/java/org/apache/impala/planner/TableSink.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java 16 files changed, 180 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/13559/2 -- To view, visit http://gerrit.cloudera.org:8080/13559 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad Gerrit-Change-Number: 13559 Gerrit-PatchSet: 2 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins
[Impala-ASF-CR] IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load
Todd Lipcon has posted comments on this change. ( http://gerrit.cloudera.org:8080/13557 ) Change subject: IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/13557/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13557/1//COMMIT_MSG@17 PS1, Line 17: This re-enables some tests that were previously disabled for > Were you able to run the end-to-end tests with the local catalog enabled? I I ran the kudu tests but didn't re-run the full suite. You think it's worth running a parameterized job and make sure I didn't regress something else? Is there an existing jenkins job that can easily be configured like that? http://gerrit.cloudera.org:8080/#/c/13557/1/fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java File fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java: http://gerrit.cloudera.org:8080/#/c/13557/1/fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java@40 PS1, Line 40: * Represents a table with incomplete metadata. The metadata may be incomplete because > Trying to understand - does this IncompleteTable implementation get used in yea I think this only gets used by the "v1" catalog. I'll add a comment. -- To view, visit http://gerrit.cloudera.org:8080/13557 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia85d6b10669866dcc9f2bf7385a4775e483dd6b0 Gerrit-Change-Number: 13557 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 07 Jun 2019 16:38:56 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13559 ) Change subject: WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables .. Patch Set 1: (2 comments) http://gerrit.cloudera.org:8080/#/c/13559/1/fe/src/main/java/org/apache/impala/analysis/InsertStmt.java File fe/src/main/java/org/apache/impala/analysis/InsertStmt.java: http://gerrit.cloudera.org:8080/#/c/13559/1/fe/src/main/java/org/apache/impala/analysis/InsertStmt.java@410 PS1, Line 410: transactionId_ = queryCtx.isSetTransaction_id() ? queryCtx.getTransaction_id() : null; line has trailing whitespace http://gerrit.cloudera.org:8080/#/c/13559/1/fe/src/main/java/org/apache/impala/analysis/InsertStmt.java@410 PS1, Line 410: transactionId_ = queryCtx.isSetTransaction_id() ? queryCtx.getTransaction_id() : null; line too long (91 > 90) -- To view, visit http://gerrit.cloudera.org:8080/13559 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad Gerrit-Change-Number: 13559 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2019 16:33:45 + Gerrit-HasComments: Yes
[Impala-ASF-CR] WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables
Zoltan Borok-Nagy has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13559 Change subject: WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables .. WIP: IMPALA-8636: Implement INSERT for insert-only ACID tables Initial prototype of INSERT into ACID tables. Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad --- M be/src/exec/hdfs-table-sink.cc M be/src/exec/hdfs-table-sink.h M be/src/runtime/coordinator.cc M be/src/service/client-request-state.cc M be/src/service/frontend.cc M be/src/service/frontend.h M be/src/util/jni-util.h M common/thrift/DataSinks.thrift M common/thrift/Frontend.thrift M common/thrift/ImpalaInternalService.thrift M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/planner/HdfsTableSink.java M fe/src/main/java/org/apache/impala/planner/TableSink.java M fe/src/main/java/org/apache/impala/service/Frontend.java M fe/src/main/java/org/apache/impala/service/JniFrontend.java 15 files changed, 144 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/59/13559/1 -- To view, visit http://gerrit.cloudera.org:8080/13559 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id6c36fa6902676f06b4e38730f737becfc7c06ad Gerrit-Change-Number: 13559 Gerrit-PatchSet: 1 Gerrit-Owner: Zoltan Borok-Nagy
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Fredy Wijaya has posted comments on this change. (
http://gerrit.cloudera.org:8080/13525 )
Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860
..
Patch Set 3:
(2 comments)
http://gerrit.cloudera.org:8080/#/c/13525/2/tests/authorization/test_ranger.py
File tests/authorization/test_ranger.py:
http://gerrit.cloudera.org:8080/#/c/13525/2/tests/authorization/test_ranger.py@566
PS2, Line 566: ctional.alltypes from user {0}"
: .format(getuser())]:
: result =
self.execute_query_expect_failure(invalid_impala_cli
> it's weird that you need two different clients, but you're passing the user
This is a known issue. I filed a bug here:
https://issues.apache.org/jira/browse/IMPALA-8640
http://gerrit.cloudera.org:8080/#/c/13525/2/tests/authorization/test_ranger.py@578
PS2, Line 578:
> is there any way we can get rid of the HTTP error part of the error message
We can but we have to parse the error message and that can be somewhat error
prone. Ranger throws a generic exception, so we pretty much just rely on the
error message returned by them.
--
To view, visit http://gerrit.cloudera.org:8080/13525
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: comment
Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576
Gerrit-Change-Number: 13525
Gerrit-PatchSet: 3
Gerrit-Owner: Fredy Wijaya
Gerrit-Reviewer: Fredy Wijaya
Gerrit-Reviewer: Impala Public Jenkins
Gerrit-Reviewer: Todd Lipcon
Gerrit-Comment-Date: Fri, 07 Jun 2019 16:30:15 +
Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8551: Bump CDP BUILD NUMBER to 1153860
Fredy Wijaya has uploaded a new patch set (#3). ( http://gerrit.cloudera.org:8080/13525 ) Change subject: IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 .. IMPALA-8551: Bump CDP_BUILD_NUMBER to 1153860 This patch bumps CDP_BUILD_NUMBER to 1153860 to pull RANGER-2437 fix that provides better error messages when granting/revoking with invalid user/group. Testing: - Added E2E test to test grant/revoke with invalid principal - Ran core tests - Ran the build with USE_CDP_HIVE=true Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 --- M bin/impala-config.sh M tests/authorization/test_ranger.py 2 files changed, 44 insertions(+), 5 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/25/13525/3 -- To view, visit http://gerrit.cloudera.org:8080/13525 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I8995f5dc88b211cd3af415713802cfeac44fe576 Gerrit-Change-Number: 13525 Gerrit-PatchSet: 3 Gerrit-Owner: Fredy Wijaya Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Todd Lipcon
[Impala-ASF-CR] IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load
Tim Armstrong has posted comments on this change. ( http://gerrit.cloudera.org:8080/13557 ) Change subject: IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load .. Patch Set 1: (2 comments) I think the solution makes sense, although I don't have a particularly deep understanding so I'd like more eyes on it. Just a couple of questions about testing and for my understanding. http://gerrit.cloudera.org:8080/#/c/13557/1//COMMIT_MSG Commit Message: http://gerrit.cloudera.org:8080/#/c/13557/1//COMMIT_MSG@17 PS1, Line 17: This re-enables some tests that were previously disabled for Were you able to run the end-to-end tests with the local catalog enabled? I had disabled local catalog in precommit because of other flakiness, so just want to make sure it works as expected. http://gerrit.cloudera.org:8080/#/c/13557/1/fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java File fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java: http://gerrit.cloudera.org:8080/#/c/13557/1/fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java@40 PS1, Line 40: * Represents a table with incomplete metadata. The metadata may be incomplete because Trying to understand - does this IncompleteTable implementation get used in LocalCatalog in some cases? Or is it only for CatalogServiceCatalog and ImpaladCatalog? The code paths are a little intermingled (at least when I'm trying to get a handle on things). If it's only used in some catalog implementations, might be good to mention briefly in the class comment. -- To view, visit http://gerrit.cloudera.org:8080/13557 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia85d6b10669866dcc9f2bf7385a4775e483dd6b0 Gerrit-Change-Number: 13557 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 07 Jun 2019 16:09:49 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8593: Prohibit write operations for bucketed tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13558 ) Change subject: IMPALA-8593: Prohibit write operations for bucketed tables .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3534/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 1 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Fri, 07 Jun 2019 14:22:18 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8593: Prohibit write operations for bucketed tables
Yongzhi Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/13558 ) Change subject: IMPALA-8593: Prohibit write operations for bucketed tables .. Patch Set 1: > Patch Set 1: > > (4 comments) Do we really want to put import into two lines? I will keep them as is. -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 1 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Fri, 07 Jun 2019 13:46:47 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8593: Prohibit write operations for bucketed tables
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13558 ) Change subject: IMPALA-8593: Prohibit write operations for bucketed tables .. Patch Set 1: (4 comments) http://gerrit.cloudera.org:8080/#/c/13558/1/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java File fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java: http://gerrit.cloudera.org:8080/#/c/13558/1/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@20 PS1, Line 20: import static org.apache.hadoop.hive.metastore.api.hive_metastoreConstants.ACCESSTYPE_NONE; line too long (91 > 90) http://gerrit.cloudera.org:8080/#/c/13558/1/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@21 PS1, Line 21: import static org.apache.hadoop.hive.metastore.api.hive_metastoreConstants.ACCESSTYPE_READONLY; line too long (95 > 90) http://gerrit.cloudera.org:8080/#/c/13558/1/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@22 PS1, Line 22: import static org.apache.hadoop.hive.metastore.api.hive_metastoreConstants.ACCESSTYPE_READWRITE; line too long (96 > 90) http://gerrit.cloudera.org:8080/#/c/13558/1/fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java@23 PS1, Line 23: import static org.apache.hadoop.hive.metastore.api.hive_metastoreConstants.ACCESSTYPE_WRITEONLY; line too long (96 > 90) -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 1 Gerrit-Owner: Yongzhi Chen Gerrit-Reviewer: Impala Public Jenkins Gerrit-Comment-Date: Fri, 07 Jun 2019 13:41:15 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8593: Prohibit write operations for bucketed tables
Yongzhi Chen has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13558 Change subject: IMPALA-8593: Prohibit write operations for bucketed tables .. IMPALA-8593: Prohibit write operations for bucketed tables This patch adds a method to check if a table bucketed. For Hive 3, integrates with HMS translation layer for capabilities checks. Implements methods ensureTableWriteSupported and ensureTableReadSupported. Tests: Added unit tests to ParserTest and AnalyzerTest. Added bucketed tables which are required by IMPALA-8439. Ran core tests. ToDo: Integrate checking bucketed tables capabilities and creating error messages with HMS translation after Hive provides the required functions. Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec --- M bin/impala-config.sh M fe/src/compat-hive-2/java/org/apache/impala/compat/MetastoreShim.java M fe/src/compat-hive-3/java/org/apache/impala/compat/MetastoreShim.java M fe/src/main/java/org/apache/impala/analysis/AlterTableStmt.java M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/BaseTableRef.java M fe/src/main/java/org/apache/impala/analysis/ComputeStatsStmt.java M fe/src/main/java/org/apache/impala/analysis/CreateTableLikeStmt.java M fe/src/main/java/org/apache/impala/analysis/DropTableOrViewStmt.java M fe/src/main/java/org/apache/impala/analysis/InsertStmt.java M fe/src/main/java/org/apache/impala/analysis/LoadDataStmt.java M fe/src/main/java/org/apache/impala/analysis/TruncateStmt.java M fe/src/main/java/org/apache/impala/catalog/MetaStoreClientPool.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java M fe/src/test/java/org/apache/impala/analysis/AnalyzerTest.java M fe/src/test/java/org/apache/impala/analysis/ParserTest.java M testdata/datasets/functional/functional_schema_template.sql M testdata/datasets/functional/schema_constraints.csv 19 files changed, 311 insertions(+), 13 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/58/13558/1 -- To view, visit http://gerrit.cloudera.org:8080/13558 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia08d01168660830b6e0d08b55a95eac129889cec Gerrit-Change-Number: 13558 Gerrit-PatchSet: 1 Gerrit-Owner: Yongzhi Chen
[Impala-ASF-CR] Bump CDP BUILD NUMBER to 1056671
Yongzhi Chen has posted comments on this change. ( http://gerrit.cloudera.org:8080/13213 ) Change subject: Bump CDP_BUILD_NUMBER to 1056671 .. Patch Set 7: (1 comment) http://gerrit.cloudera.org:8080/#/c/13213/7/bin/impala-config.sh File bin/impala-config.sh: http://gerrit.cloudera.org:8080/#/c/13213/7/bin/impala-config.sh@a164 PS7, Line 164: In order to support the new CDP feature from HMS, we need a newer version of CDP. Could you bump up the version to 1152380 or newer? 1152380 passed L1 tests. And the CDP components version will be in 7.0.0.0 ... style, for example: export CDP_HIVE_VERSION=3.1.0.7.0.0.0-105 -- To view, visit http://gerrit.cloudera.org:8080/13213 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ic758a15b33e89b6804c12356aac8e3f230e07ae0 Gerrit-Change-Number: 13213 Gerrit-PatchSet: 7 Gerrit-Owner: Vihang Karajgaonkar Gerrit-Reviewer: Fredy Wijaya Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Todd Lipcon Gerrit-Reviewer: Yongzhi Chen Gerrit-Comment-Date: Fri, 07 Jun 2019 12:52:53 + Gerrit-HasComments: Yes
[Impala-ASF-CR] IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13557 ) Change subject: IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3533/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13557 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Ia85d6b10669866dcc9f2bf7385a4775e483dd6b0 Gerrit-Change-Number: 13557 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Bharath Vissapragada Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Tim Armstrong Gerrit-Reviewer: Vihang Karajgaonkar Gerrit-Comment-Date: Fri, 07 Jun 2019 09:25:10 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load
Hello Tim Armstrong, I'd like you to do a code review. Please visit http://gerrit.cloudera.org:8080/13557 to review the following change. Change subject: IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load .. IMPALA-8459. LocalCatalog: Allow dropping tables that fail to load This adds support in LocalCatalog for IncompleteTables. A new FeIncompleteTable interface is introduced, and when a table fails to load, it produces a FeIncompleteTable instead of an immediate error. This allows DROP TABLE statements to get past analysis and execute even when tables are in bad states (eg a Kudu table with no backing table in Kudu itself). This re-enables some tests that were previously disabled for LocalCatalog. Change-Id: Ia85d6b10669866dcc9f2bf7385a4775e483dd6b0 --- M fe/src/main/java/org/apache/impala/analysis/Analyzer.java M fe/src/main/java/org/apache/impala/analysis/StmtMetadataLoader.java A fe/src/main/java/org/apache/impala/catalog/FeIncompleteTable.java M fe/src/main/java/org/apache/impala/catalog/IncompleteTable.java A fe/src/main/java/org/apache/impala/catalog/local/FailedLoadLocalTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalDb.java M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalTable.java M fe/src/main/java/org/apache/impala/service/MetadataOp.java M fe/src/test/java/org/apache/impala/catalog/local/LocalCatalogTest.java M tests/common/skip.py M tests/query_test/test_kudu.py 12 files changed, 127 insertions(+), 36 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/57/13557/1 -- To view, visit http://gerrit.cloudera.org:8080/13557 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Ia85d6b10669866dcc9f2bf7385a4775e483dd6b0 Gerrit-Change-Number: 13557 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Tim Armstrong
[Impala-ASF-CR] IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13555 ) Change subject: IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration .. Patch Set 1: Build Successful https://jenkins.impala.io/job/gerrit-code-review-checks/3532/ : Initial code review checks passed. Use gerrit-verify-dryrun-external or gerrit-verify-dryrun to run full precommit tests. -- To view, visit http://gerrit.cloudera.org:8080/13555 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: Id7a4c2cc0580f7c4dc5cfceed30b91e87c547612 Gerrit-Change-Number: 13555 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon Gerrit-Reviewer: Hao Hao Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Thomas Marshall Gerrit-Comment-Date: Fri, 07 Jun 2019 07:25:14 + Gerrit-HasComments: No
[Impala-ASF-CR] IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration
Todd Lipcon has uploaded this change for review. ( http://gerrit.cloudera.org:8080/13555 Change subject: IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration .. IMPALA-8635. Use local metastore URIs when checking for Kudu/HMS integration The patch for IMPALA-8504 (part 2) (6bb404dc359) checks to see if Impala and Kudu are configured against the same metastore to determine if the HMS integration is enabled. However, instead of using its own metastore URI config, it uses the configuration stored on the remote HMS. This is error prone because it's not common for the HMS configuration to store its own URI. Instead, we should use our own config. This patch changes to using the local configuration for this purpose. More robust would be to use the HMS "UUID" support, since it's possible that Kudu and Impala are talking to different HMS instances sharing a backing DB, but that work is deferred to a later commit since it depends on Kudu-side changes. This commit doesn't add any tests, but fixes the existing tests when running against Hive 3, where the HMS server side uses a different configuration key for the metastore URIs. Change-Id: Id7a4c2cc0580f7c4dc5cfceed30b91e87c547612 --- M fe/src/main/java/org/apache/impala/catalog/KuduTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/util/MetaStoreUtil.java 3 files changed, 9 insertions(+), 17 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/55/13555/1 -- To view, visit http://gerrit.cloudera.org:8080/13555 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newchange Gerrit-Change-Id: Id7a4c2cc0580f7c4dc5cfceed30b91e87c547612 Gerrit-Change-Number: 13555 Gerrit-PatchSet: 1 Gerrit-Owner: Todd Lipcon
[Impala-ASF-CR] IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1)
Impala Public Jenkins has posted comments on this change. ( http://gerrit.cloudera.org:8080/13522 ) Change subject: IMPALA-8561: Eliminate mtime=-1 for HDFS scan ranges (part 1) .. Patch Set 4: Verified+1 -- To view, visit http://gerrit.cloudera.org:8080/13522 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: comment Gerrit-Change-Id: I48b7ed60d6ab9104b993237b4fe23de5dc058672 Gerrit-Change-Number: 13522 Gerrit-PatchSet: 4 Gerrit-Owner: Joe McDonnell Gerrit-Reviewer: Impala Public Jenkins Gerrit-Reviewer: Joe McDonnell Gerrit-Reviewer: Lars Volker Gerrit-Reviewer: Michael Ho Gerrit-Reviewer: Tim Armstrong Gerrit-Comment-Date: Fri, 07 Jun 2019 06:04:40 + Gerrit-HasComments: No
