Hello Andrew Sherman, Kurt Deschler, Impala Public Jenkins,
I'd like you to reexamine a change. Please visit
http://gerrit.cloudera.org:8080/20110
to look at the new patch set (#5).
Change subject: IMPALA-11901: Support COPY TESTCASE in local catalog mode
......................................................................
IMPALA-11901: Support COPY TESTCASE in local catalog mode
COPY TESTCASE TO statement is used to export the metadata required for
planning a given query. It will export catalog objects cached on the
coordinator side to a file. Users can transfer the metadata file to
another debug cluster and import it by the COPY TESTCASE FROM statement.
The statement will import the metadata to the new catalogd, including
the topology of the original cluster. So an EXPLAIN on the same query
can produce the same query plan for debugging frontend issues.
However, the COPY TESTCASE statements are not supported in LocalCatalog
mode. In the legacy catalog mode, coordinator has the same catalog
representation as catalogd, i.e. coordinator just mirrors all the
metadata from catalogd. So the export is simple - we can dump the
metadata from coordinator side and they can be ingested into catalogd.
However, in the local catalog mode, coordinator caches the metadata in a
finer grained manner which differs a lot to the metadata in catalogd.
To export the metadata that can be imported in catalogd, we need some
transformation to convert them into catalog objects used in catalogd.
This patch uses the fine-grained metadata to construct the full thrift
objects of THdfsTable. So coordinator can export them in the same way as
we did in the legacy catalog mode.
Tests:
- We don't have two clusters in our dev env to verify the real use
cases. Add sanity tests for exporting and importing testcase
metadata.
Change-Id: I02c1c76d7af15f28bdbc8d98d92a1553570e9e27
---
M fe/src/main/java/org/apache/impala/analysis/CopyTestCaseStmt.java
M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java
M fe/src/main/java/org/apache/impala/catalog/Table.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java
M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java
M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java
M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java
M tests/metadata/test_testcase_builder.py
8 files changed, 126 insertions(+), 31 deletions(-)
git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/10/20110/5
--
To view, visit http://gerrit.cloudera.org:8080/20110
To unsubscribe, visit http://gerrit.cloudera.org:8080/settings
Gerrit-Project: Impala-ASF
Gerrit-Branch: master
Gerrit-MessageType: newpatchset
Gerrit-Change-Id: I02c1c76d7af15f28bdbc8d98d92a1553570e9e27
Gerrit-Change-Number: 20110
Gerrit-PatchSet: 5
Gerrit-Owner: Quanlong Huang <[email protected]>
Gerrit-Reviewer: Andrew Sherman <[email protected]>
Gerrit-Reviewer: Impala Public Jenkins <[email protected]>
Gerrit-Reviewer: Kurt Deschler <[email protected]>
Gerrit-Reviewer: Quanlong Huang <[email protected]>
