Impala Public Jenkins has submitted this change and it was merged. ( http://gerrit.cloudera.org:8080/20110 )
Change subject: IMPALA-11901: Support COPY TESTCASE in local catalog mode ...................................................................... IMPALA-11901: Support COPY TESTCASE in local catalog mode COPY TESTCASE TO statement is used to export the metadata required for planning a given query. It will export catalog objects cached on the coordinator side to a file. Users can transfer the metadata file to another debug cluster and import it by the COPY TESTCASE FROM statement. The statement will import the metadata to the new catalogd, including the topology of the original cluster. So an EXPLAIN on the same query can produce the same query plan for debugging frontend issues. However, the COPY TESTCASE statements are not supported in LocalCatalog mode. In the legacy catalog mode, coordinator has the same catalog representation as catalogd, i.e. coordinator just mirrors all the metadata from catalogd. So the export is simple - we can dump the metadata from coordinator side and they can be ingested into catalogd. However, in the local catalog mode, coordinator caches the metadata in a finer grained manner which differs a lot to the metadata in catalogd. To export the metadata that can be imported in catalogd, we need some transformation to convert them into catalog objects used in catalogd. This patch uses the fine-grained metadata to construct the full thrift objects of THdfsTable. So coordinator can export them in the same way as we did in the legacy catalog mode. Tests: - We don't have two clusters in our dev env to verify the real use cases. Add sanity tests for exporting and importing testcase metadata. Change-Id: I02c1c76d7af15f28bdbc8d98d92a1553570e9e27 Reviewed-on: http://gerrit.cloudera.org:8080/20110 Reviewed-by: Impala Public Jenkins <[email protected]> Tested-by: Impala Public Jenkins <[email protected]> --- M fe/src/main/java/org/apache/impala/analysis/CopyTestCaseStmt.java M fe/src/main/java/org/apache/impala/catalog/FeCatalogUtils.java M fe/src/main/java/org/apache/impala/catalog/Table.java M fe/src/main/java/org/apache/impala/catalog/local/LocalFsTable.java M fe/src/main/java/org/apache/impala/catalog/local/LocalKuduTable.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/test/java/org/apache/impala/testutil/PlannerTestCaseLoader.java M tests/metadata/test_testcase_builder.py 8 files changed, 126 insertions(+), 31 deletions(-) Approvals: Impala Public Jenkins: Looks good to me, approved; Verified -- To view, visit http://gerrit.cloudera.org:8080/20110 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: merged Gerrit-Change-Id: I02c1c76d7af15f28bdbc8d98d92a1553570e9e27 Gerrit-Change-Number: 20110 Gerrit-PatchSet: 7 Gerrit-Owner: Quanlong Huang <[email protected]> Gerrit-Reviewer: Andrew Sherman <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Kurt Deschler <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
