Vihang Karajgaonkar has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/17521 )
Change subject: IMPALA-10700: Add query options to skip deleting stats ...................................................................... IMPALA-10700: Add query options to skip deleting stats When a truncate table command is issued, in case of non-transactional tables, the table and column statistics for the table are also deleted by default. This can be a expensive operation especially when many truncate table commands are running concurrently. As the concurrency increases, the response time from Hive metastore slows down the delete table and column statistics RPC calls. In cases where truncate operation is used to remove the existing data and then reload new data, it is likely that users will compute stats again as soon as the new data is reloaded. This would overwrite the existing statistics and hence the additional time spent by the truncate operation to delete column and table statistics becomes unnecessary. To improve this, this change introduces a new query option: DELETE_STATS_IN_TRUNCATE. The default value of this option is 1 or true which means stats will be deleted as part of truncate operation. As the name suggests, when this query options are set to false or 0, a truncate operation will not delete the table and column statistics for the table. This change also makes a improvement to truncate operation on tables which are replicated. If the table is being replicated, previously, the statistics were not getting deleted after truncate. Now the statistics will get deleted after truncate. Testing: Modified truncate-table.test to include variations of these query options and making sure that the statistics are deleted or skipped from deletion after truncate operation. Change-Id: I9400c3586b4bdf46d9b4056ea1023aabae8cc519 --- M be/src/service/query-options.cc M be/src/service/query-options.h M common/thrift/ImpalaService.thrift M common/thrift/JniCatalog.thrift M common/thrift/Query.thrift M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M testdata/workloads/functional-query/queries/QueryTest/truncate-table.test 8 files changed, 424 insertions(+), 25 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/21/17521/2 -- To view, visit http://gerrit.cloudera.org:8080/17521 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: I9400c3586b4bdf46d9b4056ea1023aabae8cc519 Gerrit-Change-Number: 17521 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar <[email protected]> Gerrit-Reviewer: Aman Sinha <[email protected]> Gerrit-Reviewer: Impala Public Jenkins <[email protected]> Gerrit-Reviewer: Quanlong Huang <[email protected]>
