Vihang Karajgaonkar has uploaded a new patch set (#2). ( http://gerrit.cloudera.org:8080/16548 )
Change subject: IMPALA-10219: Add query option to simulate HDFS listing delay ...................................................................... IMPALA-10219: Add query option to simulate HDFS listing delay This patch introduces a new query option SIMULATE_CATALOGD_HDFS_LISTING_DELAY_MS which can be set to a numeric value. This is primarily used as a test query option to simulate conditions in catalogd when namenode response times are slow. Currently, this only applies for the "refresh table" command since it is hard to inject delays in other operations like "alter table recover..." due to the usage of RemoteIterator interface. Testing: 1. Added a simple test which sets the query option and makes sure that refresh table command on a large table takes significantly more time than without the option set. Change-Id: Ia7196b1ce76415a5faf3fa8575a26d22b2bf50b1 --- M be/src/catalog/catalog-server.cc M be/src/service/query-options.cc M be/src/service/query-options.h M be/src/util/backend-gflag-util.cc M common/thrift/BackendGflags.thrift M common/thrift/CatalogService.thrift M common/thrift/ImpalaInternalService.thrift M common/thrift/ImpalaService.thrift M fe/src/main/java/org/apache/impala/catalog/CatalogServiceCatalog.java M fe/src/main/java/org/apache/impala/catalog/FileMetadataLoader.java M fe/src/main/java/org/apache/impala/catalog/HdfsTable.java M fe/src/main/java/org/apache/impala/catalog/IcebergTable.java M fe/src/main/java/org/apache/impala/catalog/ParallelFileMetadataLoader.java M fe/src/main/java/org/apache/impala/common/FileSystemUtil.java M fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java M fe/src/main/java/org/apache/impala/service/Frontend.java M tests/common/impala_connection.py A tests/custom_cluster/test_topic_update_frequency.py 18 files changed, 206 insertions(+), 44 deletions(-) git pull ssh://gerrit.cloudera.org:29418/Impala-ASF refs/changes/48/16548/2 -- To view, visit http://gerrit.cloudera.org:8080/16548 To unsubscribe, visit http://gerrit.cloudera.org:8080/settings Gerrit-Project: Impala-ASF Gerrit-Branch: master Gerrit-MessageType: newpatchset Gerrit-Change-Id: Ia7196b1ce76415a5faf3fa8575a26d22b2bf50b1 Gerrit-Change-Number: 16548 Gerrit-PatchSet: 2 Gerrit-Owner: Vihang Karajgaonkar <[email protected]>
