[
https://issues.apache.org/jira/browse/IMPALA-10219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17218778#comment-17218778
]
ASF subversion and git services commented on IMPALA-10219:
----------------------------------------------------------
Commit 15c3b13e9730479e096275d974000ae9fe8fbb83 in impala's branch
refs/heads/master from Vihang Karajgaonkar
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=15c3b13 ]
IMPALA-10219: Expose DEBUG_ACTION query option in catalog
This patches enables DEBUG_ACTION in the catalog service's
java code. Specifically, DEBUG_ACTION query option is now
exposed to TResetMetadataRequest and TExecDdlRequest
so that we can inject delays while executing refresh
or ddl statements.
For example,
1. To inject a delay of 100ms per HDFS list operation
during refresh statement set the following query option:
set debug_action=catalogd_refresh_hdfs_listing_delay:SLEEP@100;
2. To inject a delay of 100ms in alter table recover
partitions statement:
set debug_action=catalogd_table_recover_delay:SLEEP@100;
3. To inject a delay of 100ms in compute stats statement
set debug_action=catalogd_update_stats_delay:SLEEP@100;
Note that this option only adds the delay during the
update_stats phase of the compute stats execution.
Testing:
1. Added a test which sets the query option and makes
sure that command takes more time than without query option.
2. Added unit tests for the debugAction implementation
logic.
Change-Id: Ia7196b1ce76415a5faf3fa8575a26d22b2bf50b1
Reviewed-on: http://gerrit.cloudera.org:8080/16548
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Add a query option to simulate catalogd HDFS listing delays
> -----------------------------------------------------------
>
> Key: IMPALA-10219
> URL: https://issues.apache.org/jira/browse/IMPALA-10219
> Project: IMPALA
> Issue Type: Sub-task
> Reporter: Vihang Karajgaonkar
> Assignee: Vihang Karajgaonkar
> Priority: Minor
>
> This parent issue (IMPALA-6671) caused serious query backlog on large setups
> where namenode response times are slower for whatever reasons. While you can
> tune the namenode to some extent it is still problematic that Impala HDFS
> operations which happen while holding the table lock block other unrelated
> queries.
> In order to simulate such problems in the product it would be nice to
> introduce a query option which adds a artificial delay in the RPCs to
> namenode when the table is being loaded. A query option is preferred over
> service level configuration since, that way it is easier to model a slow
> blocking query and a unrelated fast query in the test suite.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
