[
https://issues.apache.org/jira/browse/IMPALA-13912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17983764#comment-17983764
]
ASF subversion and git services commented on IMPALA-13912:
----------------------------------------------------------
Commit e0cb533c252b003e9034808140119313ce9eeb4b in impala's branch
refs/heads/master from Mihaly Szjatinya
[ https://gitbox.apache.org/repos/asf?p=impala.git;h=e0cb533c2 ]
IMPALA-13912: Use SHARED_CLUSTER_ARGS in more custom cluster tests
In addition to IMPALA-13503 which allowed having the single cluster
running for the entire test class, this attempts to minimize restarting
between the existing tests without modifying any of their code.
This changeset saves the command line with which
'start-impala-cluster.py' has been run and skips the restarting if the
command line is the same for the next test.
Some tests however do require restart due to the specific metrics being
tested. Such tests are defined with the 'force_restart' flag within the
'with_args' decorator. NOTE: there might be more tests like that
revealed after running the tests in different order resulting in test
failures.
Experimentally, this results in ~150 fewer restarts, mostly coming from
restarts between tests. As for restarts between different variants of
the same test, most of the cluster tests are restricted to single
variant, although multi-variant tests occur occasionally.
Change-Id: I7c9115d4d47b9fe0bfd9dbda218aac2fb02dbd09
Reviewed-on: http://gerrit.cloudera.org:8080/22901
Reviewed-by: Impala Public Jenkins <[email protected]>
Tested-by: Impala Public Jenkins <[email protected]>
> Use SHARED_CLUSTER_ARGS in more custom cluster tests
> ----------------------------------------------------
>
> Key: IMPALA-13912
> URL: https://issues.apache.org/jira/browse/IMPALA-13912
> Project: IMPALA
> Issue Type: Improvement
> Components: Test
> Reporter: Csaba Ringhofer
> Assignee: Mihaly Szjatinya
> Priority: Minor
> Labels: ramp-up
>
> IMPALA-13503 allowed restarting the cluster only once per test suite in
> CustomClusterTestSuites using per suite @CustomClusterTestSuite.with_args.
> There are many custom cluster tests that could be restructured to be much
> faster this way, for example
> https://github.com/apache/impala/blob/7f38c7ed61a0536c430311b3d4600aa0a16b988a/tests/custom_cluster/test_client_ssl.py#L100C7-L100C15
> {code}
> @CustomClusterTestSuite.with_args(impalad_args=SSL_ARGS,
> statestored_args=SSL_ARGS,
> catalogd_args=SSL_ARGS)
> def test_ssl(self, vector):
> {code}
> The test above is run with 4 test vectors (see add_test_dimensions) and the
> cluster is restarted each time. This is not needed as the test vector doesn't
> affect the cluster parameters. A possible fix for this is to split the test
> suite to multiple suites where CustomClusterTestSuite.with_args is set per
> suite instead of per test.
> Tests like this seem very common - the following rough estimate returns 189:
> {code}
> git grep -B5 "def test.*vector" | grep "CustomClusterTestSuite.with_args" |
> wc -l
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
