Hello, I'm about to start working on Impala's random query generator, a testing tool to help find test gaps in Impala's functional tests.
The random query generator and infra code around it has some functional and pure unit tests [0] that are not part of GVD, but it wouldn't be hard to fold them into GVD's execution. As part of the upcoming work, I plan to add even more tests: we need quick unit or functional tests to ensure a test tool is working as expected. What are people's thoughts on having these "tests for tests", or infra tests, be part of GVD? Pros: 1. Helps prevent regression in tools and infra 2. Verification procedure is the same as with the rest of Impala: run gerrit-verify-dryrun 3. Automatic Apache RAT verification Cons: 1. Patches to the random query generator tend to be self-contained. Ought we spend more AWS cycles and time building Impala and running these tests in order to run some ostensible (but growing) infra tests? 2. Flaky tests and failing builds can block test tool progress Other solutions if the cons win: 1. Separate Jenkins job for these tests (there's a separate job for submitting and verifying docs, for instance). A con of this is that this can lead to a proliferation of Jenkins jobs and confusion with contributors on which jobs apply where. Also, if there is ever a patch where Impala proper and query generator are both updated, which job wins? 2. Status quo and set Verified+1/Submitted by hand. This is much easier for committers than non-committers. I'm OK with status quo, but in the past, there have been requests to improve this situation [1] For a data point, I can cd to "tests/comparison/tests", run "impala-py.test", and 71 tests take about 10 seconds to run. Thanks for any feedback. [0] https://git-wip-us.apache.org/repos/asf?p=incubator-impala.git;a=tree;f=tests/comparison/tests;h=49e3b5d7d9a6f5f716c135bda36292e05fb0e0d3;hb=HEAD [1] https://issues.apache.org/jira/browse/IMPALA-4756
