> On Oct. 4, 2016, 6:36 a.m., Kai Huang wrote: > > I noticed that AuroraBot says the test is green but some coverage are > > missing... > > > > When I test the code locally, the code passes pants tests and style check > > in: > > ./build-support/jenkins/build.sh > > > > it also passes the e2e tests in: > > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > > > > However, I noticed some abnormalies in individual tests: > > > > The following test command is always green: > > ./pants test.pytest --options="-s -k test_thermos_executor" > > src/test/python/apache/aurora/executor:: > > > > However, the following test command with coverage flag turned-on is flaky > > for one test: test_health_check_ok. > > ./pants test.pytest --options="-s -k test_thermos_executor" > > --coverage=1 src/test/python/apache/aurora/executor:: > > > > This test is testing health check in a thermos executor with very small > > initial_interval(0.1 seconds) plus a fast StatusManager(Polling status > > every 10 milli_seconds). See > > https://github.com/apache/aurora/blob/master/src/test/python/apache/aurora/executor/test_thermos_executor.py#L216 > > and > > https://github.com/apache/aurora/blob/master/src/test/python/apache/aurora/executor/test_thermos_executor.py#L458 > > > > This may explain why AuroraBot says green tests but coverage are missing. > > > > I'm not sure if this is purely an artifact of the coverage flag or caused > > by my code change in the Aurora Executor. But I'm now concerned that the > > flakiness is not exposed by our pants test and e2e test. > > > > My guess is that it might be caused by my code > > change(https://github.com/apache/aurora/blob/master/src/main/python/apache/aurora/executor/aurora_executor.py#L120), > > and exacerbated by the --coverage flag. > > > > I was wondering if we should just proceed and ignore the warning or be more > > precautious and revisit executor change?
Currently the test case is doing a health check with initial_interval_secs=0.1 secs, interval_secs=0.1 secs. We can eliminate the flakiness by slightly increasing the initial_interval_secs in the test case (e.g. from 0.1 to 0.12 sec). But I believe that should be addressed in an separate review. - Kai ----------------------------------------------------------- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/52453/#review151303 ----------------------------------------------------------- On Oct. 4, 2016, 12:42 a.m., Kai Huang wrote: > > ----------------------------------------------------------- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/52453/ > ----------------------------------------------------------- > > (Updated Oct. 4, 2016, 12:42 a.m.) > > > Review request for Aurora, Joshua Cohen and Zameer Manji. > > > Bugs: AURORA-894 > https://issues.apache.org/jira/browse/AURORA-894 > > > Repository: aurora > > > Description > ------- > > - Add support for receiving a new HealthCheckConfig attribute > "min_consecutive_successes" in health checker. > - Add an entry in release note that describes the health check driven update > feature. > > This patch is related to https://reviews.apache.org/r/52094/, in which I > added a new configuration value "min_consecutive_successes" in > HealthCheckConfig. > > > Diffs > ----- > > RELEASE-NOTES.md 49c03e85ae4c2e3ebc8af89e9ce41df9fd52d6cd > src/main/python/apache/aurora/client/api/updater_util.py > c649316edb876565c92cc90c9f030e153c008924 > src/main/python/apache/aurora/executor/common/health_checker.py > 03fbffdc3862a94c2ba42c9b9e8f2be4094129b8 > > Diff: https://reviews.apache.org/r/52453/diff/ > > > Testing > ------- > > ./build-support/jenkins/build.sh > > ./pants test.pytest src/test/python/apache/aurora/executor:: > > ./src/test/sh/org/apache/aurora/e2e/test_end_to_end.sh > > > Thanks, > > Kai Huang > >
