adoroszlai commented on pull request #1062: URL: https://github.com/apache/hadoop-ozone/pull/1062#issuecomment-643626366
Codecov seems to agree with you. ;) It will not compare against commits with failed CI (if it know that CI passed or failed). From the [doc](https://docs.codecov.io/docs/comparing-commits#commits-with-failed-ci): > If CI fails, one or more of the following assumptions can be made: > 1. Not all tests were executed; therefore coverage is incomplete. > 2. Exceptions may call new execution paths, resulting in different coverage metrics. > 3. A failed test could produce different coverage than the same test ran successfully. I think for Ozone: 1. is the worst problem, can happen if dependent modules are skipped by Maven due to earlier failure (eg. in `common`). 2. is true even for successful CI ([example](https://codecov.io/gh/apache/hadoop-ozone/compare/bd88eec9fec06a8f8adcbe4d257c5ccaca144f4c...96f07d5c27ba8f2ab9d77df25e10d4cb94489e5f/changes), where [only javadoc was updated](https://github.com/apache/hadoop-ozone/commit/96f07d5c27ba8f2ab9d77df25e10d4cb94489e5f)). 3. is something we should try to fix (eg. split test cases that perform a long series of steps, making assertions all the way). ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
