ktmud opened a new pull request #13329: URL: https://github.com/apache/superset/pull/13329
### SUMMARY Optimize CodeCov config to 1. Always let GitHub CodeCov checks pass because we don't always report all test coverage (because of [conditional CI jobs](https://github.com/apache/superset/pull/12583)) 2. Add more detailed flags for coverage [uploads](https://codecov.io/gh/apache/superset/commit/a700205d75abf8ad628e063c3bbf12c8625330fc/build) It seems the CodeCov App has recently been enabled in this repo (#9930), which is good, but it starts to block block some PRs on status check due to false alarms of decreased test coverage. This is because: 1. The target coverage setting is set to "auto", which will report any decrease in test coverage 2. We conditionally skip Python tests for pure frontend changes (and vice versa), hence certain PRs will never have the full test coverage reported. 3. Sometimes frontend and backend workflows are delayed in queue and run in very different time. After wait for a while, CodeCov may come to believe the full CI has finished, therefore reporting only partial coverage from the already finished jobs. The [`after_n_builds`](https://docs.codecov.io/docs/notifications) config is designed to address this issue, but since we don't know which jobs will run for each PR, we can't really set a meaningful value for this config. I'm setting it to 4 because for Python tests there are 5 uploads (mysql, postgres, sqlite, hive, presto), and for frontend, there are 4 ([unit tests](https://github.com/apache/superset/blob/59d4e8759a0490e6b5bca8cfa87cbf1191b486d2/.github/workflows/superset-frontend.yml#L48) + Cypress * 3). ### BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF N/A ### TEST PLAN - CI Passes and flags correctly marked on CodeCov <img width="1166" alt="commit-flags" src="https://user-images.githubusercontent.com/335541/109105029-f9775a80-76e1-11eb-8147-7859b9f0cba7.png";> - GH checks are not blocked by CodeCov ### ADDITIONAL INFORMATION <!--- Check any relevant boxes with "x" --> <!--- HINT: Include "Fixes #nnn" if you are fixing an existing issue --> - [ ] Has associated issue: - [ ] Changes UI - [ ] Requires DB Migration. - [ ] Confirm DB Migration upgrade and downgrade tested. - [ ] Introduces new feature or API - [ ] Removes existing feature or API ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
![https://user-images.githubusercontent.com/335541/109105029-f9775a80-76e1-11eb-8147-7859b9f0cba7.png"](https://user-images.githubusercontent.com/335541/109105029-f9775a80-76e1-11eb-8147-7859b9f0cba7.png"){kind=link}