One thing codecov gives is a sense of what the coverage was previously without having to go hunt down past builds in jenkins. It is a coverage focused view essentially. It just uses the coverage data the build already calculates. It used to have some nifty graphical visualizations but those seem to have disappeared or become hard to find.
On Wed, Oct 5, 2022 at 9:24 AM Robert Muir <rcm...@gmail.com> wrote: > I would recommend a search for "github actions jacoco" to review > what's common out there. > > If we change 'gradle test' to 'gradle coverage' in our existing > PR-test action, the next step is to just not throw away the reports, > but make them available. See > > https://docs.github.com/en/actions/using-workflows/storing-workflow-data-as-artifacts > for some documentation on this. > > Seems common for PR workflows to have the action "comment on the PR" > with coverage information. Not sure if we want that as it could result > in a ton of comments. > > Finally, the current "gradle coverage" builds a separate coverage > report for each lucene module, I think. So we may want to think about > adding support to "merge" the jacoco data across all the modules and > build one monster report for all of lucene, too. This would just be > some work with the gradle build: but I think it would make the > information a lot easier to digest. This is already happening with the > "jenkins coverage build" which presents one monster report, but I > think it may be something on the jenkins side doing it? > > https://ci-builds.apache.org/job/Lucene/job/Lucene-Coverage-main/lastBuild/jacoco/ > > > On Wed, Oct 5, 2022 at 8:58 AM Patrick Zhai <zhai7...@gmail.com> wrote: > > > > Make sense to me, I'll try to look into it! > > > > On Tue, Oct 4, 2022, 16:50 Robert Muir <rcm...@gmail.com> wrote: > >> > >> We already have code coverage integrated into the build. See the > >> documentation on how to generate the reports: > >> https://github.com/apache/lucene/blob/main/help/tests.txt > >> > >> I think we should stick with jacoco and not some commercial stuff for > >> measuring coverage. Jacoco works great. We just have to put the > >> reports or stats somewhere useful. > >> > >> On Tue, Oct 4, 2022 at 5:45 PM Patrick Zhai <zhai7...@gmail.com> wrote: > >> > > >> > Hi Robert, thank you for commenting, yeah the functionality I want to > add is actually the line by line code coverage stats for the new/changed > line that are in the patch so that we don't need to wonder about "whether > that line is covered by the test?". But I'm against using the code coverage > as any kind of hard criteria, like coverage must be kept at a certain % or > all the new lines must be covered, that will drive people crazy. I think > that should be just treated as a helpful thing to check when > reviewing/creating the PR. > >> > > >> > I searched a little on google and found this: > https://about.codecov.io/, it's free for open source and seems to have > the functionality we need. Let me know if anyone has ideas about this, or > otherwise I can try it a little bit with my own repo first and then try to > add it to lucene. > >> > > >> > Best > >> > Patrick > >> > > >> > > >> > > >> > On Tue, Oct 4, 2022, 06:36 Robert Muir <rcm...@gmail.com> wrote: > >> >> > >> >> btw, you can look at the current reports created by jenkins here: > >> >> > https://ci-builds.apache.org/job/Lucene/job/Lucene-Coverage-main/lastBuild/jacoco/ > >> >> > >> >> On Tue, Oct 4, 2022 at 6:51 AM Robert Muir <rcm...@gmail.com> wrote: > >> >> > > >> >> > we can run the tests with coverage option and produce coverage > graph > >> >> > from the github actions, but need to look at the docs to see where > to > >> >> > put it so it will be available. > >> >> > > >> >> > I want us to be careful about the word "check" as I'm adamantly > >> >> > against any such automated check (e.g. coverage > N%) in the logic. > >> >> > Coverage report is just a tool to help us and the moment we do > stupid > >> >> > shit like that, is the moment people start gaming it just to make > the > >> >> > build pass. > >> >> > > >> >> > On Mon, Oct 3, 2022 at 10:57 PM Patrick Zhai <zhai7...@gmail.com> > wrote: > >> >> > > > >> >> > > Hi folks, > >> >> > > I'm not sure whether people have already discussed this but I'm > wondering whether we want to add a workflow that pulls out the code > coverage whenever a PR was created? It should be easier for both the > reviewers and the contributors to figure out what can be improved, or at > least figure out a part that is probably not covered by the tests? > >> >> > > > >> >> > > Best > >> >> > > Patrick > >> >> > >> >> --------------------------------------------------------------------- > >> >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > >> >> For additional commands, e-mail: dev-h...@lucene.apache.org > >> >> > >> > >> --------------------------------------------------------------------- > >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > >> For additional commands, e-mail: dev-h...@lucene.apache.org > >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org > For additional commands, e-mail: dev-h...@lucene.apache.org > > -- http://www.needhamsoftware.com (work) http://www.the111shift.com (play)
