We already have code coverage integrated into the build. See the documentation on how to generate the reports: https://github.com/apache/lucene/blob/main/help/tests.txt
I think we should stick with jacoco and not some commercial stuff for measuring coverage. Jacoco works great. We just have to put the reports or stats somewhere useful. On Tue, Oct 4, 2022 at 5:45 PM Patrick Zhai <zhai7...@gmail.com> wrote: > > Hi Robert, thank you for commenting, yeah the functionality I want to add is > actually the line by line code coverage stats for the new/changed line that > are in the patch so that we don't need to wonder about "whether that line is > covered by the test?". But I'm against using the code coverage as any kind of > hard criteria, like coverage must be kept at a certain % or all the new lines > must be covered, that will drive people crazy. I think that should be just > treated as a helpful thing to check when reviewing/creating the PR. > > I searched a little on google and found this: https://about.codecov.io/, it's > free for open source and seems to have the functionality we need. Let me know > if anyone has ideas about this, or otherwise I can try it a little bit with > my own repo first and then try to add it to lucene. > > Best > Patrick > > > > On Tue, Oct 4, 2022, 06:36 Robert Muir <rcm...@gmail.com> wrote: >> >> btw, you can look at the current reports created by jenkins here: >> https://ci-builds.apache.org/job/Lucene/job/Lucene-Coverage-main/lastBuild/jacoco/ >> >> On Tue, Oct 4, 2022 at 6:51 AM Robert Muir <rcm...@gmail.com> wrote: >> > >> > we can run the tests with coverage option and produce coverage graph >> > from the github actions, but need to look at the docs to see where to >> > put it so it will be available. >> > >> > I want us to be careful about the word "check" as I'm adamantly >> > against any such automated check (e.g. coverage > N%) in the logic. >> > Coverage report is just a tool to help us and the moment we do stupid >> > shit like that, is the moment people start gaming it just to make the >> > build pass. >> > >> > On Mon, Oct 3, 2022 at 10:57 PM Patrick Zhai <zhai7...@gmail.com> wrote: >> > > >> > > Hi folks, >> > > I'm not sure whether people have already discussed this but I'm >> > > wondering whether we want to add a workflow that pulls out the code >> > > coverage whenever a PR was created? It should be easier for both the >> > > reviewers and the contributors to figure out what can be improved, or at >> > > least figure out a part that is probably not covered by the tests? >> > > >> > > Best >> > > Patrick >> >> --------------------------------------------------------------------- >> To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org >> For additional commands, e-mail: dev-h...@lucene.apache.org >> --------------------------------------------------------------------- To unsubscribe, e-mail: dev-unsubscr...@lucene.apache.org For additional commands, e-mail: dev-h...@lucene.apache.org
