https://github.com/JuliaLang/julia/issues/7541
On Friday, August 08, 2014 02:23:22 AM Magnus Lie Hetland wrote: > How is --code-coverage supposed to work, exactly? In my code, I seem to > generally either get nonzero counts or "-", marking lines that *shouldn't* > be counted? Or *couldn't*? Either way, I'm not really getting any > information about the percentage of code covered, unless I manually inspect > it and scratch my head (to figure out which functions have been inlined, > and thus are covered anyway, etc.) -- only how many times some of the > covered lines were called. (I actually also sometimes get some lines that > aren't covered, but then I re-run my deterministic tests, and then they are > covered. Which is odd.) > > That is, I get the dash marks for my types, so I won't know whether they > are instantiated, for example. I get them for functions that are never > called. I get them for functions that *are* called, but that have been > inlined. Granted, if I have only *parts* of a function that isn't executed, > the coverage analysis does seem to do its job admirably (even though I do > seem to get random fluctuations there, too? Maybe just my testing code > that's doing weird stuff.) > > Is this how it's supposed to be? How it must be, perhaps? If so, how does > Coverage.jl work? From the code, as far as I can see it collects counts, > along with nothing for the dash-marked lines. Now … if the coverage will > give you "-" for lines that are actually covered, as well as lines that are > not covered (and thereby generally only nonzero counts for the remaining > lines), how are you supposed to get the correct percentage from, say, > coveralls.io? Is it, perhaps, the case that it's silently ignoring these > issues (the instantiated/uninstantiated types, the called/inlined and > uncalled functions that are all marked with dashes)? Is this a bug, or > should it perhaps just be clearly marked as part of the semantics of the > coverage analysis? Perhaps it would be possible to somehow flag functions > that are marked with "-", so we can try to figure out whether it was never > called, or whether it was just inlined? > > Or maybe I'm just going about this the wrong way. What I want is basically > something equivalent to the 100% coverage switch of dlang, where you simply > get an error if you haven't covered everything. I've implemented that in my > tests, but … the results aren't really telling me that much, given the > current semantics of the coverage :-}
