Re: [Python-Dev] We now have C code coverage!

[Hobson Lane]

`  # noqa` works with linters

--Hobson


On Fri, Jun 29, 2018 at 6:25 AM, Brett Cannon  wrote:

>
>
> On Thu, Jun 28, 2018, 21:28 Terry Reedy,  wrote:
>
>> On 6/24/2018 5:03 AM, Ammar Askar wrote:
>> >> Is it possible, given that we are not paying for those reports, to
>> >> customize the 'exclude_lines' definitions?
>> >
>> > Do you want to exclude python code or C code?
>>
>> Python code.
>>
>> > For Python code, coverage.py also has some comments you can
>> > put down to exclude lines:
>> > http://coverage.readthedocs.io/en/coverage-4.2/excluding.html
>>
>> Yes, by default, one can use '# pragma: no cover' and if one uses the
>> --branch flag, '# pragma: no branch'.  For more 'advanced exclusion',
>> one can use the following, normally in .coveragerc.
>> [report]
>> exclude_lines = ...
>> "This is useful if you have often-used constructs to exclude that can be
>> matched with a regex. You can exclude them all at once without littering
>> your code with exclusion pragmas."
>>
>> For IDLE's test suite, I use a customized .coveragerc.  I strongly
>> prefer to not abandon that and litter the code with # pragmas.
>>
>> In order to make sense of the coverage report and have it be truthful,
>> one needs to know what options are being used.
>> Is the --branch flag set?
>> Is .coveragerc or some other configuration file in use?
>> If so, what is the content?
>> Do we have any control over the use and content of exclusion settings?
>>
>
> Everything is either covered by the Travis or codecov configuration files
> which are both checked into the cpython repo. (I'm on vacation or else I
> would provide links to the files themselves.)
>
>
>
>> --
>> Terry Jan Reedy
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
>> brett%40python.org
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> hobsonlane%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Ammar Askar]

Oh whoops, sorry about that. I haven't really used mailing lists
before and so I assumed hitting reply in gmail would send it to
python-dev, not just your personal email. Just so the config file
locations are publicly documented, here's what I responded with:

> For IDLE's test suite, I use a customized .coveragerc.  I strongly prefer to 
> not abandon that and litter the code with # pragmas.


Yup, I agree completely. Having pragmas everywhere is really annoying,
its really only useful for small one-off stuff.

> In order to make sense of the coverage report and have it be truthful, one 
> needs to know what options are being used.
> Is the --branch flag set?
> Is .coveragerc or some other configuration file in use?
> If so, what is the content?
> Do we have any control over the use and content of exclusion settings?


To answer all these questions at once, yeah, we have complete control
over all the coverage parameters. Currently we aren't building with
`--branch` and don't have a coveragrc file. We could put one in the
root of CPython (which would get picked up automatically) but
personally I think CI related crud should go into its own folder.
Luckily we do have a ".github" folder, so I'd suggest putting the
coveragerc file in there and then adding the parameter
`--rcfile=.github/coveragerc`

All the parameters to coverage.py can be configured here
https://github.com/python/cpython/blob/master/.travis.yml#L82

If you wanna send over your IDLE coveragerc file, I can experiment and
try to get it working for Travis, or you can explore it yourself if
you want.

On Fri, Jun 29, 2018 at 9:14 AM, Terry Reedy  wrote:
> On 6/29/2018 9:25 AM, Brett Cannon wrote:
>
>> On Thu, Jun 28, 2018, 21:28 Terry Reedy, > > wrote:
>
> [question about our coverage bot]
>
>> Everything is either covered by the Travis or codecov configuration files
>> which are both checked into the cpython repo. (I'm on vacation or else I
>> would provide links to the files themselves.)
>
>
> Ammar Askar privately asked for my coveragerc file so he could experiment
> with the configuration.
>
> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ammar%40ammaraskar.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Terry Reedy]

On 6/29/2018 9:25 AM, Brett Cannon wrote:

On Thu, Jun 28, 2018, 21:28 Terry Reedy, > wrote:

[question about our coverage bot]

Everything is either covered by the Travis or codecov configuration 
files which are both checked into the cpython repo. (I'm on vacation or 
else I would provide links to the files themselves.)


Ammar Askar privately asked for my coveragerc file so he could 
experiment with the configuration.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Brett Cannon]

On Thu, Jun 28, 2018, 21:28 Terry Reedy,  wrote:

> On 6/24/2018 5:03 AM, Ammar Askar wrote:
> >> Is it possible, given that we are not paying for those reports, to
> >> customize the 'exclude_lines' definitions?
> >
> > Do you want to exclude python code or C code?
>
> Python code.
>
> > For Python code, coverage.py also has some comments you can
> > put down to exclude lines:
> > http://coverage.readthedocs.io/en/coverage-4.2/excluding.html
>
> Yes, by default, one can use '# pragma: no cover' and if one uses the
> --branch flag, '# pragma: no branch'.  For more 'advanced exclusion',
> one can use the following, normally in .coveragerc.
> [report]
> exclude_lines = ...
> "This is useful if you have often-used constructs to exclude that can be
> matched with a regex. You can exclude them all at once without littering
> your code with exclusion pragmas."
>
> For IDLE's test suite, I use a customized .coveragerc.  I strongly
> prefer to not abandon that and litter the code with # pragmas.
>
> In order to make sense of the coverage report and have it be truthful,
> one needs to know what options are being used.
> Is the --branch flag set?
> Is .coveragerc or some other configuration file in use?
> If so, what is the content?
> Do we have any control over the use and content of exclusion settings?
>

Everything is either covered by the Travis or codecov configuration files
which are both checked into the cpython repo. (I'm on vacation or else I
would provide links to the files themselves.)



> --
> Terry Jan Reedy
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Terry Reedy]

On 6/24/2018 5:03 AM, Ammar Askar wrote:

Is it possible, given that we are not paying for those reports, to
customize the 'exclude_lines' definitions?


Do you want to exclude python code or C code?


Python code.


For Python code, coverage.py also has some comments you can
put down to exclude lines:
http://coverage.readthedocs.io/en/coverage-4.2/excluding.html


Yes, by default, one can use '# pragma: no cover' and if one uses the 
--branch flag, '# pragma: no branch'.  For more 'advanced exclusion', 
one can use the following, normally in .coveragerc.

[report]
exclude_lines = ...
"This is useful if you have often-used constructs to exclude that can be 
matched with a regex. You can exclude them all at once without littering 
your code with exclusion pragmas."


For IDLE's test suite, I use a customized .coveragerc.  I strongly 
prefer to not abandon that and litter the code with # pragmas.


In order to make sense of the coverage report and have it be truthful, 
one needs to know what options are being used.

Is the --branch flag set?
Is .coveragerc or some other configuration file in use?
If so, what is the content?
Do we have any control over the use and content of exclusion settings?

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Dima Tisnek]

This is awesome, thank you Ammar!

On 23 June 2018 at 06:21, Brett Cannon  wrote:

> Thanks to a PR from Ammar Askar we now run Python under lcov as part of
> the code coverage build. And thanks to codecov.io automatically merging
> code coverage reports we get a complete report of our coverage (the first
> results of which can now be seen at https://codecov.io/gh/python/cpython).
>
> And funny enough the coverage average changed less than 1%. :)
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> dimaqq%40gmail.com
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Ammar Askar]

> Is it possible, given that we are not paying for those reports, to
> customize the 'exclude_lines' definitions?

Do you want to exclude python code or C code?

For C code you can mark sections that exclude coverage in lcov
with comments like "LCOV_EXCL_START"
http://ltp.sourceforge.net/coverage/lcov/geninfo.1.php

For Python code, coverage.py also has some comments you can
put down to exclude lines:
http://coverage.readthedocs.io/en/coverage-4.2/excluding.html

If you don't need line level granularity, you can always add files to
ignore in our codecov.yml file:
https://docs.codecov.io/docs/ignoring-paths
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Terry Reedy]

On 6/23/2018 5:48 PM, Zachary Ware wrote:

On Sat, Jun 23, 2018 at 2:20 PM, Terry Reedy  wrote:

Rechecking now, on Gentoo

test_idle appears and passed on these 3.6 and 3.7 pages
http://buildbot.python.org/all/#/builders/82/builds/414/steps/5/logs/stdio

Neither Firefox nor Edge can find 'test_idle' on these 3.x pages
http://buildbot.python.org/all/#/builders/103/builds/1149/steps/5/logs/stdio
http://buildbot.python.org/all/#/builders/99/builds/1130/steps/4/logs/stdio

test_tk appears on 1130 but not 1149

For your 8.1 bot: test_idle passed for 3.7
http://buildbot.python.org/all/#/builders/133/builds/339/steps/3/logs/stdio

but does is not found on this 3.x page (neither is 'test_tk')
http://buildbot.python.org/all/#/builders/12/builds/991/steps/3/logs/stdio


Click the magnifying glass icon ("load all data for use with the
browser search tool") at the upper right of the console pane and try
again; each of the above is present and passed.  This does
unfortunately seem to be another case of non-intuitive UI from
buildbot.


Presenting data on the screen that cannot be found is terrible.  With 
Edge, I had to erase and retype the search word also.  With Firefox, 
that sometimes worked without pressing the magnifier.  I thought my copy 
of Firefox was broken until I tried Edge also.



Both Appveyor and my machine run test_idle when running the full 3.x test
suite.


I am pleasantly surprised that AppVeyor does appear to be running the
GUI tests :)




--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Zachary Ware]

On Sat, Jun 23, 2018 at 2:20 PM, Terry Reedy  wrote:
> Rechecking now, on Gentoo
>
> test_idle appears and passed on these 3.6 and 3.7 pages
> http://buildbot.python.org/all/#/builders/82/builds/414/steps/5/logs/stdio
>
> Neither Firefox nor Edge can find 'test_idle' on these 3.x pages
> http://buildbot.python.org/all/#/builders/103/builds/1149/steps/5/logs/stdio
> http://buildbot.python.org/all/#/builders/99/builds/1130/steps/4/logs/stdio
>
> test_tk appears on 1130 but not 1149
>
> For your 8.1 bot: test_idle passed for 3.7
> http://buildbot.python.org/all/#/builders/133/builds/339/steps/3/logs/stdio
>
> but does is not found on this 3.x page (neither is 'test_tk')
> http://buildbot.python.org/all/#/builders/12/builds/991/steps/3/logs/stdio

Click the magnifying glass icon ("load all data for use with the
browser search tool") at the upper right of the console pane and try
again; each of the above is present and passed.  This does
unfortunately seem to be another case of non-intuitive UI from
buildbot.

> Both Appveyor and my machine run test_idle when running the full 3.x test
> suite.

I am pleasantly surprised that AppVeyor does appear to be running the
GUI tests :)

-- 
Zach
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Terry Reedy]

On 6/23/2018 1:09 PM, Zachary Ware wrote:

On Sat, Jun 23, 2018 at 11:31 AM, Terry Reedy  wrote:

I have suggested that, and before that, the same for buildbots.  The reality
is that tkinter, IDLE, or turtle could be disabled on *nix by regressions
and the official testing would not notice.


I'm looking into enabling the GUI tests on some of the CI hosts today,
not sure how far I'll make it with that.  GUI tests are currently
enabled on my Gentoo [1] and Windows [2] builders, and have been for a
couple of years at this point; I'm not sure if any other builders are
running GUI tests.


I noticed your Gentoo builders running gui tests some years ago.  When I 
re-checked perhaps a year ago, they either were not running or seem to 
have skipped test_idle, maybe the former.



[1] http://buildbot.python.org/all/#/workers/6
[2] http://buildbot.python.org/all/#/workers/11


Rechecking now, on Gentoo

test_idle appears and passed on these 3.6 and 3.7 pages
http://buildbot.python.org/all/#/builders/82/builds/414/steps/5/logs/stdio

Neither Firefox nor Edge can find 'test_idle' on these 3.x pages
http://buildbot.python.org/all/#/builders/103/builds/1149/steps/5/logs/stdio
http://buildbot.python.org/all/#/builders/99/builds/1130/steps/4/logs/stdio

test_tk appears on 1130 but not 1149

For your 8.1 bot: test_idle passed for 3.7
http://buildbot.python.org/all/#/builders/133/builds/339/steps/3/logs/stdio

but does is not found on this 3.x page (neither is 'test_tk')
http://buildbot.python.org/all/#/builders/12/builds/991/steps/3/logs/stdio

Both Appveyor and my machine run test_idle when running the full 3.x 
test suite.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Ivan Pozdeev via Python-Dev]

On 23.06.2018 13:52, Paul Moore wrote:

On 22 June 2018 at 23:21, Brett Cannon  wrote:

Thanks to a PR from Ammar Askar we now run Python under lcov as part of the
code coverage build. And thanks to codecov.io automatically merging code
coverage reports we get a complete report of our coverage (the first results
of which can now be seen at https://codecov.io/gh/python/cpython).

And funny enough the coverage average changed less than 1%. :)

Nice!

One thing I noticed, code that's Windows-specific isn't covered. I
assume that's because the coverage reports are based on runs of the
test suite on Linux. Is it possible to merge in data from the Windows
test runs? If not, what's the best way to address this? Should we be
mocking things to attempt to test Windows-specific code even on Linux,
or should we simply accept that we're not going to achieve 100%
coverage and not worry about it?


AFAICS lcov is based on gcov which is GCC-specific.


Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/vano%40mail.mipt.ru


--
Regards,
Ivan

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Zachary Ware]

On Sat, Jun 23, 2018 at 11:31 AM, Terry Reedy  wrote:
> I have suggested that, and before that, the same for buildbots.  The reality
> is that tkinter, IDLE, or turtle could be disabled on *nix by regressions
> and the official testing would not notice.

I'm looking into enabling the GUI tests on some of the CI hosts today,
not sure how far I'll make it with that.  GUI tests are currently
enabled on my Gentoo [1] and Windows [2] builders, and have been for a
couple of years at this point; I'm not sure if any other builders are
running GUI tests.

[1] http://buildbot.python.org/all/#/workers/6
[2] http://buildbot.python.org/all/#/workers/11

-- 
Zach
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Terry Reedy]

On 6/22/2018 9:40 PM, Nathaniel Smith wrote:

On Fri, Jun 22, 2018 at 6:16 PM, Terry Reedy  wrote:

I discovered the answer to 2. by shift-clicking on a text_x file to see
their coverage report for the file.  The colors actually do reflect the test
lines executed.  codecov.io excludes gui tests*, so the reported coverage
for tkinter, idlelib, and turtle is deceptive and bogus, and under-reports
the total cpython coverage by a percent or two.  It would be better to
exclude these modules.



* I assume that codecov.io uses linux servers.  I have read that there are
programs that simulate X-Windows so that gui code will execute without
actual terminals.


Codecov.io doesn't run any tests itself; it's just a service for
aggregation and reporting. The coverage information is being gathered
while running CPython's regular CI tests, and then uploaded to
codecov.io to view.


Thank you for the information.


So if you want to run the gui tests -- which seems like a good idea if
possible! -- then the way to do that would be to make them run as part
of the regular Travis/Appveyor/VSTS checks.


I have suggested that, and before that, the same for buildbots.  The 
reality is that tkinter, IDLE, or turtle could be disabled on *nix by 
regressions and the official testing would not notice.


--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Paul Moore]

On 22 June 2018 at 23:21, Brett Cannon  wrote:
> Thanks to a PR from Ammar Askar we now run Python under lcov as part of the
> code coverage build. And thanks to codecov.io automatically merging code
> coverage reports we get a complete report of our coverage (the first results
> of which can now be seen at https://codecov.io/gh/python/cpython).
>
> And funny enough the coverage average changed less than 1%. :)

Nice!

One thing I noticed, code that's Windows-specific isn't covered. I
assume that's because the coverage reports are based on runs of the
test suite on Linux. Is it possible to merge in data from the Windows
test runs? If not, what's the best way to address this? Should we be
mocking things to attempt to test Windows-specific code even on Linux,
or should we simply accept that we're not going to achieve 100%
coverage and not worry about it?

Paul
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Nathaniel Smith]

On Fri, Jun 22, 2018 at 6:16 PM, Terry Reedy  wrote:
> On 6/22/2018 8:43 PM, Terry Reedy wrote:
>>
>> On 6/22/2018 6:21 PM, Brett Cannon wrote:
>>>
>>> Thanks to a PR from Ammar Askar we now run Python under lcov as part of
>>> the code coverage build. And thanks to codecov.io 
>>> automatically merging code coverage reports we get a complete report of our
>>> coverage (the first results of which can now be seen at
>>> https://codecov.io/gh/python/cpython).
>>>
>>> And funny enough the coverage average changed less than 1%. :)
>>
>>
>> Questions:
>>
>> 1. Is it possible, given that we are not paying for those reports, to
>> customize the .coveragerc exclude_lines definitions?  Without such, the
>> idlelib measures are biased downward.
>>
>> 2. What do the colors of test files mean?  Every line of nearly all the
>> idlelib test files are executed, but over half are red.
>>
>> The Learn More page does not say anything about either.
>
>
> I discovered the answer to 2. by shift-clicking on a text_x file to see
> their coverage report for the file.  The colors actually do reflect the test
> lines executed.  codecov.io excludes gui tests*, so the reported coverage
> for tkinter, idlelib, and turtle is deceptive and bogus, and under-reports
> the total cpython coverage by a percent or two.  It would be better to
> exclude these modules.
>
> * I assume that codecov.io uses linux servers.  I have read that there are
> programs that simulate X-Windows so that gui code will execute without
> actual terminals.

Codecov.io doesn't run any tests itself; it's just a service for
aggregation and reporting. The coverage information is being gathered
while running CPython's regular CI tests, and then uploaded to
codecov.io to view.

So if you want to run the gui tests -- which seems like a good idea if
possible! -- then the way to do that would be to make them run as part
of the regular Travis/Appveyor/VSTS checks.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Terry Reedy]

On 6/22/2018 8:43 PM, Terry Reedy wrote:

On 6/22/2018 6:21 PM, Brett Cannon wrote:
Thanks to a PR from Ammar Askar we now run Python under lcov as part 
of the code coverage build. And thanks to codecov.io 
 automatically merging code coverage reports we get 
a complete report of our coverage (the first results of which can now 
be seen at https://codecov.io/gh/python/cpython).


And funny enough the coverage average changed less than 1%. :)


Questions:

1. Is it possible, given that we are not paying for those reports, to 
customize the .coveragerc exclude_lines definitions?  Without such, the idlelib 
measures are biased downward.


2. What do the colors of test files mean?  Every line of nearly all the 
idlelib test files are executed, but over half are red.


The Learn More page does not say anything about either.


I discovered the answer to 2. by shift-clicking on a text_x file to see 
their coverage report for the file.  The colors actually do reflect the 
test lines executed.  codecov.io excludes gui tests*, so the reported 
coverage for tkinter, idlelib, and turtle is deceptive and bogus, and 
under-reports the total cpython coverage by a percent or two.  It would 
be better to exclude these modules.


* I assume that codecov.io uses linux servers.  I have read that there 
are programs that simulate X-Windows so that gui code will execute 
without actual terminals.


--
Terry Jan Reedy


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] We now have C code coverage!

[Terry Reedy]

On 6/22/2018 6:21 PM, Brett Cannon wrote:
Thanks to a PR from Ammar Askar we now run Python under lcov as part of 
the code coverage build. And thanks to codecov.io  
automatically merging code coverage reports we get a complete report of 
our coverage (the first results of which can now be seen at 
https://codecov.io/gh/python/cpython).


And funny enough the coverage average changed less than 1%. :)


Questions:

1. Is it possible, given that we are not paying for those reports, to 
customize the 'exclude_lines' definitions?  Without such, the idlelib 
measures are biased downward.


2. What do the colors of test files mean?  Every line of nearly all the 
idlelib test files are executed, but over half are red.


The Learn More page does not say anything about either.

--
Terry Jan Reedy

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] We now have C code coverage!

[Brett Cannon]

Thanks to a PR from Ammar Askar we now run Python under lcov as part of the
code coverage build. And thanks to codecov.io automatically merging code
coverage reports we get a complete report of our coverage (the first
results of which can now be seen at https://codecov.io/gh/python/cpython).

And funny enough the coverage average changed less than 1%. :)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com