Re: [Python-Dev] API for bugs.python.org or some statistic about lib/module merge rate?
Hi all, thanks for all your reply, since roundup didn't contain JSON REST API, and I'm not sure did roundup robot have deal with "patched but still open" situation, thus I do some change to bpo-mergerate and have some result. GitHub repo: https://github.com/lulouie/bpo-mergerate Demo site: https://bpo-mergerate.louie.lu Now bpo-mergerate using SQLite database and no more CSV file, and an update.py script was added, on my host, it will automatically update the bpo patch status hourly. For the site, it is now using ajax and can deal with fuzzy search. The merge rate calculate is now find-grained to per-file, and will automatically count the prefix merge rate, for example: Searching: Lib/test Result: path merged open totalrate /Lib/test 3013 1359 4372 68.916 Lib/test/test_ssl.py 60 15 75 80.000 Lib/test/test_inspect.py 56 12 68 82.353 I think the site function is complete now, welcome for any suggestion. (maybe like speed.python.org make a timeline for the merge rate?) Best Regards, Louie. 2017-04-24 12:08 GMT+08:00 Nick Coghlan : > On 21 April 2017 at 06:04, Wes Turner wrote: > > Roundup has an xmlrpc API: > > > > - https://sourceforge.net/p/roundup/code/ci/default/tree/ > roundup/xmlrpc.py > > - https://sourceforge.net/p/roundup/code/ci/default/tree/ > test/test_xmlrpc.py > > There's also a pending patch to add a HTTPS+JSON REST API to our > Roundup instance: > http://psf.upfronthosting.co.za/roundup/meta/issue581 > > Definite +1 for the general idea of collecting and publishing these > kinds of statistics, though - I can see them being quite useful to a > variety of audiences. > > Cheers, > Nick. > > P.S. The upstream Roundup project is also interested in offering that > capability by default: http://issues.roundup-tracker.org/issue2550734 > > -- > Nick Coghlan | ncogh...@gmail.com | Brisbane, Australia > ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API for bugs.python.org or some statistic about lib/module merge rate?
On Monday, April 24, 2017, Louie Lu wrote:
> Hi all,
>
> thanks for all your reply, since roundup didn't contain JSON REST API, and
> I'm not sure did roundup robot have deal with "patched but still open"
> situation, thus I do some change to bpo-mergerate and have some result.
>
> GitHub repo: https://github.com/lulouie/bpo-mergerate
> Demo site: https://bpo-mergerate.louie.lu
>
> Now bpo-mergerate using SQLite database and no more CSV file, and an
> update.py script was added, on my host, it will automatically update the
> bpo patch status hourly.
> For the site, it is now using ajax and can deal with fuzzy search.
>
> The merge rate calculate is now find-grained to per-file, and will
> automatically count the prefix merge rate, for example:
>
> Searching: Lib/test
> Result:
> path merged open totalrate
> /Lib/test 3013 1359 4372 68.916
> Lib/test/test_ssl.py 60 15 75 80.000
> Lib/test/test_inspect.py 56 12 68 82.353
>
>
> I think the site function is complete now, welcome for any suggestion.
> (maybe like speed.python.org make a timeline for the merge rate?)
>
Nice work.
Additional per-file factors worth visualizing:
- change in test coverage
- change in relevant docs.rst
- build job ids (which patch on which arch)
- benchmark timings
- git log --stat
Gource is a pretty cool way to visualize file changes by author over time:
http://gource.io
"Evolution of CPython (gource visualization)"
https://youtu.be/tZVG57xFwMk
Is there something like `hg churn` for git? I haven't had need; but just
found:
- https://www.mercurial-iscm.org/wiki/ChurnExtension
-
https://gist.github.com/aras-p/8289222 (cheat sheet)
Does ChurnExtension work with hg-git?
- https://github.com/arzzen/git-quick-stats
https://github.com/arzzen/git-quick-stats/blob/master/git-quick-stats
(bash script)
- https://github.com/IonicaBizau/git-stats
Json?
JSON-LD with the file path as the resource @id really would be great (and
mergeable from various build artifacts):
- https://www.google.com/search?q=elasticsearch+git+log
- westurner/pyrpo canould write streaming JSON from `git log` but not yet
`git log --stat`
- {SQLAlchemy(SQLite), } > JSON-LD reports with @id URIs for {source, docs}
file paths
> Best Regards,
> Louie.
>
> 2017-04-24 12:08 GMT+08:00 Nick Coghlan >:
>
>> On 21 April 2017 at 06:04, Wes Turner > > wrote:
>> > Roundup has an xmlrpc API:
>> >
>> > - https://sourceforge.net/p/roundup/code/ci/default/tree/round
>> up/xmlrpc.py
>> > - https://sourceforge.net/p/roundup/code/ci/default/tree/test/
>> test_xmlrpc.py
>>
>> There's also a pending patch to add a HTTPS+JSON REST API to our
>> Roundup instance:
>> http://psf.upfronthosting.co.za/roundup/meta/issue581
>>
>> Definite +1 for the general idea of collecting and publishing these
>> kinds of statistics, though - I can see them being quite useful to a
>> variety of audiences.
>>
>> Cheers,
>> Nick.
>>
>> P.S. The upstream Roundup project is also interested in offering that
>> capability by default: http://issues.roundup-tracker.org/issue2550734
>>
>> --
>> Nick Coghlan | ncogh...@gmail.com
>>| Brisbane,
>> Australia
>>
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request review of cProfile/profile series issue
On Sun, Apr 23, 2017 at 11:12 PM, Nick Coghlan wrote: > On 23 April 2017 at 13:21, Louie Lu wrote: > > Hi all, > > > > I'm now looking for cProfile/profile lib's issue, and have solve a > series of > > dependent problem, here is the list: > > > > #9285 - Add a profile decorator to profile and cProfile > %prun -s Src: https://github.com/ipython/ipython/blob/master/IPython/core/magics/execution.py#L187 Docs: https://ipython.readthedocs.io/en/stable/interactive/magics.html#magic-prun http://pynash.org/2013/03/06/timing-and-profiling/ %time, %timeit %prun, %lprun %mprun, %memit > > #30113 - Allow helper functions to wrap sys.setprofile > > #18971 - Use argparse in the profile/cProfile modules > > #30118 - Add unittest for cProfile/profile command line interface > > > > It can divide into two categories, first is the context manager problem, > and > > the second is optparse to argparse problem. > > > > 1. context manager problem: > > > > Relative issue: #9285, #30113 > > Relative PR: #287, #1212, #1253 > > > > This is an issue since 2010, and stop at profile can't simply add a > > __enter__ and __exit__ to make it a context manager. The main problem is, > > sys.setprofile() will hit the return and get bad return in profile > > dispatch_return function. The solution is to insert a simulate call in > the > > helper function, to provide the context between helper frame and where > the > > profile is defined. > > I'd like to second Louie's call for additional review here. I've been > reviewing them so far, and they look sensible to me, but I don't > personally know the profile code base at all, so there's a strong > chance I'll miss subtle details. > https://westurner.github.io/wiki/awesome-python-testing#benchmarks - plop, https://github.com/python/cpython/blob/master/Lib/profile.py https://github.com/python/cpython/blob/master/Lib/cProfile.py ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
