Re: [Python-Dev] Tracker Stats
Hi, I added a new "stats" page to the bug tracker: http://bugs.python.org/issue?@template=stats Thanks Ezio, Two questions: how hard would be to add (or enhance) a chart with the “open issues type enhancement” and “open issues type bug” info ? In the summaries there is a link to “Issues with patch”, means that the ones not listed there are in “needs patch” or “new” status? Regards, francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tracker Stats
On 06/24/2014 03:50 AM, Ezio Melotti wrote: From the first graph you can see that out of the 4500+ open issues, about 2000 have a patch. One would like to start with the ones that are bugs ;-) and see some status line trying to drop to 0 (is that possible :-) ?) We need more reviewers and committers :) more patch writers: yes, more patch reviewers: yes, more committers: ?? automate!! :-) Regards, francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Tracker Stats
On 06/23/2014 10:12 PM, R. David Murray wrote: The stats graphs are based on the data generated for the weekly issue report. I have a patched version of that report that adds the bug/enhancement info. I'll try to dig it up this week; someone ping me if I forget :) It think the patch will need to be updated based on Ezio's changes. ping ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 4000 to explicitly declare we won't be doing a Py3k style compatibility break again?
On 08/17/2014 03:28 AM, Nick Coghlan wrote: I've seen a few people on python-ideas express the assumption that there will be another Py3k style compatibility break for Python 4.0. I've also had people express the concern that "you broke compatibility in a major way once, how do we know you won't do it again?". Why not just allow those changes that can be automatically changed by a tool/script applied on the code (a la go, 2to3, 3.Ato3.B, ...)? ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Sad status of Python 3.x buildbots
[...] But it does mean a measure of trust in some external entity, or else some very careful rules (mainly firewall), which not every coder will know about. Just curious, is there a way to mount the infrastructure the oder way around? One sets a system polling for sources changes, if so it starts a build an sends per e-mail the results to some address. Thanks in advance! francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Workflow improvement PEPs 474 & 462 updated
Hi Nick, On 02/01/2015 08:46 AM, Nick Coghlan wrote: [...] > The updates to PEP 462, which covers proposed changes to the main > CPython workflow, were more significant, as I've now rewritten that to > depend on PEP 474, and propose replacing the current Rietveld patch > review workflow with an updated approach based on Kallithea and Zuul: > https://www.python.org/dev/peps/pep-0462/ > A small question on: """ Push races would also be a thing of the past - if lots of core developers are approving patches at a sprint, then that just means the queue gets deeper in Zuul, rather than developers getting frustrated trying to merge changes and failing. """ How does the Tool Zuul resolve the case where the patches are not fully compatible. E.g. they touch the same file and some manually merging is needed? (Isn't that a push race? or I'm missing something?) And one minor detail. Early on "Current Tools": """ This proposal suggests replacing the use of Rietveld for code review with the more full-featured Kallithea-based forge.python.org service proposed in PEP 474 . """ and then later on "Perceived Benefits": """ the merge queue would allow that developer to focus more of their time on reviewing patches and helping the other contributors at the sprint, since accepting a patch for inclusion would now be a single click in the Rietveld UI, rather [...] """ Isn't "Kallithea UI" meant here? And +1 for self-hosting on: """ This proposal respects that history by recommending only tools that are available for self-hosting as sponsored or PSF funded infrastructure, and are also open source Python projects that can be customised to meet the needs of the CPython core development team """ I like the PEP. PS: Should this be forwarded to python-workflow or is that other list to be considered obsolete? Regards, francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Request for comments: [issue22941] IPv4Interface arithmetic changes subnet mask
Hi, > I, too, was thinking /24. I think that overflowing the host portion > should raise OverflowError. > Just curiosity, why not a modulo calculation on the subnet instead of raising the error? Thanks in advance! francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] (ctypes example) libffi embedded in CPython
Hi
>
> It might be a matter of taste, but I don't find declaring C functions
> any more awkward than using strange interface that ctypes comes with.
> the equivalent in cffi would be ffi.cast("double (*)()", x)
- Could you please elaborate on "using strange interface"?
- Is there an easy way to convert the current code using ctypes to
cffi?
Thanks in advance!
francis
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Accepting PEP 3154 for 3.4?
Food for thought: maybe we should have variable-encoding lengths for all opcodes, rather than the current cumbersome scheme? Funny, it sounds like UTF-8 :-) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Fwd: Python 2.x and 3.x usage survey
There are an enormous number of people that use pyton to simply get work done, and many, many, of those are still using only 2, if only because of inertia. I agree, but I expected the question: „Do you plan to write/port some of you Python 2.x code to Python 3.x next year“ (at work) Regards, francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] lambda (x, y):
On 01/26/2014 03:42 AM, Raymond Hettinger wrote: I think that is an over-simplification. The argument unpacking was handy in a number of situations where *args wouldn't suffice: lambda (px, py), (qx, qy): ((px - qx) ** 2 + (py - qy) ** 2) ** 0.5 IIRC, the original reason for the change was that it simplified the compiler a bit, not that it was broken or not useful. I'm not sure if that's applicable or other issues arise with: def fn(*p): px,py,qx,qy = p; return ((px - qx) ** 2 + (py - qy) ** 2) ** 0.5 Thanks in advance! Francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Summary of Python tracker Issues
On 02/07/2014 06:07 PM, Python tracker wrote: Open issues with patches: 2045 Has somebody done a graphic of that data againsttime? Regards, francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] PEP 465: A dedicated infix operator for matrix multiplication
> > So, I guess as far as I'm concerned, this is ready to go. Feedback welcome: > http://legacy.python.org/dev/peps/pep-0465/ > Hi, just curiosity: why is the second parameter 'o2' in: PyObject* PyObject_MatrixMultiply(PyObject *o1, PyObject o2) not a pointer to PyObject? Thanks in advance! francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue6839] zipfile can't extract file
On 04/30/2014 04:45 AM, Adam Polkosnik wrote: > Gentlemen, > On 04/30/2014 04:48 AM, Jessica McKellar wrote: Hi Adam, Gentlemen, Thanks for contributing to Python! But not everyone on this list is a guy. That was a fast reply (< 2min -> Ain't you playing with bots searching for the "Gentlemen" word?). Just a joke ;-) Regards, francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [issue6839] zipfile can't extract file
Hi Nathaniel, their fault for not having a sense of humor. Trying to further defend this position is just going to dig you in deeper, so let's drop the discussion here. You're right, the formulation wasn't correct. My excuses. I just better do the review. Thanks. Regards, francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] IDLE in the stdlib
You can use idle from the command line almost as easily as the CP interpreter: 'python -m idlelib' instead of just 'python' (I just tried it to verify). Unlike bare 'python', IDLE includes a grep. Right click on any 'hit' and it opens the file at the specified line. Unlike bare 'python', you can run tests and collect the all the output, from as many tests as you want, in a dynamically right-sized buffer. I'm just getting: ~$ python2.7 -m idlelib /usr/bin/python2.7: No module named idlelib.__main__; 'idlelib' is a package and cannot be directly executed Same with python3... ...but thank you for talking about IDLE (the possibility of reediting and using history it so easy that I'm going to give it a try at least when I'm on windows...) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
and two instances i1 = Int1() i2 = Int2() we get the following behaviour: >>> type(int(i1)) I would have expected 'Int1' >>> type(float(i1)) >>> type(float(i2)) >>> isinstance(int(i1), int) True >>> isinstance(int(i2), int) new True >>> isinstance(float(i1), float) True >>> isinstance(float(i2), float) True why is printing new ? ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Semantics of __int__(), __index__()
On 03/31/2013 09:13 PM, francis wrote: why is printing new ? read twice before you write :-) ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [PATCH] Adding braces to __future__
Hi Cedric, On 12/09/2011 09:26 PM, Cedric Sodhi wrote: It is widely known among the programmer's community that spaces and tabs are remarkably similar to eachother. So similar even, that people fight wars about which to use in a non-py context. It might strike one as an equally remarkably nonsensical idea to give them programmatic meaning - two DIFFERENT meanings, to make things even worse. While it becomes a practical impossibility to spot these kind of bugs while reviewing code -- optionally mangled through a medium which expands tabs to whitespace, not so much of a rarity -- it is still a time-consuming and tedious job to find them in a local situation. I'm not so experienced with python as the majority of people here, but I've read that the practice is: do not to mix them (spaces and tabs). If this is taking much of you time while reviewing I would recommend you to let some script run on you code first to spot that mixture. IMHO that is a rule that should go in the code rules of your project and the build process should break if this mixture if found. Don't let that code reach the sync repository. As I said I'm maybe failing to see some case. Formatting is like food, everyone has it's own taste. One has to use spicery to change it (if possible). For me the view of the code (the layout) by the programmer should be automatically changed by the tool that reads the code. Here you could have a python with braces if you want... (I thing that 'go' has some autoformater or a standard way of formatting). -- francis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Code reviews
On 01/02/2012 06:35 PM, Georg Brandl wrote:
On 01/02/2012 03:41 PM, Antoine Pitrou wrote:
On Mon, 2 Jan 2012 14:44:49 +1000
Nick Coghlan wrote:
He keeps leaving them out, I occasionally tell him they should always
be included (most recently this came up when we gave conflicting
advice to a patch contributor).
Oh, by the way, this is also why I avoid arguing too much about style
in code reviews. There are two bad things which can happen:
- your advice conflicts with advice given by another reviewer (perhaps
on another issue)
- the contributor feels drowned under tiresome requests for style
fixes ("please indent continuation lines this way")
Both are potentially demotivating. A contributor can have his/her own
style if it doesn't adversely affect code quality.
Exactly. Especially for reviews of patches from non-core people, we
should exercise a lot of restraint: as the committers, I think we can be
expected to bite the sour bullet and apply our uniform style (such as
it is).
It is tiresome, if not downright disappointing, to get reviews that
are basically "nothing wrong, but please submit again with one more
empty line between the classes", and definitely not the way to
attract more contributors.
Hi to all member of this list,
I'm not a Python-Dev (only some very small patches over core-mentorship
list.
Just my 2cents here).
I would try to relax this conflicts with a script that does the
reformatting itself. If
that reformatting where part of the process itself do you thing that
that would
be an issue anymore?
PS: I know that there’s a pep8 checker so it could be transformed into a
reformatter
but I don't know if theres a pep7 checker (reformater)
Best regards!
francis
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new dictionary implementation
On 01/29/2012 11:31 AM, Mark Shannon wrote: It passes all the tests. (I had to change a couple that relied on dict repr() ordering) Hi Mark, I've cloned the repo, build it the I've tried with ./python -m test. I got some errors: First in general: 340 tests OK. 2 tests failed: test_dis test_gdb 4 tests altered the execution environment: test_multiprocessing test_packaging test_site test_strlit 18 tests skipped: test_curses test_devpoll test_kqueue test_lzma test_msilib test_ossaudiodev test_smtpnet test_socketserver test_startfile test_timeout test_tk test_ttk_guionly test_urllib2net test_urllibnet test_winreg test_winsound test_xmlrpc_net test_zipfile64 1 skip unexpected on linux: test_lzma [1348560 refs] then test_dis: == CPython 3.3.0a0 (default:f15cf35c9922, Jan 29 2012, 18:12:19) [GCC 4.6.2] == Linux-3.1.0-1-amd64-x86_64-with-debian-wheezy-sid little-endian == /home/ci/prog/cpython/hotpy_new_dict/build/test_python_14470 Testing with flags: sys.flags(debug=0, inspect=0, interactive=0, optimize=0, dont_write_bytecode=0, no_user_site=0, no_site=0, ignore_environment=0, verbose=0, bytes_warning=0, quiet=0) [1/1] test_dis test_big_linenos (test.test_dis.DisTests) ... ok test_boundaries (test.test_dis.DisTests) ... ok test_bug_1333982 (test.test_dis.DisTests) ... ok test_bug_708901 (test.test_dis.DisTests) ... ok test_dis (test.test_dis.DisTests) ... ok test_dis_none (test.test_dis.DisTests) ... ok test_dis_object (test.test_dis.DisTests) ... ok test_dis_traceback (test.test_dis.DisTests) ... ok test_disassemble_bytes (test.test_dis.DisTests) ... ok test_disassemble_method (test.test_dis.DisTests) ... ok test_disassemble_method_bytes (test.test_dis.DisTests) ... ok test_disassemble_str (test.test_dis.DisTests) ... ok test_opmap (test.test_dis.DisTests) ... ok test_opname (test.test_dis.DisTests) ... ok test_code_info (test.test_dis.CodeInfoTests) ... FAIL test_code_info_object (test.test_dis.CodeInfoTests) ... ok test_pretty_flags_no_flags (test.test_dis.CodeInfoTests) ... ok test_show_code (test.test_dis.CodeInfoTests) ... FAIL == FAIL: test_code_info (test.test_dis.CodeInfoTests) -- Traceback (most recent call last): File "/home/ci/prog/cpython/hotpy_new_dict/Lib/test/test_dis.py", line 439, in test_code_info self.assertRegex(dis.code_info(x), expected) AssertionError: Regex didn't match: 'Name: f\nFilename: (.*)\nArgument count:1\nKw-only arguments: 0\nNumber of locals: 1\nStack size:8\nFlags: OPTIMIZED, NEWLOCALS, NESTED\nConstants:\n 0: None\nNames:\n 0: print\nVariable names:\n 0: c\nFree variables:\n 0: e\n 1: d\n 2: f\n 3: y\n 4: x\n 5: z' not found in 'Name: f\nFilename: /home/ci/prog/cpython/hotpy_new_dict/Lib/test/test_dis.py\nArgument count:1\nKw-only arguments: 0\nNumber of locals: 1\nStack size:8\nFlags: OPTIMIZED, NEWLOCALS, NESTED\nConstants:\n 0: None\nNames:\n 0: print\nVariable names:\n 0: c\nFree variables:\n 0: y\n 1: e\n 2: d\n 3: f\n 4: x\n 5: z' == FAIL: test_show_code (test.test_dis.CodeInfoTests) -- Traceback (most recent call last): File "/home/ci/prog/cpython/hotpy_new_dict/Lib/test/test_dis.py", line 446, in test_show_code self.assertRegex(output.getvalue(), expected+"\n") AssertionError: Regex didn't match: 'Name: f\nFilename: (.*)\nArgument count:1\nKw-only arguments: 0\nNumber of locals: 1\nStack size:8\nFlags: OPTIMIZED, NEWLOCALS, NESTED\nConstants:\n 0: None\nNames:\n 0: print\nVariable names:\n 0: c\nFree variables:\n 0: e\n 1: d\n 2: f\n 3: y\n 4: x\n 5: z\n' not found in 'Name: f\nFilename: /home/ci/prog/cpython/hotpy_new_dict/Lib/test/test_dis.py\nArgument count:1\nKw-only arguments: 0\nNumber of locals: 1\nStack size:8\nFlags: OPTIMIZED, NEWLOCALS, NESTED\nConstants:\n 0: None\nNames:\n 0: print\nVariable names:\n 0: c\nFree variables:\n 0: y\n 1: e\n 2: d\n 3: f\n 4: x\n 5: z\n' -- Ran 18 tests in 0.070s FAILED (failures=2) test test_dis failed 1 test failed: test_dis [111919 refs] * For test gdb: Lots of output . Ran 42 tests in 11.361s FAILED (failures=28) test test_gdb failed 1 test failed: test_gdb [109989 refs] ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscri
Re: [Python-Dev] A new dictionary implementation
I still have gdb 6.somthing, would you mail me the full output please, so I can see what the problem is. It's done, let me know if you need more output. Cheers, francis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython (3.2): remove unused import
Hi Brett, If that's the case I might as well add it as part of my mnfy project's verification run I do over the stdlib if someone doesn't beat me to it. Is that devinabox ? Thanks in advance ! francis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new dictionary implementation
Just more info: changeset is: 74843:20702d1acf17 Cheers, francis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new dictionary implementation
Hi Mark, I've just cloned : Repository: https://bitbucket.org/markshannon/cpython_new_dict Do please try it on your machine(s). that's a: Linux random 3.1.0-1-amd64 #1 SMP Tue Jan 10 05:01:58 UTC 2012 x86_64 GNU/Linux and I'm getting: gcc -pthread -c -Wno-unused-result -g -O0 -Wall -Wstrict-prototypes -I. -I./Include -DPy_BUILD_CORE -o Objects/dictobject.o Objects/dictobject.c gcc -pthread -c -Wno-unused-result -g -O0 -Wall -Wstrict-prototypes -I. -I./Include -DPy_BUILD_CORE -o Objects/memoryobject.o Objects/memoryobject.c Objects/dictobject.c: In function ‘dict_popitem’: Objects/dictobject.c:2208:5: error: ‘PyDictKeyEntry’ has no member named ‘me_value’ make: *** [Objects/dictobject.o] Error 1 make: *** Waiting for unfinished jobs Cheers francis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] A new dictionary implementation
Hi Mark, Bah... typo in assert statement. My fault for not testing the debug build (release build worked fine). Both builds working now. Yeah, now is working and passes all tests also on my machine. I've tried to run the test suite but I'm getting a SyntaxError: (may be you know it's just the first time that I try the tool): = ci@random:~/prog/cpython/benchmarks$ python perf.py -r -b apps python ../cpython_new_dict/python Running 2to3... INFO:root:Running ../cpython_new_dict/python lib/2to3/2to3 -f all lib/2to3_data Traceback (most recent call last): File "perf.py", line 2236, in main(sys.argv[1:]) File "perf.py", line 2192, in main options))) File "perf.py", line 1279, in BM_2to3 return SimpleBenchmark(Measure2to3, *args, **kwargs) File "perf.py", line 706, in SimpleBenchmark *args, **kwargs) File "perf.py", line 1275, in Measure2to3 return MeasureCommand(command, trials, env, options.track_memory) File "perf.py", line 1223, in MeasureCommand CallAndCaptureOutput(command, env=env) File "perf.py", line 1053, in CallAndCaptureOutput raise RuntimeError(u"Benchmark died: " + unicode(stderr, 'ascii')) RuntimeError: Benchmark died: Traceback (most recent call last): File "lib/2to3/2to3", line 3, in from lib2to3.main import main File "/home/ci/prog/cpython/benchmarks/lib/2to3/lib2to3/main.py", line 47 except os.error, err: ^ SyntaxError: invalid syntax = And the baseline is: Python 2.7.2+ (but it also gives me an SyntaxError running on python3 default (e50db1b7ad7b) What I'm doing wrong ? (from it's doc: “This project is intended to be an authoritative source of benchmarks for all Python implementations.”) Thanks in advance ! francis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Requesting pronouncement on PEP 0424
Hi Alex, just a small info (view pep-0424.txt @ 4491:7838a83c3ad1): - Section Proposal: [...] than the actual size >>> ofthe <<< container. [...] - Section Rationale: The first line is really long (seems to need a newline before ``__length_hint__``) Regards francis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Summary of Python tracker Issues
Most recent 15 issues waiting for review (15) = Just curious: How is a issue considered "waiting for review"? Thanks! francis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Summary of Python tracker Issues
Most recent 15 issues waiting for review (15) = Just curious: How is a issue considered "waiting for review"? Issues that have the "patch" or "needs review" keyword or are in the "patch review" stage. Thank you! Is there a easy way to automate this?: - Get a list the "waiting for review" issues - Get the last patch - Try to apply that patch to the version(s) to check if that patch already applies? Regards, francis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] question about packaging
The current effort seems to be distlib, Vinay's project to gather the "good parts" of packaging and distutils as a library API: http://packages.python.org/distlib/ (there's an active bitbucket repo) See https://bitbucket.org/vinay.sajip/distlib/ for the latest code, which is periodically pushed to http://hg.python.org/distlib/ The latest documentation is at https://distlib.readthedocs.org/en/latest/ what about the actual packaging issues from the cypthon bugtracker? should we try to port the ones that apply to https://bitbucket.org/vinay.sajip/distlib/ ? Thanks and Happy 2013! francis ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Static checker for common Python programming errors
If I may, there are prior work on JavaScript that may be worth investigating. Formal verification of dynamically typed software is a challenging endeavour, but it is very valuable to avoid errors at runtime, providing benefits from strongly type language without the rigidity. http://cs.au.dk/~amoeller/papers/tajs/ Good luck! Francis 2014-11-17 9:49 GMT-05:00 Stefan Bucur : > I'm developing a Python static analysis tool that flags common programming > errors in Python programs. The tool is meant to complement other tools like > Pylint (which perform checks at lexical and syntactic level) by going > deeper with the code analysis and keeping track of the possible control > flow paths in the program (path-sensitive analysis). > > For instance, a path-sensitive analysis detects that the following snippet > of code would raise an AttributeError exception: > > if object is None: # If the True branch is taken, we know the object is > None > object.doSomething() # ... so this statement would always fail > > I'm writing first to the Python developers themselves to ask, in their > experience, what common pitfalls in the language & its standard library > such a static checker should look for. For instance, here [1] is a list of > static checks for the C++ language, as part of the Clang static analyzer > project. > > My preliminary list of Python checks is quite rudimentary, but maybe could > serve as a discussion starter: > > * Proper Unicode handling (for 2.x) > - encode() is not called on str object > - decode() is not called on unicode object > * Check for integer division by zero > * Check for None object dereferences > > Thanks a lot, > Stefan Bucur > > [1] http://clang-analyzer.llvm.org/available_checks.html > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/francis.giraldeau%40gmail.com > > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Support for Linux perf
Hi, The PEP-418 is about performance counters, but there is no mention of performance management unit (PMU) counters, such as cache misses and instruction counts. The Linux perf tool aims at recording these samples at the system level. I ran linux perf on CPython for profiling. The resulting callstack is inside libpython.so, mostly recursive calls to PyEval_EvalFrameEx(), because the tool works at the ELF level. Here is an example with a dummy program (linux-tools on Ubuntu 14.04): $ perf record python crunch.py $ perf report --stdio # Overhead Command Shared ObjectSymbol # ... .. # 32.37% python python2.7 [.] PyEval_EvalFrameEx 13.70% python libm-2.19.so[.] __sin_avx 5.25% python python2.7 [.] binary_op1.5010 4.82% python python2.7 [.] PyObject_GetAttr While this may be insightful for the interpreter developers, it it not so for the average Python developer. The report should display Python code instead. It seems obvious, still I haven't found the feature for that. When a performance counter reaches a given value, a sample is recorded. The most basic sample only records a timestamps, thread ID and the program counter (%rip). In addition, all executable memory maps of libraries are recorded. For the callstack, frame pointers are traversed, but most of the time, they are optimized on x86, so there is a fall back to unwind, which requires saving register values and a chunk of the stack. The memory space of the process is reconstructed offline. CPython seems to allocates code and frames on mmap() pages. If the data is outside about 1k from the top of stack, it is not available offline in the trace. We need some way to reconstitute this memory space of the interpreter to resolve the symbols, probably by dumping the data on disk. In Java, there is a small HotSpot agent that spits out the symbols of JIT code: https://github.com/jrudolph/perf-map-agent The problem is that CPython does not JIT code, and executed code is the ELF library itself. The executed frames are parameters of functions of the interpreter. I don't think the same approach can be used (maybe this can be applied to PyPy?). I looked at how Python frames are handled in GDB (file cpython/Tools/gdb/libpython.py). A python frame is detected in Frame(gdbframe).is_evalframeex() by a C call to PyEval_EvalFrameEx(). However, the traceback accesses PyFrameObject on the heap (at least for f->f_back = 0xa57460), which is possible in GDB when the program is paused and the whole memory space is available, but is not recorded for offline use in perf. Here is an example of callstack from GDB: #0 PyEval_EvalFrameEx (f=Frame 0x77f1b060, for file crunch.py, line 7, in bar (num=466829), throwflag=0) at ../Python/ceval.c:1039 #1 0x00527877 in fast_function (func=, pp_stack=0x7fffd280, n=1, na=1, nk=0) at ../Python/ceval.c:4106 #2 0x00527582 in call_function (pp_stack=0x7fffd280, oparg=1) at ../Python/ceval.c:4041 We could add a kernel module that "knows" how to make samples of CPython, but it means python structures becomes sort of ABI, and kernel devs won't allow a python interpreter in kernel mode ;-). What we really want is f_code data and related objects: (gdb) print (void *)(f->f_code) $8 = (void *) 0x77e370f0 Maybe we could save these pages every time some code is loaded from the interpreter? (the memory range is about 1.7MB, but ) Anyway, I think we must change CPython to support tools such as perf. Any thoughts? Cheers, Francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Support for Linux perf
2014-11-22 7:44 GMT-05:00 Julian Taylor : > On 17.11.2014 23:09, Francis Giraldeau wrote: > > Hi, > > ... > > The PEP-418 is about performance counters, but there is no mention o > > Anyway, I think we must change CPython to support tools such as perf. > > Any thoughts? > > > > there are some patches available adding systemtap and dtrace probes, > which should at least help getting function level profiles: > > http://bugs.python.org/issue21590 Thanks for these links, the patches looks interesting. As Jonas mentioned, Perf should be able to unwind a Python stack. It does at the interpreter level, and the frame info is scattered in virtual memory. It needs to be access offline. I think it could be possible to use the function entry and exit hooks in the interpreter to save important frame info, such as function name, file and line number, in a memory map known to perf. Then, we can tell Perf to record this compact zone of data in the sample as "extra" field for offline use. Then, at the analysis time, each ELF interpreter frame could be matched with the corresponding Python frame info. I think the perf handler can't sleep, and accesses on each function entry/exit will also ensure the page is present in main memory when the sample is recorded. Thanks again for your inputs, I'll post any further developments. Cheers, Francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] LTTng-UST support for CPython
Here is a working prototype for CPython to record all function call/return using LTTng-UST, a fast tracer. https://github.com/giraldeau/python-profile-ust However, there are few issues and questions: - I was not able to get PyTrace_EXCEPTION using "raise" or other error conditions. How can we trigger this event in Python code (PyTrace_C_EXCEPTION works)? - How could be the best way to get the full name of an object (such as package, module, class and function). Maybe it's too Java-ish, and it is better to record file/lineno instead? - On the C-API side: I did a horrible and silly function show_type() to run every Py*_Check() to determine the type of a PyObject *. What would be the sane way to do that? Your comments are very valuable. Thanks! Francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Encoding of PyFrameObject members
I need to access frame members from within a signal handler for tracing purpose. My first attempt to access co_filename was like this (omitting error checking): PyFrameObject *frame = PyEval_GetFrame(); PyObject *ob = PyUnicode_AsUTF8String(frame->f_code->co_filename) char *str = PyBytes_AsString(ob) However, the function PyUnicode_AsUTF8String() calls PyObject_Malloc(), which is not reentrant. If the signal handler nest over PyObject_Malloc(), it causes a segfault, and it could also deadlock. Instead, I access members directly: char *str = PyUnicode_DATA(frame->f_code->co_filename); size_t len = PyUnicode_GET_DATA_SIZE(frame->f_code->co_filename); Is it safe to assume that unicode objects co_filename and co_name are always UTF-8 data for loaded code? I looked at the PyTokenizer_FromString() and it seems to convert everything to UTF-8 upfront, and I would like to make sure this assumption is valid. Thanks! Francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Encoding of PyFrameObject members
2015-02-06 6:04 GMT-05:00 Armin Rigo : > Hi, > > On 6 February 2015 at 08:24, Maciej Fijalkowski wrote: > > I don't think it's safe to assume f_code is properly filled by the > > time you might read it, depending a bit where you find the frame > > object. Are you sure it's not full of garbage? > Yes, before discussing how to do the utf8 decoding, we should realize > that it is really unsafe code starting from the line before. From a > signal handler you're only supposed to read data that was written to > "volatile" fields. So even PyEval_GetFrame(), which is done by > reading the thread state's "frame" field, is not safe: this is not a > volatile. This means that the compiler is free to do crazy things > like *first* write into this field and *then* initialize the actual > content of the frame. The uninitialized content may be garbage, not > just NULLs. > Thanks for these comments. Of course accessing frames withing a signal handler is racy. I confirm that code encoded in non-ascii is not accessible from the uft8 buffer pointer. However, a call to PyUnicode_AsUTF8() encodes the data and caches it in the unicode object. Later access returns the byte buffer without memory allocation and re-encoding. I think it is possible to solve both safety problems by registering a handler with PyPyEval_SetProfile(). On function entry, the handler will call PyUnicode_AsUTF8() on the required frame members to make sure the utf8 encoded string is available. Then, we increment the refcount of the frame and assign it to a thread local pointer. On function return, the refcount is decremented. These operations occurs in the normal context and they are not racy. The signal handler will use the thread local frame pointer instead of calling PyEval_GetFrame(). Does that sounds good? Thanks again for your feedback! Francis ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Encoding of PyFrameObject members
2015-02-08 4:01 GMT-05:00 Maciej Fijalkowski : > I'm working on vmprof (github.com/vmprof/vmprof-python) which works > for both cpython and pypy (pypy has special support, cpython is > patched on-the fly) This looks interesting. I'm working on a profiler that is similar, but not based on timer. Instead, the signal is generated when an hardware performance counter overflows. It required a special linux kernel module, and the tracepoint is recorded using LTTng-UST. https://github.com/giraldeau/perfuser https://github.com/giraldeau/perfuser-modules https://github.com/giraldeau/python-profile-ust This is of course very experimental, requires a special setup, an I don't even know if it's going to produce good results. I'll report the results in the coming weeks. Cheers, Francis Giraldeau ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
