Re: [Python-Dev] Understanding the buffer API
- Summary:
The PEP, or sometimes just the documentation, definitely requires that
features not requested shall be NULL.
The API would benefit from:
a. stored flags that tell you the actual structural features.
b. requiring exporters to provide full information (e.g. strides =
{1}, format = "B") even when trivial.
It could and possibly should work this way in Python 4.0.
Nick thinks we could *allow* exporters to behave this way (PEP change)
in Python 3.x. Stefan thinks not, because "Perhaps there is code that
tests for shape==NULL to determine C-contiguity."
Jython exporters should return full information unconditionally from the
start: "any implementation that doesn't use the Py_buffer struct
directly in a C-API should just always return a full buffer" (Stefan);
"I think that's the way Jython should go: *require* that those fields be
populated appropriately" (Nick).
- But what I now think is:
_If the only problem really is_ "code that tests for shape==NULL to
determine C-contiguity", or makes similar deductions, I agree that
providing unasked-for information is_safe_. I think the stipulation in
PEP/documentation has some efficiency value: on finding shape!=NULL the
code has to do a more complicated test, as inPyBuffer_IsContiguous(). I
have the option to provide an isContiguous that has the answer written
down already, so the risk is only from/to ported code. If it is only a
risk to the efficiency of ported code, I'm relaxed: I hesitate only to
check that there's no circumstance that logically requires nullity for
correctness. Whether it was safe that was the key question.
In the hypothetical Python 4.0 buffer API (and in Jython) where feature
flags are provided, the efficiency is still useful, but complicated
deductive logic in the consumer should be deprecated in favour of
(functions for) interrogating the flags.
An example illustrating the semantics would then be:
1. consumer requests a buffer, saying "I can cope with a strided arrays"
(PyBUF_STRIDED);
2. exporter provides a strides array, but in the feature flags
STRIDED=0, meaning "you don't need the strides array";
3. exporter (optionally) uses efficient, non-strided access.
_I do not think_ that full provision by the exporter has to be
_mandatory_, as the discussion has gone on to suggest. I know your
experience is that you have often had to regenerate the missing
information to write generic code, but I think this does not continue
once you have the feature flags. An example would be:
1. consumer requests a buffer, saying "I can cope with a N-dimensional
but not strided arrays" (PyBUF_ND);
2. exporter sets strides=NULL, and the feature flag STRIDED=0;
3. exporter accesses the data, without reference to the strides array,
as it planned;
4. new generic code that respects the feature flag STRIDED=0, does not
reference the strides array;
5. old generic code, ignorant of the feature flags, finds the
strides=NULL and so does not dereference strides.
Insofar as it is not necessary, there is some efficiency in not
providing it. There would only be a problem with broken code that both
ignores the feature flag and uses the strides array unchecked. But this
code was always broken.
Really useful discussion this.
Jeff
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Close #15559: Implementing __index__ creates a nasty interaction with the bytes
On Sun, 5 Aug 2012 10:20:36 +0200 (CEST) nick.coghlan wrote: > http://hg.python.org/cpython/rev/5abea8a43f19 > changeset: 78426:5abea8a43f19 > user:Nick Coghlan > date:Sun Aug 05 18:20:17 2012 +1000 > summary: > Close #15559: Implementing __index__ creates a nasty interaction with the > bytes constructor. At least for 3.3, ipaddress objects must now be explicitly > converted with int() and thus can't be passed directly to the hex() builtin. __index__, as the name implies, allows instances to be used as sequence indices, which does sound like a weird thing to serve as for an IP address :-) Regards Antoine. -- Software development and contracting: http://pro.pitrou.net ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] cpython: Close #15559: Implementing __index__ creates a nasty interaction with the bytes
On Sun, Aug 5, 2012 at 11:00 PM, Antoine Pitrou wrote:
> On Sun, 5 Aug 2012 10:20:36 +0200 (CEST)
> nick.coghlan wrote:
>
>> http://hg.python.org/cpython/rev/5abea8a43f19
>> changeset: 78426:5abea8a43f19
>> user:Nick Coghlan
>> date:Sun Aug 05 18:20:17 2012 +1000
>> summary:
>> Close #15559: Implementing __index__ creates a nasty interaction with the
>> bytes constructor. At least for 3.3, ipaddress objects must now be
>> explicitly converted with int() and thus can't be passed directly to the
>> hex() builtin.
I noticed this when I tried
"bytes(ipaddress.Ipv4Address('192.168.0.1')" Apparently allocating and
initialising a 3.2 GB array on an ASUS Zenbook consumes large amounts
of time and makes the X server rather unresponsive. Even
faulthandler's timeout thread took more than ten times the specified
timeout to actually kill the operation. Who knew? :)
> __index__, as the name implies, allows instances to be used as sequence
> indices, which does sound like a weird thing to serve as for an IP
> address :-)
I expect the original reasoning had to do with the hex() builtin. In
2.x you could selectively support that by implementing __hex__
directly. In 3.x, the __oct__ and __hex__ methods are gone and the
only way to support those builtins (as well as bin()) is by
implementing __index__ instead. However, implementing __index__ makes
the type usable in a whole host of other contexts as well, so the
naive __hex__ -> __index__ conversion really wasn't a good idea.
I'm thinking it may make sense to eventually implement __bytes__, as
having bytes(address) be equivalent to address.packed *does* make
sense. No hurry on that, though.
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] devguide: It has been pointed out this paragraph was incorrect. One of the Windows devs
On Sun, Aug 5, 2012 at 1:48 AM, nick.coghlan wrote: > http://hg.python.org/devguide/rev/f518f23d06d5 > changeset: 539:f518f23d06d5 > summary: > It has been pointed out this paragraph was incorrect. One of the Windows > devs will need to fill in more accurate info > diff --git a/setup.rst b/setup.rst > -For Windows systems, all the necessary components should be included in the > -CPython checkout. This issue may not provide the information you're looking for, but it is related: http://bugs.python.org/issue14873 It has a patch from a few days ago awaiting review. Incidentally, the UNIX-specific information added in http://hg.python.org/devguide/rev/80358cdac0a6 might be good to link to in the UNIX section here: "Do take note of what modules were not built as stated at the end of your build. More than likely you are missing a dependency for the module(s) that were not built, and so you can install the dependencies and re-run both configure and make (if available for your OS)." (from http://docs.python.org/devguide/setup.html#unix ) Or else move the UNIX-specific information into a subsection of the UNIX section. --Chris ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] cpython (merge 3.2 -> default): Make TextIOWrapper's documentation clearer by copying the newline argument's
On Sat, Aug 4, 2012 at 11:51 AM, Victor Stinner wrote: > 2012/8/4 Chris Jerdonek : >> Now that this change is made, it may make sense to update the >> subprocess documentation to reference TextIOWrapper's documentation >> instead of open()'s (since use of the 'U' flag to open() is >> discouraged in new code). > > Good idea, can you please open an issue? The documentation is wrong: > UTF-8 is not used, it's the locale encoding. I created an issue for this (with patch) here: http://bugs.python.org/issue15561 --Chris ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] No summary of tracker issues this week?
Hi all, I keep an eye open for this but can't find one for Saturday 03/08/2012. Have I missed it, has it been stopped, has something gone wrong with its production or what? -- Cheers. Mark Lawrence. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] No summary of tracker issues this week?
On 05/08/2012 23:31, Mark Lawrence wrote: Hi all, I keep an eye open for this but can't find one for Saturday 03/08/2012. Have I missed it, has it been stopped, has something gone wrong with its production or what? I haven't seen it either. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Summary of Python tracker Issues
ACTIVITY SUMMARY (2012-07-30 - 2012-08-06)
Python tracker at http://bugs.python.org/
To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.
Issues counts and deltas:
open3559 (+18)
closed 23781 (+51)
total 27340 (+69)
Open issues with patches: 1534
Issues opened (42)
==
#13072: Getting a buffer from a Unicode array uses invalid format
http://bugs.python.org/issue13072 reopened by haypo
#15494: Move test/support.py into a test.support subpackage
http://bugs.python.org/issue15494 opened by cjerdonek
#15495: enable type truncation warnings for gcc builds
http://bugs.python.org/issue15495 opened by jkloth
#15496: harden directory removal for tests on Windows
http://bugs.python.org/issue15496 opened by jkloth
#15497: correct characters in TextWrapper.replace_whitespace docs
http://bugs.python.org/issue15497 opened by cjerdonek
#15498: Eliminate the use of deprecated OS X APIs in getpath.c
http://bugs.python.org/issue15498 opened by ned.deily
#15500: Python should support naming threads
http://bugs.python.org/issue15500 opened by bra
#15501: Document exception classes in subprocess module
http://bugs.python.org/issue15501 opened by anton.barkovsky
#15502: Meta path finders and path entry finders are different, but sh
http://bugs.python.org/issue15502 opened by ncoghlan
#15504: pickle/cPickle saves invalid/incomplete data
http://bugs.python.org/issue15504 opened by Philipp.Lies
#15505: unittest.installHandler incorrectly assumes SIGINT handler is
http://bugs.python.org/issue15505 opened by twouters
#15506: configure should use PKG_PROG_PKG_CONFIG
http://bugs.python.org/issue15506 opened by vapier
#15507: test_subprocess assumes SIGINT is not being ignored.
http://bugs.python.org/issue15507 opened by twouters
#15509: webbrowser.open sometimes passes zero-length argument to the b
http://bugs.python.org/issue15509 opened by anton.barkovsky
#15510: textwrap.wrap('') returns empty list
http://bugs.python.org/issue15510 opened by cjerdonek
#15511: _decimal does not build in PGUpdate mode
http://bugs.python.org/issue15511 opened by skrah
#15513: Correct __sizeof__ support for pickle
http://bugs.python.org/issue15513 opened by storchaka
#15516: exception-handling bug in PyString_Format
http://bugs.python.org/issue15516 opened by tromey
#15518: Provide test coverage for filecmp.dircmp.report methods.
http://bugs.python.org/issue15518 opened by cbc
#15520: Document datetime.timestamp() in 3.3 What's New
http://bugs.python.org/issue15520 opened by djc
#15522: impove 27 percent performance on stringpbject.c( by prefetch a
http://bugs.python.org/issue15522 opened by abael
#15523: Block on close TCP socket in SocketServer.py
http://bugs.python.org/issue15523 opened by jarvisliang
#15526: regrtest crash on Windows 7 AMD64
http://bugs.python.org/issue15526 opened by ncoghlan
#15527: Double parens in functions references
http://bugs.python.org/issue15527 opened by storchaka
#15528: Better support for finalization with weakrefs
http://bugs.python.org/issue15528 opened by sbt
#15533: subprocess.Popen(cwd) documentation
http://bugs.python.org/issue15533 opened by cjerdonek
#15535: Fix pickling of named tuples in 2.7.3
http://bugs.python.org/issue15535 opened by thomie
#15539: Fixing Tools/scripts/pindent.py
http://bugs.python.org/issue15539 opened by storchaka
#15542: Documentation incorrectly suggests __init__ called after direc
http://bugs.python.org/issue15542 opened by Aaron.Staley
#15543: central documentation for 'universal newlines'
http://bugs.python.org/issue15543 opened by cjerdonek
#15544: math.isnan fails with some Decimal NaNs
http://bugs.python.org/issue15544 opened by stevenjd
#15545: sqlite3.Connection.iterdump() does not work with row_factory =
http://bugs.python.org/issue15545 opened by plemarre
#15548: Mention all new os functions in What's New in Python 3.3
http://bugs.python.org/issue15548 opened by haypo
#15549: openssl version in windows builds does not support renegotiati
http://bugs.python.org/issue15549 opened by cory.mintz
#15550: Trailing white spaces
http://bugs.python.org/issue15550 opened by storchaka
#15552: gettext: if looking for .mo in default locations, also look in
http://bugs.python.org/issue15552 opened by Dominique.Leuenberger
#15553: Segfault in test_6_daemon_threads() of test_threading, on Mac
http://bugs.python.org/issue15553 opened by haypo
#15554: correct and clarify str.splitlines() documentation
http://bugs.python.org/issue15554 opened by cjerdonek
#1: Default newlines of io.TextIOWrapper
http://bugs.python.org/issue1 opened by ishimoto
#15556: os.stat fails for file pending delete on Windows
http://bugs.python.org/issue15556 opened by jkloth
#15557: Tests for webbrowser module
http://bugs.python.org/issue15557 opened by anton.barkovsky
#15561: update subprocess docs to reference io.TextIOWrapper
http://bugs.pyt
Re: [Python-Dev] No summary of tracker issues this week?
On Mon, 06 Aug 2012 00:39:22 +0100, MRAB wrote: > On 05/08/2012 23:31, Mark Lawrence wrote: > > Hi all, > > > > I keep an eye open for this but can't find one for Saturday 03/08/2012. > >Have I missed it, has it been stopped, has something gone wrong with > > its production or what? > > > I haven't seen it either. Thanks for noticing. It should be fixed now. I triggered a run of the report manually, but be aware that it reports the last week, so it starts from last Monday early AM UTC, and ends now...so there will be some reporting overlap in next Friday's report. (Not that that should matter much unless someone is using them to track week-to-week statistics.) --David ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
