[Python-Dev] The online documentation is no more updated?
Hi, I modified the Python documentaton (asyncio module) 6 days ago, and my changes are not online yet: http://docs.python.org/dev/library/asyncio-eventloop.html#running-subprocesses Is it a problem with the server generating the documentation? By the way, would it be possible to regenerate the documentation at each commit, or maybe every hours, instead of having to wait 24h to see the modified documentation online? Thanks. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] The docstring hack for signature information has to go
A quick summary of the context: currently in CPython 3.4, a builtin
function can publish its "signature" as a specially encoded line at the
top of its docstring. CPython internally detects this line inside
PyCFunctionObject.__doc__ and skips past it, and there's a new getter at
PyCFunctionObject.__text_signature__ that returns just this line. As an
example, the signature for os.stat looks like this:
sig=($module, path, *, dir_fd=None, follow_symlinks=True)
The convention is, if you have this signature, you shouldn't have your
docstring start with a handwritten signature like 3.3 and before.
help() on a callable displays the signature automatically if it can, so
if you *also* had a handwritten signature, help() would show two
signatures. That would look dumb.
-
So here's the problem. Let's say you want to write an extension that
will work with Python 3.3 and 3.4, using the stable ABI. If you don't
add this line, then in 3.4 you won't have introspection information,
drat. But if you *do* add this line, your docstring will look mildly
stupid in 3.3, because it'll have this unsightly "sig=(" line at the
top. And it *won't* have a nice handwritten docstring. (And if you
added both a sig= signature *and* a handwritten signature, in 3.4 it
would display both. That would also look dumb.)
I can't figure out any way to salvage this "first line of the docstring"
approach. So I think we have to abandon it, and do this the hard way:
extend the PyMethodDef structure. I propose three different
variations. I prefer B, but I'm guessing Guido would prefer the YAGNI
approach, which is A:
A: We create a PyMethodDefEx structure with an extra field: "const char
*signature". We add a new METH_SIGNATURE (maybe just METH_SIG?) flag to
the flags, indicating that this is an extended structure. When
iterating over the PyMethodDefs, we know how far to advance the pointer
based on this flag.
B: Same as A, but we add three unused pointers (void *reserved1 etc) to
PyMethodDefEx to give us some room to grow.
C: Same as A, but we add two fields to PyMethodDefEx. The second new
field identifies the "version" of the structure, telling us its size
somehow. Like the lStructSize field of the OPENFILENAME structure in
Win32. I suspect YAGNI.
-
But that only fixes part of the problem. Our theoretical extension that
wants to be binary-compatible with 3.3 and 3.4 still has a problem: how
can they support signatures? They can't give PyMethodDefEx structures
to 3.3, it will blow up. But if they don't use PyMethodDefEx, they
can't have signatures.
Solution: we write a function (which users would have to copy into their
extension) that gives a PyMethodDefEx array to 3.4+, but converts it
into a PyMethodDef array for 3.3. The tricky part there: what do we do
about the docstring? The convention for builtins is to have the first
line(s) contain a handwritten signature. But you *don't* want that if
you provide a signature, because help() will read that signature and
automatically render this first line for you.
I can suggest four options here, and of these I like P best:
M: Don't do anything. Docstrings with real signature information and a
handwritten signature in the docstring will show two signatures in 3.4+,
docstrings without any handwritten signature won't display their
signature in help in 3.3. (Best practice for modules compiled for 3.4+
is probably: skip the handwritten signature. Users would have to do
without in 3.3.)
N: Leave the handwritten signature in the docstring, then when
registering for 3.4+ add a second flag called METH_33_COMPAT that means
"when displaying help for this function, don't automatically generate
that first line."
O: Have the handwritten signature in the docstring. When registering
the function for 3.3, have the PyMethodDef docstring point to the it
starting at the signature. When registering the function for 3.4+, have
the docstring in the PyMethodDefEx point to the first byte after the
handwritten signature. Note that automatically skipping the signature
with a heuristic is mildly complicated, so this may be hard to get right.
P: Have the handwritten signature in the docstring, and have separate
static PyMethodDef and PyMethodDefEx arrays. The PyMethodDef docstring
points to the docstring like normal. The PyMethodDefEx docstring field
points to the first byte after the handwritten signature. This makes
the registration "function" very simple: if it's 3.3 or before, use the
PyMethodDef array, if it's 3.4+ use the PyMethodDefEx array. (Argument
Clinic could theoretically automate coding some or all of this.)
It's late and my brain is only working so well. I'd be interested in
other approaches if people can suggest something good.
Sorry about the mess,
//arry/
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailma
Re: [Python-Dev] The docstring hack for signature information has to go
On Feb 03, 2014, at 06:43 AM, Larry Hastings wrote: >But that only fixes part of the problem. Our theoretical extension that >wants to be binary-compatible with 3.3 and 3.4 still has a problem: how can >they support signatures? They can't give PyMethodDefEx structures to 3.3, it >will blow up. But if they don't use PyMethodDefEx, they can't have >signatures. Can't an extension writer #ifdef around this? Yeah, it's ugly, but it's a pretty standard approach for making C extensions multi-version compatible. -Barry ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The online documentation is no more updated?
On Mon, Feb 3, 2014, at 02:22 AM, Victor Stinner wrote: > Hi, > > I modified the Python documentaton (asyncio module) 6 days ago, and my > changes are not online yet: > http://docs.python.org/dev/library/asyncio-eventloop.html#running-subprocesses > > Is it a problem with the server generating the documentation? Hopefully fixed now. > > By the way, would it be possible to regenerate the documentation at > each commit, or maybe every hours, instead of having to wait 24h to > see the modified documentation online? It was twice a day before, I've now switched the build to be every hour. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] _PyUnicode_CheckConsistency() too strict?
_PyUnicode_CheckConsistency() checks that the contents of the string matches the _KIND of the string. However it does this in a very strict manner, ie. that the contents *exactly* match the _KIND rather than just detecting an inconsistency between the contents and the _KIND. For example, a string created with a maxchar of 255 (ie. a Latin-1 string) must contain at least one character in the range 128-255 otherwise you get an assertion failure. As it stands, when converting Latin-1 strings in my C extension module I must first check each character and specify a maxchar of 127 if the strings happens to only contain ASCII characters. What is the reasoning behind the checks being so strict? Phil ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
2014-02-03 Phil Thompson : > For example, a string created with a maxchar of 255 (ie. a Latin-1 string) > must contain at least one character in the range 128-255 otherwise you get > an assertion failure. Yes, it's the specification of the PEP 393. > As it stands, when converting Latin-1 strings in my C extension module I > must first check each character and specify a maxchar of 127 if the strings > happens to only contain ASCII characters. Use PyUnicode_FromKindAndData(PyUnicode_1BYTE_KIND, latin1_str, length) which computes the kind for you. > What is the reasoning behind the checks being so strict? Different Python functions rely on the exact kind to compare strings. For example, if you search a latin1 substring in an ASCII string, the search returns immediatly instead of searching in the string. A latin1 string cannot be found in an ASCII string. The main reason in the PEP 393 itself, a string must be compact to not waste memory. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
On 03-02-2014 3:35 pm, Victor Stinner wrote: 2014-02-03 Phil Thompson : For example, a string created with a maxchar of 255 (ie. a Latin-1 string) must contain at least one character in the range 128-255 otherwise you get an assertion failure. Yes, it's the specification of the PEP 393. As it stands, when converting Latin-1 strings in my C extension module I must first check each character and specify a maxchar of 127 if the strings happens to only contain ASCII characters. Use PyUnicode_FromKindAndData(PyUnicode_1BYTE_KIND, latin1_str, length) which computes the kind for you. What is the reasoning behind the checks being so strict? Different Python functions rely on the exact kind to compare strings. For example, if you search a latin1 substring in an ASCII string, the search returns immediatly instead of searching in the string. A latin1 string cannot be found in an ASCII string. The main reason in the PEP 393 itself, a string must be compact to not waste memory. Victor Are you saying that code will fail if a particular Latin-1 string just happens not to contains any character greater than 127? I would be very surprised if that was the case. If it isn't the case then I think that particular check shouldn't be made. Phil ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
2014-02-03 Phil Thompson : > Are you saying that code will fail if a particular Latin-1 string just > happens not to contains any character greater than 127? PyUnicode_FromKindAndData(PyUnicode_1BYTE_KIND, latin1_str, length) accepts latin1 and ASCII strings. It computes the maximum code point and then use ASCII or latin1 unicode string. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
On 03-02-2014 4:04 pm, Victor Stinner wrote: 2014-02-03 Phil Thompson : Are you saying that code will fail if a particular Latin-1 string just happens not to contains any character greater than 127? PyUnicode_FromKindAndData(PyUnicode_1BYTE_KIND, latin1_str, length) accepts latin1 and ASCII strings. It computes the maximum code point and then use ASCII or latin1 unicode string. Victor That doesn't answer my original question, that just works around the use case I presented. To restate... Why is a Latin-1 string considered inconsistent just because it doesn't happen to contain any characters in the range 128-255? Phil ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docstring hack for signature information has to go
On 02/03/2014 07:08 AM, Barry Warsaw wrote: On Feb 03, 2014, at 06:43 AM, Larry Hastings wrote: But that only fixes part of the problem. Our theoretical extension that wants to be binary-compatible with 3.3 and 3.4 still has a problem: how can they support signatures? They can't give PyMethodDefEx structures to 3.3, it will blow up. But if they don't use PyMethodDefEx, they can't have signatures. Can't an extension writer #ifdef around this? Yeah, it's ugly, but it's a pretty standard approach for making C extensions multi-version compatible. For source compatibility, yes. But I thought the point of the binary ABI was to allow compiling a single extension that worked unmodified with multiple versions of Python. If we simply don't support that, then an ifdef would be fine. //arry/ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docstring hack for signature information has to go
Larry Hastings wrote:
> So here's the problem. Let's say you want to write an extension that will
> work
> with Python 3.3 and 3.4, using the stable ABI. If you don't add this line,
> then in 3.4 you won't have introspection information, drat. But if you *do*
> add this line, your docstring will look mildly stupid in 3.3, because it'll
> have this unsightly "sig=(" line at the top. And it *won't* have a nice
> handwritten docstring. (And if you added both a sig= signature *and* a
> handwritten signature, in 3.4 it would display both. That would also look
> dumb.)
I think we may slowly get into PEP territory here. Just imagine that
we settle on X, then decide at a later point to have a standard way of
adding type annotations, then find that X does not work because of (unknown).
I'm mentioning this because signatures get really interesting for me
if they contain type information.
Stefan Krah
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The online documentation is no more updated?
2014-02-03 Benjamin Peterson : >> Is it a problem with the server generating the documentation? > > Hopefully fixed now. The documentation is still outdated. For example, I don't see the new subprocess page in http://docs.python.org/dev/library/asyncio.html > It was twice a day before, I've now switched the build to be every hour. Nice. I'm just waiting until it works :-) Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
On 3 February 2014 16:10, Phil Thompson wrote: > That doesn't answer my original question, that just works around the use > case I presented. > > To restate... > > Why is a Latin-1 string considered inconsistent just because it doesn't > happen to contain any characters in the range 128-255? Butting in here (sorry) but I thought what Victor was trying to say is that being able to say that a string marked as Latin1 "kind" definitely has characters >127 allows the code to optimise some tests (for example, two strings cannot be equal if their kinds differ). Obviously, requiring this kind of constraint makes it somewhat harder for user code to construct string objects that conform to the spec. That's why the PyUnicode_FromKindAndData function has the convenience feature of doing the check and setting the kind correctly for you - you should use that rather than trying to get the details right yourself.. Paul. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Guidance regarding what counts as breaking backwards compatibility
On Sat, Feb 1, 2014 at 9:14 PM, Nick Coghlan wrote:
> On 2 February 2014 11:06, Steven D'Aprano wrote:
> > Hi all,
> >
> > Over on the Python-ideas list, there's a thread about the new statistics
> > module, and as the author of that module, I'm looking for a bit of
> > guidance regarding backwards compatibility. Specifically two issues:
> >
> >
> > (1) With numeric code, what happens if the module become more[1]
> > accurate in the future? Does that count as breaking backwards
> > compatibility?
> >
> > E.g. Currently I use a particular algorithm for calculating variance.
> > Suppose that for a particular data set, this algorithm is accurate to
> > (say) seven decimal places:
> >
> > # Python 3.4
> > variance(some_data) == 1.2345671
> >
> > Later, I find a better algorithm, which improves the accuracy of the
> > result:
> >
> > # Python 3.5 or 3.6
> > variance(some_data) == 1.23456789001
> >
> >
> > Would this count as breaking backwards compatibility? If so, how should
> > I handle this? I don't claim that the current implementation of the
> > statistics module is optimal, as far as precision and accuracy is
> > concerned. It may improve in the future.
>
> For this kind of case, we tend to cover it in the "Porting to Python
> X.Y" section of the What's New guide. User code *shouldn't* care about
> this kind of change, but it *might*, so we split the difference and
> say "It's OK in a feature release, but not in a maintenance release".
> There have been multiple changes along these lines in our floating
> handling as Tim Peters, Mark Dickinson et al have made various
> improvements to reduce platform dependent behaviour (especially around
> overflow handling, numeric precision, infinity and NaN handling, etc).
>
I agree with Nick that it's a feature release change, not a bugfix one.
I think the key way to rationalize this particular case is the use of the
words "better" and "improves". Notice you never said "broken" or "wrong",
just that you were making the estimate better. Since the previous behaviour
was not fundamentally broken and going to cause errors in correct code it
then should only go in a feature release.
-Brett
>
> However, we also sometimes have module specific disclaimers - the
> decimal module, for example, has an explicit caveat that updates to
> the General Decimal Arithmetic Specification will be treated as bug
> fixes, even if they would normally not be allowed in maintenance
> releases.
>
> For a non-math related example, a comment from Michael Foord at the
> PyCon US 2013 sprints made me realise that the implementation of
> setting the __wrapped__ attribute in functools was just flat out
> broken - when applied multiple times it was supposed to create a chain
> of references that eventually terminated in a callable without the
> attribute set, but due to the bug every layer actually referred
> directly to the innermost callable (the one without the attribute
> set). Unfortunately, the docs I wrote for it were also ambiguous, so a
> lot of folks (including Michael) assumed it was working as intended. I
> have fixed the bug in 3.4, but there *is* a chance it will break
> introspection code that assumed the old behaviour was intentional and
> doesn't correctly unravel __wrapped__ chains.
>
> > Or would that count as a bug-fix? "Variance function was inaccurate, now
> > less wrong", perhaps.
> >
> > I suppose the math module has the same issue, except that it just wraps
> > the C libraries, which are mature and stable and unlikely to change.
>
> They may look that way *now*, but that's only after Tim, Mark et al
> did a lot of work on avoiding platform specific issues and
> inconsistencies
>
> > The random module has a similar issue:
> >
> > http://docs.python.org/3/library/random.html#notes-on-reproducibility
>
> I think a disclaimer in the statistics module similar to the ones in
> the math module and this one in the random module would be appropriate
> - one of the key purposes of the library/language reference is to let
> us distinguish between "guaranteed behaviour user code can rely on"
> and "implementation details that user code should not assume will
> remain unchanged forever".
>
> In this case, it would likely be appropriate to point out that the
> algorithms used internally may change over time, thus potentially
> changing the error bounds in the module output.
>
> > (2) Mappings[2] are iterable. That means that functions which expect
> > sequences or iterators may also operate on mappings by accident. For
> > example, sum({1: 100, 2: 200}) returns 3. If one wanted to reserve the
> > opportunity to handle mappings specifically in the future, without being
> > locked in by backwards-compatibility, how should one handle it?
> >
> > a) document that behaviour with mappings is unsupported and may
> >change in the future;
> >
> > b) raise a warning when passed a mapping, but still iterate over it;
> >
> > c) raise an exception and refuse to iterate over the mapping;
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
On 03-02-2014 4:38 pm, Paul Moore wrote: On 3 February 2014 16:10, Phil Thompson wrote: That doesn't answer my original question, that just works around the use case I presented. To restate... Why is a Latin-1 string considered inconsistent just because it doesn't happen to contain any characters in the range 128-255? Butting in here (sorry) but I thought what Victor was trying to say is that being able to say that a string marked as Latin1 "kind" definitely has characters >127 allows the code to optimise some tests (for example, two strings cannot be equal if their kinds differ). So there *is* code that will fail if a particular Latin-1 string just happens not to contains any character greater than 127? Obviously, requiring this kind of constraint makes it somewhat harder for user code to construct string objects that conform to the spec. That's why the PyUnicode_FromKindAndData function has the convenience feature of doing the check and setting the kind correctly for you - you should use that rather than trying to get the details right yourself.. Paul. I see now... The docs for PyUnicode_FromKindAndData() say... "Create a new Unicode object *with* the given kind" ...and so I didn't think is was useful to me. If they said... "Create a new Unicode object *from* the given kind" ...then I might have got it. Thanks - I'm happy now. Phil ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
On Mon, 03 Feb 2014 16:10:03 + Phil Thompson wrote: > > Why is a Latin-1 string considered inconsistent just because it doesn't > happen to contain any characters in the range 128-255? Because as Victor said, it allows for some optimization shortcuts (e.g. a non-ASCII latin1 string cannot be equal to an ASCII string - no need for a memcmp). Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docstring hack for signature information has to go
Larry, Can you summarize why neither of the two schemes you tried so far worked? AFAIR the original scheme was to support the 3.3-compatible syntax; there was some kind of corner-case problem with this, so you switched to the new "sig=..." syntax, but obviously this has poor compatibility with 3.3. Can you remind us of what the corner-case was? How bad would it be if we decided to just live with it or if we added a new flag bit (only recognized by 3.4) to disambiguate corner-cases? -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
Can we provide a convenience API (or even a few lines of code one could copy+paste) that determines if a particular 8-bit string should have max-char equal to 127 or 255? I can easily imagine a number of use cases where this would come in handy (e.g. a list of strings produced by translation, or strings returned in Latin-1 by some other non-Python C-level API) -- and let's not get into a debate about whether UTF-8 wouldn't be better, I can also easily imagine legacy APIs where that isn't (yet) an option. On Mon, Feb 3, 2014 at 9:35 AM, Antoine Pitrou wrote: > On Mon, 03 Feb 2014 16:10:03 + > Phil Thompson wrote: > > > > Why is a Latin-1 string considered inconsistent just because it doesn't > > happen to contain any characters in the range 128-255? > > Because as Victor said, it allows for some optimization shortcuts (e.g. > a non-ASCII latin1 string cannot be equal to an ASCII string - no need > for a memcmp). > > Regards > > Antoine. > > > ___ > Python-Dev mailing list > [email protected] > https://mail.python.org/mailman/listinfo/python-dev > Unsubscribe: > https://mail.python.org/mailman/options/python-dev/guido%40python.org > -- --Guido van Rossum (python.org/~guido) ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
On 02/03/2014 09:16 AM, Phil Thompson wrote: So there *is* code that will fail if a particular Latin-1 string just happens not to contains any character greater than 127? Yes, because if it does not contain a character > 127 it is not a latin-1 string as far as Python is concerned. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
On 03-02-2014 5:52 pm, Guido van Rossum wrote: Can we provide a convenience API (or even a few lines of code one could copy+paste) that determines if a particular 8-bit string should have max-char equal to 127 or 255? I can easily imagine a number of use cases where this would come in handy (e.g. a list of strings produced by translation, or strings returned in Latin-1 by some other non-Python C-level API) -- and lets not get into a debate about whether UTF-8 wouldnt be better, I can also easily imagine legacy APIs where that isnt (yet) an option. For my particular use case PyUnicode_FromKindAndData() (once I'd interpreted the docs correctly) should have made such code unnecessary. However I've just discovered that it doesn't support surrogates in UCS2 so I'm going to have to roll my own anyway. Phil ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] _PyUnicode_CheckConsistency() too strict?
On 02/03/2014 09:52 AM, Guido van Rossum wrote: Can we provide a convenience API (or even a few lines of code one could copy+paste) that determines if a particular 8-bit string should have max-char equal to 127 or 255? Isn't that what this is? PyObject* PyUnicode_FromKindAndData( int kind, const void *buffer, Py_ssize_t size) Create a new Unicode object with the given kind (possible values are PyUnicode_1BYTE_KIND etc., as returned by PyUnicode_KIND()). The buffer must point to an array of size units of 1, 2 or 4 bytes per character, as given by the kind. -- ~Ethan~ ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The online documentation is no more updated?
On Mon, Feb 3, 2014, at 08:36 AM, Victor Stinner wrote: > 2014-02-03 Benjamin Peterson : > >> Is it a problem with the server generating the documentation? > > > > Hopefully fixed now. > > The documentation is still outdated. For example, I don't see the new > subprocess page in > http://docs.python.org/dev/library/asyncio.html Finally fixed. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] BugFix for "time" native module in Python 2.7
Hi, guys! I have found a bug in module "time" function "sleep" in Python 2.7 under Windows XP / Server 2003 and lower. I fix this bug locally. But how could I send you hg patch or pull request or, maybe, commit to sandbox? P.S.: Sorry for my English :-) Best Regards, Victor Scherba, C++ developer. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] BugFix for "time" native module in Python 2.7
On 03/02/2014 19:41, Виктор Щерба wrote: Hi, guys! I have found a bug in module "time" function "sleep" in Python 2.7 under Windows XP / Server 2003 and lower. I fix this bug locally. But how could I send you hg patch or pull request or, maybe, commit to sandbox? The best thing is to raise an bug on our issue tracker: http://bugs.python.org/ and attach the patch together with a test which reproduces the issue TJG ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] BugFix for "time" native module in Python 2.7
On Mon, Feb 3, 2014 at 2:41 PM, Виктор Щерба wrote: > Hi, guys! > > I have found a bug in module "time" function "sleep" in Python 2.7 under > Windows XP / Server 2003 and lower. > I fix this bug locally. But how could I send you hg patch or pull request > or, maybe, commit to sandbox? > Please file bugs and patches at bugs.python.org. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The online documentation is no more updated?
2014-02-03 Benjamin Peterson : >> The documentation is still outdated. For example, I don't see the new >> subprocess page in >> http://docs.python.org/dev/library/asyncio.html > > Finally fixed. Thanks. Victor ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] BugFix for "time" native module in Python 2.7
Hi Victor, On Mon, Feb 3, 2014 at 1:41 PM, Виктор Щерба wrote: > Hi, guys! > > I have found a bug in module "time" function "sleep" in Python 2.7 under > Windows XP / Server 2003 and lower. > I fix this bug locally. But how could I send you hg patch or pull request > or, maybe, commit to sandbox? Please open an issue on the bug tracker (http://bugs.python.org/) and attach your patch. > P.S.: Sorry for my English :-) Your English is better than that of most (American) teenagers I know, don't worry :) -- Zach ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] BugFix for "time" native module in Python 2.7
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 02/03/2014 02:41 PM, ?? ? wrote: > Hi, guys! > > I have found a bug in module "time" function "sleep" in Python 2.7 > under Windows XP / Server 2003 and lower. I fix this bug locally. But > how could I send you hg patch or pull request or, maybe, commit to > sandbox? > > P.S.: Sorry for my English :-) Create an issue on the Python tracker: http://bugs.python.org/ and attach your patch there. See the Python Developers' Guide for more details: http://docs.python.org/devguide/ Tres. - -- === Tres Seaver +1 540-429-0999 [email protected] Palladion Software "Excellence by Design"http://palladion.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.11 (GNU/Linux) Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iEYEARECAAYFAlLv9dAACgkQ+gerLs4ltQ6JdQCfbQ5lFmW9OQceVZJFW8Zs3fIz 0m4An3ttIZbdWtVLtOtloMfsrnW4nanE =Cd+P -END PGP SIGNATURE- ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] BugFix for "time" native module in Python 2.7
Hi Victor, You can find instructions here to produce a patch: http://docs.python.org/devguide/ The first thing to do would be to create an issue on http://bugs.python.org/, and post your patch there. Regards Antoine. On Mon, 3 Feb 2014 23:41:22 +0400 Виктор Щерба wrote: > Hi, guys! > > I have found a bug in module "time" function "sleep" in Python 2.7 under > Windows XP / Server 2003 and lower. > I fix this bug locally. But how could I send you hg patch or pull request > or, maybe, commit to sandbox? > > P.S.: Sorry for my English :-) > > Best Regards, > Victor Scherba, > C++ developer. > ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docstring hack for signature information has to go
On 2/3/2014 9:43 AM, Larry Hastings wrote:
A quick summary of the context: currently in CPython 3.4, a builtin
function can publish its "signature" as a specially encoded line at the
top of its docstring. CPython internally detects this line inside
PyCFunctionObject.__doc__ and skips past it, and there's a new getter at
PyCFunctionObject.__text_signature__ that returns just this line. As an
example, the signature for os.stat looks like this:
sig=($module, path, *, dir_fd=None, follow_symlinks=True)
The convention is, if you have this signature, you shouldn't have your
docstring start with a handwritten signature like 3.3 and before.
help() on a callable displays the signature automatically if it can, so
if you *also* had a handwritten signature, help() would show two
signatures. That would look dumb.
-
So here's the problem. Let's say you want to write an extension that
will work with Python 3.3 and 3.4, using the stable ABI. If you don't
add this line, then in 3.4 you won't have introspection information,
drat. But if you *do* add this line, your docstring will look mildly
stupid in 3.3, because it'll have this unsightly "sig=(" line at the
top. And it *won't* have a nice handwritten docstring. (And if you
added both a sig= signature *and* a handwritten signature, in 3.4 it
would display both. That would also look dumb.)
I think the solution adopted should be future-oriented (ie, clean in the
future) even if the cost is slight awkwardness in 3.3. To me, an
temporary 'unsightly' extra 'sig=' at the start of some docstrings, in
one release, is better that any of the permanent contortions you propose
to avoid it. For 3.3.5 Idle, I could add a check to detect and remove
'sig=' from calltips, but I would not consider it a disaster for it to
appear with earlier versions. In 3.3.5 (assuming no change is possible
for 3.3.4), help (or pydoc) could make the same check and deletion. As
with calltips, help is for interactive viewing by humans.
[snip]
O: Have the handwritten signature in the docstring. When registering
the function for 3.3, have the PyMethodDef docstring point to the it
starting at the signature. When registering the function for 3.4+, have
the docstring in the PyMethodDefEx point to the first byte after the
handwritten signature. Note that automatically skipping the signature
with a heuristic is mildly complicated, so this may be hard to get right.
The old convention builtins was a one line handwritten signature
followed by a blank line. For Python functions, one line describing the
return value. The 'heuristic' for Idle was to grab the first line of the
docstring. If than ended in mid-sentence because someone did not follow
the convention, too bad.
The newer convention for builtins is multiple lines followed by a blank
line. So I recently changed the heuristic to all lines up to the first
blank, but with a limit of 5 (needed for bytes), as protection against
doctrings that start with a long paragraph.
--
Terry Jan Reedy
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docstring hack for signature information has to go
On Mon, Feb 3, 2014 at 8:43 AM, Larry Hastings wrote:
>
>
> A quick summary of the context: currently in CPython 3.4, a builtin function
> can publish its "signature" as a specially encoded line at the top of its
> docstring. CPython internally detects this line inside
> PyCFunctionObject.__doc__ and skips past it, and there's a new getter at
> PyCFunctionObject.__text_signature__ that returns just this line. As an
> example, the signature for os.stat looks like this:
>
> sig=($module, path, *, dir_fd=None, follow_symlinks=True)
>
> The convention is, if you have this signature, you shouldn't have your
> docstring start with a handwritten signature like 3.3 and before. help() on
> a callable displays the signature automatically if it can, so if you *also*
> had a handwritten signature, help() would show two signatures. That would
> look dumb.
>
> -
>
> So here's the problem. Let's say you want to write an extension that will
> work with Python 3.3 and 3.4, using the stable ABI. If you don't add this
> line, then in 3.4 you won't have introspection information, drat. But if
> you *do* add this line, your docstring will look mildly stupid in 3.3,
> because it'll have this unsightly "sig=(" line at the top. And it *won't*
> have a nice handwritten docstring. (And if you added both a sig= signature
> *and* a handwritten signature, in 3.4 it would display both. That would
> also look dumb.)
What about just choosing a marker value that is somewhat less
unsightly? "signature = (", or "parameters: (", or something (better)
to that effect? It may not be beautiful in 3.3, but we can at least
make it make sense.
--
Zach
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docstring hack for signature information has to go
On Mon, Feb 3, 2014 at 8:04 AM, Larry Hastings wrote: > On 02/03/2014 07:08 AM, Barry Warsaw wrote: > > On Feb 03, 2014, at 06:43 AM, Larry Hastings wrote: > > > But that only fixes part of the problem. Our theoretical extension that > wants to be binary-compatible with 3.3 and 3.4 still has a problem: how can > they support signatures? They can't give PyMethodDefEx structures to 3.3, it > will blow up. But if they don't use PyMethodDefEx, they can't have > signatures. > > Can't an extension writer #ifdef around this? Yeah, it's ugly, but it's a > pretty standard approach for making C extensions multi-version compatible. > > > For source compatibility, yes. But I thought the point of the binary ABI > was to allow compiling a single extension that worked unmodified with > multiple versions of Python. If we simply don't support that, then an > ifdef would be fine. > Wouldn't your proposal to extend the PyMethodDef structure would require ifdef's and make it impossible to include the type information in something compiled against the 3.3 headers that you want to use in 3.4 without recompiling? If you don't like seeing an sig= at the front of the docstring couldn't you just move it to the end of the docstring. I don't think messiness in docstrings when running something read for 3.4 under 3.3 is a big deal. [side note] I consider it CRAZY for anyone to load a binary extension module compiled for one version in a later version of Python. People do it, I know, but they're insane. I wish we didn't bother trying to support that crap. I know this isn't going to change in 3.4. Just ranting. [/side note] -gps ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docstring hack for signature information has to go
On Mon, 03 Feb 2014 06:43:31 -0800 Larry Hastings wrote: > > A: We create a PyMethodDefEx structure with an extra field: "const char > *signature". We add a new METH_SIGNATURE (maybe just METH_SIG?) flag to > the flags, indicating that this is an extended structure. When > iterating over the PyMethodDefs, we know how far to advance the pointer > based on this flag. How do you create an array that mixes PyMethodDefs and PyMethodDefExs together? It sounds like METH_SIGNATURE is the wrong mechanism. Instead, you may want a tp_methods_ex as well as as a Py_TPFLAGS_HAVE_METHODS_EX. > B: Same as A, but we add three unused pointers (void *reserved1 etc) to > PyMethodDefEx to give us some room to grow. Note that this constrains future growth to only add pointer fields, unless you also add a couple of long fields. But at least it sounds workable. > C: Same as A, but we add two fields to PyMethodDefEx. The second new > field identifies the "version" of the structure, telling us its size > somehow. Like the lStructSize field of the OPENFILENAME structure in > Win32. I suspect YAGNI. That doesn't work. The size of elements of a C array is constant, so you can't "mix and match" PyMethodDefExs of different versions with different sizes each. > Solution: we write a function (which users would have to copy into their > extension) that gives a PyMethodDefEx array to 3.4+, but converts it > into a PyMethodDef array for 3.3. The tricky part there: what do we do > about the docstring? The convention for builtins is to have the first > line(s) contain a handwritten signature. But you *don't* want that if > you provide a signature, because help() will read that signature and > automatically render this first line for you. Uh... If you write a "conversion function", you may as well make it so it converts the "sig=" line to a plain signature line in 3.3, which avoids the issue entirely. (and how would that conversion function be shipped to the user anyway? Python 3.3 and the stable ABI don't have it) Regards Antoine. ___ Python-Dev mailing list [email protected] https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docstring hack for signature information has to go
On 02/03/2014 09:46 AM, Guido van Rossum wrote:
Can you summarize why neither of the two schemes you tried so far worked?
Certainly.
In the first attempt, the signature looked like this:
(arguments)\n
The "(arguments)" part of the string was 100% compatible with Python
syntax. So much so that I didn't write my own parser. Instead, I would
take the whole line, strip off the \n, prepend it with "def ", append it
with ": pass", and pass in the resulting string to ast.parse().
This had the advantage of looking great if the signature was not
mechanically separated from the rest of the docstring: it looked like
the old docstring with the handwritten signature on top.
The problem: false positives. This is also exactly the traditional
format for handwritten signatures. The function in C that mechanically
separated the signature from the rest of the docstring had a simple
heuristic: if the docstring started with "(", it
assumed it had a valid signature and separated it from the rest of the
docstring. But most of the functions in CPython passed this test, which
resulted in complaints like "help(open) eats first line":
http://bugs.python.org/issue20075
I opened an issue, writing a long impassioned plea to change this syntax:
http://bugs.python.org/issue20326
Which we did.
In the second attempt, the signature looked like this:
sig=(arguments)\n
In other words, the same as the first attempt, but with "sig=" instead
of the name of the function. Since you never see docstrings that start
with "sig=" in the wild, the false positives dropped to zero.
I also took the opportunity to modify the signature slightly. Signatures
were a little inconsistent about whether they specified the "self"
parameter or not, so there were some complicated heuristics in
inspect.Signature about when to keep or omit the first argument. In the
new format I made this more explicit: if the first argument starts with
a dollar sign ("$"), that means "this is a special first argument" (self
for methods, module for module-level callables, type for class methods
and __new__). That removed all the guesswork from inspect.Signature;
now it works great. (In case you're wondering: I still use ast.parse to
parse the signature, I just strip out the "$" first.)
I want to mention: we anticipate modifying the syntax further in 3.5,
adding square brackets around parameters to indicate "optional groups".
This all has caused no problems so far. But my panicky email last night
was me realizing a problem we may see down the road. To recap: if a
programmer writes a module using the binary ABI, in theory they can use
it with different Python versions without modification. If this
programmer added Python 3.4+ compatible signatures, they'd have to
insert this "sig=(" line at the top of their docstring. The downside:
Python 3.3 doesn't understand that this is a signature and would happily
display it to the user as part of help().
How bad would it be if we decided to just live with it or if we added
a new flag bit (only recognized by 3.4) to disambiguate corner-cases?
A new flag might solve the problem cheaply. Let's call it METH_SIG, set
in the flags portion of the PyMethodDef. It would mean "This docstring
contains a computer-readable signature". One could achieve source
compatibility with 3.3 easily by adding "#ifndef METH_SIG / #define
METH_SIG 0 / #endif"; the next version of 3.3 could add that itself. We
could then switch back to the original approach of
"(", so the signature would look presentable when
displayed to the user. It would still have the funny dollar-sign, a la
"$self" or "$module" or "$type", but perhaps users could live with
that. Though perhaps this time maybe the end delimiter should be two
newlines in a row, so that we can text-wrap long signature lines to
enhance their readability if/when they get shown to users.
I have two caveats:
A: for binary compatibility, would Python 3.3 be allergic to this
unfamiliar flag in PyMethodDef? Or does it ignore flags it doesn't
explicitly look for?
B: I had to modify four (or was it five?) different types in Python to
add support for mechanically separating the __text_signature__. Although
all of them originally started with a PyMethodDef structure, I'm not
sure that all of them carry the "flags" parameter around with them. We
might have to add a "flags" to a couple of these. Fortunately I believe
they're all part of Py_LIMITED_API.
//arry/
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] The docstring hack for signature information has to go
Hmm... I liked the original scheme because it doesn't come out so badly if
some tool doesn't special-case the first line of the docstring at all. (I
have to fess up that I wrote such a tool for a limited case not too long
ago, and I wrote it to search for a blank line if the docstring starts with
followed by '('.)
Adding a flag sounds harmless, all the code I could find that looks at them
just checks whether specific flags it knows about are set.
But why do you even need a flag? Reading issue 20075 where the complaint
started, it really feels that the change was an overreaction to a very
minimal problem. A few docstrings appear truncated. Big deal. We can
rewrite the ones that are reported as broken (either by adjusting the
docstring to not match the patter or by adjusting it to match the pattern
better, depending on the case). Tons of docstrings contain incorrect info,
we just fix them when we notice the issue, we don't declare the language
broken.
On Mon, Feb 3, 2014 at 5:29 PM, Larry Hastings wrote:
>
>
> On 02/03/2014 09:46 AM, Guido van Rossum wrote:
>
> Can you summarize why neither of the two schemes you tried so far worked?
>
>
> Certainly.
>
> In the first attempt, the signature looked like this:
>
> (arguments)\n
>
> The "(arguments)" part of the string was 100% compatible with Python
> syntax. So much so that I didn't write my own parser. Instead, I would
> take the whole line, strip off the \n, prepend it with "def ", append it
> with ": pass", and pass in the resulting string to ast.parse().
>
> This had the advantage of looking great if the signature was not
> mechanically separated from the rest of the docstring: it looked like the
> old docstring with the handwritten signature on top.
>
> The problem: false positives. This is also exactly the traditional format
> for handwritten signatures. The function in C that mechanically separated
> the signature from the rest of the docstring had a simple heuristic: if the
> docstring started with "(", it assumed it had a valid
> signature and separated it from the rest of the docstring. But most of the
> functions in CPython passed this test, which resulted in complaints like
> "help(open) eats first line":
>
> http://bugs.python.org/issue20075
>
> I opened an issue, writing a long impassioned plea to change this syntax:
>
> http://bugs.python.org/issue20326
>
> Which we did.
>
>
> In the second attempt, the signature looked like this:
>
> sig=(arguments)\n
>
> In other words, the same as the first attempt, but with "sig=" instead of
> the name of the function. Since you never see docstrings that start with
> "sig=" in the wild, the false positives dropped to zero.
>
> I also took the opportunity to modify the signature slightly. Signatures
> were a little inconsistent about whether they specified the "self"
> parameter or not, so there were some complicated heuristics in
> inspect.Signature about when to keep or omit the first argument. In the
> new format I made this more explicit: if the first argument starts with a
> dollar sign ("$"), that means "this is a special first argument" (self for
> methods, module for module-level callables, type for class methods and
> __new__). That removed all the guesswork from inspect.Signature; now it
> works great. (In case you're wondering: I still use ast.parse to parse the
> signature, I just strip out the "$" first.)
>
> I want to mention: we anticipate modifying the syntax further in 3.5,
> adding square brackets around parameters to indicate "optional groups".
>
> This all has caused no problems so far. But my panicky email last night
> was me realizing a problem we may see down the road. To recap: if a
> programmer writes a module using the binary ABI, in theory they can use it
> with different Python versions without modification. If this programmer
> added Python 3.4+ compatible signatures, they'd have to insert this "sig=("
> line at the top of their docstring. The downside: Python 3.3 doesn't
> understand that this is a signature and would happily display it to the
> user as part of help().
>
>
>
> How bad would it be if we decided to just live with it or if we added a
> new flag bit (only recognized by 3.4) to disambiguate corner-cases?
>
>
> A new flag might solve the problem cheaply. Let's call it METH_SIG, set
> in the flags portion of the PyMethodDef. It would mean "This docstring
> contains a computer-readable signature". One could achieve source
> compatibility with 3.3 easily by adding "#ifndef METH_SIG / #define
> METH_SIG 0 / #endif"; the next version of 3.3 could add that itself. We
> could then switch back to the original approach of "(",
> so the signature would look presentable when displayed to the user. It
> would still have the funny dollar-sign, a la "$self" or "$module" or
> "$type", but perhaps users could live with that. Though perhaps this time
> maybe the end delimiter should be two newlines in a row, so that we can
> text-wrap long signature lines t
Re: [Python-Dev] The docstring hack for signature information has to go
Larry Hastings writes:
> In the second attempt, the signature looked like this:
>
>sig=(arguments)\n
>
[...]
> This all has caused no problems so far. But my panicky email last
> night was me realizing a problem we may see down the road. To recap:
> if a programmer writes a module using the binary ABI, in theory they
> can use it with different Python versions without modification. If
> this programmer added Python 3.4+ compatible signatures, they'd have
> to insert this "sig=(" line at the top of their docstring. The
> downside: Python 3.3 doesn't understand that this is a signature and
> would happily display it to the user as part of help().
I think this is not a bug, it's a feature. Since 3.3 users don't have
the special signature parser either, this gives them exactly the
information they need and without any duplication. The only drawback is
in the cosmetic "sig=" prefix -- but that's the right amount of
non-intrusive, kind nudging to get people to eventually update.
>> How bad would it be if we decided to just live with it or if we
>> added a new flag bit (only recognized by 3.4) to disambiguate
>> corner-cases?
>
> A new flag might solve the problem cheaply. Let's call it METH_SIG,
> set in the flags portion of the PyMethodDef. It would mean "This
> docstring contains a computer-readable signature". One could achieve
> source compatibility with 3.3 easily by adding "#ifndef METH_SIG /
> #define METH_SIG 0 / #endif"; the next version of 3.3 could add that
> itself. We could then switch back to the original approach of
> "(", so the signature would look presentable when
[...]
That much effort to fix a purely cosmetic problem showing up only in
older releases? Note that it's going to be a while until machine
generated signatures have actually trickled down to end-users, so it's
not as if every 3.3 installation would suddenly show different
docstrings for all modules.
Just my $0.02 of course.
Best,
Nikolaus
--
Encrypted emails preferred.
PGP fingerprint: 5B93 61F8 4EA2 E279 ABF6 02CF A9AD B7F8 AE4E 425C
»Time flies like an arrow, fruit flies like a Banana.«
___
Python-Dev mailing list
[email protected]
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
