Re: [Python-Dev] Status of the Argument Clinic DSL

[Larry Hastings]

On 08/04/2016 11:58 PM, Nick Coghlan wrote:

I occasionally wonder if we should document the "/" notation in
https://docs.python.org/3/library/inspect.html#introspecting-callables-with-the-signature-object
as it can sometimes show up in the text representation of signature
objects:

 >>> print(inspect.signature(format))
 (value, format_spec='', /)


I think we probably should.  After all, this same string is used for pydoc:

>>> import os
>>> help(os.execv)
   Help on built-in function execv in module posix:

   execv(path, argv, /)
Execute an executable path with arguments, replacing current
   process.

path
  Path of executable file.
argv
  Tuple or list of strings.

so it's easily user-visible.

I've always found it  a little strange that the signatures for functions 
using Py_ArgParseTuple() had this / in them that wasn't Python syntax.  
On the other hand, it accurately reflects the fact that these functions 
have signatures that you can't write in Python.


(And, FWIW, I wasn't the person who added the code that made "/" start 
showing up in the text representations of signatures.  I was waffling on 
it, then someone else JFDI, to quote Barry.)



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the Argument Clinic DSL

[Nick Coghlan]

On 5 August 2016 at 09:12, Larry Hastings  wrote:
> / is the delimiter between positional-only parameters and
> positional-or-keyword arguments.  It's not actual Python syntax, but Guido
> said (somewhere) that *if* Python ever sprouted a syntax for positional-only
> parameters, that was as good a delimiter as any.  I think he picked it
> because / is the inverse of *.

I was thinking Guido picked the "/" symbol when we were discussing the
Argument Clinic DSL at PyCon 2014 and you had also been working on
https://www.python.org/dev/peps/pep-0457/

However, re-reading the latter shows it dates back earlier than that:
https://www.python.org/dev/peps/pep-0457/#guido

> If you have more questions about __text_signature__, I recommend reading the
> implementation of inspect.signature, since that's the one and only consumer
> of it.

I occasionally wonder if we should document the "/" notation in
https://docs.python.org/3/library/inspect.html#introspecting-callables-with-the-signature-object
as it can sometimes show up in the text representation of signature
objects:

>>> print(inspect.signature(format))
(value, format_spec='', /)

Likewise for describing the option of using __text_signature__ to
override the default inspect.signature() output (and perhaps adding a
public "inspect.Signature.from_text" alternate constructor at the same
time).

However, either notion is a bit tricky while PEP 457 is still in draft
rather than accepted - even if it's "just" a syntax for overriding
inspect.signature, we'd still be effectively locking this in as *the*
syntax for positional-only arguments (if they're ever added)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the Argument Clinic DSL

[eryk sun]

On Thu, Aug 4, 2016 at 11:33 PM, Alexander Belopolsky
 wrote:
>
> On Thu, Aug 4, 2016 at 7:12 PM, Larry Hastings  wrote:
>>
>> C extension functions get the module passed in automatically, but this is
>> done internally and from the Python level you can't see it.
>
> Always something new to learn!  This was not so in Python 2.x - self was
> passed as NULL to the C module functions.  When did this change?

In 2.x this is the `self` parameter (actually named "passthrough" in
the source) of Py_InitModule4 [1, 2]. You probably use the
Py_InitModule or Py_InitModule3 macros, which pass NULL for this
parameter:

#define Py_InitModule(name, methods) \
Py_InitModule4(name, methods, (char *)NULL, (PyObject *)NULL, \
   PYTHON_API_VERSION)

#define Py_InitModule3(name, methods, doc) \
Py_InitModule4(name, methods, doc, (PyObject *)NULL, \
   PYTHON_API_VERSION)

Python 3's PyModule_Create2 [3-5] API makes this a reference to the
module. It's currently implemented in
PyModule_AddFunctions [6, 7].

[1]: https://docs.python.org/2/c-api/allocation.html#c.Py_InitModule4
[2]: https://hg.python.org/cpython/file/v2.7.12/Python/modsupport.c#l31
[3]: https://docs.python.org/3/c-api/module.html#c.PyModule_Create2
[4]: https://hg.python.org/cpython/file/v3.5.2/Objects/moduleobject.c#l133
[5]: https://hg.python.org/cpython/file/v3.0b1/Objects/moduleobject.c#l63
[6]: https://docs.python.org/3/c-api/module.html#c.PyModule_AddFunctions
[7]: https://hg.python.org/cpython/file/v3.5.2/Objects/moduleobject.c#l387
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the Argument Clinic DSL

[Alexander Belopolsky]

On Thu, Aug 4, 2016 at 7:12 PM, Larry Hastings  wrote:

> C extension functions get the module passed in automatically, but this is
> done internally and from the Python level you can't see it.


Always something new to learn!  This was not so in Python 2.x - self was
passed as NULL to the C module functions.  When did this change?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the Argument Clinic DSL

[Larry Hastings]

On 08/04/2016 03:45 PM, Alexander Belopolsky wrote:


On Thu, Aug 4, 2016 at 2:19 PM, Larry Hastings > wrote:


AFAIK the Clinic DSL can handle all of Python's C extensions.  I
have no plans to "revise the whole approach"; if someone else does
I haven't heard about it.


I was just wondering that with so much effort to bring typing to the 
mainstream, a more "pythonic" DSL may emerge for describing the 
signatures of functions in C modules.


BTW, is there any document describing the syntax of "text signatures" 
such as:


>>> os.rename.__text_signature__
'($module, /, src, dst, *, src_dir_fd=None, dst_dir_fd=None)'

?

What does the "$module, /," part mean?



/ is the delimiter between positional-only parameters and 
positional-or-keyword arguments.  It's not actual Python syntax, but 
Guido said (somewhere) that *if* Python ever sprouted a syntax for 
positional-only parameters, that was as good a delimiter as any.  I 
think he picked it because / is the inverse of *.


The "$" in "$module" means it's what we called a "bound parameter", a 
parameter which gets bound to a value before Python ever sees it.  C 
extension functions get the module passed in automatically, but this is 
done internally and from the Python level you can't see it.  So it's 
accurate to present it there, but we suppress it before we compute the 
inspect.signature.  For example, os_chdir_impl in Modules/posixmodule.c 
takes two arguments, the first being the module, the second being the 
path; inspect.signature(os.chdir) only shows one parameter, the path.


__text_signature__, although user-visible, is not considered information 
for users.  It's an internal implementation detail like co_code.  
However, with the exception of things like "$module" and "/" it's just a 
Python function declaration. Literally we remove the funny bits like 
"$module" and "/", prepend that string with "def foo", append that 
string with ": pass", and parse the result with ast.parse.


If you have more questions about __text_signature__, I recommend reading 
the implementation of inspect.signature, since that's the one and only 
consumer of it.



//arry/

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the Argument Clinic DSL

[Brett Cannon]

On Thu, 4 Aug 2016 at 15:47 Alexander Belopolsky <
alexander.belopol...@gmail.com> wrote:

>
> On Thu, Aug 4, 2016 at 2:19 PM, Larry Hastings  wrote:
>
>> AFAIK the Clinic DSL can handle all of Python's C extensions.  I have no
>> plans to "revise the whole approach"; if someone else does I haven't heard
>> about it.
>
>
> I was just wondering that with so much effort to bring typing to the
> mainstream, a more "pythonic" DSL may emerge for describing the signatures
> of functions in C modules.
>
> BTW, is there any document describing the syntax of "text signatures" such
> as:
>
> >>> os.rename.__text_signature__
> '($module, /, src, dst, *, src_dir_fd=None, dst_dir_fd=None)'
>
> ?
>
> What does the "$module, /," part mean?
>

Don't remember what the $module means but the / means positional-only
arguments.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the Argument Clinic DSL

[Alexander Belopolsky]

On Thu, Aug 4, 2016 at 7:00 PM, Brett Cannon  wrote:
>> >>> os.rename.__text_signature__
>> '($module, /, src, dst, *, src_dir_fd=None, dst_dir_fd=None)'
>>
>> ?
>>
>> What does the "$module, /," part mean?
>
>
> Don't remember what the $module means but the / means positional-only
> arguments.

Not in this case.  Apparently the / applies to the $module thingy
because src and dst are not positional only (and the / would be after
them if they were, I think.)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the Argument Clinic DSL

[Alexander Belopolsky]

On Thu, Aug 4, 2016 at 2:19 PM, Larry Hastings  wrote:

> AFAIK the Clinic DSL can handle all of Python's C extensions.  I have no
> plans to "revise the whole approach"; if someone else does I haven't heard
> about it.


I was just wondering that with so much effort to bring typing to the
mainstream, a more "pythonic" DSL may emerge for describing the signatures
of functions in C modules.

BTW, is there any document describing the syntax of "text signatures" such
as:

>>> os.rename.__text_signature__
'($module, /, src, dst, *, src_dir_fd=None, dst_dir_fd=None)'

?

What does the "$module, /," part mean?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the Argument Clinic DSL

[Larry Hastings]

On 08/04/2016 09:11 AM, Alexander Belopolsky wrote:

Furthermore, the 3.4 release notes contain a warning
saying "The Argument Clinic PEP is not fully up to date with the state
of the implementation."  It does not look like this situation has
improved since.


It hasn't.  The PEP lags behind the implementation.  The HOWTO is 
up-to-date though.




The practical question is: does it make sense to invest time in
learning the clinic DSL and converting more modules (e.g. datetime) to
it or it is likely that the whole approach will be revised in the
future?


AFAIK the Clinic DSL can handle all of Python's C extensions.  I have no 
plans to "revise the whole approach"; if someone else does I haven't 
heard about it.



//arry/
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Status of the Argument Clinic DSL

[Brett Cannon]

On Thu, 4 Aug 2016 at 09:14 Alexander Belopolsky <
alexander.belopol...@gmail.com> wrote:

> What is the current status of the Argument Clinic DSL?  The clinic
> preprocessor was released [1] with Python 3.4 with a promise [2] "that
> signature metadata for programmatic introspection will be added to
> additional callables implemented in C as part of Python 3.4
> maintenance releases."  We are now close to 3.6 beta and clinic
> coverage in stdlib is sparse at best.  For example, the math module
> makes no use of the clinic (but interestingly a few methods in cmath
> do).  The documentation is very sparse as well.  While one would
> expect PEP 436 to serve as *the* documentation while clinic is being
> developed, but the PEP has the "Draft" status and has not been updated
> since 2013.  Furthermore, the 3.4 release notes contain a warning
> saying "The Argument Clinic PEP is not fully up to date with the state
> of the implementation."  It does not look like this situation has
> improved since.
>

I always read https://docs.python.org/3/howto/clinic.html as the
documentation and not the PEP.


>
> The practical question is: does it make sense to invest time in
> learning the clinic DSL and converting more modules (e.g. datetime) to
> it


Yes. Part of the issue is there was a big push to convert modules for the
3.4 release and some of us helped convert however many we can, but then
people stopped putting the time in to convert code a module at a time once
3.4 was released. Serhiy has actually been working on a patch to speed up
keyword parsing which will automatically be applied to code using Argument
Clinic: http://bugs.python.org/issue27574 .


> or it is likely that the whole approach will be revised in the
> future?
>
>
I don't see it being revised, but maybe Larry has something to say on the
matter.


> [1] http://bugs.python.org/issue16612
> [2]
> https://docs.python.org/3.6/whatsnew/3.4.html?highlight=clinic#pep-436-argument-clinic
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Status of the Argument Clinic DSL

[Alexander Belopolsky]

What is the current status of the Argument Clinic DSL?  The clinic
preprocessor was released [1] with Python 3.4 with a promise [2] "that
signature metadata for programmatic introspection will be added to
additional callables implemented in C as part of Python 3.4
maintenance releases."  We are now close to 3.6 beta and clinic
coverage in stdlib is sparse at best.  For example, the math module
makes no use of the clinic (but interestingly a few methods in cmath
do).  The documentation is very sparse as well.  While one would
expect PEP 436 to serve as *the* documentation while clinic is being
developed, but the PEP has the "Draft" status and has not been updated
since 2013.  Furthermore, the 3.4 release notes contain a warning
saying "The Argument Clinic PEP is not fully up to date with the state
of the implementation."  It does not look like this situation has
improved since.

The practical question is: does it make sense to invest time in
learning the clinic DSL and converting more modules (e.g. datetime) to
it or it is likely that the whole approach will be revised in the
future?

[1] http://bugs.python.org/issue16612
[2] 
https://docs.python.org/3.6/whatsnew/3.4.html?highlight=clinic#pep-436-argument-clinic
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com