Re: [Python-Dev] API bloat
Thanks to everyone for all their comments so far. Martin v. Löwis wrote: The functions may have been add to CPython 8 years ago, but they were added to the API when they appeared in the docs, between 3.1 and 3.1.3. How is the API defined, if not by the documentation? Just to stress and support Georg's explanation: the API is *not* defined through the documentation, but instead primarily through the header files. All functions declared as PyAPI_FUNC and not starting with _Py are public API. There used to be a lot of undocumented API (up to 1.4, there was no API documentation at all, only the extension module tutorial); these days, more and more API gets documented. Doing a search for the regex: "PyAPI_FUNC\([^)]*\) *Py" in .h files, which should match API functions (functions starting _Py are excluded) gives the following result: Version matches 3.0 717 3.1.3 728 3.2b2 743 It would appear the API bloat is real, not just an artefact of updated docs. The "what's new for 3.2" API section: http://docs.python.org/dev/py3k/whatsnew/3.2.html#build-and-c-api-changes lists 6 new functions, yet according to my search, 15 have been added between 3.1.3 and 3.2b2. Of course 743 functions is about 700 too many, but that's a discussion for Python-ideas and PEP 384. Mark. HTH, Martin ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
On Thu, Feb 10, 2011 at 8:16 PM, Mark Shannon wrote: > Doing a search for the regex: "PyAPI_FUNC\([^)]*\) *Py" in .h files, > which should match API functions (functions starting _Py are excluded) gives > the following result: > > Version matches > 3.0 717 > 3.1.3 728 > 3.2b2 743 > > It would appear the API bloat is real, > not just an artefact of updated docs. Since it doesn't account for #ifdef, a naive count like that isn't a valid basis for comparison. I would hazard a guess that a substantial amount of the additional numbers there are going to be from new alternative definitions of Py_hash_t and some of the Py_UNICODE macros. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
Mark Shannon, 10.02.2011 11:16: Of course 743 functions is about 700 too many, Sorry, but that's so wrong, it's just being mean. What do you expect from a platform that has grown for more than 20 years? And which has been the only (real) Python implementation for most of that time. I'm sure it'll be easy to find, say, 200 functions that are not really required (PySequence_Fast?), but many of those make the life easier for C programmers and/or have helped with portability in the past. Most of those "C helper functions" long predate the dedicated extension writer tools that bring relieve from calling them. Mind you, even the Cython project is only some 4 years old, and even there you will still want to call a C-API function or two from time to time. Seriously, CPython's C-API has always been a *reason* for it to be popular. Many important tools wouldn't even exist without it, and sure wouldn't be in their current state if the C-API had not evolved together with them. Just think of NumPy, which is one (if not *the* one) of the main attractors for other Python implementations to get their view of the C-API implemented. Stefan ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88385 - python/branches/py3k/Doc/whatsnew/3.2.rst
On Thu, Feb 10, 2011 at 7:20 PM, raymond.hettinger
wrote:
> +The :func:`logging.basicConfig` set-up function gained a *style* argument to
> +support three different types of string formatting. It defaults to "%" for
> +traditional %-formatting, can be set to "{" for the new :meth:`str.format`
> style, or
> +can be set to "$" for the shell-style formatting provided by
> +:class:`string.Template`. The following three configurations are
> equivalent::
> +
> + >>> from logging import basicConfig
> + >>> basicConfig(style='%', format="%(name)s -> %(levelname)s:
> %(message)s")
> + >>> basicConfig(style='{', format="{name} -> {levelname} {message}")
> + >>> basicConfig(style='$', format="$name -> $levelname: $message")
It may be worth noting here that:
1. the "style" parameter also exists for logging.Formatter objects
2. it only applies to the output formatting for output, individual
logging calls are still constrained to using %-formatting by backwards
compatibility issues
(Especially point 2 - I briefly forgot that distinction myself, and I
helped review this feature when Vinay added it)
Cheers,
Nick.
--
Nick Coghlan | [email protected] | Brisbane, Australia
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
Nick Coghlan wrote: On Thu, Feb 10, 2011 at 8:16 PM, Mark Shannon wrote: Doing a search for the regex: "PyAPI_FUNC\([^)]*\) *Py" in .h files, which should match API functions (functions starting _Py are excluded) gives the following result: Version matches 3.0 717 3.1.3 728 3.2b2 743 It would appear the API bloat is real, not just an artefact of updated docs. Since it doesn't account for #ifdef, a naive count like that isn't a valid basis for comparison. OK. How about this: egrep -ho '#.*PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h finds no matches. egrep -ho 'PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h | sort -u This finds all matches and removes duplicates, so anything defined multiple time in branches of #ifdef blocks, will only be counted once. Version matches 3.0 714 3.1.3 725 3.2b2 739 So given, the revised numbers; The "what's new for 3.2" API section: http://docs.python.org/dev/py3k/whatsnew/3.2.html#build-and-c-api-changes lists 6 new functions, yet 14 have been added between 3.1.3 and 3.2b2. I would hazard a guess that a substantial amount of the additional numbers there are going to be from new alternative definitions of Py_hash_t and some of the Py_UNICODE macros. Unless there is a PyAPI_FUNC involved, these won't match the regex. Cheers, Nick. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
Mark Shannon wrote: > Nick Coghlan wrote: >> On Thu, Feb 10, 2011 at 8:16 PM, Mark Shannon >> wrote: >>> Doing a search for the regex: "PyAPI_FUNC\([^)]*\) *Py" in .h files, >>> which should match API functions (functions starting _Py are >>> excluded) gives >>> the following result: >>> >>> Version matches >>> 3.0 717 >>> 3.1.3 728 >>> 3.2b2 743 >>> >>> It would appear the API bloat is real, >>> not just an artefact of updated docs. >> >> Since it doesn't account for #ifdef, a naive count like that isn't a >> valid basis for comparison. >> > OK. How about this: > > egrep -ho '#.*PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h > finds no matches. > > egrep -ho 'PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h | sort -u > > This finds all matches and removes duplicates, so anything defined > multiple time in branches of #ifdef blocks, will only be counted once. > > Version matches > 3.0 714 > 3.1.3 725 > 3.2b2 739 Given these numbers, I don't think the subject line really captures the problem accurately enough ... a 2% increase in number of API function per release can hardly be called API bloat :-) > So given, the revised numbers; > > The "what's new for 3.2" API section: > http://docs.python.org/dev/py3k/whatsnew/3.2.html#build-and-c-api-changes > lists 6 new functions, yet 14 have been added between 3.1.3 and 3.2b2. Could you identify the ones that are not yet documented ? That would be useful. Thanks, -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 10 2011) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
On Thu, Feb 10, 2011 at 11:51 PM, M.-A. Lemburg wrote: >> The "what's new for 3.2" API section: >> http://docs.python.org/dev/py3k/whatsnew/3.2.html#build-and-c-api-changes >> lists 6 new functions, yet 14 have been added between 3.1.3 and 3.2b2. > > Could you identify the ones that are not yet documented ? >From Misc/NEWS: PyUnicode_TransformDecimalToASCII PyErr_SyntaxLocationEx PyArg_ValidateKeywordArguments PyFrame_GetLineNumber PyCode_NewEmpty I would guess that a lot of the other ones that aren't explicitly documented in NEWS or What's New are the assorted utilities Victor added in order to properly support non-ASCII entries in sys.path, as well as all the additional APIs to handle both bytes and unicode interfaces to the external environment. In scanning NEWS, I also noticed that there were quite a few ancient APIs that were finally killed off in 3.2, so the number of new functions is likely to actually exceed the 14 picked up by Mark's measurements. Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
M.-A. Lemburg wrote: Mark Shannon wrote: Nick Coghlan wrote: On Thu, Feb 10, 2011 at 8:16 PM, Mark Shannon wrote: Doing a search for the regex: "PyAPI_FUNC\([^)]*\) *Py" in .h files, which should match API functions (functions starting _Py are excluded) gives the following result: Version matches 3.0 717 3.1.3 728 3.2b2 743 It would appear the API bloat is real, not just an artefact of updated docs. Since it doesn't account for #ifdef, a naive count like that isn't a valid basis for comparison. OK. How about this: egrep -ho '#.*PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h finds no matches. egrep -ho 'PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h | sort -u This finds all matches and removes duplicates, so anything defined multiple time in branches of #ifdef blocks, will only be counted once. Version matches 3.0 714 3.1.3 725 3.2b2 739 Given these numbers, I don't think the subject line really captures the problem accurately enough ... a 2% increase in number of API function per release can hardly be called API bloat :-) So given, the revised numbers; The "what's new for 3.2" API section: http://docs.python.org/dev/py3k/whatsnew/3.2.html#build-and-c-api-changes lists 6 new functions, yet 14 have been added between 3.1.3 and 3.2b2. Could you identify the ones that are not yet documented ? That would be useful. Here's the details: The following API functions were removed from 3.1.3: PyAST_Compile PyCObject_AsVoidPtr PyCObject_FromVoidPtr PyCObject_FromVoidPtrAndDesc PyCObject_GetDesc PyCObject_Import PyCObject_SetVoidPtr PyCode_CheckLineNumber Py_CompileStringFlags PyEval_CallObject PyOS_ascii_atof PyOS_ascii_formatd PyOS_ascii_strtod PyThread_exit_prog PyThread__PyThread_exit_prog PyThread__PyThread_exit_thread PyUnicode_SetDefaultEncoding And the following were added to 3.2, of which only 2 are documented: PyArg_ValidateKeywordArguments PyAST_CompileEx Py_CompileString Py_CompileStringExFlags PyErr_NewExceptionWithDoc(documented) PyErr_SyntaxLocationEx PyErr_WarnFormat PyFrame_GetLineNumber PyImport_ExecCodeModuleWithPathnames PyImport_GetMagicTag PyLong_AsLongLongAndOverflow(documented) PyModule_GetFilenameObject Py_SetPath PyStructSequence_GetItem PyStructSequence_NewType PyStructSequence_SetItem PySys_AddWarnOptionUnicode PySys_AddXOption PySys_FormatStderr PySys_FormatStdout PySys_GetXOptions PyThread_acquire_lock_timed PyType_FromSpec PyUnicode_AsUnicodeCopy PyUnicode_AsWideCharString PyUnicode_EncodeFSDefault PyUnicode_FSDecoder Py_UNICODE_strcat Py_UNICODE_strncmp Py_UNICODE_strrchr PyUnicode_TransformDecimalToASCII For added confusion PySys_SetArgvEx is documented as new in 3.2, but exists in 3.1.3 That should keep someone busy ;) Note that this only include functions. The API also includes a number of macros such as Py_False and Py_RETURN_FALSE, types , and data like PyBool_Type. I've not tried to analyse any of these. Mark. Thanks, ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
Mark Shannon wrote: > M.-A. Lemburg wrote: >> Mark Shannon wrote: >>> Nick Coghlan wrote: On Thu, Feb 10, 2011 at 8:16 PM, Mark Shannon wrote: > Doing a search for the regex: "PyAPI_FUNC\([^)]*\) *Py" in .h files, > which should match API functions (functions starting _Py are > excluded) gives > the following result: > > Version matches > 3.0 717 > 3.1.3 728 > 3.2b2 743 > > It would appear the API bloat is real, > not just an artefact of updated docs. Since it doesn't account for #ifdef, a naive count like that isn't a valid basis for comparison. >>> OK. How about this: >>> >>> egrep -ho '#.*PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h >>> finds no matches. >>> >>> egrep -ho 'PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h | sort -u >>> >>> This finds all matches and removes duplicates, so anything defined >>> multiple time in branches of #ifdef blocks, will only be counted once. >>> >>> Version matches >>> 3.0 714 >>> 3.1.3 725 >>> 3.2b2 739 >> >> Given these numbers, I don't think the subject line really >> captures the problem accurately enough ... a 2% increase >> in number of API function per release can hardly be called >> API bloat :-) >> >>> So given, the revised numbers; >>> >>> The "what's new for 3.2" API section: >>> http://docs.python.org/dev/py3k/whatsnew/3.2.html#build-and-c-api-changes >>> >>> lists 6 new functions, yet 14 have been added between 3.1.3 and 3.2b2. >> >> Could you identify the ones that are not yet documented ? >> >> That would be useful. > > Here's the details: > > The following API functions were removed from 3.1.3: > > PyAST_Compile > PyCObject_AsVoidPtr > PyCObject_FromVoidPtr > PyCObject_FromVoidPtrAndDesc > PyCObject_GetDesc > PyCObject_Import > PyCObject_SetVoidPtr > PyCode_CheckLineNumber > Py_CompileStringFlags > PyEval_CallObject > PyOS_ascii_atof > PyOS_ascii_formatd > PyOS_ascii_strtod > PyThread_exit_prog > PyThread__PyThread_exit_prog > PyThread__PyThread_exit_thread > PyUnicode_SetDefaultEncoding > > And the following were added to 3.2, > of which only 2 are documented: > > PyArg_ValidateKeywordArguments > PyAST_CompileEx > Py_CompileString > Py_CompileStringExFlags > PyErr_NewExceptionWithDoc(documented) > PyErr_SyntaxLocationEx > PyErr_WarnFormat > PyFrame_GetLineNumber > PyImport_ExecCodeModuleWithPathnames > PyImport_GetMagicTag > PyLong_AsLongLongAndOverflow(documented) > PyModule_GetFilenameObject > Py_SetPath > PyStructSequence_GetItem > PyStructSequence_NewType > PyStructSequence_SetItem > PySys_AddWarnOptionUnicode > PySys_AddXOption > PySys_FormatStderr > PySys_FormatStdout > PySys_GetXOptions > PyThread_acquire_lock_timed > PyType_FromSpec > PyUnicode_AsUnicodeCopy > PyUnicode_AsWideCharString > PyUnicode_EncodeFSDefault > PyUnicode_FSDecoder > Py_UNICODE_strcat > Py_UNICODE_strncmp > Py_UNICODE_strrchr > PyUnicode_TransformDecimalToASCII > > For added confusion PySys_SetArgvEx is documented as > new in 3.2, but exists in 3.1.3 > > That should keep someone busy ;) > > Note that this only include functions. > The API also includes a number of macros such as > Py_False and Py_RETURN_FALSE, types , > and data like PyBool_Type. > > I've not tried to analyse any of these. Thanks. I opened http://bugs.python.org/issue11173 for this. -- Marc-Andre Lemburg eGenix.com Professional Python Services directly from the Source (#1, Feb 10 2011) >>> Python/Zope Consulting and Support ...http://www.egenix.com/ >>> mxODBC.Zope.Database.Adapter ... http://zope.egenix.com/ >>> mxODBC, mxDateTime, mxTextTools ...http://python.egenix.com/ ::: Try our new mxODBC.Connect Python Database Interface for free ! eGenix.com Software, Skills and Services GmbH Pastor-Loeh-Str.48 D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg Registered at Amtsgericht Duesseldorf: HRB 46611 http://www.egenix.com/company/contact/ ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
Le jeudi 10 février 2011 à 16:19 +0100, M.-A. Lemburg a écrit : > > And the following were added to 3.2, > > of which only 2 are documented: > > > > PyArg_ValidateKeywordArguments > > PyAST_CompileEx > > Py_CompileString > > Py_CompileStringExFlags > > PyErr_NewExceptionWithDoc(documented) > > PyErr_SyntaxLocationEx > > PyErr_WarnFormat > > PyFrame_GetLineNumber > > PyImport_ExecCodeModuleWithPathnames > > PyImport_GetMagicTag > > PyLong_AsLongLongAndOverflow(documented) > > PyModule_GetFilenameObject > > Py_SetPath > > PyStructSequence_GetItem > > PyStructSequence_NewType > > PyStructSequence_SetItem > > PySys_AddWarnOptionUnicode > > PySys_AddXOption > > PySys_FormatStderr > > PySys_FormatStdout > > PySys_GetXOptions > > PyThread_acquire_lock_timed > > PyType_FromSpec > > PyUnicode_AsUnicodeCopy > > PyUnicode_AsWideCharString > > PyUnicode_EncodeFSDefault > > PyUnicode_FSDecoder > > Py_UNICODE_strcat > > Py_UNICODE_strncmp > > Py_UNICODE_strrchr > > PyUnicode_TransformDecimalToASCII PyErr_WarnFormat, PyImport_ExecCodeModuleWithPathnames, PyModule_GetFilenameObject, PySys_AddWarnOptionUnicode, PySys_FormatStderr, PySys_FormatStdout, PyUnicode_AsUnicodeCopy, PyUnicode_AsWideCharString, PyUnicode_EncodeFSDefault and PyUnicode_FSDecoder are documented (I wrote most of these functions). I added references in the issue #11173. So there are a little bit less than 29/31 of new undocumented functions. But yes, most Py_UNICODE_* functions are not documented: see issue #10435 (which has a patch). victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
And the following were added to 3.2, of which only 2 are documented: PyArg_ValidateKeywordArguments PyAST_CompileEx Py_CompileString Py_CompileStringExFlags PyErr_NewExceptionWithDoc(documented) PyErr_SyntaxLocationEx PyErr_WarnFormat PyFrame_GetLineNumber PyImport_ExecCodeModuleWithPathnames PyImport_GetMagicTag PyLong_AsLongLongAndOverflow(documented) PyModule_GetFilenameObject Py_SetPath PyStructSequence_GetItem PyStructSequence_NewType PyStructSequence_SetItem PySys_AddWarnOptionUnicode PySys_AddXOption PySys_FormatStderr PySys_FormatStdout PySys_GetXOptions PyThread_acquire_lock_timed PyType_FromSpec PyUnicode_AsUnicodeCopy PyUnicode_AsWideCharString PyUnicode_EncodeFSDefault PyUnicode_FSDecoder Py_UNICODE_strcat Py_UNICODE_strncmp Py_UNICODE_strrchr PyUnicode_TransformDecimalToASCII For added confusion PySys_SetArgvEx is documented as new in 3.2, but exists in 3.1.3 That should keep someone busy ;) Note that this only include functions. The API also includes a number of macros such as Py_False and Py_RETURN_FALSE, types , and data like PyBool_Type. I've not tried to analyse any of these. Thanks. I opened http://bugs.python.org/issue11173 for this. Please, don't just document all these. Don't add them to the API, unless they are really needed. For example, PySys_FormatStdout and PySys_FormatStderr get exactly zero hits between them on google code search. PySys_FormatStdout doesn't even get used in the 3.2 source. I'm not picking on PySys_FormatStderr, or its author here, I'm just using it as an example, it seems fairly typical. According to http://bugs.python.org/issue9599 "I only need PySys_FormatStderr(), but I added also PySys_FormatStdout() just to be consistent with PySys_Write*() and because it only costs a few line of code." This seems a little cavalier to me. That little PyAPI_FUNC() macro carries a lot of obligation in terms of documentation, future support, and the cost to other implementations. Cheers, Mark. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
Victor Stinner wrote: Le jeudi 10 février 2011 à 16:19 +0100, M.-A. Lemburg a écrit : And the following were added to 3.2, of which only 2 are documented: PyArg_ValidateKeywordArguments PyAST_CompileEx Py_CompileString Py_CompileStringExFlags PyErr_NewExceptionWithDoc(documented) PyErr_SyntaxLocationEx PyErr_WarnFormat PyFrame_GetLineNumber PyImport_ExecCodeModuleWithPathnames PyImport_GetMagicTag PyLong_AsLongLongAndOverflow(documented) PyModule_GetFilenameObject Py_SetPath PyStructSequence_GetItem PyStructSequence_NewType PyStructSequence_SetItem PySys_AddWarnOptionUnicode PySys_AddXOption PySys_FormatStderr PySys_FormatStdout PySys_GetXOptions PyThread_acquire_lock_timed PyType_FromSpec PyUnicode_AsUnicodeCopy PyUnicode_AsWideCharString PyUnicode_EncodeFSDefault PyUnicode_FSDecoder Py_UNICODE_strcat Py_UNICODE_strncmp Py_UNICODE_strrchr PyUnicode_TransformDecimalToASCII PyErr_WarnFormat, PyImport_ExecCodeModuleWithPathnames, PyModule_GetFilenameObject, PySys_AddWarnOptionUnicode, PySys_FormatStderr, PySys_FormatStdout, PyUnicode_AsUnicodeCopy, PyUnicode_AsWideCharString, PyUnicode_EncodeFSDefault and PyUnicode_FSDecoder are documented (I wrote most of these functions). I added references in the issue #11173. I meant documented in the "What's new in 3.2" section. Gathering all these in one place might make the extent of the changes clearer for all. So there are a little bit less than 29/31 of new undocumented functions. But yes, most Py_UNICODE_* functions are not documented: see issue #10435 (which has a patch). victor ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/marks%40dcs.gla.ac.uk ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
> > Please, don't just document all these. > Don't add them to the API, unless they are really needed. We only add functions when they are actually needed (by us, usually). > I'm not picking on PySys_FormatStderr, or its author here, > I'm just using it as an example, it seems fairly typical. You've found an exception. Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
Antoine Pitrou wrote: Please, don't just document all these. Don't add them to the API, unless they are really needed. We only add functions when they are actually needed (by us, usually). If only you (I presume you mean the CPython devs) need them, why put them in the API. That underscore prefix saves a lot of trouble in the long run ;) I'm not picking on PySys_FormatStderr, or its author here, I'm just using it as an example, it seems fairly typical. You've found an exception. What about this one then, PyFrame_GetLineNumber was added because people were using PyCode_Addr2Line to get the current line number. The API will contain then both PyFrame_GetLineNumber *and* PyCode_Addr2Line. The API then has even more redundancy. PyObject_GetAttrString(frame, "f_lineno") should do the job. Mark. Antoine. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/marks%40dcs.gla.ac.uk ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
Am 10.02.2011 15:26, schrieb Mark Shannon: > M.-A. Lemburg wrote: >> Mark Shannon wrote: >>> Nick Coghlan wrote: On Thu, Feb 10, 2011 at 8:16 PM, Mark Shannon wrote: > Doing a search for the regex: "PyAPI_FUNC\([^)]*\) *Py" in .h files, > which should match API functions (functions starting _Py are > excluded) gives > the following result: > > Version matches > 3.0 717 > 3.1.3 728 > 3.2b2 743 > > It would appear the API bloat is real, > not just an artefact of updated docs. Since it doesn't account for #ifdef, a naive count like that isn't a valid basis for comparison. >>> OK. How about this: >>> >>> egrep -ho '#.*PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h >>> finds no matches. >>> >>> egrep -ho 'PyAPI_FUNC\([^)]*\)( |\n)*Py\w+' Include/*.h | sort -u >>> >>> This finds all matches and removes duplicates, so anything defined >>> multiple time in branches of #ifdef blocks, will only be counted once. >>> >>> Version matches >>> 3.0 714 >>> 3.1.3 725 >>> 3.2b2 739 >> >> Given these numbers, I don't think the subject line really >> captures the problem accurately enough ... a 2% increase >> in number of API function per release can hardly be called >> API bloat :-) >> >>> So given, the revised numbers; >>> >>> The "what's new for 3.2" API section: >>> http://docs.python.org/dev/py3k/whatsnew/3.2.html#build-and-c-api-changes >>> lists 6 new functions, yet 14 have been added between 3.1.3 and 3.2b2. >> >> Could you identify the ones that are not yet documented ? >> >> That would be useful. > > Here's the details: > > The following API functions were removed from 3.1.3: > > PyAST_Compile This is not correct, for example: this function is still present as a #define (and there is also a stub implementation so that extensions compiled against earlier versions can import the symbol). Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] devguide: Fix a silly statement.
On Wed, Feb 9, 2011 at 23:10, Georg Brandl wrote: > Am 09.02.2011 23:58, schrieb brett.cannon: >> brett.cannon pushed 7101df1bd817 to devguide: >> >> http://hg.python.org/devguide/rev/7101df1bd817 >> changeset: 291:7101df1bd817 >> branch: hg_transition >> tag: tip >> user: Brett Cannon >> date: Wed Feb 09 14:58:17 2011 -0800 >> summary: >> Fix a silly statement. >> >> files: >> setup.rst >> >> diff --git a/setup.rst b/setup.rst >> --- a/setup.rst >> +++ b/setup.rst >> @@ -34,8 +34,7 @@ >> :abbr:`VCS`. It also means you will have better tool >> support through the VCS as it will provide a diff tool, etc. >> >> -To get a read-only checkout of CPython's source, you need a working copy the >> -source code. To get a read-only checkout of >> +To get a read-only checkout of >> the :ref:`in-development ` branch of Python, run:: >> >> hg clone http://hg.python.org/cpython > > This statement is still somewhat silly, as a) you get a clone, not a checkout > and b) it is not read only in any way: you can commit just fine. The only > difference will be the entry in .hg/hgrc pointing the default repo to > something > you can't push to. > > Skimming through, the whole section "Checking out the code" is still way too > SVN-point of viewy (e.g. you always get all branches anyway). I'll take another pass, but do realize this needs to be something that can easily be understood by someone who has never used hg before, so I can't get too technically accurate while ignoring a possible base ignorance of hg and DVCSs as a whole. ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] devguide: Fix a silly statement.
Am 10.02.2011 19:27, schrieb Brett Cannon: > On Wed, Feb 9, 2011 at 23:10, Georg Brandl wrote: >> Am 09.02.2011 23:58, schrieb brett.cannon: >>> brett.cannon pushed 7101df1bd817 to devguide: >>> >>> http://hg.python.org/devguide/rev/7101df1bd817 >>> changeset: 291:7101df1bd817 >>> branch: hg_transition >>> tag: tip >>> user:Brett Cannon >>> date:Wed Feb 09 14:58:17 2011 -0800 >>> summary: >>> Fix a silly statement. >>> >>> files: >>> setup.rst >>> >>> diff --git a/setup.rst b/setup.rst >>> --- a/setup.rst >>> +++ b/setup.rst >>> @@ -34,8 +34,7 @@ >>> :abbr:`VCS`. It also means you will have better tool >>> support through the VCS as it will provide a diff tool, etc. >>> >>> -To get a read-only checkout of CPython's source, you need a working copy >>> the >>> -source code. To get a read-only checkout of >>> +To get a read-only checkout of >>> the :ref:`in-development ` branch of Python, run:: >>> >>> hg clone http://hg.python.org/cpython >> >> This statement is still somewhat silly, as a) you get a clone, not a checkout >> and b) it is not read only in any way: you can commit just fine. The only >> difference will be the entry in .hg/hgrc pointing the default repo to >> something >> you can't push to. >> >> Skimming through, the whole section "Checking out the code" is still way too >> SVN-point of viewy (e.g. you always get all branches anyway). > > I'll take another pass, but do realize this needs to be something that > can easily be understood by someone who has never used hg before, so I > can't get too technically accurate while ignoring a possible base > ignorance of hg and DVCSs as a whole. Well, it's no good to keep using CVCS terms and mislead users. That the "checkout" is not a checkout but a full repository is about the most important fact about a hg (or any DVCS) clone. Georg ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] devguide: Move to using mq for basic usage.
Hi, On Wed, Feb 9, 2011 at 23:19, brett.cannon wrote: > +project's preference. We present here a very simple solution based on mq_ > +(Mercurial Queue) non-core developers. for non-core devs (ok, this is not the last patch applied to the file, but it's still there after the next changes). > +This will update the patch to contain all of the changes you have made up to > +this point. If you have any you have added or removed, use ``hg add`` or ``hg > +remove``, respectively, before running ``hg qrefresh``. If you have any > This will check and/or fix various common things people forget to do for > -patches, such as adding any new files needing for the patch to work. > +patches, such as adding any new files needing for the patch to work (do not (do note Cheers, -- Sandro Tosi (aka morph, morpheus, matrixhasu) My website: http://matrixhasu.altervista.org/ Me at Debian: http://wiki.debian.org/SandroTosi ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
[Python-Dev] Proposal / Questions about OrderedDict literals and/or faster C implementation
I have a proposal for a literal syntax for OrderedDicts which is just
replacing the braces with square brackets:
['a': 1,'b': 2] == OrderedDict([('a', 1),('b', 2)])
OrderedDict literals would follow:
[x : x for x in foo()] == OrderedDict([(x,x) for x in foo()])
The rationale for the syntax is that it follows the set / list /
dict precedent of curly braces for unordered collections, square brackets
for ordered collections, and otherwise aping the normal dict syntax.
OrderedDict is arguable one of the best recent additions to the Python
standard library. Looking at the py3k codebase, it seems like adding
OrderedDicts as a native C implementation at the same time as introducing a
literal syntax with all the additions to the grammar and ast makes sense. A
native implementation would make the memory usage closer to normal dicts
(plus two pointers per element) but be potentially faster for many
operations than even regular dictionaries and certainly much, much faster
than the current Python-only implementation of OrderedDict.
I've started working on this in my free time, but I'm not a seasoned CPython
hacker. Any feedback or pointers would be helpful. Originally, I was
planning on creating a patch first before suggesting this to the mailing
list, but given the scope of the feature and the number of concerns I figure
I might as well test the waters first.
Even if the idea of a literal syntax is dismissed, I think a C
implementation of OrderedDict would be a great addition and I'd love to
help.
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Proposal / Questions about OrderedDict literals and/or faster C implementation
Hello, Thanks for wanting to contribute and welcome to python-dev :) Ideas are usually discussed first on python-ideas to assess usefulness, get the pulse of the community, beat the API into shape and such things before coming up to python-dev. (A number of core devs are on both lists.) You may want to search the mail archive for all the previous threads about adding a C version and literal notation for ordered dicts. Regards ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
Le jeudi 10 février 2011 à 17:25 +, Mark Shannon a écrit :
> What about this one then,
>
> PyFrame_GetLineNumber was added because people were using
> PyCode_Addr2Line to get the current line number.
>
> The API will contain then both
> PyFrame_GetLineNumber *and* PyCode_Addr2Line.
> The API then has even more redundancy.
>
> PyObject_GetAttrString(frame, "f_lineno") should do the job.
Not exactly:
int
PyFrame_GetLineNumber(PyFrameObject *f)
{
if (f->f_trace)
return f->f_lineno;
else
return PyCode_Addr2Line(f->f_code, f->f_lasti);
}
Victor
___
Python-Dev mailing list
[email protected]
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] API bloat
On Fri, Feb 11, 2011 at 3:25 AM, Mark Shannon wrote: > Antoine Pitrou wrote: >>> >>> Please, don't just document all these. >>> Don't add them to the API, unless they are really needed. >> >> We only add functions when they are actually needed (by us, usually). > > If only you (I presume you mean the CPython devs) need them, > why put them in the API. > That underscore prefix saves a lot of trouble in the long run ;) Keep in mind that you're asking us to change our audience here. When we modify the C API, we have precisely *three* audiences in mind: 1. CPython developers 2. authors of CPython extensions 3. developers embedding a CPython interpreter (or interpreters) into their application So if we find something that makes life easier for us in group 1, our natural inclination is to make it available to the people in groups 2 and 3 as well, rather than selfishly reserving it for ourselves. We will also take aesthetics and obvious symmetries into account when providing convenience functions (as in the case of PySys_FormatStdout). It's only when we consider something to be an ugly hack, or have some other reason to think we may want to change it in the future, that we opt to make it a private implementation detail. The audience consisting of "authors of other implementations trying to mimic the CPython C API" has never even been on the radar. That's quite a significant philosophical shift, so coming in and asking us to apply it retroactively to a release in its RC phase comes across as being *very* presumptuous. Now that the issue has been brought up, it can certainly be taken into consideration for 3.3. The idea of defining a Py_PORTABLE_API that is even more restrictive than PEP 384 (e.g. eliminating lots of old cruft that is a legacy of CPython's long history of development when it was the *only* viable Python implementation) may also be worth exploring. But no, at this late stage nothing significant is going to be done in the context of 3.2, except perhaps describing the C API changes in more detail in the What's New document (whether or not that happens is up to Raymond - the C API is of very little interest to most Python users, who will either use pre-existing C extensions, or else use something like ctypes, Cython, Pyrex, Boost or SWIG to deal with the details of the C/Python boundary). Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] [Python-checkins] r88387 - python/branches/py3k/Lib/asyncore.py
On Fri, Feb 11, 2011 at 4:42 AM, giampaolo.rodola wrote: > Author: giampaolo.rodola > Date: Thu Feb 10 19:42:36 2011 > New Revision: 88387 > > Log: > get rid of asyncore.dispatcher's debug attribute, which is no longer used > (assuming it ever was). Reviewer? NEWS entry? RM approval? Tracker issue? Removing a public attribute seems like an odd change to be making at this stage of a release. What if there is third party code that uses that attribute to enable additional debugging info in asyncore? Cheers, Nick. -- Nick Coghlan | [email protected] | Brisbane, Australia ___ Python-Dev mailing list [email protected] http://mail.python.org/mailman/listinfo/python-dev Unsubscribe: http://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
