[Python-Dev] What is the difference between Py_BUILD_CORE and Py_BUILD_CORE_BUILTIN?

2018-11-06 Thread Victor Stinner
Hi, I'm trying to cleanup the Python C API, especially strictly separate the public C API, the stable C API (Py_LIMITED_API), and the internal C API (Py_BUILD_CORE). Move internal headers to Include/internal/ : https://bugs.python.org/issue35081 Move !Py_LIMITED_API to Include/pycapi/: https://b

Re: [Python-Dev] Get a running instance of the doc for a PR.

2018-11-04 Thread Victor Stinner
OpenStack does that on review.openstack.org PRs. If I recall correctly, the CI produces files which are online on a static web server. Nothing crazy. And it works. Old files are removed, I don't know when exactly. I don't think that it matters where the static files are hosted. Victor Le dimanch

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-11-02 Thread Victor Stinner
Le ven. 2 nov. 2018 à 18:32, Neil Schemenauer a écrit : > A simple approach would be to introduce something like > Python-internal.h. If you are a Python internal unit, you can > include both Python.h and Python-internal.h. We could, over time, > split Python-iternal.h into smaller modular inclu

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-11-01 Thread Victor Stinner
Le jeu. 1 nov. 2018 à 16:38, Steve Dower a écrit : > I assume the redundancy was there to handle the naming collision > problem, but a better way to do that is to have only header files that > users should ever use in "include/", and put the rest inside > "include/python/" and "include/internal/"

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-31 Thread Victor Stinner
I pushed many changes. I moved most of the Py_BUILD_CORE code out of Include/ header files. Le jeu. 1 nov. 2018 à 02:35, Eric Snow a écrit : > On the one hand dropping redundancy in the filename is fine. On the other > having the names mirror the public header files is valuable. Current conten

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-31 Thread Victor Stinner
" with #include "pycore_mem.h". Victor Le mer. 31 oct. 2018 à 23:38, Eric Snow a écrit : > > B > On Wed, Oct 31, 2018 at 4:28 PM Victor Stinner wrote: > > > > Le mer. 31 oct. 2018 à 22:19, Eric Snow a > > écrit : > > > > Maybe we can keep &q

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-31 Thread Victor Stinner
Le mer. 31 oct. 2018 à 22:19, Eric Snow a écrit : > > Maybe we can keep "Include/internal/" directory name, but add > > "pycore_" rather than "internal_" to header filenames? > > this sounds good to me. thanks for chasing this down. What do you prefer? (A Include/internal/internal_pystate.h (B)

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-31 Thread Victor Stinner
Le mer. 31 oct. 2018 à 20:40, Eric Snow a écrit : > On Mon, Oct 29, 2018 at 6:39 AM Victor Stinner wrote: > > Le lun. 29 oct. 2018 à 06:32, Benjamin Peterson a > > écrit : > > > How does the current Include/internal/ directory fail at accomplishing > > &g

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-31 Thread Victor Stinner
Ok, I proposed to rename internal header files, add an "internal_" prefix, to avoid this issue: https://github.com/python/cpython/pull/10263 My PR also adds Include/internal/ to search paths of header files. Extract of the change: diff --git a/Include/internal/pystate.h b/Include/internal/intern

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-31 Thread Victor Stinner
Le mer. 31 oct. 2018 à 19:22, Eric Snow a écrit : > > I propose a practical solution for that: Include/*.h files would only > > be be public API. > > As we've already discussed, I'm entirely in favor of this. :) In > fact, I was thinking along those same lines when I originally created > "Include

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-30 Thread Victor Stinner
Le mar. 30 oct. 2018 à 02:59, Benjamin Peterson a écrit : > > To me, it seems wrong that a function or macro defined in > > Include/objimpl.h requires an explicit #include "internal/pystate.h". > > objimpl.h should be self-sufficient. > > I agree. I would say nothing in Include/*.h should be inclu

Re: [Python-Dev] Custom AIX build last week...

2018-10-30 Thread Victor Stinner
I used a custom build to check if a fix repaired a buildbot. It's unrelated to AIX, but inlining on Visual Studio in Debug mode, so specific to Windows. Victor Le mar. 30 oct. 2018 à 07:11, Michael Felt a écrit : > > I noticed that there was a "custom" build queued for my AIX build-bot last > we

Re: [Python-Dev] Standardize error message for non-pickleable types

2018-10-30 Thread Victor Stinner
Le lun. 29 oct. 2018 à 22:20, MRAB a écrit : > 1. If you're pickling, then saying "pickle" is more helpful. I'm not sure that it's really possible to know if the error occurs while pickle is trying to serialize an object, or if it's a different serialization protocol. We are talking about the ver

Re: [Python-Dev] Standardize error message for non-pickleable types

2018-10-29 Thread Victor Stinner
Le lun. 29 oct. 2018 à 20:42, Serhiy Storchaka a écrit : > 1. "pickle" or "serialize"? serialize > 2. "can't", "Cannot", "can not" or "cannot"? cannot > 3. "object" or "objects"? object > 4. Use the "a" article or not? no: "cannot serialize xxx object" (but i'm not a native english speaker,

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-29 Thread Victor Stinner
Le lun. 29 oct. 2018 à 06:32, Benjamin Peterson a écrit : > > My overall approach is to make sure that we don't leak functions by > > mistakes into the public API or into the stable API anymore. For > > example, if a function is really for the core, put it in pycore/. It > > will be more explicit

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-28 Thread Victor Stinner
Le dim. 28 oct. 2018 à 21:50, Benjamin Peterson a écrit : > I don't think more or less API should be magically included based on whether > Py_BUILD_CORE is defined or not. If we want to have private headers, we > should include them where needed and not install them. Really, Py_BUILD_CORE > sho

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-28 Thread Victor Stinner
. 2018 à 17:20, Victor Stinner a écrit : > > Hi, > > Python C API has no strict separation between the 3 levels of API: > > * core: Py_BUILD_CORE define > * stable: Py_LIMITED_API define (it has a value) > * regular: no define > > IMHO the current design of header file

[Python-Dev] Rename Include/internals/ to Include/pycore/

2018-10-28 Thread Victor Stinner
Hi, Python C API has no strict separation between the 3 levels of API: * core: Py_BUILD_CORE define * stable: Py_LIMITED_API define (it has a value) * regular: no define IMHO the current design of header files is done backward: by default, everything is public. To exclude an API from core or sta

Re: [Python-Dev] "Deprecation" of os.system in favor of subprocess?

2018-10-24 Thread Victor Stinner
I like os.system() and use it everyday. I prefer to write os.system("grep ^Vm /proc/%s/status" % os.getpid()) than... much more Python code to do the same thing, or subprocess.run("grep ^Vm /proc/%s/status" % os.getpid(), shell=True): os.system() is shorter and only requires "import os" :-) I'm n

Re: [Python-Dev] ​Re: Python Language Governance Proposals

2018-10-23 Thread Victor Stinner
Hi, Le mar. 23 oct. 2018 à 18:26, eamanu15 a écrit : > Are there any list of candidates? No PEP proposes directly candidates. The process to nominate candidates is part of discussed PEPs. So first core developers must vote for the governance PEP, and then a new process will be organized to selec

[Python-Dev] Python Language Governance Proposals

2018-10-23 Thread Victor Stinner
Hi, Last July, Guido van Rossum decided to resign from his role of BDFL. Python core developers decided to design a new governance/organization for Python. 6 governance PEPs have been proposed. It has been decided that discussions are reserved to core developers (everyone can read, but only core d

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Victor Stinner
Le lun. 22 oct. 2018 à 15:24, Steve Dower a écrit : > Yes, that's true. But "should reduce ... footprint" is also an > optimisation that deserves a benchmark by that standard. pyperformance has a mode to mesure the memory usage (mostly the memory peak) if someone wants to have a look. > Also, I'

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Victor Stinner
Le lun. 22 oct. 2018 à 15:08, Steve Dower a écrit : > Agreed the cache is useless here, but since the listdir() result came in > as wchar_t we could keep it that way (assuming we'd only be changing it > to char), and then there wouldn't have to be a conversion when we > immediately pass it back to

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Victor Stinner
Le sam. 20 oct. 2018 à 18:02, Steve Dower a écrit : > I don't have numbers, but my instinct says the most impacted operations > would be retrieving collections of strings from the OS (avoiding a > scan/conversion for each one), comparisons against these collections > (in-memory handling for hash/c

Re: [Python-Dev] The future of the wchar_t cache

2018-10-22 Thread Victor Stinner
Hi Serhiy, +1 to remove wchar_t cache. IMHO it wastes memory for no real performance gain. By the way, can we start to schedule the *removal* of the Py_UNICODE API? For example, decide when Py_DEPRECATED is used in the C API? Should we start to deprecate when Python 2 reachs its end of life? Or c

Re: [Python-Dev] Some PRs to merge?

2018-10-22 Thread Victor Stinner
Le sam. 20 oct. 2018 à 13:15, Serhiy Storchaka a écrit : > Thank you Victor! I prefer to merge my PRs and PRs assigned to me > myself, but I am not sure that I would merge all PRs that can be merged > in the nearest future. ;) Some PRs were blocked by me because I was nitpicking on something. I d

Re: [Python-Dev] Some PRs to merge?

2018-10-19 Thread Victor Stinner
Le ven. 19 oct. 2018 à 19:01, Stephane Wirtel a écrit : > total: 49 PRs > is:open is:pr review:approved status:success label:"awaiting merge" > -label:"DO-NOT-MERGE" label:""LA signed"" I merged many PRs and closed a few (2 if I recall correctly). Your query now counts 24 PRs. Victor __

Re: [Python-Dev] dear core-devs

2018-10-04 Thread Victor Stinner
Hi, If IBM wants a better Python support, it would help a lot if IBM pays for this development. With money, you can easily find core dev contractors. Antoine Pitrou has been paid in the past to enhance Python support in Solaris and it worked well. Victor Le mercredi 3 octobre 2018, Michael Felt

Re: [Python-Dev] Communication channels

2018-10-01 Thread Victor Stinner
Le lun. 1 oct. 2018 à 22:32, Tres Seaver a écrit : > I'm pretty strongly -1 on the notion that folks who subscribe python-dev, > BPO, and the github repositories should need to *also* follow an > arbitrarily-growing set of Twitter accounts: how would one know if a new > one popped into being? Ho

Re: [Python-Dev] Communication channels

2018-10-01 Thread Victor Stinner
Le lun. 1 oct. 2018 à 14:14, Chris Jerdonek a écrit : > Also, if you are trying to be complete, another communication channel > is in-person events and conferences, etc. Right, there are two main events for CPython core developers which are only in the US: * Language Summit during Pycon US (one

[Python-Dev] Communication channels

2018-10-01 Thread Victor Stinner
Hi, Last months, new communication channels appear. This is just a reminder that they exist: * Zulip: https://python.zulipchat.com/ (exist since 1 year?) * Discourse: http://discuss.python.org/ (I'm not sure if it's fully official yet ;-)) * IRC: #python-dev on FreeNode, only for development *of*

Re: [Python-Dev] What is the purpose of the _PyThreadState_Current symbol in Python 3?

2018-09-27 Thread Victor Stinner
Hi, Le mer. 26 sept. 2018 à 23:27, Gabriele a écrit : > In trying to find the location of a valid instance of PyInterpreterState in > the virtual memory of a running Python (3.6) application (using > process_vm_read on Linux), I understand that you are writing a debugger and you can only *read

Re: [Python-Dev] Documenting the private C API (was Re: Questions about signal handling.)

2018-09-25 Thread Victor Stinner
directory.) Victor Le mar. 25 sept. 2018 à 16:05, Barry Warsaw a écrit : > > On Sep 25, 2018, at 03:44, Victor Stinner wrote: > > > By the way, recently, we had to fix yet another bug in signal > > handling. A new function has been added: > > > > voi

Re: [Python-Dev] Questions about signal handling.

2018-09-25 Thread Victor Stinner
Please don't rely on this ugly API. *By design*, Py_AddPendingCall() tries 100 times to acquire the lock: if it fails to acquire the lock, it does notthing... your callback is ignored... By the way, recently, we had to fix yet another bug in signal handling. A new function has been added: void _P

Re: [Python-Dev] Questions about signal handling.

2018-09-21 Thread Victor Stinner
Le sam. 22 sept. 2018 à 01:05, Eric Snow a écrit : > 3. Why does signal handling operate via the "pending calls" machinery > and not distinctly? Signals can be received anytime, between two instructions at the machine code level. But the Python code base is rarely reentrant. Moreover, you can get

Re: [Python-Dev] Late Python 3.7.1 changes to fix the C locale coercion (PEP 538) implementation

2018-09-19 Thread Victor Stinner
Le mardi 18 septembre 2018, Victor Stinner a écrit : > Hi Unicode and locales lovers, > > tl; dr Nick, Ned, INADA-san: I modified 3.7.1 to add a new "-X > coerce_c_locale=value" option and make sure that the C locale coercion > cannot be when Python in embedded: are

Re: [Python-Dev] Late Python 3.7.1 changes to fix the C locale coercion (PEP 538) implementation

2018-09-19 Thread Victor Stinner
> IMHO the implementation is really a secondary concern here, the main > question is: what is the correct behavior? > > Nick: > > * Do we agree that we need to provide a way to disable C locale > coercion (PEP 538) even when -E is used? > * Do you agree that Py_Initialize() and Py_Main() must not e

Re: [Python-Dev] Late Python 3.7.1 changes to fix the C locale coercion (PEP 538) implementation

2018-09-19 Thread Victor Stinner
Le mer. 19 sept. 2018 à 09:50, Nick Coghlan a écrit : > I think the changes to both master and the 3.7 branch should be reverted. Ok, I prepared a PR to revert the 3.7 change: https://github.com/python/cpython/pull/9416 > For 3.7, I already said that I think we should just accept that that > sh

[Python-Dev] Late Python 3.7.1 changes to fix the C locale coercion (PEP 538) implementation

2018-09-17 Thread Victor Stinner
Hi Unicode and locales lovers, tl; dr Nick, Ned, INADA-san: I modified 3.7.1 to add a new "-X coerce_c_locale=value" option and make sure that the C locale coercion cannot be when Python in embedded: are you ok with these changes? Before 3.7.0 release, during the implementation of the UTF-8 Mode

Re: [Python-Dev] bpo-34595: How to format a type name?

2018-09-13 Thread Victor Stinner
Le ven. 14 sept. 2018 à 00:09, Eric V. Smith a écrit : > f'{type(obj)}' becomes type(obj).__format__(''), so you can return > something other than __str__ or __repr__ does. It's only by convention > that an object's __format__ returns __str__: it need not do so. What's New in Python 3.7 contains:

Re: [Python-Dev] bpo-34595: How to format a type name?

2018-09-13 Thread Victor Stinner
Le jeu. 13 sept. 2018 à 16:01, Eric V. Smith a écrit : > > * Add !t conversion to format string > > I'm strongly opposed to this. This !t conversion would not be widely > applicable enough to be generally useful, and would need to be exposed > in the f-string and str.format() documentation, even t

Re: [Python-Dev] bpo-34595: How to format a type name?

2018-09-12 Thread Victor Stinner
Hi, For the type name, sometimes, we only get a type (not an instance), and we want to format its FQN. IMHO we need to provide ways to format the FQN of a type for *types* and for *instances*. Here is my proposal: * Add !t conversion to format string * Add ":T" format to type.__format__() * Add "

Re: [Python-Dev] Stop automerging

2018-09-12 Thread Victor Stinner
t. Victor Le mer. 12 sept. 2018 à 17:56, Victor Stinner a écrit : > > Hi Benjamin, > > https://github.com/python/cpython/commit/d13e59c1b512069d90efe7ee9b613d3913e79c56 > > Le mer. 12 sept. 2018 à 17:19, Benjamin Peterson a > écrit : > > (Just checking) Is there somethin

Re: [Python-Dev] Stop automerging

2018-09-12 Thread Victor Stinner
Hi Benjamin, https://github.com/python/cpython/commit/d13e59c1b512069d90efe7ee9b613d3913e79c56 Le mer. 12 sept. 2018 à 17:19, Benjamin Peterson a écrit : > (Just checking) Is there something wrong with this message besides the <-- > comment? Since the commit is described as a follow-up of 90fc

[Python-Dev] bpo-34595: How to format a type name?

2018-09-11 Thread Victor Stinner
Hi, Last week, I opened an issue to propose to add a new %T formatter to PyUnicode_FromFormatV() and so indirectly to PyUnicode_FromFormat() and PyErr_Format(): https://bugs.python.org/issue34595 I merged my change, but then Serhiy Storchaka asked if we can add something to get the "fully qua

Re: [Python-Dev] Fwd: We cannot fix all issues: let's close XML security issues (not fix them)

2018-09-07 Thread Victor Stinner
Le ven. 7 sept. 2018 à 17:02, PMS PMS a écrit : > XML support in Python is critical and desired for many sectors like banking > or telecoms, > and code base based on XML is still on rise in such world. Would it be possible to send money to the PSF? I'm sure that the PSF will be able to find you

Re: [Python-Dev] We cannot fix all issues: let's close XML security issues (not fix them)

2018-09-07 Thread Victor Stinner
Le ven. 7 sept. 2018 à 10:23, Christian Heimes a écrit : > Back in the days, I didn't push hard for the necessary fixes, because > all fixes were breaking changes. After all I'd have to disable some > features that people may have relied upon. The XML security stuff was my > first major security t

Re: [Python-Dev] We cannot fix all issues: let's close XML security issues (not fix them)

2018-09-07 Thread Victor Stinner
Le jeu. 6 sept. 2018 à 21:10, Steve Dower a écrit : > If Christian is not able to keep maintaining the defused* packages, then > I may take a look at this next week at the sprints. The built-in XML > packages actually don't meet Microsoft's internal security requirements, > so I have some business

Re: [Python-Dev] We cannot fix all issues: let's close XML security issues (not fix them)

2018-09-06 Thread Victor Stinner
Are you volunteer to fix the XML modules? Victor Le jeu. 6 sept. 2018 à 16:50, Antoine Pitrou a écrit : > > > Le 06/09/2018 à 16:40, Victor Stinner a écrit : > > Le jeu. 6 sept. 2018 à 16:33, Antoine Pitrou a écrit : > >> If we consider fixing these issues to be de

Re: [Python-Dev] We cannot fix all issues: let's close XML security issues (not fix them)

2018-09-06 Thread Victor Stinner
Le jeu. 6 sept. 2018 à 16:33, Antoine Pitrou a écrit : > If we consider fixing these issues to be desirable, then the issues > should be kept open. Closing issues because no-one is working on them > sounds a bit silly to me. I forgot to mention that closing these issues is my reply to Larry's ca

[Python-Dev] We cannot fix all issues: let's close XML security issues (not fix them)

2018-09-06 Thread Victor Stinner
Hi, The Python bug tracker is full of bugs, and sadly we don't have enough people to take care of all of them. There are 3 open bugs about security issues in XML and I simply propose to close it: https://bugs.python.org/issue17318 https://bugs.python.org/issue17239 https://bugs.python.or

Re: [Python-Dev] Workflow blocked on the 3.6 because of AppVeyor; who owns the AppVeyor project?

2018-09-05 Thread Victor Stinner
Le mer. 5 sept. 2018 à 15:47, Zachary Ware a écrit : > For the actual issue at hand, the problem arises from doing builds on > 3.6 with both the VS2015 and VS2017 images. Apparently something > built in `/externals` by the VS2015 build gets cached, which then > breaks the VS2017 build; I haven't

Re: [Python-Dev] Workflow blocked on the 3.6 because of AppVeyor; who owns the AppVeyor project?

2018-09-05 Thread Victor Stinner
I wrote some notes about our CIs. Link to AppVeyor notes: https://pythondev.readthedocs.io/ci.html#appveyor Victor Le mer. 5 sept. 2018 à 12:04, Paul Moore a écrit : > > On Wed, 5 Sep 2018 at 10:55, Victor Stinner wrote: > > > > Hi, > > > > It's no longer poss

[Python-Dev] Schedule of the Python 3.7.1 release?

2018-09-05 Thread Victor Stinner
Hi, Someone asked somewhere (oops, I forgot where!) when is Python 3.7.1 scheduled. I wanted to reply when I saw that it was scheduled for 2 months ago: "3.7.1: 2018-07-xx" https://www.python.org/dev/peps/pep-0537/#maintenance-releases Is there any blocker for 3.7.1? I fixed dozens of bugs in th

[Python-Dev] Workflow blocked on the 3.6 because of AppVeyor; who owns the AppVeyor project?

2018-09-05 Thread Victor Stinner
Hi, It's no longer possible to merge any change in the 3.6 branch of CPython, because the AppVeyor job fails: https://bugs.python.org/issue34575 It seems like AppVeyor has a build cache and this cache is outdated. I tried to use the REST API but I'm not allowed to invalidate the cache: even the m

[Python-Dev] UTF-8 Mode now also enabled by the POSIX locale

2018-08-28 Thread Victor Stinner
Hi, While working on test_utf8_mode on AIX (bpo-34347) and HP-UX (bpo-34403), I noticed that FreeBSD doesn't work properly with the POSIX locale (bpo-34527). I also noticed that my implementation of my PEP 540 "UTF-8 Mode" doesn't respect the PEP: the UTF-8 Mode should be enabled by the POSIX loca

Re: [Python-Dev] [Python-checkins] bpo-34171: Prevent creating Lib/trace.cover when run the trace module. (GH-8841)

2018-08-27 Thread Victor Stinner
Jemery: would you mind to revert this commit since nobody fixed the buildbot since 2 days? https://pythondev.readthedocs.io/ci.html#revert-on-fail Victor Le sam. 25 août 2018 à 22:50, Jeremy Kloth a écrit : > > On Sat, Aug 25, 2018 at 1:28 AM Serhiy Storchaka > wrote: > > > > https://github.com/

Re: [Python-Dev] Python 2.7 EOL date

2018-08-23 Thread Victor Stinner
Hi, The reference is the PEP 373 "Python 2.7 Release Schedule". See the "Update" section: https://www.python.org/dev/peps/pep-0373/#update Victor 2018-08-23 20:53 GMT+02:00 Collin Anderson : > Hi All, > > Sorry if this has been mentioned before, but I noticed the Python 2.7 EOL > date was recent

Re: [Python-Dev] [python-committers] Winding down 3.4

2018-08-20 Thread Victor Stinner
> "shutil copy* unsafe on POSIX - they preserve setuid/setgit bits" > https://bugs.python.org/issue17180 There is no fix. A fix may break the backward compatibility. Is it really worth it for the last 3.4 release? > "XML vulnerabilities in Python" > https://bugs.python.org/issue17239 Bug inactiv

Re: [Python-Dev] New _Py_InitializeFromConfig() function (PEP 432)

2018-08-03 Thread Victor Stinner
It seems like the PEP 432 proposes an API designed from scratch as the target API. I started from the 28 years old C code and I tried to cleanup the code. Our method is different, so it's not surprising that the result is different :-) My intent is to get: * a function to read *all* configuration

Re: [Python-Dev] New _Py_InitializeFromConfig() function (PEP 432)

2018-08-02 Thread Victor Stinner
2018-08-02 1:18 GMT+02:00 Eric Snow : > The "core" config is basically the config for the runtime. In fact, > PEP 432 renamed "core" to "runtime". Please keep the firm distinction > between the runtime and the (main) interpreter. There is already something called _PyRuntime but it's shared betwe

Re: [Python-Dev] New _Py_InitializeFromConfig() function (PEP 432)

2018-08-02 Thread Victor Stinner
2018-08-02 17:17 GMT+02:00 Eric Snow : > Note that there are backward compatibility issues to deal with. AFAIU > if we start ignoring those global variables during initialization then > it's going to cause problems for embedders. One of the first operation of Py_Initialize(), Py_Main() and _PyCor

Re: [Python-Dev] [python-committers] [RELEASED] Python 3.4.9 and Python 3.5.6 are now available

2018-08-02 Thread Victor Stinner
Hi, 2018-08-02 16:00 GMT+02:00 Larry Hastings : > On behalf of the Python development community, I'm happy to announce the > availability of Python 3.4.9 and Python 3.5.6. Great! FYI these versions fix two security vulnerabilities: (*) CVE-2018-1000117: Buffer overflow vulnerability in os.symlin

Re: [Python-Dev] New _Py_InitializeFromConfig() function (PEP 432)

2018-08-02 Thread Victor Stinner
2018-08-02 1:18 GMT+02:00 Eric Snow : > Backporting shouldn't be so risky since it's all private API and there > are few other changes in the relevant code since 3.7, right? It > depends on if Ned's okay with it or not. :) I'm still doing further bug fixes and cleanup in the master branch: https:

Re: [Python-Dev] Accessing mailing list archives

2018-07-31 Thread Victor Stinner
ul 30, 2018 at 3:29 PM Victor Stinner wrote: >> >> Hi Bob, >> >> I wrote a basic script to compute the number of emails per PEP. It >> requires to download gzipped mbox files from the web page of archives per >> month, then ungzip them: >> >> https://

[Python-Dev] New _Py_InitializeFromConfig() function (PEP 432)

2018-07-31 Thread Victor Stinner
Hi, I finished my work on the _PyCoreConfig structure: it's a C structure in Include/pystate.h which has many fields used to configure Python initialization. In Python 3.6 and older, these parameters were scatted around the code, and it was hard to get an exhaustive list of it. This work is linke

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Victor Stinner
I replied on capi-sig. 2018-07-31 18:03 GMT+02:00 Antoine Pitrou : > On Tue, 31 Jul 2018 15:34:05 +0200 > Victor Stinner wrote: >> Antoine: would you mind to subscribe to the capi-sig mailing list? As >> expected, they are many interesting points discussed here, but I would &

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Victor Stinner
2018-07-31 14:01 GMT+02:00 Jeroen Demeyer : > Anyway, I know that this is probably not going to happen, but I just wanted > to bring it up in case people would find it a great idea. But maybe not many > CPython core developers actually know and use Cython? I know that Yury wants to use Cython for

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Victor Stinner
Antoine: would you mind to subscribe to the capi-sig mailing list? As expected, they are many interesting points discussed here, but I would like to move all C API discussions to capi-sig. I only continue on python-dev since you started here (and ignored my request to start discussing my idea on ca

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Victor Stinner
2018-07-31 9:27 GMT+02:00 Jeroen Demeyer : > On 2018-07-31 08:58, Antoine Pitrou wrote: >> >> I think Stefan is right that we >> should push people towards Cython and alternatives, rather than direct >> use of the C API (which people often fail to use correctly, in my >> experience). > > > I know t

Re: [Python-Dev] Let's change to C API!

2018-07-31 Thread Victor Stinner
2018-07-31 8:58 GMT+02:00 Antoine Pitrou : > What exactly in the C API made it slow or non-promising? > >> The C API requires that your implementations make almost all the same >> design choices that CPython made 25 years ago (C structures, memory >> allocators, reference couting, specific GC imple

Re: [Python-Dev] Let's change to C API!

2018-07-30 Thread Victor Stinner
>> Last year, I gave a talk at the Language Summit (during Pycon) to >> explain that CPython should become 2x faster to remain competitive. >> IMHO all attempts to optimize Python (CPython forks) have failed >> because they have been blocked by the C API which implies strict >> constraints. > > Wel

Re: [Python-Dev] Testing C API

2018-07-30 Thread Victor Stinner
ib/test/test_*_capi.py > tests; ./python -m test --fromfile tests". There are different options. By the way, running the full test suite just takes 5 min on my laptop, it isn't so long ;-) Victor Le lundi 30 juillet 2018, Nick Coghlan a écrit : > On 30 July 2018 at 21:23, Victor Stinne

Re: [Python-Dev] Accessing mailing list archives

2018-07-30 Thread Victor Stinner
Hi Bob, I wrote a basic script to compute the number of emails per PEP. It requires to download gzipped mbox files from the web page of archives per month, then ungzip them: https://github.com/vstinner/misc/blob/master/python/parse_mailman_mbox_peps.py Results: https://mail.python.org/pipermail/p

Re: [Python-Dev] Testing C API

2018-07-30 Thread Victor Stinner
Or maybe test__capi.py so you can more easily discover test_unicode_cami while working on Unicode. You can use -m "test_*_capi" to run all C API tests. Victor Le lundi 30 juillet 2018, Victor Stinner a écrit : > Buildbots have a timeout of 15 min per test. I suggest to use multip

Re: [Python-Dev] Testing C API

2018-07-30 Thread Victor Stinner
Buildbots have a timeout of 15 min per test. I suggest to use multiple test_capi_.py files rather than a directory which behaves as a single test. Or regrtest should be modified to implement timeout differently. Victor Le dimanche 29 juillet 2018, Serhiy Storchaka a écrit : > Currently C API is

Re: [Python-Dev] Using Python on a fork-less POSIX-like OS

2018-07-30 Thread Victor Stinner
2018, Barath Aron a écrit : > On 07/30/2018 10:23 AM, Victor Stinner wrote: >> >> Python 3.8 will support os.posix_spawn(). I would like to see it used whenever possible instead of fork+exec, since it's faster and it can be safer on some platforms. Pablo Salgado is your g

Re: [Python-Dev] Using Python on a fork-less POSIX-like OS

2018-07-30 Thread Victor Stinner
Python 3.8 will support os.posix_spawn(). I would like to see it used whenever possible instead of fork+exec, since it's faster and it can be safer on some platforms. Pablo Salgado is your guy for that. Victor ___ Python-Dev mailing list Python-Dev@pytho

Re: [Python-Dev] Let's change to C API!

2018-07-30 Thread Victor Stinner
8 at 10:46 AM, Victor Stinner wrote: >> 2018-07-29 23:41 GMT+02:00 Jeroen Demeyer : >>> For example, you mention that you want to make Py_INCREF() a function call >>> instead of a macro. But since Py_INCREF is very common, I would guess that >>> this would make p

Re: [Python-Dev] Let's change to C API!

2018-07-29 Thread Victor Stinner
2018-07-29 23:41 GMT+02:00 Jeroen Demeyer : > For example, you mention that you want to make Py_INCREF() a function call > instead of a macro. But since Py_INCREF is very common, I would guess that > this would make performance worse (not by much maybe but surely measurable). For the very specific

[Python-Dev] Let's change to C API!

2018-07-29 Thread Victor Stinner
Hi, I just sent an email to the capi-sig mailing list. Since this mailing list was idle for months, I copy my email here to get a wider audience. But if possible, I would prefer that you join me on capi-sig to reply ;-) -- Hi, Last year, I gave a talk at the Language Summit (during Pycon) to ex

Re: [Python-Dev] Exporting Python functions on AIX

2018-07-27 Thread Victor Stinner
Yes, all symbols starting with _Py are private and must not be used outside CPython internals. Victor Le vendredi 27 juillet 2018, WILSON, MICHAEL a écrit : > All, > > My excuse if this is not the appropriate list for a question essentially concerning the AIX port of Python. > > The current port

[Python-Dev] What's the status of the PEP 572 implementation?

2018-07-26 Thread Victor Stinner
Hi, Is the PEP 572 implemented? If no, who is working on that? Is there a WIP pull request? An open issue? One month ago, I tried Chis Angelo's implementation but it implemented an old version of the PEP which evolved in the meanwhile. Victor ___ Pytho

[Python-Dev] Status of Python CIs (buildbots, Travis CI, AppVeyor): july 2018

2018-07-18 Thread Victor Stinner
Hi, It seems like my latest status of Python CIs was already one year ago! https://mail.python.org/pipermail/python-dev/2017-June/148511.html Since last year, Zachary Ware (with the help of others, but I forgot names, sorry!) migrated our buildbot server from buildbot 0.8 (Python 2.7) to buil

Re: [Python-Dev] Const access to CPython objects outside of GIL?

2018-07-17 Thread Victor Stinner
2018-07-17 6:18 GMT+02:00 Radim Řehůřek : > one of our Python projects calls for pretty heavy, low-level optimizations. > > We went down the rabbit hole and determined that having access to > PyList_GET_ITEM(list), PyInt_AS_LONG(int) and PyDict_GetItem(dict, unicode) > on Python objects **outside o

Re: [Python-Dev] PEP 572 and assert

2018-07-17 Thread Victor Stinner
2018-07-17 10:50 GMT+02:00 Serhiy Storchaka : > assert len(subdirs := list(path.iterdir())) == 0, subdirs > > Does PEP 572 encourages writing such code, discourages this, or completely > forbids? If I understood correctly Guido, Python the language must not prevent developers to experiment var

Re: [Python-Dev] PEP 572: Do we really need a ":" in ":="?

2018-07-12 Thread Victor Stinner
2018-07-12 17:14 GMT+02:00 Abdur-Rahmaan Janhangeer : > sorry for reviving the dead but community acceptance, a fundamental pep > principle has not been respected for 572 > > also 29 core devs dislike vs 3 like You are referring to a *poll* that I ran in May. I don't see any community issue, the P

Re: [Python-Dev] Can I make marshal.dumps() slower but stabler?

2018-07-12 Thread Victor Stinner
2018-07-12 8:21 GMT+02:00 Serhiy Storchaka : >> Is there any real application which marshal.dumps() performance is >> critical? > > EVE Online is a well known example. EVE Online has been created in 2003. I guess that it still uses Python 2.7. I'm not sure that a video game would pick marshal in

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-11 Thread Victor Stinner
2018-07-11 2:12 GMT+02:00 INADA Naoki : > If my idea has 50% gain and current PEP 580 has only 5% gain, > why we should accept PEP 580? > But no one know real gain, because there are no realistic application > which bottleneck is calling overhead. I'm skeptical about "50% gain": I want to see a wo

Re: [Python-Dev] Micro-benchmarks for function calls (PEP 576/579/580)

2018-07-11 Thread Victor Stinner
2018-07-11 9:19 GMT+02:00 Andrea Griffini : > May be is something obvious but I find myself forgetting often about > the fact that most modern CPUs can change speed (and energy consumption) > depending on a moving average of CPU load. > > If you don't disable this "green" feature and the benchmarks

Re: [Python-Dev] On the METH_FASTCALL calling convention

2018-07-10 Thread Victor Stinner
2018-07-07 10:55 GMT+02:00 Serhiy Storchaka : > There is my idea. Split every of keyword argument parsing functions on two > parts. The first part linearize keyword arguments, it converts positional > and keyword arguments (in whatever form they were presented) into a linear > array of PyObject* (w

Re: [Python-Dev] On the METH_FASTCALL calling convention

2018-07-10 Thread Victor Stinner
2018-07-07 0:26 GMT+02:00 Victor Stinner : > I designed FASTCALL with the help of Serhiy for keywords. I prepared a long > email reply, but I found an opportunity for optimisation on **kwargs and I > need time to see how to optimize it. I just created: "Python function call optim

Re: [Python-Dev] Micro-benchmarks for function calls (PEP 576/579/580)

2018-07-10 Thread Victor Stinner
The pyperformance benchmark suite had micro benchmarks on function calls, but I removed them because they were sending the wrong signal. A function call by itself doesn't matter to compare two versions of CPython, or CPython to PyPy. It's also very hard to measure the cost of a function call when y

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Victor Stinner
About your benchmark results: "FASTCALL unbound method(obj, 1, two=2): Mean +- std dev: 42.6 ns +- 29.6 ns" That's a very big standard deviation :-( Are you using CPU pinning and other technics explaining in my doc? http://perf.readthedocs.io/en/latest/run_benchmark.html#how-to-get-reproductible-

Re: [Python-Dev] Micro-benchmarks for PEP 580

2018-07-10 Thread Victor Stinner
2018-07-10 14:59 GMT+02:00 INADA Naoki : > PyObject_CallFunction(func, "n", 42); > > Currently, we create temporary long object for passing argument. > If there is protocol for exposeing format used by PyArg_Parse*, we can > bypass temporal Python object and call myfunc_impl directly. I'm not sure

Re: [Python-Dev] why is not 64-bit installer the default download link for Windows?

2018-07-10 Thread Victor Stinner
2018-07-09 18:01 GMT+02:00 Steve Dower : > The difficulty is that they *definitely* can use the 32-bit version, and > those few who are on older machines or older installs of Windows may not > understand why the link we provide didn't work for them. Let's say that only 10% of users still use 32-bi

Re: [Python-Dev] On the METH_FASTCALL calling convention

2018-07-06 Thread Victor Stinner
Naoki?) who thought METH_FASTCALL could use improvements. Maybe that person can write back to this thread? Or perhaps Victor Stinner (who seems to have touched it last) has a suggestion for what could be improved about it? > --Guido > > On Thu, Jul 5, 2018 at 7:55 AM Jeroen Demeyer wrote: >

Re: [Python-Dev] Removal of install_misc command from distutils

2018-07-06 Thread Victor Stinner
Hello, I am not sure of what you propose. Do you want to get the feature back in Python 3.7.1? If yes, should it start to emit a deprection warning? Did you manage to workaround the removal? If yes, maybe we can add more doc to the Porting section of What's New in Python 3.7? Victor Le jeudi 5

Re: [Python-Dev] A "day of silence" on PEP 572?

2018-07-06 Thread Victor Stinner
Last week I proposed to create a mailing list dedicated to discuss only the PEP 572, but nobody reacted to my idea (on python-commiters). Then Guido van Rossum announced his intent to approve it. So I am not sure if a mailing list is still needed if the PEP will be approved soon. Victor Le vendre

Re: [Python-Dev] Assignment expression and coding style: the while True case

2018-07-05 Thread Victor Stinner
Hi, I wrote more pull requests in the meanwhile to see how assignment expressions could be used in the standard library. I combined my 5 PR into a new single PR to see all changes at once: https://github.com/python/cpython/pull/8122/files Again, all these PR must not be merged. I only wrote t

<    4   5   6   7   8   9   10   11   12   13   >