[Python-Dev] Re: Accepting PEP 675 - Arbitrary Literal String Type

On 2022-03-21, Gregory P. Smith wrote: > TL;DR - PEP 675 allows type checkers to help prevent bugs allowing > attacker-controlled data to be passed to APIs that declare themselves as > requiring literal, in-code strings. Great idea. I did something like this for HTML templating in the Quixote

[Python-Dev] Re: Move the pythoncapi_compat project under the GitHub Python or PSF organization?

On 2022-02-11 06:14, Petr Viktorin wrote: Sounds reasonable, but... The implication of endorsing code like this is that *we cannot change private API even in patch releases*, which I don't think is documented anywhere, and might be a bit controversial. I think we are still allowed to

[Python-Dev] Re: Move the pythoncapi_compat project under the GitHub Python or PSF organization?

On 2022-02-10 2:58 p.m., Victor Stinner wrote: Would it make sense to move the pythoncapi_compat project under the GitHub Python or PSF organization to make it more "official" and a little bit more sustainable? I think that makes sense.  Extensions typically have this kind of compatibility

[Python-Dev] Re: Request to revert unittest and configparser incompatible changes in Python 3.11

On 2022-01-18 23:14, Gregory P. Smith wrote: Our stdlib unittest already enables warnings by default per https://bugs.python.org/issue10535. Getting the right people to pay attention to them is always the hard part. I wonder if we can do a bit better in that regard.  When I install 3rd

[Python-Dev] Re: my plans for subinterpreters (and a per-interpreter GIL)

On 2021-12-15 2:57 p.m., Guido van Rossum wrote: But as long as the imbalance is less than 0x_2000_, the refcount will remain in the inclusive range [ 0x_4000_ , 0x_7FFF_ ] and we can test for immortality by testing a single bit: if (o->ob_refcnt & 0x_4000_) Could we have a

[Python-Dev] Difficulty of testing beta releases now available

On 2021-05-04, Łukasz Langa wrote: > We strongly encourage maintainers of third-party Python projects > to test with 3.10 during the beta phase and report issues found to > the Python bug tracker as soon as > possible. Testing with Python 3.10b1 is not easy, at least

[Python-Dev] Re: Future PEP: Include Fine Grained Error Locations in Tracebacks

On 2021-05-07, Pablo Galindo Salgado wrote: > Technically the main concern may be the size of the unmarshalled > pyc files in memory, more than the storage size of disk. It would be cool if we could mmap the pyc files and have the VM run code without an unmarshal step. One idea is something

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

On 2021-01-12, Petr Viktorin wrote: > Unfortunately, it's not just the creation that needs to be changed. > You also need to decref Foo_Type somewhere. Add the type to the module dict? ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

On 2021-01-12, Pablo Galindo Salgado wrote: > One worry that I have in general with this move is the usage of > _PyType_GetModuleByDef to get the type object from the module > definition. This normally involves getting a TLS in every instance > creation, which can impact notably performance for

[Python-Dev] Re: Heap types (PyType_FromSpec) must fully implement the GC protocol

On 2021-01-12, Victor Stinner wrote: > It seems like a safer approach is to continue the work on > bpo-40077: "Convert static types to PyType_FromSpec()". I agree that trying to convert static types is a good idea. Another possible bonus might be that we can gain some performance by integrating

[Python-Dev] Re: PEP: Deferred Evaluation Of Annotations Using Descriptors

On 2021-01-11, Łukasz Langa wrote: > The stringification process which your PEP describes as costly > only happens during compilation of a .py file to .pyc. Since > pip-installing pre-compiles modules for the user at installation > time, there is very little runtime penalty for a fully annotated >

[Python-Dev] Re: Pattern matching reborn: PEP 622 is dead, long live PEP 634, 635, 636

Woah, this is both exciting and scary.  If adopted, it will be a major change to how Python programs are written.  It seems a lot of work has been put into polishing the design.  That's good because if we do this, will not be easy to fix things later if we made design errors. One of my first

[Python-Dev] Re: Remove module's __version__ attributes in the stdlib

On 2020-10-15, Serhiy Storchaka wrote: > [..] it seems that there are no usages the __version__ variable in > top 4K pypi packages. Given that, I think it's fine to remove them. If we find broken code during the alpha release we still have a chance to revert. However, it would seem quite

[Python-Dev] Re: Remove module's __version__ attributes in the stdlib

On 2020-10-14, Serhiy Storchaka wrote: > I propose to remove __version__ in all stdlib modules. Are there any > exceptions? I agree that these kinds of meta attributes are not useful and it would be nice to clean them up. However, IMHO, maybe the cleanup is not worth breaking Python programs.

[Python-Dev] Re: Deferred, coalescing, and other very recent reference counting optimization

On 2020-09-02, Greg Ewing wrote: > On 2/09/20 8:32 am, Neil Schemenauer wrote: > > The most obvious approach is to adopt a multi-threaded model like is > > done by modern Java. I.e. no GIL and non-thread safe core data > > structures. That sounds a bit scary but bas

[Python-Dev] Re: Deferred, coalescing, and other very recent reference counting optimization

On 2020-09-01, Larry Hastings wrote: > Personally I think the future of CPython is to change completely > over to tracing garbage collection.  It's so much friendlier to > multicore, which is clearly the future of programming.  I'd rather > see efforts in this area directed towards that goal. I

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

On 2020-06-23, Thomas Wouters wrote: > I think the ability for per-type allocation/deallocation routines isn't > really about efficiency, but more about giving more control to embedding > systems (or libraries wrapped by extension modules) about how *their* > objects are allocated. It doesn't make

[Python-Dev] Re: PEP 620: Hide implementation details from the C API

Hi Victor, Thanks for putting work into this. I support the idea of slowly evolving the C API. It must be done carefully so as to not unnecessarily break 3rd party extensions. Changes must be made for well founded reasons and not just because we think it makes a "cleaner" API. I believe you

[Python-Dev] Re: Pass the Python thread state to internal C functions

On AMD64 Linux, the location of the thread local data seems to be stored in the GS CPU register[1]. It seems likely other platforms and other operating systems could do something similar. Passing threadstate as an explicit argument could be either faster or slower depending on how often you use

[Python-Dev] Re: What to do about invalid escape sequences

On 2019-08-10, Serhiy Storchaka wrote: > Actually we need to distinguish the the author and the user of the code and > show warnings only to the author. Using .pyc files was just an heuristic: > the author compiles the Python code, and the user uses compiled .pyc files. > Would be nice to have

[Python-Dev] Re: How to extricate large set of diffs from hg.python.org/sandbox?

On 2019-08-07, Skip Montanaro wrote: > Victor's experiments into a register-based virtual machine live here: > > https://hg.python.org/sandbox/registervm > > I'd like to revive them, if for no other reason to understand what he > did. I see no obvious way to collect them all as a massive diff.

[Python-Dev] Re: What to do about invalid escape sequences

Making it an error so soon would be mistake, IMHO. That will break currently working code for small benefit. When Python was a young language with a few thousand users, it was easier to make these kinds of changes. Now, we should be much more conservative and give people a long time and a lot

[Python-Dev] Re: Optimizing pymalloc (was obmalloc

On 2019-07-09, Inada Naoki wrote: > PyObject_Malloc inlines pymalloc_alloc, and PyObject_Free inlines > pymalloc_free. > But compiler doesn't know which is the hot part in pymalloc_alloc and > pymalloc_free. Hello Inada, I don't see this on my PC. I'm using GCC 8.3.0. I have configured the

[Python-Dev] Re: Optimizing pymalloc (was obmalloc

On 2019-07-09, Inada Naoki wrote: > So I tried to use LIKELY/UNLIKELY macro to teach compiler hot part. > But I need to use > "static inline" for pymalloc_alloc and pymalloc_free yet [1]. I think LIKELY/UNLIKELY is not helpful if you compile with LTO/PGO enabled. So, I would try that first.

[Python-Dev] Re: Removing dead bytecode vs reporting syntax errors

On 2019-07-06, Victor Stinner wrote: > More people seems to expect "if 0: ..." to be removed, than people who > care of syntax errors on "if 0". One small data point: I have shipped code that depended on 'if 0' removing code from the .pyc file. The code inside was not meant to be released

[Python-Dev] Re: obmalloc (was Have a big machine and spare time? Here's a possible Python bug.)

For those who would like to test with something compatible with Python 3.7.3, I made re-based branches here: https://github.com/nascheme/cpython/tree/obmalloc_radix_v37 https://github.com/nascheme/cpython/tree/obmalloc_big_pools_v37 They should be ABI compatible with Python 3.7.3. So,

[Python-Dev] Re: obmalloc (was Have a big machine and spare time? Here's a possible Python bug.)

On 2019-06-21, Tim Peters wrote: > [Thomas Wouters ] > > Getting rid of address_in_range sounds like a nice idea, and I > > would love to test how feasible it is -- I can run such a change > > against a wide selection of code at work, including a lot of > > third-party extension modules, but I

[Python-Dev] Re: radix tree arena map for obmalloc

On 2019-06-15, Tim Peters wrote: > At the start, obmalloc never returned arenas to the system. The vast > majority of users were fine with that. Yeah, I was totally fine with that back in the day. However, I wonder now if there is a stronger reason to try to free memory back to the OS. Years

[Python-Dev] Re: radix tree arena map for obmalloc

On 2019-06-15, Antoine Pitrou wrote: > We should evaluate what problem we are trying to solve here, instead > of staring at micro-benchmark numbers on an idle system. I think a change to obmalloc is not going to get accepted unless we can show it doesn't hurt these micro-benchmarks. To displace

[Python-Dev] Re: radix tree arena map for obmalloc

Here are benchmark results for 64 MB arenas and 16 kB pools. I ran without the --fast option and on a Linux machine in single user mode. The "base" columm is the obmalloc-big-pools branch with ARENA_SIZE = 64 MB and POOL_SIZE = 16 kB. The "radix" column is obmalloc_radix_tree (commit 5e00f6041)

[Python-Dev] Re: radix tree arena map for obmalloc

On 2019-06-14, Tim Peters wrote: > However, last I looked there Neil was still using 4 KiB obmalloc > pools, all page-aligned. But using much larger arenas (16 MiB, 16 > times bigger than my branch, and 64 times bigger than Python currently > uses). I was testing it verses your obmalloc-big-pool

[Python-Dev] Re: radix tree arena map for obmalloc

On 2019-06-15, Inada Naoki wrote: > Oh, do you mean your branch doesn't have headers in each page? That's right. Each pool still has a header but pools can be larger than the page size. Tim's obmalloc-big-pool idea writes something to the head of each page within a pool. The radix tree doesn't

[Python-Dev] radix tree arena map for obmalloc

I've been working on this idea for a couple of days. Tim Peters has being helping me out and I think it has come far enough to get some more feedback. It is not yet a good replacement for the current address_in_range() test. However, performance wise, it is very close. Tim figures we are not

[Python-Dev] Re: obmalloc (was Have a big machine and spare time? Here's a possible Python bug.)

On 2019-06-09, Tim Peters wrote: > And now there's a PR that removes obmalloc's limit on pool sizes, and, > for a start, quadruples pool (and arena!) sizes on 64-bit boxes: Neat. > As the PR says, > > """ > It would be great to get feedback from 64-bit apps that do massive > amounts of

[Python-Dev] Re: Expected stability of PyCode_New() and types.CodeType() signatures

On 2019-05-31, Simon Cross wrote: > As the maintainer of Genshi, one the libraries affected by the CodeType and > similar changes, I thought I could add a users perspective to the > discussion: [...] Thanks. I think this change to PyCode_New() could have been handled a bit better. Couldn't we

[Python-Dev] Re: obmalloc (was Have a big machine and spare time? Here's a possible Python bug.)

On 2019-06-06, Tim Peters wrote: > The doubly linked lists in gc primarily support efficient > _partitioning_ of objects for gc's purposes (a union of disjoint sets, > with constant-time moving of an object from one set to another, and > constant-time union of disjoint sets). "All objects" is

[Python-Dev] Re: obmalloc (was Have a big machine and spare time? Here's a possible Python bug.)

On 2019-06-06, Tim Peters wrote: > Like now: if the size were passed in, obmalloc could test the size > instead of doing the `address_in_range()` dance(*). But if it's ever > possible that the size won't be passed in, all the machinery > supporting `address_in_range()` still needs to be there,

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

On 2019-05-21, Brett Cannon wrote: > On Tue., May 21, 2019, 13:07 Neil Schemenauer, > wrote: > > Here is an alternative, straw man, proposal. Split the CPython repo > > into two parts: > > > > - core Python: minimal possible stdlib > > - ever

Re: [Python-Dev] PEP 594: Removing dead batteries from the standard library

On 2019-05-21, Terry Reedy wrote: > The problem with this argument, taken by itself, it that it would argue for > adding to the stdlib 100s or 1000s of modules or packages that would be > useful to many more people than the modules proposed to be dropped. I don't think it does. We are not

Re: [Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

On 2019-04-27, Nathaniel Smith wrote: > For Py_TRACE_REFS specifically, IIUC the only goal is to be able to produce > a list of all live objects on demand. If that's the goal, then static type > objects aren't a huge deal. You can't add extra data into the type objects > themselves, but since

Re: [Python-Dev] Use C extensions compiled in release mode on a Python compiled in debug mode

On 2019-04-24, Victor Stinner wrote: > The current blocker issue is that the Py_DEBUG define imply the > Py_TRACE_REFS define I think your change to make Py_TRACE_REFS as separate configure flag is fine. I've used the trace fields to debug occasionally but I don't use it often enough to need it

Re: [Python-Dev] PEP 556 threaded garbage collection & linear recursion in gc

On 2019-03-28, Antoine Pitrou wrote: > On Wed, 27 Mar 2019 15:59:25 -0700 > "Gregory P. Smith" wrote: > > > > That had a C++ stack trace 1000+ levels deep repeating the pattern of > > > > ... > > @ 0x564d59bd21de 32 func_dealloc > > @ 0x564d59bce0c1 32

Re: [Python-Dev] Register-based VM [Was: Possible performance regression]

On 2019-02-27, Victor Stinner wrote: > The compiler begins with using static single assignment form (SSA) but > then uses a register allocator to reduce the number of used registers. > Usually, at the end you have less than 5 registers for a whole > function. In case anyone is interested on

Re: [Python-Dev] [RELEASE] Python 3.8.0a1 is now available for testing

On 2019-02-26, Stephane Wirtel wrote: > I also filled an issue [2] for brotlipy (used by httpbin and requests). > The problem is with PyInterpreterState. I tried compiling psycopg2 today and it has a similar problem: psycopg/psycopgmodule.c: In function ‘psyco_is_main_interp’:

Re: [Python-Dev] Register-based VM [Was: Possible performance regression]

On 2019-02-27, Greg Ewing wrote: > Joe Jevnik via Python-Dev wrote: > > If Python switched to a global stack and global registers we may be able > > to eliminate a lot of instructions that just shuffle data from the > > caller's stack to the callee's stack. > > That would make implementing

Re: [Python-Dev] Possible performance regression

On 2019-02-26, Raymond Hettinger wrote: > That said, I'm only observing the effect when building with the > Mac default Clang (Apple LLVM version 10.0.0 (clang-1000.11.45.5). > When building GCC 8.3.0, there is no change in performance. My guess is that the code in _PyEval_EvalFrameDefault() got

[Python-Dev] Compile-time resolution of packages [Was: Another update for PEP 394...]

On 2019-02-26, Gregory P. Smith wrote: > On Tue, Feb 26, 2019 at 9:55 AM Barry Warsaw wrote: > For an OS distro provided interpreter, being able to restrict its use to > only OS distro provided software would be ideal (so ideal that people who > haven't learned the hard distro maintenance lessons

[Python-Dev] Register-based VM [Was: Possible performance regression]

On 2019-02-26, Victor Stinner wrote: > I made an attempt once and it was faster: > https://faster-cpython.readthedocs.io/registervm.html Interesting. I don't think I have seen that before. Were you aware of "Rattlesnake" before you started on that? It seems your approach is similar. Probably

Re: [Python-Dev] Possible performance regression

On 2019-02-25, Eric Snow wrote: > So it looks like commit ef4ac967 is not responsible for a performance > regression. I did a bit of exploration myself and that was my conclusion as well. Perhaps others would be interested in how to use "perf" so I did a little write up:

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

On 2019-02-13, Terry Reedy wrote: > It appears python is already python3 for a large majority of human users (as > opposed to machines). IMHO, the question about where /usr/bin/python points is more important for machines than for humans. Think about changing /bin/sh to some different version of

Re: [Python-Dev] Another update for PEP 394 -- The "python" Command on Unix-Like Systems

On 2019-02-13, Barry Warsaw wrote: > I personally would like for `python` to be the latest Python 3 > version (or perhaps Brett’s launcher), `python2` to be Python 2.7 > where installed (and not mandatory). `python3` would be an alias > for the latest Python 3. To me, having 'py' on Unix would

Re: [Python-Dev] About the future of multi-process Python

On 2019-02-06, Antoine Pitrou wrote: > For maximum synergy between these initiatives and the resulting APIs, > it is better if things are done in the open ;-) Hi Antoine, It would be good if we could have some feedback from alternative Python implementations as well. I suspect they might want

Re: [Python-Dev] Asking for reversion

I wrote: > Could we somehow mark these APIs as experimental in 3.8? It seems the change "e5ef45b8f519a9be9965590e1a0a587ff584c180" the one we are discussing. It adds two new files: Lib/multiprocessing/shared_memory.py Modules/_multiprocessing/posixshmem.c It doesn't introduce new C APIs.

Re: [Python-Dev] Asking for reversion

On 2019-02-05, Giampaolo Rodola' wrote: > The main problem I have with this PR is that it seems to introduce > 8 brand new APIs, but since there is no doc, docstrings or tests > it's unclear which ones are supposed to be used, how or whether > they are supposed to supersede or deprecate older

Re: [Python-Dev] Add more SyntaxWarnings?

On 2019-01-24, Terry Reedy wrote: > Serhiy Storchaka suggested a compiler SyntaxWarning and uploaded a > proof-of-concept diff that handled the above and many similar cases. I believe that in general we should give better errors or warnings if we can do it without huge difficulty. Serhiy's patch

Re: [Python-Dev] C API changes

On 2018-11-29, Armin Rigo wrote: > ...Also, although I'm discussing it here, I think the whole approach > would be better if done as a third-party extension for now, without > requiring changes to CPython---just use the existing C API to > implement the CPython version. Hello Armin, Thank you

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

On 2018-11-19, Antoine Pitrou wrote: > There are important use cases for the C API where it is desired to have > fast type-specific access to Python objects such as tuples, ints, > strings, etc. This is relied upon by modules such as _json and _pickle, > and third-party extensions as well. Thank

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

On 2018-11-19, Victor Stinner wrote: > Moreover, I failed to find anyone who can explain me how the C API > is used in the wild, which functions are important or not, what is > the C API, etc. One idea is to download a large sample of extension modules from PyPI and then analyze them with some

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

On 2018-11-16, Nathaniel Smith wrote: > [..] it seems like you should investigate (a) whether you can make > Py_LIMITED_API *be* that API, instead of having two different > ifdefs That might be a good idea. One problem is that we might like to make backwards incompatible changes to

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

On 2018-11-16, Brett Cannon wrote: > I think part of the challenge here (and I believe it has been > brought up elsewhere) is no one knows what kind of API is > necessary for some faster VM other than PyPy. I think we have some pretty good ideas as to what are the problematic parts of the current

Re: [Python-Dev] Experiment an opt-in new C API for Python? (leave current API unchanged)

On 2018-11-09, Dino Viehland wrote: > Rather than adding yet another pre-processor directive for this I would > suggest just adding a new header file that only has the new stable API. > For example it could just be "py.h" or "pyapi.h". It would have all of the > definitions for the stable API. I

Re: [Python-Dev] Rename Include/internals/ to Include/pycore/

On 2018-10-28, Benjamin Peterson wrote: > I don't think more or less API should be magically included based > on whether Py_BUILD_CORE is defined or not. I agree. > If we want to have private headers, we should include them where > needed and not install them. Really, Py_BUILD_CORE should go

Re: [Python-Dev] dear core-devs

On 2018-10-02, Michael Felt wrote: > I am sorry, for myself obviously - but also for Python. Obviously, I am > doing it all wrong - as I see lots of other issues being picked up > immediately. I'm not sure that's the case. There are a lot of PRs or bugs that sit there without getting reviews.

Re: [Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

On 2018-09-18, Carl Shapiro wrote: > How might people feel about using the linker to bundle a list of pre-loaded > modules into a single-file executable? The users of Python are pretty diverse so it depends on who you ask. Some would like a giant executable that includes everything they need (so

Re: [Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

On 2018-09-15, Paul Moore wrote: > On Fri, 14 Sep 2018 at 23:28, Neil Schemenauer wrote: > > We could have a new format, .pya (compiled python archive) that has > > data for many .pyc files in it. [..] > Isn't that essentially what putting the stdlib in a zipfile does? (See >

Re: [Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

On 2018-09-14, Larry Hastings wrote: > [..] adding the stat calls back in costs you half the startup.  So > any mechanism where we're talking to the disk _at all_ simply > isn't going to be as fast. Okay, so if we use hundreds of small .pyc files scattered all over the disk, that's bad? Who

Re: [Python-Dev] Store startup modules as C structures for 20%+ startup speed improvement?

On 2018-09-14, Larry Hastings wrote: [...] > improvement 0.21242667903482038 % I assume that should be 21.2 % othewise I recommend you abandon the idea. ;-P > The downside of the patch: for these modules it ignores the Python files on > disk--it doesn't even stat them. Having a command-line/env

Re: [Python-Dev] Heap-allocated StructSequences

On 2018-09-04, Eddie Elizondo wrote: > Solution: > > * Fix the implementation of PyStructSequence_NewType: > > The best solution would be to fix the implementation of this > function. This can easily be done by dynamically creating a > PyType_Spec and calling PyType_FromSpec Hello Eddie,

Re: [Python-Dev] bpo-34595: How to format a type name?

On 2018-09-13, Victor Stinner wrote: > Right, that's a side effect of the discussion on the C API. It seems > like Py_TYPE() has to go in the new C API. Sorry, the rationale is not > written down yet, but Dino convinced me that Py_TYPE() has to go :-) My understanding is that using Py_TYPE()

Re: [Python-Dev] Let's change to C API!

On 2018-07-31, Victor Stinner wrote: > I would be nice to be able to use something to "generate" C > extensions, maybe even from pure Python code. But someone has to > work on a full solution to implement that. Perhaps a "argument clinic on steroids" would be the proper approach. So, extensions

Re: [Python-Dev] When tp_clear returns non-zero?

On 2018-05-28, Serhiy Storchaka wrote: > I'm interesting what the result of this function means. In what > cases it can return non-zero, and can it set an exception? My memory is fuzzy (nearly 20 years since I wrote that code). My best guess is that I thought a return value might be useful

Re: [Python-Dev] PEP 574 (pickle 5) implementation and backport available

On 2018-05-25, Antoine Pitrou wrote: > The question is what purpose does it serve for pickle to do it rather > than for the user to compress the pickle themselves. You're basically > saving one line of code. It's one line of code everywhere pickling or unpicking happens. And you probably need

Re: [Python-Dev] PEP 574 (pickle 5) implementation and backport available

On 2018-05-25, Antoine Pitrou wrote: > Do you have something specific in mind? I think compressed by default is a good idea. My quick proposal: - Use fast compression like lz4 or zlib with Z_BEST_SPEED - Add a 'compress' keyword argument with a default of None. For protocol 5, None means to

Re: [Python-Dev] Python startup time

On 2018-05-03, Lukasz Langa wrote: > > On May 2, 2018, at 8:57 PM, INADA Naoki wrote: > > * Add lazy compiling API or flag in `re` module. The pattern is compiled > > when first used. > > How about go the other way and allow compiling at Python > *compile*-time? That

Re: [Python-Dev] Python startup time

Antoine: > The overhead of importing is not in trying too many names, but in > loading the module and executing its bytecode. That was my conclusion as well when I did some profiling last fall at the Python core sprint. My lazy execution experiments are an attempt to solve this:

Re: [Python-Dev] 'continue'/'break'/'return' inside 'finally' clause

On 2018-01-04, Guido van Rossum wrote: > We should interview you for the paper we may be writing for HOPL. History of Programming Languages? I did some more digging this afternoon, trying to find source code between versions 1.0.1 and 0.9.1. No luck though. It looks like 0.9.1 might have been

Re: [Python-Dev] 'continue'/'break'/'return' inside 'finally' clause

On 2018-01-03, Guido van Rossum wrote: > I'm sorry, I don't think more research can convince me either way. > I want all three of return/break/continue to work inside finally > clauses, despite there being few use cases. That's fine. The history of 'continue' inside 'finally' is interesting.

Re: [Python-Dev] 'continue'/'break'/'return' inside 'finally' clause

On 2018-01-03, Serhiy Storchaka wrote: > I haven't found 'finally' clauses in > https://github.com/gevent/gevent/blob/master/src/gevent/libev/corecffi.py. > Perhaps this code was changed in recent versions. Yes, I was looking at was git revision bcf4f65e. I reran my AST checker and found this:

Re: [Python-Dev] 'continue'/'break'/'return' inside 'finally' clause

Generally I think programming language implementers don't get to decide how the language works. You just have to implement it as specified, inconvenient as that might be. However, from a languge design prespective, I think there is a good argument that this is a corner of the language we should

Re: [Python-Dev] 'continue'/'break'/'return' inside 'finally' clause

Serhiy Storchaka wrote: > Currently 'break' and 'return' are never used inside 'finally' > clause in the stdlib. See the _recv_bytes() function: Lib/multiprocessing/connection.py: 316 > I would want to see a third-party code that uses them. These are the only ones I found

[Python-Dev] Enabling depreciation warnings feature code cutoff

On 2017-11-06, Nick Coghlan wrote: > Gah, seven years on from Python 2.7's release, I still get caught by > that. I'm tempted to propose we reverse that decision and go back to > enabling them by default :P Either enable them by default or make them really easy to enable for development

Re: [Python-Dev] Reorganize Python categories (Core, Library, ...)?

Antoine Pitrou wrote: > There is no definite "correct category" when you're mixing different > classification schemes (what kind of bug it is -- > bug/security/enhancement/etc. --, what functional domain it pertains > to -- networking/concurrency/etc. --, which stdlib API it

Re: [Python-Dev] Python startup optimization: script vs. service

Christian Heimes wrote: > That approach could work, but I think that it is the wrong > approach. I'd rather keep Python optimized for long-running > processes and introduce a new mode / option to optimize for > short-running scripts. Another idea is to run a fake

[Python-Dev] Lazy initialization of module global state

This is an idea that came out of the lazy module loading (via AST analysis), posted to python-ideas. The essential idea is to split the marshal data stored in the .pyc into smaller pieces and only load the parts as they are accessed. E.g. use a __getattr__ hook on the module to unmarshal+exec

Re: [Python-Dev] PEP 549: Instance Properties (aka: module properties)

Larry Hastings wrote: > The TL;DR summary: add support for property objects to modules. > I've already posted a prototype. I posted an idea to python-ideas about lazy module loading. If the lazy loading idea works, having properties would allow modules to continue to be

Re: [Python-Dev] Consolidate stateful runtime globals

Is there any issue with unit-at-a-time optimization? I would imagine that a static global would allow optimizations that are not safe for a exported global (not sure the C term for it). I suspect it doesn't matter and I support the idea in general. Global variables in extension modules kills the

[Python-Dev] Memory bitmaps for the Python cyclic garbage collector

Python objects that participate in cyclic GC (things like lists, dicts, sets but not strings, ints and floats) have extra memory overhead. I think it is possible to mostly eliminate this overhead. Also, while the GC is running, this GC state is mutated, which destroys copy-on-write

Re: [Python-Dev] To reduce Python "application" startup time

INADA Naoki wrote: > Current `python -v` is not useful to optimize import. > So I use this patch to profile import time. > https://gist.github.com/methane/e688bb31a23bcc437defcea4b815b1eb I have implemented DTrace probes that do almost the same thing. Your patch is better

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

Nick Coghlan wrote: > It could be very interesting to add an "ascii-warn" codec to Python > 2.7, and then set that as the default encoding when the -3 flag is > set. I don't think that can work. The library code in Python would spew out warnings even in the cases when

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On 6/10/2016 10:49 AM, Nick Coghlan wrote: What Brett said is mostly accurate for me, except with one slight caveat: I've been explicitly trying to nudge you towards making the *existing tools better*, rather than introducing new tools. With modernize and futurize we have a fairly clear

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On 2016-06-09, Brett Cannon wrote: > I don't think you meant for what you said to sound insulting, > Neil, but it did feel like it upon first reading. Sorry, I think I misunderstood what you and Nick were saying. I've experienced a fair amount of negative feedback on my idea so I'm pretty cranky

Re: [Python-Dev] Smoothing the transition from Python 2 to 3

On 2016-06-09, Brett Cannon wrote: > On Thu, 9 Jun 2016 at 14:56 Nick Coghlan wrote: > > Once you switch to those now recommended more conservative migration > > tools, the tool suite you request already exists: > > > > - update your code with modernize or futurize > > - check

[Python-Dev] Smoothing the transition from Python 2 to 3

[I've posted something about this on python-ideas but since I now have some basic working code, I think it is more than an idea.] I think the uptake of Python 3 is starting to accelerate. That's good. However, there are still millions or maybe billions of lines of Python code that still needs

Re: [Python-Dev] PyGC_Collect ignores state of `enabled`

Nick Coghlan wrote: > PEP 3121 is insufficient, since a lot of extension modules can't (or > at least haven't) adopted it in practice. > https://www.python.org/dev/peps/pep-0489/ has some more background on > that (since it was the first step towards tackling the problem in a

Re: [Python-Dev] PyGC_Collect ignores state of `enabled`

Benjamin Peterson wrote: > Adding PyGC_CollectIfEnabled() and calling it in Py_Finalize is probably > fine. I don't think the contract of PyGC_Collect itself (or gc.collect() > for that matter) should be changed. You might want to disable GC but > invoke it yourself. Yes,

Re: [Python-Dev] PyGC_Collect ignores state of `enabled`

Hi, I intended for gc.collect() to actually collect cycles even if the auto-GC was disabled. Having Py_Finalize() run GC even when it has been disabled seems wrong to me. Originally, cyclic GC was supposed to be optional. Back then, most programs did not leak cycles. I would vote for

[Python-Dev] obmalloc mmap/munmap thrashing

I was running Python 2.4.11 under strace and I noticed some odd looking system calls: mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f9848681000 munmap(0x7f9848681000, 262144) = 0 mmap(NULL, 262144, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0)

Re: [Python-Dev] Migration from Python 2.7 and bytes formatting

On 2014-01-18, Stephen J. Turnbull wrote: The above are descriptions of current behavior (ie, unchanged by PEPs 460, 461), and this: [..] is the content of this proposal, is that right? The proposal is that -2 enables the following: - %r as an alias for %a (i.e. calls ascii()) - %s will

Re: [Python-Dev] PEP 461 Final?

Ethan Furman et...@stoneleaf.us wrote: So, if %a is added it would act like: - %a % some_obj - tmp = str(some_obj) res = b'' for ch in tmp: if ord(ch) 256: res += bytes([ord(ch)] else: res += unicode_escape(ch) -

Re: [Python-Dev] PEP 461 Final?

Steven D'Aprano st...@pearwood.info wrote: To properly handle int and float subclasses, int(), index(), and float() will be called on the objects intended for (d, i, u), (b, o, x, X), and (e, E, f, F, g, G). -1 on this idea. This is a rather large violation of the principle of least

  1   2   3   >