Re: [Python-Dev] PyGC_Collect ignores state of `enabled`
Hi, I intended for gc.collect() to actually collect cycles even if the auto-GC was disabled. Having Py_Finalize() run GC even when it has been disabled seems wrong to me. Originally, cyclic GC was supposed to be optional. Back then, most programs did not leak cycles. I would vote for Py_Finalize() checking the 'enabled' flag and not calling PyGC_Collect if false. Regards, Neil ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding Type[C] to PEP 484
Right. I still have no good intuition for what Type[BasicUser, ProUser]
would mean so I think you should be required to use the Union, which is
clear. Type[] should only allow one parameter.
Most real uses of Type[] use a type variable anyway.
I really recommend that you read through
https://github.com/python/typing/issues/107 even though it's long and
meanders a bit; it comes to the right conclusion in the end.
On Sun, May 15, 2016 at 10:24 AM, Peter Ludemann
wrote:
>
>
> On 14 May 2016 at 13:28, Guido van Rossum wrote:
>
>> On Sat, May 14, 2016 at 11:47 AM, Peter Ludemann
>> wrote:
>>
>>> I think that Type[C,D] would mean Union[Type[C], Type[D]], but I'm not
>>> sure ... I should probably talk to a typing expert about this.
>>> (Sorry for thinking out loud; but if we decide that Type[C,D] doesn't
>>> make sense, we need to prohibit it)
>>>
>>
>> It sounds iffy enough to prohibit.
>>
>>
>>> I suppose I'm asking: do we allow new_user(Type[BasicUser, ProUser])?
>>>
>>
>> That looks like a call, but Type[] is something to use in the definition.
>>
>>
>>> Here's a trivial example that off the top of my head makes sense:
>>>
>>> BasicOrProUser = TypeVar('BasicOrProUser', BasicUser, ProUser)
>>> def new_user(name: str, user_factory: Type[BasicOrProuser]) ->
>>> BasicOrProUser:
>>> return user_factory(name)
>>>
>>
>> That looks reasonable and only has one parameter to Type[].
>>
>
> So, should we allow this (by analogy to TypeVar allowing multiple types):
>
>
> def new_user(name: str, user_factory: Type[BasicUser, ProUser]) ->
> Union[BasicUser, ProUser]:
>
>
> return user_factory(name)
>
> ?
> ...
> or should it be:
>
>
> def new_user(name: str,
>
> user_factory: Type[
>
> Union[
> BasicUser, ProUser
> ]
> ]) -> Union[BasicUser, ProUser]:
> return user_factory(name)
>
> ?
>
> Although both of these lose the information that the output type is
> related to the input type; and hopefully the verbosity would encourage
> people to use TypeVar.
>
> (From these examples, my preference is for Type to be limited to a single
> choice and anything more complicated should be done with TypeVar, Union,
> etc.)
>
>
>
>>
>>
>>> Or must the Type[...] item have been defined with a TypeVar(...,
>>> bound=...), in which case multiple types aren't allowed with Type[...]?
>>>
>>
>> You can use whatever you want. If you want to write Type[int] go right
>> ahead. Even Type[Union[int, str]] should be allowed. There is *usually* a
>> good reason to use a type variable though -- without one the return type
>> would be the base class. (But then again maybe that could be what you want,
>> or you might not even have a return type, e.g. when the thing just gets
>> thrown into the database.)
>>
>> --Guido
>>
>>
>>>
>>>
>>> On 14 May 2016 at 11:30, Guido van Rossum wrote:
>>>
What would Type[C, D] mean?
--Guido (mobile)
On May 14, 2016 11:21 AM, "Peter Ludemann via Python-Dev" <
python-dev@python.org> wrote:
> Is Type[C,D] allowed? Or should multiple types be restricted to
> TypeVar?
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>
>
--
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Adding Type[C] to PEP 484
On 14 May 2016 at 13:28, Guido van Rossum wrote:
> On Sat, May 14, 2016 at 11:47 AM, Peter Ludemann
> wrote:
>
>> I think that Type[C,D] would mean Union[Type[C], Type[D]], but I'm not
>> sure ... I should probably talk to a typing expert about this.
>> (Sorry for thinking out loud; but if we decide that Type[C,D] doesn't
>> make sense, we need to prohibit it)
>>
>
> It sounds iffy enough to prohibit.
>
>
>> I suppose I'm asking: do we allow new_user(Type[BasicUser, ProUser])?
>>
>
> That looks like a call, but Type[] is something to use in the definition.
>
>
>> Here's a trivial example that off the top of my head makes sense:
>>
>> BasicOrProUser = TypeVar('BasicOrProUser', BasicUser, ProUser)
>> def new_user(name: str, user_factory: Type[BasicOrProuser]) ->
>> BasicOrProUser:
>> return user_factory(name)
>>
>
> That looks reasonable and only has one parameter to Type[].
>
So, should we allow this (by analogy to TypeVar allowing multiple types):
def new_user(name: str, user_factory: Type[BasicUser, ProUser]) ->
Union[BasicUser, ProUser]:
return user_factory(name)
?
...
or should it be:
def new_user(name: str,
user_factory: Type[
Union[
BasicUser, ProUser
]
]) -> Union[BasicUser, ProUser]:
return user_factory(name)
?
Although both of these lose the information that the output type is related
to the input type; and hopefully the verbosity would encourage people to
use TypeVar.
(From these examples, my preference is for Type to be limited to a single
choice and anything more complicated should be done with TypeVar, Union,
etc.)
>
>
>> Or must the Type[...] item have been defined with a TypeVar(...,
>> bound=...), in which case multiple types aren't allowed with Type[...]?
>>
>
> You can use whatever you want. If you want to write Type[int] go right
> ahead. Even Type[Union[int, str]] should be allowed. There is *usually* a
> good reason to use a type variable though -- without one the return type
> would be the base class. (But then again maybe that could be what you want,
> or you might not even have a return type, e.g. when the thing just gets
> thrown into the database.)
>
> --Guido
>
>
>>
>>
>> On 14 May 2016 at 11:30, Guido van Rossum wrote:
>>
>>> What would Type[C, D] mean?
>>>
>>> --Guido (mobile)
>>> On May 14, 2016 11:21 AM, "Peter Ludemann via Python-Dev" <
>>> python-dev@python.org> wrote:
>>>
Is Type[C,D] allowed? Or should multiple types be restricted to
TypeVar?
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] file system path protocol PEP
Greg Ewing writes: > Paul Moore wrote: > > On 13 May 2016 at 17:57, Ethan Furman wrote: > > > >>1) What is a wallet garden? > > > > I assumed he meant "walled garden" > > Works either way -- you'd want a wall around your wallet > garden to stop people stealing your wallets. Actually, I expect it's a wall intended to keep your clients' wallets in a garden where you can harvest them at your convenience. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Speeding up CPython 5-10%
2016-02-01 17:54 GMT+01:00 Yury Selivanov : > Thanks for bringing this up! > > IIRC wpython was about using "fat" bytecodes, i.e. using 64bits per > bytecode instead of 8. No, it used 16, 32, and 48-bit per opcode (1, 2, or 3 16-bit words). > That allows to minimize the number of bytecodes, thus having some > performance increase. TBH, I don't think it was "significantly faster". > Please, take a look at the benchmarks, or compile it and check yourself. ;-) If I were to do some big refactoring of the ceval loop, I'd probably > consider implementing a register VM. While register VMs are a bit faster > than stack VMs (up to 20-30%), they would also allow us to apply more > optimizations, and even bolt on a simple JIT compiler. > > Yury WPython was an hybrid-VM: it supported both a stack-based and a register-based approach. I think that it's needed, since the nature of Python, because you can have operations with intermixed operands: constants, locals, globals, names. It's quite difficult to handle all possible cases with a register-based VM. Regards, Cesare ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Wordcode: new regular bytecode using 16-bit units
2016-04-13 23:23 GMT+02:00 Victor Stinner : > Hopefully, I don't expect 32-bit parameters in the wild, only 24-bit > parameter for function with annotation. > I never found 32-bit parameters, and not even 24-bit ones. I think that their usage is as rare as all planets alignment. ;-) That's why with in WPython I supported only 8, 16, and 32-bit parameters (which are 6 bytes long). Regards, Cesare ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Wordcode: new regular bytecode using 16-bit units
2016-04-13 18:24 GMT+02:00 Victor Stinner : > Demur Rumed proposes a different change to use a regular bytecode > using 16-bit units: an instruction has always one 8-bit argument, it's > zero if the instruction doesn't have an argument: > >http://bugs.python.org/issue26647 > > According to benchmarks, it looks faster: > > http://bugs.python.org/issue26647#msg263339 > > IMHO it's a nice enhancement: it makes the code simpler. The most > interesting change is made in Python/ceval.c: > > -if (HAS_ARG(opcode)) > -oparg = NEXTARG(); > +oparg = NEXTARG(); > > This code is the very hot loop evaluating Python bytecode. I expect > that removing a conditional branch here can reduce the CPU branch > misprediction. > Correct. The old bytecode format wasn't so much predictable for the CPU. > > Right now, ceval.c still fetchs opcode and then oparg with two 8-bit > instructions. Later, we can discuss if it would be possible to ensure > that the bytecode is always aligned to 16-bit in memory to fetch the > two bytes using a uint16_t* pointer. > > Maybe we can overallocate 1 byte in codeobject.c and align manually > the memory block if needed. Or ceval.c should maybe copy the code if > it's not aligned? > > Raymond Hettinger proposes something like that, but it looks like > there are concerns about non-aligned memory accesses: > >http://bugs.python.org/issue25823 > > The cost of non-aligned memory accesses depends on the CPU > architecture, but it can raise a SIGBUS on some arch (MIPS and > SPARC?). > > Victor > It should not be a problem, since every PyObject is allocated with PyAlloc (however I don't remember if it's the correct name) which AFAIK guarantees a base 8 bytes alignment. So, it's safe to use an unsigned int for keeping/referencing a word at the time. The only problem with such approach is related to the processor endianess, but it can be solved with proper macros (like I did with WPython). Regards, Cesare ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Wordcode v2
2016-02-17 12:04 GMT+01:00 Antoine Pitrou : > Demur Rumed gmail.com> writes: > > I've personally benchmarked this fork with positive results. > > I'm skeptical of claims like this. What did you benchmark exactly, and with > which results? > I don't think changing the opcode encoding per se will bring any large > benefit... > > Regards > > Antoine. With WPython I've introduced several optimizations which improved a lot the execution speed (+25% with PyStone, at the time, compared to CPython 2.6), but most of the benefits came from the new opcode format. Regards, Cesare ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Wordcode v2
2016-02-15 8:14 GMT+01:00 Andrew Barnert via Python-Dev < python-dev@python.org>: > Despite the name (and inspiration), my fork has very little to do with > WPython. I'm just focused on simpler (hopefully = faster) fetch code; he > started with that, but ended up going the exact opposite direction, > accepting more complicated (and much slower) fetch code as a reasonable > cost for drastically reducing the number of instructions. (If you double > the 30% fetch-and-parse overhead per instruction, but cut the number of > instructions to 40%, the net is a huge win.) I don't know why you consider slower the WPython's code that fetches the more complicated instructions. On the contrary, I've structured such "superinstructions" in order to simplify their decoding. Arguments are decoded as they are needed in a specific moment, in order to reduce or completely avoid the usage of temporary variables to keep such values. Can you provide some example about your claim? Regarding the WPython goal, it wasn't only about introducing simpler instructions. As I've written also in my presentation, it's an hybrid VM: stack and register-based. I've introduced a new instruction format for the existing CPython's instructions, which are now easier to fetch, decode, execute, and provide a better density too (for the most common case: arguments with a maximum of 255 as value/index). However I've also added superinstructions to better pack more "useful work", which provides more code density and they are the primary responsible for improving the execution speed. Regards, Cesare ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Wordcode v2
2016-02-15 1:20 GMT+01:00 Demur Rumed : > Saw recent discussion: > https://mail.python.org/pipermail/python-dev/2016-February/143013.html > > I remember trying WPython; it was fast. Unfortunately it feels it came at > the wrong time when development was invested in getting py3k out the door. > Not only that. IMO the primary problem was related to the fact the "patch" was too big to be reviewed. Unfortunately it was my first attempt, and having worked alone I introduced too much optimizations and (heavy) changes to the code. An incremental approach should have worked better, albeit I believe that such drastic move from the consolidated bytecodes to the new wordcodes would have produced strong resistance anyway. > It also had a lot of other ideas like *_INT instructions which allowed > having oparg to be a constant int rather than needing to LOAD_CONST one. > This, specifically, was an experiment that I made with WPython 1.1, which I recommend to do not follow. There are other, more general, ways to speedup the execution when dealing with integers. ___ Python-Dev mailing list Python-Dev@python.org https://mail.python.org/mailman/listinfo/python-dev Unsubscribe: https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] file system path protocol PEP
On Fri, May 13, 2016 at 7:43 PM, Chris Angelico wrote:
[...]
> "Check" accepts subclasses; "CheckExact" doesn't (it's like "type(x)
> is str"). The question is, which one SHOULD be being done?
Indeed it should do "Check", so that path libraries that do inherit
from str will still work (see also below).
> What should this do:
>
> class TmpPath(str):
> def __fspath__(self):
> return "/tmp/"+self
> x = TmpPath("foo/bar")
> open(x, "w")
>
> Does that respect __fspath__, or respect the fact that it's a string?
It should not respect __fspath__ because that would mean the behavior
is different in older versions of Python(!). This also means that such
code should never be written (except for that example you wrote ;-).
So indeed, composition would be the way to go.
So yes, I still think str subclasses should not implement __fspath__,
and that, at the very least, the docs should say that :).
-- Koos
> ChrisA
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/k7hoven%40gmail.com
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
Re: [Python-Dev] Speeding up CPython 5-10%
2016-02-02 10:28 GMT+01:00 Victor Stinner :
> 2016-01-27 19:25 GMT+01:00 Yury Selivanov :
> > tl;dr The summary is that I have a patch that improves CPython
> performance
> > up to 5-10% on macro benchmarks. Benchmarks results on Macbook Pro/Mac
> OS
> > X, desktop CPU/Linux, server CPU/Linux are available at [1]. There are
> no
> > slowdowns that I could reproduce consistently.
>
> That's really impressive, great job Yury :-) Getting non-negligible
> speedup on large macrobenchmarks became really hard in CPython.
> CPython is already well optimized in all corners.
It's long time since I took a look at CPython (3.2), but if it didn't
changed a lot then there might be some corner cases still waiting to be
optimized. ;-)
Just one thing that comes to my mind: is the stack depth calculation
routine changed? It was suboptimal, and calculating a better number
decreases stack allocation, and increases the frame usage.
> It looks like the
> overall Python performance still depends heavily on the performance of
> dictionary and attribute lookups. Even if it was well known, I didn't
> expect up to 10% speedup on *macro* benchmarks.
>
True, but it might be mitigated in some ways, at least for built-in types.
There are ideas about that, but they are a bit complicated to implement.
The problem is with functions like len, which IMO should become attribute
lookups ('foo'.len) or method executions ('foo'.len()). Then it'll be
easier to accelerate their execution, with one of the above ideas.
However such kind of changes belong to Guido, which defines the language
structure/philosophy. IMO something like len should be part of the
attributes exposed by an object: it's more "object-oriented". Whereas other
things like open, file, sum, etc., are "general facilities".
Regards,
Cesare
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe:
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com
