Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-05 Thread Koos Zevenhoven 
On Tue, Oct 3, 2017 at 1:11 AM, Koos Zevenhoven  wrote:

> On Oct 3, 2017 01:00, "Guido van Rossum"  wrote:
>
>  Mon, Oct 2, 2017 at 2:52 PM, Koos Zevenhoven  wrote
>
> I don't mind this (or Nathaniel ;-) being academic. The backwards
>> incompatibility issue I've just described applies to any extension via
>> composition, if the underlying type/protocol grows new members (like the CM
>> protocol would have gained __suspend__ and __resume__ in PEP521).
>>
>
> Since you seem to have a good grasp on this issue, does PEP 550 suffer
> from the same problem? (Or PEP 555, for that matter? :-)
>
>
>
> Neither has this particular issue, because they don't extend an existing
> protocol. If this thread has any significance, it will most likely be
> elsewhere.
>

​Actually, I realize I should be more precise with terminology regarding
"extending an existing protocol"/"growing new members". Below, I'm still
using PEP 521 as an example (sorry).

In fact, in some sense, "adding" __suspend__ and __resume__ to context
managers *does not* extend the context manager protocol, even though it
kind of looks like it does.

There would instead be two separate protocols:

(A) The traditional PEP 343 context manager:
__enter__
__exit__

(B) The hyphothetical PEP 521 context manager:
__enter__
__suspend__
__resume__
__exit__

Protocols A and B are incompatible in both directions:

* It is generally not safe to use a type-A context manager assuming it
implements B.

* It is generally not safe to use a type-B context manager assuming it
implements A.

But if you now have a type-B object, it looks like it's also type-A,
especially for code that is not aware of the existence of B. This is where
the problems come from: a wrapper for type A does the wrong thing when
wrapping a type-B object (except when using inheritance).


[Side note:

Another interpretation of the situation is that, instead of adding protocol
B, A is removed and is replaced with:

(C) The hypothetical PEP 521 context manager with optional members:
__enter__
__suspend__ (optional)
__resume__  (optional)
__exit__

But now the same problems just come from the fact that A no longer exists
while there is code out there that assumes A. But this is only a useful
interpretation if you are the only user of the protocol or if it's
otherwise ok to remove A. So let's go back to the A-B interpretation.]


Q: Could the problem of protocol conflict be solved?

One way to tell A and B apart would be to always explicitly mark the
protocol with a base class. Obviously this is not the case with existing
uses of context managers.

But there's another way, which is to change the naming:

(A) The traditional PEP 343 context manager:
__enter__
__exit__

(Z) The *modified* hyphothetical PEP 521 context manager:
__begin__
__suspend__
__resume__
__end__

Now, A and Z are easy to tell apart. A context manager wrapper designed for
type A immediately fails if used to wrap a type-Z object. But of course the
whole context manager concept now suddenly became a lot more complicated.


It is interesting that, in the A-B scheme, making a general context manager
wrapper using inheritance *just works*, even if A is not a subprotocol of B
and B is not a subprotocol of A.

Anyway, a lot of this is amplified by the fact that the methods of the
context manager protocols are not independent functionality. Instead,
calling one of them leads to the requirement that the other methods are
also called at the right moments.

--Koos

-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-04 Thread Koos Zevenhoven 
On Wed, Oct 4, 2017 at 4:04 PM, Nick Coghlan  wrote:

> On 4 October 2017 at 22:45, Koos Zevenhoven  wrote:
> > On Wed, Oct 4, 2017 at 3:33 PM, Nick Coghlan  wrote:
> >> That's not a backwards compatibility problem, because the only way to
> >> encounter it is to update your code to rely on the new extended
> >> protocol - your *existing* code will continue to work fine, since
> >> that, by definition, can't be relying on the new protocol extension.
> >>
> >
> > No, not all code is "your" code. Clearly this is not a well-known
> problem.
> > This is a backwards-compatibility problem for the author of the wrappeR,
> not
> > for the author of the wrappeD object.
>
> No, you're misusing the phrase "backwards compatibility", and
> confusing it with "feature enablement".
>
> Preserving backwards compatibility just means "existing code and
> functionality don't break". It has nothing to do with whether or not
> other support libraries and frameworks might need to change in order
> to enable full access to a new language feature.
>
>
​It's not about full access to a new language feature. It's about the
wrappeR promising it can wrap any ​context manager, which it then no longer
can. If the __suspend__ and __resume__ methods are ignored, that is not
about "not having full access to a new feature" — that's broken code. The
error message you get (if any) may not contain any hint of what went wrong.

Take the length hint protocol defined in PEP 424 for example: that
> extended the iterator protocol to include a new optional
> __length_hint__ method, such that container constructors can make a
> more reasonable guess as to how much space they should pre-allocate
> when being initialised from an iterator or iterable rather than
> another container.
>
>
​This is slightly similar, but not really. Not using __length_hint__ does
not affect the correctness of code.


> That protocol means that many container wrappers break the
> optimisation. That's not a compatibility problem, it just means those
> wrappers don't support the feature, and it would potentially be a
> useful enhancement if they did.
>
>
​Again, ignoring __length_hint__ does not lead to broken code, so that just
means the wrapper is as slow or as fast as it was before.

​So I still think it's an issue for the author of the wrapper to fix––even
if just by documenting that the wrapper does not support the new protocol
members. But that would not be necessary if the wrapper uses inheritance.

(Of course there may be another reason to not use inheritance, but just
overriding two methods seems like a good case for inheritance.).
​
​This discussion seems pretty pointless by now. It's true that *some* code
needs to change for this to be a problem. Updating only the Python version
does not break a codebase if libraries aren't updated, and even then,
breakage is not very likely, I suppose.

It all depends on the kind of change that is made. For __length_hint__, you
only risk not getting the performance improvement. For __suspend__ and
__resume__, there's a small chance of problems. For some other change, it
might be even riskier. But this is definitely not the most dangerous type
of compatibility issue.

––Koos


-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-04 Thread Nick Coghlan 
On 4 October 2017 at 22:45, Koos Zevenhoven  wrote:
> On Wed, Oct 4, 2017 at 3:33 PM, Nick Coghlan  wrote:
>> That's not a backwards compatibility problem, because the only way to
>> encounter it is to update your code to rely on the new extended
>> protocol - your *existing* code will continue to work fine, since
>> that, by definition, can't be relying on the new protocol extension.
>>
>
> No, not all code is "your" code. Clearly this is not a well-known problem.
> This is a backwards-compatibility problem for the author of the wrappeR, not
> for the author of the wrappeD object.

No, you're misusing the phrase "backwards compatibility", and
confusing it with "feature enablement".

Preserving backwards compatibility just means "existing code and
functionality don't break". It has nothing to do with whether or not
other support libraries and frameworks might need to change in order
to enable full access to a new language feature.

Take the length hint protocol defined in PEP 424 for example: that
extended the iterator protocol to include a new optional
__length_hint__ method, such that container constructors can make a
more reasonable guess as to how much space they should pre-allocate
when being initialised from an iterator or iterable rather than
another container.

That protocol means that many container wrappers break the
optimisation. That's not a compatibility problem, it just means those
wrappers don't support the feature, and it would potentially be a
useful enhancement if they did.

Similarly, when context managers were added, folks needed to add
appropriate implementations of the protocol in order to be able to
actually make use of the feature. If a library didn't support it
natively, then they either needed to write their own context manager,
or else contribute an enhancement to that library.

This pattern applies whenever a new protocol is added or an existing
protocol is extended: whether or not you can actually rely on the new
feature will depend on whether or not all your dependencies also
support it.

The best case scenarios are those where we can enable a new feature in
a few key standard library APIs, and then most third party APIs will
transparently pick up the new behaviour (e.g. as we did for the fspath
protocol). However, even in situations like that, there may still be
other code that makes no longer correct assumptions, and blocks access
to the new feature (e.g. by including an explicit isinstance check).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-04 Thread Koos Zevenhoven 
On Wed, Oct 4, 2017 at 3:33 PM, Nick Coghlan  wrote:

> On 4 October 2017 at 20:22, Koos Zevenhoven  wrote:
> > On Wed, Oct 4, 2017 at 8:07 AM, Nick Coghlan  wrote:
> >>
> >> On 3 October 2017 at 03:13, Koos Zevenhoven  wrote:
> >> > Well, it's not completely unrelated to that. The problem I'm talking
> >> > about
> >> > is perhaps most easily seen from a simple context manager wrapper that
> >> > uses
> >> > composition instead of inheritance:
> >> >
> >> > class Wrapper:
> >> > def __init__(self):
> >> > self._wrapped = SomeContextManager()
> >> >
> >> > def __enter__(self):
> >> > print("Entering context")
> >> > return self._wrapped.__enter__()
> >> >
> >> > def __exit__(self):
> >> > self._wrapped.__exit__()
> >> > print("Exited context")
> >> >
> >> >
> >> > Now, if the wrapped contextmanager becomes a PEP 521 one with
> >> > __suspend__
> >> > and __resume__, the Wrapper class is broken, because it does not
> respect
> >> > __suspend__ and __resume__. So actually this is a backwards
> compatiblity
> >> > issue.
> >>
> >> This is a known problem, and one of the main reasons that having a
> >> truly transparent object proxy like
> >> https://wrapt.readthedocs.io/en/latest/wrappers.html#object-proxy as
> >> part of the standard library would be highly desirable.
> >>
> >
> > This is barely related to the problem I describe. The wrapper is not
> > supposed to pretend to *be* the underlying object. It's just supposed to
> > extend its functionality.
>
> If a wrapper *isn't* trying to act as a transparent object proxy, and
> is instead adapting it to a particular protocol, then yes, you'll need
> to update the wrapper when the protocol is extended.
>
>
​Yes, but it still means that the change in the dependency (in this case a
standard Python protocol) breaks the wrapper code.​

Remember that the wrappeR class and the wrappeD class can be implemented in
different libraries.



> That's not a backwards compatibility problem, because the only way to
> encounter it is to update your code to rely on the new extended
> protocol - your *existing* code will continue to work fine, since
> that, by definition, can't be relying on the new protocol extension.
>
>
​No, not all code is "your" code. Clearly this is not a well-known problem.
This is a backwards-compatibility problem for the author of the wrappeR,
not for the author of the wrappeD object.

––Koos
​

-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-04 Thread Nick Coghlan 
On 4 October 2017 at 20:22, Koos Zevenhoven  wrote:
> On Wed, Oct 4, 2017 at 8:07 AM, Nick Coghlan  wrote:
>>
>> On 3 October 2017 at 03:13, Koos Zevenhoven  wrote:
>> > Well, it's not completely unrelated to that. The problem I'm talking
>> > about
>> > is perhaps most easily seen from a simple context manager wrapper that
>> > uses
>> > composition instead of inheritance:
>> >
>> > class Wrapper:
>> > def __init__(self):
>> > self._wrapped = SomeContextManager()
>> >
>> > def __enter__(self):
>> > print("Entering context")
>> > return self._wrapped.__enter__()
>> >
>> > def __exit__(self):
>> > self._wrapped.__exit__()
>> > print("Exited context")
>> >
>> >
>> > Now, if the wrapped contextmanager becomes a PEP 521 one with
>> > __suspend__
>> > and __resume__, the Wrapper class is broken, because it does not respect
>> > __suspend__ and __resume__. So actually this is a backwards compatiblity
>> > issue.
>>
>> This is a known problem, and one of the main reasons that having a
>> truly transparent object proxy like
>> https://wrapt.readthedocs.io/en/latest/wrappers.html#object-proxy as
>> part of the standard library would be highly desirable.
>>
>
> This is barely related to the problem I describe. The wrapper is not
> supposed to pretend to *be* the underlying object. It's just supposed to
> extend its functionality.

If a wrapper *isn't* trying to act as a transparent object proxy, and
is instead adapting it to a particular protocol, then yes, you'll need
to update the wrapper when the protocol is extended.

That's not a backwards compatibility problem, because the only way to
encounter it is to update your code to rely on the new extended
protocol - your *existing* code will continue to work fine, since
that, by definition, can't be relying on the new protocol extension.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-04 Thread Koos Zevenhoven 
On Wed, Oct 4, 2017 at 8:07 AM, Nick Coghlan  wrote:

> On 3 October 2017 at 03:13, Koos Zevenhoven  wrote:
> > Well, it's not completely unrelated to that. The problem I'm talking
> about
> > is perhaps most easily seen from a simple context manager wrapper that
> uses
> > composition instead of inheritance:
> >
> > class Wrapper:
> > def __init__(self):
> > self._wrapped = SomeContextManager()
> >
> > def __enter__(self):
> > print("Entering context")
> > return self._wrapped.__enter__()
> >
> > def __exit__(self):
> > self._wrapped.__exit__()
> > print("Exited context")
> >
> >
> > Now, if the wrapped contextmanager becomes a PEP 521 one with __suspend__
> > and __resume__, the Wrapper class is broken, because it does not respect
> > __suspend__ and __resume__. So actually this is a backwards compatiblity
> > issue.
>
> This is a known problem, and one of the main reasons that having a
> truly transparent object proxy like
> https://wrapt.readthedocs.io/en/latest/wrappers.html#object-proxy as
> part of the standard library would be highly desirable.
>
>
This is barely related to the problem I describe. The wrapper is not
supposed to pretend to *be* the underlying object. It's just supposed to
extend its functionality.

Maybe it's just me, but using a transparent object proxy for this sounds
like someone trying to avoid inheritance for no reason and at any cost.
Inheritance probably has faster method access, and makes it more obvious
what's going on:

def Wrapper(contextmanager):
class Wrapper(type(contextmanager)):
def __enter__(self):
print("Entering context")
return contextmanager.__enter__()

def __exit__(self):
contextmanager.__exit__()
print("Exited context")
return Wrapper()


A wrapper based on a transparent object proxy is just a non-transparent
replacement for inheritance. Its wrapper nature is non-transparent because
it pretends to `be` the original object, while it's actually a wrapper.

But an object cannot `be` another object as long as the `is` operator won't
return True. And any straightforward way to implement that would add
performance overhead for normal objects.

I do remember sometimes wanting a transparent object proxy. But not for
normal wrappers. But I don't think I've gone as far as looking for a
library to do that, because it seems that you can only go half way anyway.

––Koos

​--
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-03 Thread Nick Coghlan 
On 3 October 2017 at 03:13, Koos Zevenhoven  wrote:
> Well, it's not completely unrelated to that. The problem I'm talking about
> is perhaps most easily seen from a simple context manager wrapper that uses
> composition instead of inheritance:
>
> class Wrapper:
> def __init__(self):
> self._wrapped = SomeContextManager()
>
> def __enter__(self):
> print("Entering context")
> return self._wrapped.__enter__()
>
> def __exit__(self):
> self._wrapped.__exit__()
> print("Exited context")
>
>
> Now, if the wrapped contextmanager becomes a PEP 521 one with __suspend__
> and __resume__, the Wrapper class is broken, because it does not respect
> __suspend__ and __resume__. So actually this is a backwards compatiblity
> issue.

This is a known problem, and one of the main reasons that having a
truly transparent object proxy like
https://wrapt.readthedocs.io/en/latest/wrappers.html#object-proxy as
part of the standard library would be highly desirable.

Actually getting such a proxy defined, implemented, and integrated
isn't going to be easy though, so while Graham (Dumpleton, the author
of wrapt) is generally amenable to the idea, he doesn't have the time
or inclination to do that work himself.

In the meantime, we mostly work around the problem by defining new
protocols rather than extending existing ones, but it still means it
takes longer than it otherwise for full support for new interfaces to
ripple out through various object proxying libraries (especially for
hard-to-proxy protocols like the new asynchronous ones that require
particular methods to be defined as coroutines).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-02 Thread Koos Zevenhoven 
On Oct 3, 2017 01:11, "Koos Zevenhoven"  wrote:

On Oct 3, 2017 01:00, "Guido van Rossum"  wrote:

 Mon, Oct 2, 2017 at 2:52 PM, Koos Zevenhoven  wrote

I don't mind this (or Nathaniel ;-) being academic. The backwards
> incompatibility issue I've just described applies to any extension via
> composition, if the underlying type/protocol grows new members (like the CM
> protocol would have gained __suspend__ and __resume__ in PEP521).
>

Since you seem to have a good grasp on this issue, does PEP 550 suffer from
the same problem? (Or PEP 555, for that matter? :-)



Neither has this particular issue, because they don't extend an existing
protocol. If this thread has any significance, it will most likely be
elsewhere.


That said, I did come across this thought while trying to find flaws in my
own PEP ;)

-- Koos
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-02 Thread Koos Zevenhoven 
On Oct 3, 2017 01:00, "Guido van Rossum"  wrote:

 Mon, Oct 2, 2017 at 2:52 PM, Koos Zevenhoven  wrote

I don't mind this (or Nathaniel ;-) being academic. The backwards
> incompatibility issue I've just described applies to any extension via
> composition, if the underlying type/protocol grows new members (like the CM
> protocol would have gained __suspend__ and __resume__ in PEP521).
>

Since you seem to have a good grasp on this issue, does PEP 550 suffer from
the same problem? (Or PEP 555, for that matter? :-)



Neither has this particular issue, because they don't extend an existing
protocol. If this thread has any significance, it will most likely be
elsewhere.

-- Koos (mobile)


-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-02 Thread Guido van Rossum 
 Mon, Oct 2, 2017 at 2:52 PM, Koos Zevenhoven  wrote

> I don't mind this (or Nathaniel ;-) being academic. The backwards
> incompatibility issue I've just described applies to any extension via
> composition, if the underlying type/protocol grows new members (like the CM
> protocol would have gained __suspend__ and __resume__ in PEP521).
>

Since you seem to have a good grasp on this issue, does PEP 550 suffer from
the same problem? (Or PEP 555, for that matter? :-)

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-02 Thread Koos Zevenhoven 
On Oct 3, 2017 00:02, "Guido van Rossum"  wrote:

On Mon, Oct 2, 2017 at 10:13 AM, Koos Zevenhoven  wrote:

> Hi all, It was suggested that I start a new thread, because the other
> thread drifted away from its original topic. So here, in case someone is
> interested:
>
> On Oct 2, 2017 17:03, "Koos Zevenhoven  wrote:
>
> On Mon, Oct 2, 2017 at 6:42 AM, Guido van Rossum  wrote:
>
> On Sun, Oct 1, 2017 at 1:52 PM, Koos Zevenhoven  wrote:
>
> On Oct 1, 2017 19:26, "Guido van Rossum"  wrote:
>
> Your PEP is currently incomplete. If you don't finish it, it is not even a
> contender. But TBH it's not my favorite anyway, so you could also just
> withdraw it.
>
>
> I can withdraw it if you ask me to, but I don't want to withdraw it
> without any reason. I haven't changed my mind about the big picture. OTOH,
> PEP 521 is elegant and could be used to implement PEP 555, but 521 is
> almost certainly less performant and has some problems regarding context
> manager wrappers that use composition instead of inheritance.
>
>
> It is my understanding that PEP 521 (which proposes to add optional
> __suspend__ and __resume__ methods to the context manager protocol, to be
> called whenever a frame is suspended or resumed inside a `with` block) is
> no longer a contender because it would be way too slow. I haven't read it
> recently or thought about it, so I don't know what the second issue you
> mention is about (though it's presumably about the `yield` in a context
> manager implemented using a generator decorated with
> `@contextlib.contextmanager`).
>
>
> ​Well, it's not completely unrelated to that. The problem I'm talking
> about is perhaps most easily seen from a simple context manager wrapper
> that uses composition instead of inheritance:
>
> class Wrapper:
> def __init__(self):
> self._wrapped = SomeContextManager()
>
> def __enter__(self):
> print("Entering context")
> return self._wrapped.__enter__()
>
> def __exit__(self):
> self._wrapped.__exit__()
> print("Exited context")
>
>
> Now, if the wrapped contextmanager becomes a PEP 521 one with __suspend__
> and __resume__, the Wrapper class is broken, because it does not respect
> __suspend__ and __resume__. So actually this is a backwards compatiblity
> issue.
>
>
Why is it backwards incompatible? I'd think that without PEP 521 it would
be broken in exactly the same way because there's no __suspend__/__resume__
at all.



The wrapper is (would be) broken because it depends on the internal
implementation of the wrapped CM.

Maybe the author of SomeContextManager wants to upgrade the CM to also work
in coroutines and generators. But it could be a more subtle change in the
CM implementation.

The problem becomes more serious and more obvious if you don't know which
context manager you are wrapping:

class Wrapper:
def __init__(self, contextmanager):
self._wrapped = contextmanager

def __enter__(self):
print("Entering context")
return self._wrapped.__enter__()

def __exit__(self):
self._wrapped.__exit__()
print("Exited context")


The wrapper is (would be) broken because it does not work for all CMs
anymore.


But if the wrapper is made using inheritance, the problem goes away:
>
>
> class Wrapper(SomeContextManager):
> def __enter__(self):
> print("Entering context")
> return super().__enter__()
>
> def __exit__(self):
> super().__exit__()
> print("Exited context")
>
>
> Now the wrapper cleanly inherits the new optional __suspend__ and
> __resume__ from the wrapped context manager type.
>
>
In any case this is completely academic because PEP 521 is not going to
happen. Nathaniel himself has said so (I think in the context of discussing
PEP 550).


I don't mind this (or Nathaniel ;-) being academic. The backwards
incompatibility issue I've just described applies to any extension via
composition, if the underlying type/protocol grows new members (like the CM
protocol would have gained __suspend__ and __resume__ in PEP521).


-- Koos (mobile)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-02 Thread Guido van Rossum 
On Mon, Oct 2, 2017 at 10:13 AM, Koos Zevenhoven  wrote:

> Hi all, It was suggested that I start a new thread, because the other
> thread drifted away from its original topic. So here, in case someone is
> interested:
>
> On Oct 2, 2017 17:03, "Koos Zevenhoven  wrote:
>
> On Mon, Oct 2, 2017 at 6:42 AM, Guido van Rossum  wrote:
>
> On Sun, Oct 1, 2017 at 1:52 PM, Koos Zevenhoven  wrote:
>
> On Oct 1, 2017 19:26, "Guido van Rossum"  wrote:
>
> Your PEP is currently incomplete. If you don't finish it, it is not even a
> contender. But TBH it's not my favorite anyway, so you could also just
> withdraw it.
>
>
> I can withdraw it if you ask me to, but I don't want to withdraw it
> without any reason. I haven't changed my mind about the big picture. OTOH,
> PEP 521 is elegant and could be used to implement PEP 555, but 521 is
> almost certainly less performant and has some problems regarding context
> manager wrappers that use composition instead of inheritance.
>
>
> It is my understanding that PEP 521 (which proposes to add optional
> __suspend__ and __resume__ methods to the context manager protocol, to be
> called whenever a frame is suspended or resumed inside a `with` block) is
> no longer a contender because it would be way too slow. I haven't read it
> recently or thought about it, so I don't know what the second issue you
> mention is about (though it's presumably about the `yield` in a context
> manager implemented using a generator decorated with
> `@contextlib.contextmanager`).
>
>
> ​Well, it's not completely unrelated to that. The problem I'm talking
> about is perhaps most easily seen from a simple context manager wrapper
> that uses composition instead of inheritance:
>
> class Wrapper:
> def __init__(self):
> self._wrapped = SomeContextManager()
>
> def __enter__(self):
> print("Entering context")
> return self._wrapped.__enter__()
>
> def __exit__(self):
> self._wrapped.__exit__()
> print("Exited context")
>
>
> Now, if the wrapped contextmanager becomes a PEP 521 one with __suspend__
> and __resume__, the Wrapper class is broken, because it does not respect
> __suspend__ and __resume__. So actually this is a backwards compatiblity
> issue.
>
>
Why is it backwards incompatible? I'd think that without PEP 521 it would
be broken in exactly the same way because there's no __suspend__/__resume__
at all.


> But if the wrapper is made using inheritance, the problem goes away:
>
>
> class Wrapper(SomeContextManager):
> def __enter__(self):
> print("Entering context")
> return super().__enter__()
>
> def __exit__(self):
> super().__exit__()
> print("Exited context")
>
>
> Now the wrapper cleanly inherits the new optional __suspend__ and
> __resume__ from the wrapped context manager type.
>
>
In any case this is completely academic because PEP 521 is not going to
happen. Nathaniel himself has said so (I think in the context of discussing
PEP 550).

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Inheritance vs composition in backcompat (PEP521)

2017-10-02 Thread Koos Zevenhoven 
Hi all, It was suggested that I start a new thread, because the other
thread drifted away from its original topic. So here, in case someone is
interested:

On Oct 2, 2017 17:03, "Koos Zevenhoven  wrote:

On Mon, Oct 2, 2017 at 6:42 AM, Guido van Rossum  wrote:

On Sun, Oct 1, 2017 at 1:52 PM, Koos Zevenhoven  wrote:

On Oct 1, 2017 19:26, "Guido van Rossum"  wrote:

Your PEP is currently incomplete. If you don't finish it, it is not even a
contender. But TBH it's not my favorite anyway, so you could also just
withdraw it.


I can withdraw it if you ask me to, but I don't want to withdraw it without
any reason. I haven't changed my mind about the big picture. OTOH, PEP 521
is elegant and could be used to implement PEP 555, but 521 is almost
certainly less performant and has some problems regarding context manager
wrappers that use composition instead of inheritance.


It is my understanding that PEP 521 (which proposes to add optional
__suspend__ and __resume__ methods to the context manager protocol, to be
called whenever a frame is suspended or resumed inside a `with` block) is
no longer a contender because it would be way too slow. I haven't read it
recently or thought about it, so I don't know what the second issue you
mention is about (though it's presumably about the `yield` in a context
manager implemented using a generator decorated with
`@contextlib.contextmanager`).


​Well, it's not completely unrelated to that. The problem I'm talking about
is perhaps most easily seen from a simple context manager wrapper that uses
composition instead of inheritance:

class Wrapper:
def __init__(self):
self._wrapped = SomeContextManager()

def __enter__(self):
print("Entering context")
return self._wrapped.__enter__()

def __exit__(self):
self._wrapped.__exit__()
print("Exited context")


Now, if the wrapped contextmanager becomes a PEP 521 one with __suspend__
and __resume__, the Wrapper class is broken, because it does not respect
__suspend__ and __resume__. So actually this is a backwards compatiblity
issue.

But if the wrapper is made using inheritance, the problem goes away:


class Wrapper(SomeContextManager):
def __enter__(self):
print("Entering context")
return super().__enter__()

def __exit__(self):
super().__exit__()
print("Exited context")


Now the wrapper cleanly inherits the new optional __suspend__ and
__resume__ from the wrapped context manager type.


––Koos




-- 
+ Koos Zevenhoven + http://twitter.com/k7hoven +
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com