Re: [Python-Dev] [ssl] The weird case of IDNA

[Guido van Rossum]

This being a security issue I think it's okay to break 3.6. might even
backport to 3.5 if it's easy?

On Dec 29, 2017 1:59 PM, "Christian Heimes"  wrote:

> Hi,
>
> tl;dr
> This mail is about internationalized domain names and TLS/SSL. It
> doesn't concern you if you live in ASCII-land. Me and a couple of other
> developers like to change the ssl module in a backwards-incompatible way
> to fix IDN support for TLS/SSL.
>
>
> Simply speaking the IDNA standards (internationalized domain names for
> applications) describe how to encode non-ASCII domain names. The DNS
> system and X.509 certificates cannot handle non-ASCII host names. Any
> non-ASCII part of a hostname is punyencoded. For example the host name
> 'www.bücher.de ' (books) is translated into '
> www.xn--bcher-kva.de'. In
> IDNA terms, 'www.bücher.de ' is called an
> IDN U-label (unicode) and
> 'www.xn--bcher-kva.de' an IDN A-label (ASCII). Please refer to the TR64
> document [1] for more information.
>
> In a perfect world, it would be very simple. We'd only had one IDNA
> standard. However there are multiple standards that are incompatible
> with each other. The German TLD .de demands IDNA-2008 with UTS#46
> compatibility mapping. The hostname 'www.straße.de '
> maps to
> 'www.xn--strae-oqa.de'. However in the older IDNA 2003 standard,
> 'www.straße.de ' maps to 'www.strasse.de', but '
> strasse.de' is a totally
> different domain!
>
>
> CPython has only support for IDNA 2003.
>
> It's less of an issue for the socket module. It only converts text to
> IDNA bytes on the way in. All functions support bytes and text. Since
> IDNA encoding does change ASCII and IDNA-encoded data is ASCII, it is
> also no problem to pass IDNA2008-encoded text or bytes to all socket
> functions.
>
> Example:
>
> >>> import socket
> >>> import idna  # from PyPI
> >>> names = ['straße.de ', b'strasse.de', idna.encode('
> straße.de '),
> idna.encode('straße.de ').encode('ascii')]
> >>> for name in names:
> ... print(name, socket.getaddrinfo(name, None, socket.AF_INET,
> socket.SOCK_STREAM, 0, socket.AI_CANONNAME)[0][3:5])
> ...
> straße.de  ('strasse.de', ('89.31.143.1', 0))
> b'strasse.de' ('strasse.de', ('89.31.143.1', 0))
> b'xn--strae-oqa.de' ('xn--strae-oqa.de', ('81.169.145.78', 0))
> xn--strae-oqa.de ('xn--strae-oqa.de', ('81.169.145.78', 0))
>
> As you can see, 'straße.de ' is canonicalized as '
> strasse.de'. The IDNA
> 2008 encoded hostname maps to a different IP address.
>
>
> On the other hand ssl module is currently completely broken. It converts
> hostnames from bytes to text with 'idna' codec in some places, but not
> in all. The SSLSocket.server_hostname attribute and callback function
> SSLContext.set_servername_callback() are decoded as U-label.
> Certificate's common name and subject alternative name fields are not
> decoded and therefore A-labels. The *must* stay A-labels because
> hostname verification is only defined in terms of A-labels. We even had
> a security issue once, because partial wildcard like 'xn*.example.org'
> must not match IDN hosts like 'xn--bcher-kva.example.org'.
>
> In issue [2] and PR [3], we all agreed that the only sensible fix is to
> make 'SSLContext.server_hostname' an ASCII text A-label. But this is an
> backwards incompatible fix. On the other hand, IDNA is totally broken
> without the fix. Also in my opinion, PR [3] is not going far enough.
> Since we have to break backwards compatibility anyway, I'd like to
> modify SSLContext.set_servername_callback() at the same time.
>
> Questions:
> - Is everybody OK with breaking backwards compatibility? The risk is
> small. ASCII-only domains are not affected and IDNA users are broken
> anyway.
> - Should I only fix 3.7 or should we consider a backport to 3.6, too?
>
> Regards,
> Christian
>
> [1] https://www.unicode.org/reports/tr46/
> [2] https://bugs.python.org/issue28414
> [3] https://github.com/python/cpython/pull/3010
>
>
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Nick Coghlan]

On 30 December 2017 at 11:48, Ned Batchelder  wrote:
> On 12/29/17 1:59 PM, Guido van Rossum wrote:
>>
>> Regarding whether this should live on PyPI first, in this case that would
>> not be helpful, since attrs is already the category killer on PyPI. So we
>> are IMO taking the best course possible given that we want something in the
>> stdlib but not exactly attrs.
>
>
> It always seemed to me that the reason to recommend putting something on
> PyPI first wasn't so that it would climb up some kind of leaderboard, but so
> that people could get real-world experience with it before freezing it into
> the stdlib.  If we think people won't start using data classes from PyPI,
> why do we think it's important to get into the stdlib?
>
> It still seems to me like there are open questions about how data classes
> should work. Getting people using it will be a good way to get the best
> design before our hands are tied with backward compatibility in the stdlib.
> What is the rush to put a new design into the stdlib?  Presumably it is
> better than attrs (or we would have simply adopted attrs).  Having data
> classes on PyPI will be a good way to gauge acceptance.

attrs has already proved the utility of the approach, and the
differences between the two (such as they are) are mostly cosmetic
(attrs even already has a release out that supports the annotation
based syntax).

The cosmetic differences matter for educational purposes (i.e. "data
classes" with "fields", vs trying to explain that "attributes",
"attrs", "attr.s", and "attr.ib" are all different things), but
"available by default" matters even more on that front.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Ethan Smith]

Okay, I think Guido's proposal is a good compromise.

I already have a branch of dataclasses that should implement that behavior,
so perhaps it was meant to be. :)

~>Ethan Smith

On Fri, Dec 29, 2017 at 5:13 PM, Nick Coghlan  wrote:

>
>
> On 30 Dec. 2017 11:01 am, "Ethan Smith"  wrote:
>
>
>
> On Fri, Dec 29, 2017 at 4:52 PM, Guido van Rossum 
> wrote:
>
>> I still think it should overrides anything that's just inherited but
>> nothing that's defined in the class being decorated.
>>
>>
> Could you explain why you are of this opinion? Is it a concern about
> complexity of implementation?
>
>
> Adding a new method to a base class shouldn't risk breaking existing
> subclasses.
>
> If folks want to retain the base class implementation, they can request
> that explicitly (and doing so isn't redundant at the point of subclass
> definition the way it is for methods defined in the class body).
>
> Cheers,
> Nick.
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Ned Batchelder]

On 12/29/17 1:59 PM, Guido van Rossum wrote:
Regarding whether this should live on PyPI first, in this case that 
would not be helpful, since attrs is already the category killer on 
PyPI. So we are IMO taking the best course possible given that we want 
something in the stdlib but not exactly attrs.


It always seemed to me that the reason to recommend putting something on 
PyPI first wasn't so that it would climb up some kind of leaderboard, 
but so that people could get real-world experience with it before 
freezing it into the stdlib.  If we think people won't start using data 
classes from PyPI, why do we think it's important to get into the stdlib?


It still seems to me like there are open questions about how data 
classes should work. Getting people using it will be a good way to get 
the best design before our hands are tied with backward compatibility in 
the stdlib.  What is the rush to put a new design into the stdlib?  
Presumably it is better than attrs (or we would have simply adopted 
attrs).  Having data classes on PyPI will be a good way to gauge acceptance.


--Ned.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Nick Coghlan]

On 30 Dec. 2017 11:01 am, "Ethan Smith"  wrote:



On Fri, Dec 29, 2017 at 4:52 PM, Guido van Rossum  wrote:

> I still think it should overrides anything that's just inherited but
> nothing that's defined in the class being decorated.
>
>
Could you explain why you are of this opinion? Is it a concern about
complexity of implementation?


Adding a new method to a base class shouldn't risk breaking existing
subclasses.

If folks want to retain the base class implementation, they can request
that explicitly (and doing so isn't redundant at the point of subclass
definition the way it is for methods defined in the class body).

Cheers,
Nick.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Guido van Rossum]

No, I am concerned about the rule being too complex to explain, and about
surprising effects when the base changes (action at a distance).

I also don't necessarily think "we all agree" that what attrs does is
wrong, but the rule I propose seems reasonable.

On Dec 29, 2017 5:58 PM, "Ethan Smith"  wrote:

>
>
> On Fri, Dec 29, 2017 at 4:52 PM, Guido van Rossum 
> wrote:
>
>> I still think it should overrides anything that's just inherited but
>> nothing that's defined in the class being decorated.
>>
>>
> Could you explain why you are of this opinion? Is it a concern about
> complexity of implementation?
>
>
>> On Dec 29, 2017 5:43 PM, "Nathaniel Smith"  wrote:
>>
>>> On Fri, Dec 29, 2017 at 12:30 PM, Ethan Furman 
>>> wrote:
>>> > Good point.  So auto-generate a new __repr__ if:
>>> >
>>> > - one is not provided, and
>>> > - existing __repr__ is either:
>>> >   - object.__repr__, or
>>> >   - a previous dataclass __repr__
>>> >
>>> > And if the auto default doesn't work for one's use-case, use the
>>> keyword
>>> > parameter to specify what you want.
>>>
>>> What does attrs do here?
>>>
>>> -n
>>>
>>> --
>>> Nathaniel J. Smith -- https://vorpus.org
>>> ___
>>> Python-Dev mailing list
>>> Python-Dev@python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe: https://mail.python.org/mailma
>>> n/options/python-dev/guido%40python.org
>>>
>>
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/ethan%
>> 40ethanhs.me
>>
>>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Ethan Furman]

On Fri, Dec 29, 2017 at 12:30 PM, Ethan Furman wrote:
> Good point.  So auto-generate a new __repr__ if:
>
> - one is not provided, and
> - existing __repr__ is either:
>   - object.__repr__, or
>   - a previous dataclass __repr__
>
> And if the auto default doesn't work for one's use-case, use the keyword
> parameter to specify what you want.

On Dec 29, 2017 5:43 PM, "Nathaniel Smith" wrote:
> What does attrs do here?

On 12/29/2017 04:52 PM, Ethan Smith wrote:
> attrs just silently overwrites any user provided __repr__ unless you provide
> repr=False to attr.s.

On 12/29/2017 04:52 PM, Guido van Rossum wrote:
> I still think it should overrides anything that's just inherited but nothing
> that's defined in the class being decorated.

I can certainly live with that.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Ethan Smith]

On Fri, Dec 29, 2017 at 4:52 PM, Guido van Rossum  wrote:

> I still think it should overrides anything that's just inherited but
> nothing that's defined in the class being decorated.
>
>
Could you explain why you are of this opinion? Is it a concern about
complexity of implementation?


> On Dec 29, 2017 5:43 PM, "Nathaniel Smith"  wrote:
>
>> On Fri, Dec 29, 2017 at 12:30 PM, Ethan Furman 
>> wrote:
>> > Good point.  So auto-generate a new __repr__ if:
>> >
>> > - one is not provided, and
>> > - existing __repr__ is either:
>> >   - object.__repr__, or
>> >   - a previous dataclass __repr__
>> >
>> > And if the auto default doesn't work for one's use-case, use the keyword
>> > parameter to specify what you want.
>>
>> What does attrs do here?
>>
>> -n
>>
>> --
>> Nathaniel J. Smith -- https://vorpus.org
>> ___
>> Python-Dev mailing list
>> Python-Dev@python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%
>> 40python.org
>>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> ethan%40ethanhs.me
>
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Ethan Smith]

attrs just silently overwrites any user provided __repr__ unless you
provide repr=False to attr.s.

I think we can all agree that if nothing else, silently overwriting
unconditionally is not what we want for dataclasses.

On Fri, Dec 29, 2017 at 4:38 PM, Nathaniel Smith  wrote:

> On Fri, Dec 29, 2017 at 12:30 PM, Ethan Furman  wrote:
> > Good point.  So auto-generate a new __repr__ if:
> >
> > - one is not provided, and
> > - existing __repr__ is either:
> >   - object.__repr__, or
> >   - a previous dataclass __repr__
> >
> > And if the auto default doesn't work for one's use-case, use the keyword
> > parameter to specify what you want.
>
> What does attrs do here?
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> ethan%40ethanhs.me
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Guido van Rossum]

I still think it should overrides anything that's just inherited but
nothing that's defined in the class being decorated.

On Dec 29, 2017 5:43 PM, "Nathaniel Smith"  wrote:

> On Fri, Dec 29, 2017 at 12:30 PM, Ethan Furman  wrote:
> > Good point.  So auto-generate a new __repr__ if:
> >
> > - one is not provided, and
> > - existing __repr__ is either:
> >   - object.__repr__, or
> >   - a previous dataclass __repr__
> >
> > And if the auto default doesn't work for one's use-case, use the keyword
> > parameter to specify what you want.
>
> What does attrs do here?
>
> -n
>
> --
> Nathaniel J. Smith -- https://vorpus.org
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Nathaniel Smith]

On Fri, Dec 29, 2017 at 12:30 PM, Ethan Furman  wrote:
> Good point.  So auto-generate a new __repr__ if:
>
> - one is not provided, and
> - existing __repr__ is either:
>   - object.__repr__, or
>   - a previous dataclass __repr__
>
> And if the auto default doesn't work for one's use-case, use the keyword
> parameter to specify what you want.

What does attrs do here?

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Ethan Smith]

On Fri, Dec 29, 2017 at 12:30 PM, Ethan Furman  wrote:

> On 12/29/2017 11:55 AM, Ethan Smith wrote:
>
>> On Fri, Dec 29, 2017 at 11:37 AM, Ethan Furman wrote:
>>
>
> It is possible to determine whether an existing __repr__ is from 'object'
>>>
>> >> or not, and only provide one if that is the case.  I think that should
> be
> >> the default, with 'repr = True' for those cases where a new,
> auto-generated,
>
>> __repr__ is desired.
>>>
>>
>> The only other thing you'd want to handle is to cover inheriting from
>> another dataclass. e.g., if I have dataclass with
>> attribute a, and subclass it to add attribute b, I'd want both in the
>> repr.
>>
>
> Good point.  So auto-generate a new __repr__ if:
>
> - one is not provided, and
> - existing __repr__ is either:
>   - object.__repr__, or
>   - a previous dataclass __repr__
>
> And if the auto default doesn't work for one's use-case, use the keyword
> parameter to specify what you want.
>
>
> --
> ~Ethan~
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/ethan%
> 40ethanhs.me
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] [ssl] The weird case of IDNA

[Christian Heimes]

Hi,

tl;dr
This mail is about internationalized domain names and TLS/SSL. It
doesn't concern you if you live in ASCII-land. Me and a couple of other
developers like to change the ssl module in a backwards-incompatible way
to fix IDN support for TLS/SSL.


Simply speaking the IDNA standards (internationalized domain names for
applications) describe how to encode non-ASCII domain names. The DNS
system and X.509 certificates cannot handle non-ASCII host names. Any
non-ASCII part of a hostname is punyencoded. For example the host name
'www.bücher.de' (books) is translated into 'www.xn--bcher-kva.de'. In
IDNA terms, 'www.bücher.de' is called an IDN U-label (unicode) and
'www.xn--bcher-kva.de' an IDN A-label (ASCII). Please refer to the TR64
document [1] for more information.

In a perfect world, it would be very simple. We'd only had one IDNA
standard. However there are multiple standards that are incompatible
with each other. The German TLD .de demands IDNA-2008 with UTS#46
compatibility mapping. The hostname 'www.straße.de' maps to
'www.xn--strae-oqa.de'. However in the older IDNA 2003 standard,
'www.straße.de' maps to 'www.strasse.de', but 'strasse.de' is a totally
different domain!


CPython has only support for IDNA 2003.

It's less of an issue for the socket module. It only converts text to
IDNA bytes on the way in. All functions support bytes and text. Since
IDNA encoding does change ASCII and IDNA-encoded data is ASCII, it is
also no problem to pass IDNA2008-encoded text or bytes to all socket
functions.

Example:

>>> import socket
>>> import idna  # from PyPI
>>> names = ['straße.de', b'strasse.de', idna.encode('straße.de'),
idna.encode('straße.de').encode('ascii')]
>>> for name in names:
... print(name, socket.getaddrinfo(name, None, socket.AF_INET,
socket.SOCK_STREAM, 0, socket.AI_CANONNAME)[0][3:5])
...
straße.de ('strasse.de', ('89.31.143.1', 0))
b'strasse.de' ('strasse.de', ('89.31.143.1', 0))
b'xn--strae-oqa.de' ('xn--strae-oqa.de', ('81.169.145.78', 0))
xn--strae-oqa.de ('xn--strae-oqa.de', ('81.169.145.78', 0))

As you can see, 'straße.de' is canonicalized as 'strasse.de'. The IDNA
2008 encoded hostname maps to a different IP address.


On the other hand ssl module is currently completely broken. It converts
hostnames from bytes to text with 'idna' codec in some places, but not
in all. The SSLSocket.server_hostname attribute and callback function
SSLContext.set_servername_callback() are decoded as U-label.
Certificate's common name and subject alternative name fields are not
decoded and therefore A-labels. The *must* stay A-labels because
hostname verification is only defined in terms of A-labels. We even had
a security issue once, because partial wildcard like 'xn*.example.org'
must not match IDN hosts like 'xn--bcher-kva.example.org'.

In issue [2] and PR [3], we all agreed that the only sensible fix is to
make 'SSLContext.server_hostname' an ASCII text A-label. But this is an
backwards incompatible fix. On the other hand, IDNA is totally broken
without the fix. Also in my opinion, PR [3] is not going far enough.
Since we have to break backwards compatibility anyway, I'd like to
modify SSLContext.set_servername_callback() at the same time.

Questions:
- Is everybody OK with breaking backwards compatibility? The risk is
small. ASCII-only domains are not affected and IDNA users are broken anyway.
- Should I only fix 3.7 or should we consider a backport to 3.6, too?

Regards,
Christian

[1] https://www.unicode.org/reports/tr46/
[2] https://bugs.python.org/issue28414
[3] https://github.com/python/cpython/pull/3010




___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Ethan Furman]

On 12/29/2017 11:55 AM, Ethan Smith wrote:

On Fri, Dec 29, 2017 at 11:37 AM, Ethan Furman wrote:



It is possible to determine whether an existing __repr__ is from 'object'

>> or not, and only provide one if that is the case.  I think that should be
>> the default, with 'repr = True' for those cases where a new, auto-generated,

__repr__ is desired.


The only other thing you'd want to handle is to cover inheriting from another 
dataclass. e.g., if I have dataclass with
attribute a, and subclass it to add attribute b, I'd want both in the repr.


Good point.  So auto-generate a new __repr__ if:

- one is not provided, and
- existing __repr__ is either:
  - object.__repr__, or
  - a previous dataclass __repr__

And if the auto default doesn't work for one's use-case, use the keyword 
parameter to specify what you want.

--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Ethan Smith]

On Fri, Dec 29, 2017 at 11:37 AM, Ethan Furman  wrote:

> On 12/29/2017 02:23 AM, Ethan Smith wrote:
>
> The first is that needing both a keyword and method is duplicative and
>> unnecessary. Eric agreed it was a hassle, but
>> felt it was justified considering someone may accidentally override a
>> dataclass method. I disagree with this point of
>> view as dataclasses are billed as providing automatic methods. Overriding
>> via method definition is very natural and
>> idiomatic. I don't really see how someone could accidentally override a
>> dataclass method if methods were not generated
>> by the dataclass decorator that are already defined in the class at
>> definition time.
>>
>
> Accidental or not, the decorator should not be replacing methods defined
> by the class.
>
> The second concern, which I came across more recently, is if I have a base
>> class, and dataclasses inherit from this base
>> class, inherited __repr__ & co are silently overridden by dataclass. This
>> is both unexpected, and also means I need to
>> pass a repr=False to each subclass' decorator to get correct behavior,
>> which somewhat defeats the utility of
>> subclassing. Im not as sure a whole lot can be done about this though.
>>
>
> It is possible to determine whether an existing __repr__ is from 'object'
> or not, and only provide one if that is the case.  I think that should be
> the default, with 'repr = True' for those cases where a new,
> auto-generated, __repr__ is desired.
>

The only other thing you'd want to handle is to cover inheriting from
another dataclass. e.g., if I have dataclass with attribute a, and subclass
it to add attribute b, I'd want both in the repr.

~>Ethan Smith

>
> --
> ~Ethan~
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/ethan%
> 40ethanhs.me
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Ethan Furman]

On 12/29/2017 02:23 AM, Ethan Smith wrote:


The first is that needing both a keyword and method is duplicative and 
unnecessary. Eric agreed it was a hassle, but
felt it was justified considering someone may accidentally override a dataclass 
method. I disagree with this point of
view as dataclasses are billed as providing automatic methods. Overriding via 
method definition is very natural and
idiomatic. I don't really see how someone could accidentally override a 
dataclass method if methods were not generated
by the dataclass decorator that are already defined in the class at definition 
time.


Accidental or not, the decorator should not be replacing methods defined by the 
class.


The second concern, which I came across more recently, is if I have a base 
class, and dataclasses inherit from this base
class, inherited __repr__ & co are silently overridden by dataclass. This is 
both unexpected, and also means I need to
pass a repr=False to each subclass' decorator to get correct behavior, which 
somewhat defeats the utility of
subclassing. Im not as sure a whole lot can be done about this though.


It is possible to determine whether an existing __repr__ is from 'object' or not, and only provide one if that is the 
case.  I think that should be the default, with 'repr = True' for those cases where a new, auto-generated, __repr__ is 
desired.


--
~Ethan~
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Antoine Pitrou]

On Fri, 29 Dec 2017 11:12:11 -0800
Ethan Smith  wrote:
> 
> > Agreed as well.  If I make the effort of having a dataclass inherit
> > from a base class, I probably don't want the base class' methods to be
> > silently overriden by machine-generated methods.  Of course, that can
> > be worked around by using multiple inheritance, you just need to be
> > careful and add a small amount of class definition boilerplate.
> 
> I am not sure exactly what you mean by "worked around by using multiple
> inheritance".

I mean you can write:

class _BaseClass:
def __repr__(self):
# ...

@dataclass
class _DataclassMixin:
# your attribute definitions here

class FinalClass(_BaseClass, _BaseDataclass):
pass


Yes, it's tedious and verbose :-)

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Guido van Rossum]

But you always inherit __repr__, from object. The base class might also
itself be a dataclass. I think it should only skip when the decorated class
itself defines __repr__.

On Dec 29, 2017 3:47 AM, "Antoine Pitrou"  wrote:

> On Fri, 29 Dec 2017 02:23:56 -0800
> Ethan Smith  wrote:
> >
> > In a few cases I want to override the repr of the AST nodes. I wrote a
> > __repr__ and ran the code but lo and behold I got a type error. I
> couldn't
> > override it. I quickly learned that one needs to pass a keyword to the
> > dataclass decorator to tell it *not* to auto generate methods you
> override.
> >
> > I have two usability concerns with the current implementation. I emailed
> > Eric about the first, and he said I should ask for thoughts here. The
> > second I found after a couple of days sitting on this message.
> >
> > The first is that needing both a keyword and method is duplicative and
> > unnecessary. Eric agreed it was a hassle, but felt it was justified
> > considering someone may accidentally override a dataclass method. I
> > disagree with this point of view as dataclasses are billed as providing
> > automatic methods. Overriding via method definition is very natural and
> > idiomatic.
>
> Agreed.  We shouldn't take magic too far just for the sake of
> protecting users against their own (alleged) mistakes.  And I'm not
> sure how you "accidentally" override a dataclass method (if I'm
> implementing a __repr__ I'm doing so deliberately :-)).
>
> > The second concern, which I came across more recently, is if I have a
> base
> > class, and dataclasses inherit from this base class, inherited __repr__ &
> > co are silently overridden by dataclass. This is both unexpected, and
> also
> > means I need to pass a repr=False to each subclass' decorator to get
> > correct behavior, which somewhat defeats the utility of subclassing. Im
> not
> > as sure a whole lot can be done about this though.
>
> Agreed as well.  If I make the effort of having a dataclass inherit
> from a base class, I probably don't want the base class' methods to be
> silently overriden by machine-generated methods.  Of course, that can
> be worked around by using multiple inheritance, you just need to be
> careful and add a small amount of class definition boilerplate.
>
> I would expect dataclass parameters such as `repr` to be tri-state:
>
> * repr=None (the default): only provide a machine-generated
>   implementation if none is already defined (either on a base class or
>   in the dataclass namespace... ignoring runtime-provided defaults such
>   as object.__repr__)
> * repr=False: never provide a machine-generated implementation
> * repr=True: always provide a machine-generated implementation, even
>   overriding a previous user-defined implementation
>
> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Ethan Smith]

On Fri, Dec 29, 2017 at 2:45 AM, Antoine Pitrou  wrote:

> On Fri, 29 Dec 2017 02:23:56 -0800
> Ethan Smith  wrote:
> >
> > In a few cases I want to override the repr of the AST nodes. I wrote a
> > __repr__ and ran the code but lo and behold I got a type error. I
> couldn't
> > override it. I quickly learned that one needs to pass a keyword to the
> > dataclass decorator to tell it *not* to auto generate methods you
> override.
> >
> > I have two usability concerns with the current implementation. I emailed
> > Eric about the first, and he said I should ask for thoughts here. The
> > second I found after a couple of days sitting on this message.
> >
> > The first is that needing both a keyword and method is duplicative and
> > unnecessary. Eric agreed it was a hassle, but felt it was justified
> > considering someone may accidentally override a dataclass method. I
> > disagree with this point of view as dataclasses are billed as providing
> > automatic methods. Overriding via method definition is very natural and
> > idiomatic.
>
> Agreed.  We shouldn't take magic too far just for the sake of
> protecting users against their own (alleged) mistakes.  And I'm not
> sure how you "accidentally" override a dataclass method (if I'm
> implementing a __repr__ I'm doing so deliberately :-)).
>

My thinking exactly.

>
> > The second concern, which I came across more recently, is if I have a
> base
> > class, and dataclasses inherit from this base class, inherited __repr__ &
> > co are silently overridden by dataclass. This is both unexpected, and
> also
> > means I need to pass a repr=False to each subclass' decorator to get
> > correct behavior, which somewhat defeats the utility of subclassing. Im
> not
> > as sure a whole lot can be done about this though.
>
> Agreed as well.  If I make the effort of having a dataclass inherit
> from a base class, I probably don't want the base class' methods to be
> silently overriden by machine-generated methods.  Of course, that can
> be worked around by using multiple inheritance, you just need to be
> careful and add a small amount of class definition boilerplate.
>

I am not sure exactly what you mean by "worked around by using multiple
inheritance". Do you mean you think
the dataclass decorator should be made into a dataclass base class for a
dataclass class, or that it should look
at the MRO?


>
> I would expect dataclass parameters such as `repr` to be tri-state:
>
> * repr=None (the default): only provide a machine-generated
>   implementation if none is already defined (either on a base class or
>   in the dataclass namespace... ignoring runtime-provided defaults such
>   as object.__repr__)
> * repr=False: never provide a machine-generated implementation
> * repr=True: always provide a machine-generated implementation, even
>   overriding a previous user-defined implementation
>

This is sensible to me.


Thanks for your thoughts!

~>Ethan Smith


> Regards
>
> Antoine.
>
>
> ___
> Python-Dev mailing list
> Python-Dev@python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> ethan%40ethanhs.me
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Guido van Rossum]

On Fri, Dec 29, 2017 at 7:10 AM, Stephen J. Turnbull <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

> Recently a question has been raised about the decorator overriding
> methods defined in the class (especially __repr__).  People feel that
> if the class defines a method, the decorator should not override it.
> The current API requires passing "repr=false" to the decorator.
>

I think this is a reasonable question, though I'm not sure how "big" it is.
Note that if the *base* class defines __repr__ the decorator should still
override it (unless repr=False), since there's always object.__repr__ (and
same for most other dunders). We should also (like we did with most
questions big and small) look at what attrs does and why.

Regarding whether this should live on PyPI first, in this case that would
not be helpful, since attrs is already the category killer on PyPI. So we
are IMO taking the best course possible given that we want something in the
stdlib but not exactly attrs.

-- 
--Guido van Rossum (python.org/~guido)
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supporting functools.singledispatch with classes.

[Ethan Smith]

On Fri, Dec 29, 2017 at 7:02 AM, Nick Coghlan  wrote:

> On 28 December 2017 at 04:22, Ethan Smith  wrote:
> > Okay, if there is no further feedback, I will work on a
> singledispatchmethod
> > decorator like partialmethod.
> >
> > For the future perhaps, would it not be possible to tell that the passed
> > argument is a descriptor/function and dispatch to the correct
> > implementation, thus not needing two functions for essentially the same
> > thing?
> >
> > It seems more straightforward to make the implementation a bit more
> complex
> > to provide a single, simple API to users.
>
> "Add 'method' to the decorator name when decorating a method" is a
> pretty simple rule to remember - it's much easier than "Add
> 'arg_index=1'" (which is a comparatively arbitrary adjustment that
> requires a fairly in depth understanding of both the descriptor
> protocol and type-based function dispatch to follow).
>
> And you need the change to be explicitly opt-in *somehow*, in order to
> avoid breaking any existing code that relies on methods decorated with
> "singledispatch" dispatching on the bound class.
>

Good points. I will start working on the singledispatchmethod
implementation.

~>Ethan Smith

>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
>
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Summary of Python tracker Issues

[Python tracker]

ACTIVITY SUMMARY (2017-12-22 - 2017-12-29)
Python tracker at https://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open6355 (+13)
  closed 37843 (+24)
  total  44198 (+37)

Open issues with patches: 2472 


Issues opened (31)
==

#30722: Tools/demo/redemo.py broken
https://bugs.python.org/issue30722  reopened by serhiy.storchaka

#32411: Idlelib.browser: stop sorting dicts created by pyclbr
https://bugs.python.org/issue32411  opened by terry.reedy

#32412: help() of bitwise operators should mention sets as well
https://bugs.python.org/issue32412  opened by steven.daprano

#32413: Document that locals() may return globals()
https://bugs.python.org/issue32413  opened by steven.daprano

#32414: PyCapsule_Import fails when name is in the form 'package.modul
https://bugs.python.org/issue32414  opened by lekma

#32417: fromutc does not respect datetime subclasses
https://bugs.python.org/issue32417  opened by p-ganssle

#32418: Implement Server.get_loop() method
https://bugs.python.org/issue32418  opened by asvetlov

#32419: Add unittest support for pyc projects
https://bugs.python.org/issue32419  opened by brgirgis

#32420: LookupError : unknown encoding : [0x7FF092395AD0] ANOMALY
https://bugs.python.org/issue32420  opened by Kitamura

#32421: Keeping an exception in cache can segfault the interpreter
https://bugs.python.org/issue32421  opened by zunger

#32423: The Windows SDK version 10.0.15063.0 was not found
https://bugs.python.org/issue32423  opened by isuruf

#32424: Synchronize copy methods between Python and C implementations 
https://bugs.python.org/issue32424  opened by gphemsley

#32425: Allow non-default XML parsers to take advantage of a _parse_wh
https://bugs.python.org/issue32425  opened by gphemsley

#32426: Tkinter.ttk Widget does not define wich option exists to set t
https://bugs.python.org/issue32426  opened by alex.75

#32427: Rename and expose dataclasses._MISSING
https://bugs.python.org/issue32427  opened by eric.smith

#32428: dataclasses: make it an error to have initialized non-fields i
https://bugs.python.org/issue32428  opened by eric.smith

#32429: Outdated Modules/Setup warning is invisible
https://bugs.python.org/issue32429  opened by mdk

#32430: Simplify Modules/Setup{,.dist,.local}
https://bugs.python.org/issue32430  opened by mdk

#32431: Two bytes objects of zero length don't compare equal
https://bugs.python.org/issue32431  opened by jonathanunderwood

#32433: Provide optimized HMAC digest
https://bugs.python.org/issue32433  opened by christian.heimes

#32434: pathlib.WindowsPath.reslove(strict=False) returns absoulte pat
https://bugs.python.org/issue32434  opened by mliska

#32435: tarfile recognizes .gz file as tar
https://bugs.python.org/issue32435  opened by spetrunin

#32436: Implement PEP 567
https://bugs.python.org/issue32436  opened by yselivanov

#32438: PyLong_ API cleanup
https://bugs.python.org/issue32438  opened by erik.bray

#32439: Clean up the code for compiling comparison expressions
https://bugs.python.org/issue32439  opened by serhiy.storchaka

#32441: os.dup2 should return the new fd
https://bugs.python.org/issue32441  opened by benjamin.peterson

#32443: Add Linux's signalfd() to the signal module
https://bugs.python.org/issue32443  opened by gregory.p.smith

#32444: python -m venv symlink dependency on how python binary is call
https://bugs.python.org/issue32444  opened by seliger

#32445: Skip creating redundant wrapper functions in ExitStack.callbac
https://bugs.python.org/issue32445  opened by ncoghlan

#32446: ResourceLoader.get_data() should accept a PathLike
https://bugs.python.org/issue32446  opened by barry

#32447: IDLE shell won't open on Mac OS 10.13.1
https://bugs.python.org/issue32447  opened by sm1979



Most recent 15 issues with no replies (15)
==

#32447: IDLE shell won't open on Mac OS 10.13.1
https://bugs.python.org/issue32447

#32446: ResourceLoader.get_data() should accept a PathLike
https://bugs.python.org/issue32446

#32445: Skip creating redundant wrapper functions in ExitStack.callbac
https://bugs.python.org/issue32445

#32441: os.dup2 should return the new fd
https://bugs.python.org/issue32441

#32439: Clean up the code for compiling comparison expressions
https://bugs.python.org/issue32439

#32436: Implement PEP 567
https://bugs.python.org/issue32436

#32433: Provide optimized HMAC digest
https://bugs.python.org/issue32433

#32427: Rename and expose dataclasses._MISSING
https://bugs.python.org/issue32427

#32426: Tkinter.ttk Widget does not define wich option exists to set t
https://bugs.python.org/issue32426

#32423: The Windows SDK version 10.0.15063.0 was not found
https://bugs.python.org/issue32423

#32418: Implement Server.get_loop() method
https://bugs.python.org/issue32418

#32410: Implement loop.sock_sendfile method
https://bugs.python.org/issue32410

Re: [Python-Dev] Is static typing still optional?

[Stephen J. Turnbull]

Brett Cannon writes:

 > I wouldn't characterize it as "big questions". For some people there's a
 > question as to how to make them work without type hints, but otherwise how
 > they function is settled.

Recently a question has been raised about the decorator overriding
methods defined in the class (especially __repr__).

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Supporting functools.singledispatch with classes.

[Nick Coghlan]

On 28 December 2017 at 04:22, Ethan Smith  wrote:
> Okay, if there is no further feedback, I will work on a singledispatchmethod
> decorator like partialmethod.
>
> For the future perhaps, would it not be possible to tell that the passed
> argument is a descriptor/function and dispatch to the correct
> implementation, thus not needing two functions for essentially the same
> thing?
>
> It seems more straightforward to make the implementation a bit more complex
> to provide a single, simple API to users.

"Add 'method' to the decorator name when decorating a method" is a
pretty simple rule to remember - it's much easier than "Add
'arg_index=1'" (which is a comparatively arbitrary adjustment that
requires a fairly in depth understanding of both the descriptor
protocol and type-based function dispatch to follow).

And you need the change to be explicitly opt-in *somehow*, in order to
avoid breaking any existing code that relies on methods decorated with
"singledispatch" dispatching on the bound class.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncogh...@gmail.com   |   Brisbane, Australia
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Is static typing still optional?

[Stephen J. Turnbull]

Brett Cannon writes:

 > I wouldn't characterize it as "big questions". For some people there's a
 > question as to how to make them work without type hints, but otherwise how
 > they function is settled.

Recently a question has been raised about the decorator overriding
methods defined in the class (especially __repr__).  People feel that
if the class defines a method, the decorator should not override it.
The current API requires passing "repr=false" to the decorator.





___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Heap allocate type structs in native extension modules?

[Hugh Fisher]

> Date: Fri, 29 Dec 2017 11:25:14 +0200
> From: Serhiy Storchaka 
> To: python-dev@python.org
> Subject: Re: [Python-Dev] Heap allocate type structs in native
> extension modules?
[ munch ]
> Yes, you can create heap types by using PyType_FromSpecWithBases().
>
> But be aware of caveats (https://bugs.python.org/issue26979).

Thanks! I'll give it a go. I've already run into and solved the tp_new
issue, so that won't be a problem.

-- 

cheers,
Hugh Fisher
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Antoine Pitrou]

On Fri, 29 Dec 2017 02:23:56 -0800
Ethan Smith  wrote:
> 
> In a few cases I want to override the repr of the AST nodes. I wrote a
> __repr__ and ran the code but lo and behold I got a type error. I couldn't
> override it. I quickly learned that one needs to pass a keyword to the
> dataclass decorator to tell it *not* to auto generate methods you override.
> 
> I have two usability concerns with the current implementation. I emailed
> Eric about the first, and he said I should ask for thoughts here. The
> second I found after a couple of days sitting on this message.
> 
> The first is that needing both a keyword and method is duplicative and
> unnecessary. Eric agreed it was a hassle, but felt it was justified
> considering someone may accidentally override a dataclass method. I
> disagree with this point of view as dataclasses are billed as providing
> automatic methods. Overriding via method definition is very natural and
> idiomatic.

Agreed.  We shouldn't take magic too far just for the sake of
protecting users against their own (alleged) mistakes.  And I'm not
sure how you "accidentally" override a dataclass method (if I'm
implementing a __repr__ I'm doing so deliberately :-)).

> The second concern, which I came across more recently, is if I have a base
> class, and dataclasses inherit from this base class, inherited __repr__ &
> co are silently overridden by dataclass. This is both unexpected, and also
> means I need to pass a repr=False to each subclass' decorator to get
> correct behavior, which somewhat defeats the utility of subclassing. Im not
> as sure a whole lot can be done about this though.

Agreed as well.  If I make the effort of having a dataclass inherit
from a base class, I probably don't want the base class' methods to be
silently overriden by machine-generated methods.  Of course, that can
be worked around by using multiple inheritance, you just need to be
careful and add a small amount of class definition boilerplate.

I would expect dataclass parameters such as `repr` to be tri-state:

* repr=None (the default): only provide a machine-generated
  implementation if none is already defined (either on a base class or
  in the dataclass namespace... ignoring runtime-provided defaults such
  as object.__repr__)
* repr=False: never provide a machine-generated implementation
* repr=True: always provide a machine-generated implementation, even
  overriding a previous user-defined implementation

Regards

Antoine.


___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Where to report performance of re? [was: (no subject)]

[Stephen J. Turnbull]

Franklin? Lee writes:
 > On Tue, Dec 26, 2017 at 2:01 AM, Yogev Hendel  wrote:
 > >
 > > I don't know if this is the right place to put this,
 > > but I've found the following lines of code results in an incredibly long
 > > processing time.

 > (I think the correct place is python-list. python-dev is primarily for
 > the developers of Python itself.

I read this as a report of a performance problem, perhaps a
regression.  IMO YMMV, Python-list would be appropriate only if the
reporter was already aware that some regular expressions are expected
to be disastrously non-performant on some target strings, and wanted
help dealing with that.  That doesn't seem to be the case (and your
response also presumes ignorance of the performance properties of re).

 > Bug reports and feature requests belong in
 > https://bugs.python.org/ (where your post could also have gone).)

If you really meant that, you should have avoided rewarding him with a
wordy explanation. ;-)  N.B. I've bookmarked yours, it's excellent!
Thank you very much!

I think even in this case an experienced community member probably
should report on the tracker, though.  If all they got was, "WONTFIX:
that's how the algorithm works" there, *then* they go to python-list.

 > (assuming P != NP): Perl allows (introduced?) backreferences, which

I've been using them in Emacsen since 1987 or so.

 > are NP-hard[1]. Perl also added some other features which complicate
 > things, but backreferences are enough.
 > 
 > The user-level solution is to understand how regexes are executed, and
 > to work around it.

This is the Pythonic approach<0.5 wink/>, in the sense that we haven't
gone to the trouble of trying to improve the algorithm yet.

 > 2. In theory, we can use the textbook algorithm when possible, and the
 > backtracking algorithm when necessary. However, the textbook version
 > won't necessarily be faster, and may take more time to create, so
 > there's a tradeoff here.

There may be a question of the difficulty of maintenance, as well.
The linear time algorithms are somewhat more delicate, and we're still
occasionally discussing what semantics a regular expression should
have (null matches), let alone whether they're implemented correctly.

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

[Python-Dev] Concerns about method overriding and subclassing with dataclasses

[Ethan Smith]

Hello all,

I've recently been experimenting with dataclasses. They totally rock! A lot
of the boilerplate for the AST I've designed in Python is automatically
taken care of, it's really great! However, I have a few concerns about the
implementation.

In a few cases I want to override the repr of the AST nodes. I wrote a
__repr__ and ran the code but lo and behold I got a type error. I couldn't
override it. I quickly learned that one needs to pass a keyword to the
dataclass decorator to tell it *not* to auto generate methods you override.

I have two usability concerns with the current implementation. I emailed
Eric about the first, and he said I should ask for thoughts here. The
second I found after a couple of days sitting on this message.

The first is that needing both a keyword and method is duplicative and
unnecessary. Eric agreed it was a hassle, but felt it was justified
considering someone may accidentally override a dataclass method. I
disagree with this point of view as dataclasses are billed as providing
automatic methods. Overriding via method definition is very natural and
idiomatic. I don't really see how someone could accidentally override a
dataclass method if methods were not generated by the dataclass decorator
that are already defined in the class at definition time.

The second concern, which I came across more recently, is if I have a base
class, and dataclasses inherit from this base class, inherited __repr__ &
co are silently overridden by dataclass. This is both unexpected, and also
means I need to pass a repr=False to each subclass' decorator to get
correct behavior, which somewhat defeats the utility of subclassing. Im not
as sure a whole lot can be done about this though.


I appreciate any thoughts folks have related to this.


Cheers,
~>Ethan Smith
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] Heap allocate type structs in native extension modules?

[Serhiy Storchaka]

26.12.17 12:16, Hugh Fisher пише:

I have a Python program which generates the boilerplate code for
native extension modules from a Python source definition.
(http://bitbucket.org/hugh_fisher/fullofeels if interested.)

The examples in the Python doco and the "Python Essential Reference"
book all use a statically declared PyTypeObject struct and
PyType_Ready in the module init func, so I'm doing the same. Then
Python 3.5 added a check for statically allocated types inheriting
from heap types, which broke a couple of my classes. And now I'm
trying to add a __dict__ to native classes so end users can add their
own attributes, and this is turning out to be painful with static
PyTypeObject structs

Would it be better to use dynamically allocated type structs in native modules?


Yes, you can create heap types by using PyType_FromSpecWithBases().

But be aware of caveats (https://bugs.python.org/issue26979).

___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com