Re: [Python-Dev] [ssl] The weird case of IDNA

2017-12-31 Thread Steven D'Aprano 
On Sun, Dec 31, 2017 at 05:51:47PM -0800, Nathaniel Smith wrote:
> On Sun, Dec 31, 2017 at 5:39 PM, Steven D'Aprano  wrote:
> > On Sun, Dec 31, 2017 at 09:07:01AM -0800, Nathaniel Smith wrote:
> >
> >> This is another reason why we ought to let users do their own IDNA handling
> >> if they want...
> >
> > I expect that letting users do their own IDNA handling will correspond
> > to not doing any IDNA handling at all.
> 
> You did see the words "if they want", right?

Yes. Its the people who don't know that they ought to handle IDNA that 
concern me. They would "want to" if they knew they ought to, but they 
don't because they never even thought of non-ASCII URLs and consequently 
they write libraries or applications open to IDNA security issues.


> I'm not talking about
> removing the stdlib's default IDNA handling, I'm talking about fixing
> the cases where the stdlib goes out of its way to prevent users from
> overriding its IDNA handling.

That wasn't clear to me. I completely agree that the stdlib preventing 
people from overriding the IDNA is a bad thing that ought to be fixed, 
and that users should be able to opt out of it (presumably if they know 
enough to do that, they know enough to avoid IDNA vulnerabilities). I 
thought you meant it ought to be opt-in.

Sorry for misunderstanding you, but your wording suggested to me that 
you meant that the stdlib shouldn't do IDNA handling at all unless the 
user did it themselves (perhaps by calling an IDNA library in the std 
lib). I see now that's not what you meant.


-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [ssl] The weird case of IDNA

2017-12-31 Thread Chris Angelico 
On Mon, Jan 1, 2018 at 12:39 PM, Steven D'Aprano  wrote:
> On Sun, Dec 31, 2017 at 09:07:01AM -0800, Nathaniel Smith wrote:
>
>> This is another reason why we ought to let users do their own IDNA handling
>> if they want...
>
> I expect that letting users do their own IDNA handling will correspond
> to not doing any IDNA handling at all.
>

That'll lead to one of two possibilities:

1) People use Unicode strings to represent domain names. Python's
existing IDNA handling will happen; they're not doing their own. Not
what you're talking about. Or:

2) People use byte strings to represent domain names. Any non-ASCII
characters will simply cause an exception, if I'm not mistaken. Safe,
but not as functional.

ChrisA
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [ssl] The weird case of IDNA

2017-12-31 Thread Nathaniel Smith 
On Sun, Dec 31, 2017 at 5:39 PM, Steven D'Aprano  wrote:
> On Sun, Dec 31, 2017 at 09:07:01AM -0800, Nathaniel Smith wrote:
>
>> This is another reason why we ought to let users do their own IDNA handling
>> if they want...
>
> I expect that letting users do their own IDNA handling will correspond
> to not doing any IDNA handling at all.

You did see the words "if they want", right? I'm not talking about
removing the stdlib's default IDNA handling, I'm talking about fixing
the cases where the stdlib goes out of its way to prevent users from
overriding its IDNA handling.

And "users" here is a very broad category; it includes libraries like
requests, twisted, trio, ... that are already doing better IDNA
handling than the stdlib, except in cases where the stdlib actively
prevents it.

-n

-- 
Nathaniel J. Smith -- https://vorpus.org
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [ssl] The weird case of IDNA

2017-12-31 Thread Steven D'Aprano 
On Sun, Dec 31, 2017 at 09:07:01AM -0800, Nathaniel Smith wrote:

> This is another reason why we ought to let users do their own IDNA handling
> if they want...

I expect that letting users do their own IDNA handling will correspond 
to not doing any IDNA handling at all.



-- 
Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [ssl] The weird case of IDNA

2017-12-31 Thread Nathaniel Smith 
On Dec 31, 2017 7:37 AM, "Stephen J. Turnbull" <
turnbull.stephen...@u.tsukuba.ac.jp> wrote:

Nathaniel Smith writes:

 > Issue 1: Python's built-in IDNA implementation is wrong (implements
 > IDNA 2003, not IDNA 2008).

Is "wrong" the right word here?  I'll grant you that 2008 is *better*,
but typically in practice versions coexist for years.  Ie, is there no
backward compatibility issue with registries that specified IDNA 2003?


Well, yeah, I was simplifying, but at the least we can say that always and
only using IDNA 2003 certainly isn't right :-). I think in most cases the
preferred way to deal with these kinds of issues is not to carry around an
IDNA 2003 implementation, but instead to use an IDNA 2008 implementation
with the "transitional compatibility" flag enabled in the UTS46
preprocessor? But this is rapidly exceeding my knowledge.

This is another reason why we ought to let users do their own IDNA handling
if they want...


This is not entirely an idle question: I'd like to tool up on the
RFCs, research existing practice (especially in the East/Southeast Asian
registries), and contribute to the implementation if there may be an
issue remaining.  (Interpreting RFCs is something I'm reasonably good
at.)


Maybe this is a good place to start:

https://github.com/kjd/idna/blob/master/README.rst

-n

[Sorry if my quoting is messed up; posting from my phone and Gmail for
Android apparently generates broken text/plain.]
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [ssl] The weird case of IDNA

2017-12-31 Thread Stephen J. Turnbull 
Nathaniel Smith writes:

 > Issue 1: Python's built-in IDNA implementation is wrong (implements
 > IDNA 2003, not IDNA 2008).

Is "wrong" the right word here?  I'll grant you that 2008 is *better*,
but typically in practice versions coexist for years.  Ie, is there no
backward compatibility issue with registries that specified IDNA 2003?

This is not entirely an idle question: I'd like to tool up on the
RFCs, research existing practice (especially in the East/Southeast Asian
registries), and contribute to the implementation if there may be an
issue remaining.  (Interpreting RFCs is something I'm reasonably good
at.)

Steve
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com

Re: [Python-Dev] [ssl] The weird case of IDNA

2017-12-31 Thread Antoine Pitrou 
On Sat, 30 Dec 2017 23:27:04 -0800
Nathaniel Smith  wrote:
> 
> We can bikeshed what the new name should be. Maybe set_sni_callback?
> or set_server_hostname_callback, since the corresponding client-mode
> argument is server_hostname?

Or set_idna_servername_callback().

Regards

Antoine.
___
Python-Dev mailing list
Python-Dev@python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: 
https://mail.python.org/mailman/options/python-dev/archive%40mail-archive.com