[Python-Dev] Re: When to remove BytesWarning?
11.11.20 15:05, John Hagen пише: > If I recall, it was str(bytes) warning that flagged in a few places and was > missing a .decode() call or similar. > > It seems like the bytes== warnings could be implemented in a type checker > such as mypy, if it doesn't > already do this. Assuming you have correct type coverage/inference on your > project, you could > potentially catch this at static analysis time rather than runtime. There were several bugs like sep=='/' (where sep can be bytes) in the stdlib. These cases were not covered by tests, so they were fixed only in 3.3 or even later. I hope all such bugs are already fixed, but I cannot guarantee. And there were bugs with str(bytes) too. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/OQNQIEKS3UTPIHROL66OTQIRG3NX67D6/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
If I recall, it was str(bytes) warning that flagged in a few places and was missing a .decode() call or similar. It seems like the bytes== warnings could be implemented in a type checker such as mypy, if it doesn't already do this. Assuming you have correct type coverage/inference on your project, you could potentially catch this at static analysis time rather than runtime. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/SN2HQG6KQGGQB2SOF2NKCIWQVF6UWKKU/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
On Wed, Nov 11, 2020 at 10:19 AM John Hagen wrote: > > I admit I am in a very small minority however. That being said, I have > discovered > a few minor bugs in my code or in third party libraries over the years using > -bb. > But I would understand still wanting to remove this feature to lower > maintenance burden. Which warning helped you? str(bytes)? bytes == str? or bytes == int? I am not much concerned about removing str(bytes) warning anytime soon. Only bytes== warning is significant maintenance burden. Regards, -- Inada Naoki ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/KROXMCDQGYAJDFH7ZSMZG2SW22TTWJ4Y/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
> I'm quite sure that almost nobody uses -b or -bb when running their test suite or to develop I noticed this thread and just thought I'd give one example of someone who does use -bb: https://github.com/johnthagen/python-blueprint/blob/210b89fe011d172104e9f1ba2553e4245f76a77b/tox.ini#L44 I admit I am in a very small minority however. That being said, I have discovered a few minor bugs in my code or in third party libraries over the years using -bb. But I would understand still wanting to remove this feature to lower maintenance burden. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QZYDKPEDN6FKJ4ZRPOUS6D6HACCGMW7O/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
On Tue, Oct 27, 2020 at 5:23 AM Victor Stinner wrote:
>
> os.get_exec_path() must modify temporarily warnings filters to ignore
> BytesWarning when it looks for 'PATH' (unicode) or b'PATH' (bytes) in
> the 'env' dictionary which may contain unicode or bytes strings.
> Modifying warnings filters impact all threads which is bad.
>
> I dislike having to workaround this annoying behavior for dict lookup
> when -b or -bb is used.
>
Completely agree with you.
> I'm quite sure that almost nobody uses -b or -bb when running their
> test suite or to develop. I expect that nobody uses it. According to
> replies, it seems like porting Python 2 code to Python 3 is the only
> use case. Python 3.9 and older can be used for that, no?
>
I think so. But I became a bit conservative when writing this proposal.
> > When can we remove it? My idea is:
> >
> > 3.10: Deprecate the -b option.
>
> Do you mean writing a message into stderr? Or just deprecate it in the
> documentation?
I thought document only.
>
> > 3.11: Make the -b option no-op. Bytes warning never emits.
> > 3.12: Remove the -b option.
>
> There is no _need_ to raise an error when -b is used. The -t option
> was kept even after the feature was removed (in Python 3.0 ?). -J
> ("used by Jython" says a comment) is a second command line option
> which is silently ignored.
>
I see.
>
> > BytesWarning will be deprecated in the document, but not to be removed.
>
> I don't see what you mean here. I dislike the idea of deprecating a
> feature without scheduling its removal. I don't see the point of
> deprecating it in this case. I only see that as an annoyance.
>
Document only deprecation is useful for readers. Readers can know "I
can just ignore this.".
> I'm fine with removing the exception. If you don't plan to remove it,
> just leave it unchanged (not deprecated), no?
>
OK, my new proposal is:
3.10: Stop emitting BytesWarning for bytes == unicode case, because
this is the most annoying part.
3.11: Stop emitting BytesWarning in core and stdlib.
4.0: Remove `-b` option, `sys.flags.bytes_warning`, and `BytesWarning`.
Regards,
--
Inada Naoki
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/OQOCZISWWKRFAFMZJI5GMA3SNEQ2TYIJ/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
Le sam. 24 oct. 2020 à 15:13, Christian Heimes a écrit : > In my experience it would be useful to keep the bytes warning for > implicit representation of bytes in string formatting. It's still a > common source of issues in code. IMO it's not a big deal to investigate such bugs without the -b / -bb command line option. It should be easy to identify where bytes are formatted as string. Victor -- Night gathers, and now my watch begins. It shall not end until my death. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/QCZ5F2BTEQAAX6GJKBQGWHXPOCQZKLAJ/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
Hi,
Which operations are impacted by -b and -bb? str == bytes, bytes ==
str, dict lookup using str or bytes keys? 'unicode' < b'bytes' always
raises a TypeError.
Le sam. 24 oct. 2020 à 05:20, Inada Naoki a écrit :
> To avoid BytesWarning, the compiler needs to do some hack when they
> need to store bytes and str constants in one dict or set.
> BytesWarning has maintenance costs. It is not huge, but significant.
os.get_exec_path() must modify temporarily warnings filters to ignore
BytesWarning when it looks for 'PATH' (unicode) or b'PATH' (bytes) in
the 'env' dictionary which may contain unicode or bytes strings.
Modifying warnings filters impact all threads which is bad.
I dislike having to workaround this annoying behavior for dict lookup
when -b or -bb is used.
I'm quite sure that almost nobody uses -b or -bb when running their
test suite or to develop. I expect that nobody uses it. According to
replies, it seems like porting Python 2 code to Python 3 is the only
use case. Python 3.9 and older can be used for that, no?
> When can we remove it? My idea is:
>
> 3.10: Deprecate the -b option.
Do you mean writing a message into stderr? Or just deprecate it in the
documentation?
> 3.11: Make the -b option no-op. Bytes warning never emits.
> 3.12: Remove the -b option.
There is no _need_ to raise an error when -b is used. The -t option
was kept even after the feature was removed (in Python 3.0 ?). -J
("used by Jython" says a comment) is a second command line option
which is silently ignored.
> BytesWarning will be deprecated in the document, but not to be removed.
I don't see what you mean here. I dislike the idea of deprecating a
feature without scheduling its removal. I don't see the point of
deprecating it in this case. I only see that as an annoyance.
I'm fine with removing the exception. If you don't plan to remove it,
just leave it unchanged (not deprecated), no?
Victor
--
Night gathers, and now my watch begins. It shall not end until my death.
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/YL4DITFFUYBNXF7EFZKO4IRZRDRMRIVP/
Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
24.10.20 06:19, Inada Naoki пише: > To avoid BytesWarning, the compiler needs to do some hack when they > need to store bytes and str constants in one dict or set. > BytesWarning has maintenance costs. It is not huge, but significant. > > When can we remove it? My idea is: > > 3.10: Deprecate the -b option. > 3.11: Make the -b option no-op. Bytes warning never emits. > 3.12: Remove the -b option. > > BytesWarning will be deprecated in the document, but not to be removed. > Users who want to use the -b option during 2->3 conversion need to use > Python ~3.10 for a while. I agree that it should be removed, and that BytesWarning should be kept (maybe we will reuse it for other purposes in future). But I do not see how deprecating it before removing could help. Using it with -We will no longer work, and without -We it will just add a noise. We can just make -b a no-op at any moment and remove it few versions later. Or maybe first make it no-op, then deprecate, then remove. But it looks too much. -b is still usable in 3.9, so it can be removed not earlier than EOL of 3.9. Users that use it should be able to use it with all maintained Python versions if it makes sense with at least one of them. 3.x: Make the -b option no-op. Bytes warning never emits. 3.x+4: Remove the -b option. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/6LBAYEQSWUEFKNR6LMJ35OAF2YZXAVWE/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
26.10.20 10:10, Inada Naoki пише: > Of course, there are some runtime costs too. > > https://github.com/python/cpython/blob/fb5db7ec58624cab0797b4050735be865d380823/Modules/_functoolsmodule.c#L802 > https://github.com/python/cpython/blob/fb5db7ec58624cab0797b4050735be865d380823/Objects/codeobject.c#L724 > (maybe more, but I'm not sure) It will not help much in these cases because we still need to distinguish 1 from True and 1.0 and -0.0 from 0.0. But if keys only can be str or bytes, we pay additional cost. An example is the re cache. ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/S4B7KIIEB5XRPE4WKDXMSN424DJGNGBC/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
On Mon, Oct 26, 2020 at 4:35 PM Senthil Kumaran wrote: > > On Sat, Oct 24, 2020 at 6:18 AM Christian Heimes wrote: >> >> In my experience it would be useful to keep the bytes warning for >> implicit representation of bytes in string formatting. It's still a >> common source of issues in code. > > I am with Christian here. Do you mean you are OK to remove BytesWarning from b"abc" == u"def" and b"abc" == 42? > Still notice a possibility of people running into this because all the > Python2 code is not dead yet. > Perhaps this warning might stay for a long time. > I never proposed to remove it "now", but 3.11. 3.10 will become security only mode at 2022-04, and EOL at 2026-10. But you can use Python 3.10 after EOL for porting Python 2 code, because security fix is not required while porting. > > BytesWarning has maintenance costs. It is not huge, but significant. > > Should we know by how much so that the proposal of `-b` switch can be > weighted against? > It is difficult to say "how much". We need to keep it in mind that `a == b` is not safe even for builtin types everytime we write a patch or review pull request. Especially, when u"foo" and b"bar" are used as keys of the same dict, BytesWarnings happens only when (randomized) hash collision. It is very hard to find this bug. Of course, there are some runtime costs too. https://github.com/python/cpython/blob/fb5db7ec58624cab0797b4050735be865d380823/Modules/_functoolsmodule.c#L802 https://github.com/python/cpython/blob/fb5db7ec58624cab0797b4050735be865d380823/Objects/codeobject.c#L724 (maybe more, but I'm not sure) Regards, -- Inada Naoki ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/52WLUUIL2LS27R5UFYFICJH5OX3ETSTA/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
On Sat, Oct 24, 2020 at 6:18 AM Christian Heimes wrote: > > > In my experience it would be useful to keep the bytes warning for > implicit representation of bytes in string formatting. It's still a > common source of issues in code. > I am with Christian here. Still notice a possibility of people running into this because all the Python2 code is not dead yet. Perhaps this warning might stay for a long time. > BytesWarning has maintenance costs. It is not huge, but significant. Should we know by how much so that the proposal of `-b` switch can be weighted against? Thank you, Senthil ___ Python-Dev mailing list -- python-dev@python.org To unsubscribe send an email to python-dev-le...@python.org https://mail.python.org/mailman3/lists/python-dev.python.org/ Message archived at https://mail.python.org/archives/list/python-dev@python.org/message/Z62L5ABNQK5AYPEE6I3KTZMKEY3BC65R/ Code of Conduct: http://python.org/psf/codeofconduct/
[Python-Dev] Re: When to remove BytesWarning?
On 24/10/2020 05.19, Inada Naoki wrote:
> Hi, all.
>
> To avoid BytesWarning, the compiler needs to do some hack when they
> need to store bytes and str constants in one dict or set.
> BytesWarning has maintenance costs. It is not huge, but significant.
>
> When can we remove it? My idea is:
>
> 3.10: Deprecate the -b option.
> 3.11: Make the -b option no-op. Bytes warning never emits.
> 3.12: Remove the -b option.
>
> BytesWarning will be deprecated in the document, but not to be removed.
> Users who want to use the -b option during 2->3 conversion need to use
> Python ~3.10 for a while.
In my experience it would be useful to keep the bytes warning for
implicit representation of bytes in string formatting. It's still a
common source of issues in code. Bytes / str comparison or dict lookup
is a less common issue.
>>> b = b'bytes'
>>> f"{b}"
Traceback (most recent call last):
File "", line 1, in
BytesWarning: str() on a bytes instance
>>> str(b)
Traceback (most recent call last):
File "", line 1, in
BytesWarning: str() on a bytes instance
Christian
___
Python-Dev mailing list -- python-dev@python.org
To unsubscribe send an email to python-dev-le...@python.org
https://mail.python.org/mailman3/lists/python-dev.python.org/
Message archived at
https://mail.python.org/archives/list/python-dev@python.org/message/M2NJZTGLUA5IAGYDJZOZCPOL4AIXKMY6/
Code of Conduct: http://python.org/psf/codeofconduct/
