[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[Éric Araujo]

Éric Araujo  added the comment:

> it is useful to build old versions, sometimes 7 year old versions, to confirm 
> if/when the behaviour regressed. 

I think the sticking point is this: it is not expected that old versions should 
work as-is with the newest Python.  Can you do your build and test with the 
Python version that was current 7 years ago?

--
nosy: +eric.araujo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Let's see how it goes. In the worst case, we have time before Python 3.11 final 
to revert it one more time.

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 19ba2122ac7313ac29207360cfa864a275b9489e by Victor Stinner in 
branch 'main':
bpo-37330: open() no longer accept 'U' in file mode (GH-28118)
https://github.com/python/cpython/commit/19ba2122ac7313ac29207360cfa864a275b9489e


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> docutils is already fixed in the development version. I requested a release: 
> https://sourceforge.net/p/docutils/bugs/364/

At 2019-07-22,  Günter Milde wrote: "Docutils 0.15 is released" (with the fix). 
The latest docutils version is 0.17.1.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> This breaks Samba's build:
> https://bugzilla.samba.org/show_bug.cgi?id=14266

Samba has been updated in the meanwhile, buildtools/wafsamba/samba_utils.py:

-function_code = node.read('rU', None)
+function_code = node.read('r', None)

https://github.com/samba-team/samba/blob/1209c89dcf6371bbfa4f3929a47a573ef2916c1a/buildtools/wafsamba/samba_utils.py#L692

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I reopen the issue. I created PR 28118 to try again to remove the "U" mode in 
Python 3.11.

--
resolution: fixed -> 
status: closed -> open
versions: +Python 3.11 -Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +26559
pull_request: https://github.com/python/cpython/pull/28118

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 942f7a2dea2e95a0fa848329565c0d0288d92e47 by Victor Stinner in 
branch 'master':
bpo-39674: Revert "bpo-37330: open() no longer accept 'U' in file mode 
(GH-16959)" (GH-18767)
https://github.com/python/cpython/commit/942f7a2dea2e95a0fa848329565c0d0288d92e47


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Le mer. 4 mars 2020 à 03:01, Andrew Bartlett  a écrit :
> (...) but that a valid build script that operates correctly on Python 3.8 
> will no longer, without warning to us, (...)

Displaying warnings by default or not is an old debate.

I added the "Python Development Mode" (-X dev) to help the situation:

https://docs.python.org/dev/library/devmode.html#devmode

I just added a very explicit "You should check for DeprecationWarning in your 
code" section in What's New In Python 3.9:

https://docs.python.org/dev/whatsnew/3.9.html#you-should-check-for-deprecationwarning-in-your-code

Moreover, DeprecationWarning are now (Python 3.7) displayed *by default* in the 
__main__ module:
https://www.python.org/dev/peps/pep-0565/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[Andrew Bartlett]

Andrew Bartlett  added the comment:

...Python 2.9 or at best 2.10. 

Of course I mean Python 3.9 or at best 3.10. 

Sorry for the confusion.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[Andrew Bartlett]

Andrew Bartlett  added the comment:

I think we are speaking past each other. 

Yes, Python 2 is no longer being changed, which is awesome as we need for fear 
breakage of the older builds that use that for the build system.

The issue isn't the particular language feature, or that there is no way to 
further adapt code, but that a valid build script that operates correctly on 
Python 3.8 will no longer, without warning to us, operate on Python 2.9 or at 
best 2.10. 

Thankfully the only Samba versions with support for Python3 are still in 
maintenance, so course I've back-ported patches to the tip of each release 
branch to work around this change.  But again, that doesn't help when I need to 
build an arbitrary git revision in the future. 

(That warnings were available, if we knew were to look, is an entirely moot 
point at this stage).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +18125
pull_request: https://github.com/python/cpython/pull/18767

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> Thankfully we don't need universal newlines, but no matter if we need them, 
> old versions of Samba will fail to build without a fix-up patch.

io.open() is available since Python 2.6 and handles different kinds of newlines 
transparently. Example:

$ python2
Python 2.7.17 (default, Oct 20 2019, 00:00:00) 
>>> f=open("test", "wb"); f.write("a\rb\rc\r"); f.close()
>>> import io; f=io.open("test"); lines=list(f); f.close(); lines
[u'a\n', u'b\n', u'c\n']

Do you suggest to change the documentation to suggest to open io.open() for 
applications which still care about Python 2?
https://docs.python.org/dev/whatsnew/3.9.html#changes-in-the-python-api

Note: Python 2 is no longer supported upstream, I strongly advice you to 
upgrade to Python 3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[Andrew Bartlett]

Andrew Bartlett  added the comment:

Thanks for the reply.

To clarify, it is best to understand where Samba thinks of itself in terms of 
Python.  We use python, but we don't really think of ourselves as a python 
project, and while we have some developers with a depth of experience in the 
language, most of us just use it as a tool to get things done, in particular to 
build Samba.

Samba has a build system built in Python, called Waf.  Samba uses waf to build 
the project (particularly the C parts), and our build of the C binaries is 
portable back to Python 2.6.  

Often in Samba we find bugs.  This might come as a shock! ;-).  Many of those 
bugs are regressions, and so it is useful to build old versions, sometimes 7 
year old versions, to confirm if/when the behaviour regressed.  We are able to 
do this, with only one patch to address a perl behaviour change, even on a 
modern Ubuntu 18.04 system, all the way back to 2013.  

I terms of 'but this has been deprecated for years', Samba has only had 
releases been able to even use Python3 in the build system for a year (first 
committed to master in July 2018).

Nobody in Samba was aware of any DeprecationWarning and for better or worse we 
only knew about the issue once our builds failed in Fedora.  We are incredibly 
grateful that Fedora tried a rebuild with the alpha version of Python 3.9 as 
otherwise we would find out only after this was well baked into a release.

Thankfully we don't need universal newlines, but no matter if we need them, old 
versions of Samba will fail to build without a fix-up patch.

More importantly, I think it is unlikely that Samba is the only software to 
have used this feature, intentionally or otherwise, or to be in this position.  

While not as critical here, for context as to why we support Python 2.6 to 
build, it should be noted that retaining this compatibility was a critical part 
of being able to get consensus to otherwise move to Python 3.4.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> While we are of course able to patch new versions of Samba, this will make it 
> harder to bisect back though Samba history.

I don't understand the bisect part. Would you mind to elaborate? What do you 
want to bisect?

>  It would be really great if this kind of change could be reverted or at 
> least deferred so Samba can rely on a stable language platform for it's build 
> system.

The "U" mode was deprecated since September 2012 (Python 3.3): you had 8 years 
to handle the DeprecationWarning. 8 years before actually removing a deprecated 
feature sounds like a stable language platform to me.

I'm not sure of the benefit of deferring the change. Does it mean that Sambda 
would not be updated?

Well, your comment fits exactly into bpo-39674 scope, except that revert the 
"U" mode is not scheduled yet.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[Andrew Bartlett]

Andrew Bartlett  added the comment:

This breaks Samba's build:

https://bugzilla.samba.org/show_bug.cgi?id=14266

While we are of course able to patch new versions of Samba, this will make it 
harder to bisect back though Samba history.  It would be really great if this 
kind of change could be reverted or at least deferred so Samba can rely on a 
stable language platform for it's build system.

--
nosy: +abartlet

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

> should we backport a documentation change for this? (the deprecatedremoved 
> says 4.0 currently)

Done. Thanks for the reminder.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[miss-islington]

miss-islington  added the comment:


New changeset 4cb08b6c0ae6989d169dd48c2b24087941f6d0b0 by Miss Skeleton (bot) 
in branch '3.7':
bpo-37330: open(): "U" mode is removed in Python 3.9 (GH-16972)
https://github.com/python/cpython/commit/4cb08b6c0ae6989d169dd48c2b24087941f6d0b0


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[miss-islington]

Change by miss-islington :


--
pull_requests: +16508
pull_request: https://github.com/python/cpython/pull/16982

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 1d2862a323ae1387e306942253b1e487018e2b7f by Victor Stinner in 
branch '3.8':
bpo-37330: open(): "U" mode is removed in Python 3.9 (GH-16972)
https://github.com/python/cpython/commit/1d2862a323ae1387e306942253b1e487018e2b7f


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +16499
pull_request: https://github.com/python/cpython/pull/16972

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[Anthony Sottile]

Anthony Sottile  added the comment:

should we backport a documentation change for this? (the deprecatedremoved says 
4.0 currently)

--
nosy: +Anthony Sottile

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

Thanks for the review Serhiy ;-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:


New changeset e471e72977c83664f13d041c78549140c86c92de by Victor Stinner in 
branch 'master':
bpo-37330: open() no longer accept 'U' in file mode (GH-16959)
https://github.com/python/cpython/commit/e471e72977c83664f13d041c78549140c86c92de


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I cited this change in the PEP 608 "Coordinated Python release".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +16487
pull_request: https://github.com/python/cpython/pull/16959

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

docutils is already fixed in the development version. I requested a release:
https://sourceforge.net/p/docutils/bugs/364/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I reported the issue to docutils. In fact, docutils FileInput class was already 
fixed to prevent the deprecation warning, but there was no release yet:
https://sourceforge.net/p/docutils/bugs/363/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

STINNER Victor  added the comment:

I prefer to push backward incompatible changes at the beginning of a dev cycle 
(Python 3.9), so we have room to revert it later if it causes too much trouble.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

Change by STINNER Victor :


--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +14042
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/14204

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37330] open(): remove 'U' mode, deprecated since Python 3.3

[STINNER Victor]

New submission from STINNER Victor :

The builtin open() function accepts 'U' in the file mode, but it emits a 
DeprecationWarning since Python 3.4, change made in bpo-15204:

commit 6787a3806ee6a85a6f21ede70c10e15a6df267c4
Author: Serhiy Storchaka 
Date:   Sat Nov 23 22:12:06 2013 +0200

Issue #15204: Deprecated the 'U' mode in file-like objects.

$ cat x.py 
f = open(__file__, 'U')
line = f.readline()
print(type(line))
f.close()

$ python3.7 x.py 
x.py:1: DeprecationWarning: 'U' mode is deprecated
  f = open(__file__, 'U')


The 'U' mode is documented as deprecated since Python 3.3

The flag is basically only there for backward compatibility with Python 2. 
Python 3.9.0 final release is currently scheduled at 2020-06-08: 6 months after 
Python 2 end-of-life (2020-01-01).

Currently, the 'U' mode is documented as it will be removed from Python 4.0. 
But I dislike having a bunch of backward incompatible changes in Python 4.0. I 
would prefer Python 4.0 to be simply the version after Python 3.X.

Deprecation warnings are now displayed in the __main__ module by default, 
thanks to the PEP 565 -- Show DeprecationWarning in __main__.
https://www.python.org/dev/peps/pep-0565/

More and more projects run their test suite using -Werror and so have to fix 
deprecation warnings.

--
components: Library (Lib)
messages: 345991
nosy: serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: open(): remove 'U' mode, deprecated since Python 3.3
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com