[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[Antti Haapala]

Antti Haapala added the comment:

Yeah, it definitely is a bug in CPython. open(mode='a') should always append to 
the end of the given file. 

If you're writing an append-only text log to some file-like object, that's the 
mode you use, not some version/platform/filesystem specific voodoo to find out 
what's the least incorrect way to work around Python implementation 
deficiencies.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[Martin Panter]

Martin Panter added the comment:

Handling ESPIPE for append mode seems reasonable to me, even as a bug fix for 
existing versions.

But there is a similar problem with "r+" and "w+" modes and unseekable files 
(unless buffering=0). See Issue 20074. So we can’t say in general that Python 3 
faithfully implements all aspects of Python 2’s / C’s / Posix’s file modes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[R. David Murray]

Changes by R. David Murray :


--
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[R. David Murray]

R. David Murray added the comment:

And the fact that python deviates from posix in this regard seems like a bug to 
me, since the posix behavior is, as noted, very useful.  Can't we handle ESPIPE 
like the C library does, instead of forcing users to learn that arcane 
incantation?

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[Shane Hathaway]

Shane Hathaway added the comment:

Thanks for the analysis. I have already started a pull request to fix this in 
Supervisor, but I also thought this change to Python might be gratuitous and 
accidental. It seems like open('/dev/stdout', 'a') ought to work the same as 
Python 2. If not, the Python documentation should warn people that using 'a' 
with character devices and FIFOs will cause an OSError.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[STINNER Victor]

STINNER Victor added the comment:

Antti Haapala added the comment:
> Presumably the case was that a *named* log file is opened with 'a' mode, and 
> one could pass '/dev/stdout' just like any other name of a file, and it did 
> work, but not in Python 3.5.

Oh ok, in this case, you need something smarter like:

mode = 'a' if not stat.S_ISCHR(os.stat(filename).st_mode) else 'w'
fp = open(filename, mode)

or something like (emulate append mode, but catch ESPIPE):

fp = open(filename, 'w')
try:
  fp.seek(0, os.SEEK_END)
except OSError as exc:
  if exc.errno != errno.ESPIPE: raise

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[Antti Haapala]

Antti Haapala added the comment:

Presumably the case was that a *named* log file is opened with 'a' mode, and 
one could pass '/dev/stdout' just like any other name of a file, and it did 
work, but not in Python 3.5.

--
nosy: +ztane

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[STINNER Victor]

STINNER Victor added the comment:

> Users of Supervisor have been depending on that for a long time. 

Hum, the workaround is very simple no? Just open /dev/stdout with mode "w", no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[STINNER Victor]

STINNER Victor added the comment:

Syscalls made by open("/dev/stdout", "a") in Python 2:

  open("/dev/stdout", O_WRONLY|O_CREAT|O_APPEND, 0666) = 3
  lseek(3, 0, SEEK_END)   = -1 ESPIPE (Illegal seek)
  fstat(3, {st_mode=S_IFCHR|0620, st_rdev=makedev(136, 0), ...}) = 0

I used this comand:

  $ strace -o trace python2 -c 'import os; os.uname();os.uname();os.uname(); 
f=open("/dev/stdout", "a"); os.uname(); f.close()'

It looks like the C library simply ignores ESPIPE on lseek(fd, 0, SEEK_END) 
when opening a file in append mode:
https://sourceware.org/git/?p=glibc.git;a=blob;f=libio/fileops.c;h=13157354272ff9ab1832d4a619a81f05898fcd69;hb=HEAD#l242

  if ((read_write & _IO_IS_APPENDING) && (read_write & _IO_NO_READS))
if (_IO_SEEKOFF (fp, (_IO_off64_t)0, _IO_seek_end, _IOS_INPUT|_IOS_OUTPUT)
== _IO_pos_BAD && errno != ESPIPE)
  {
close_not_cancel (fdesc);
return NULL;
  }

from _IO_file_open() file operation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[Martin Panter]

Martin Panter added the comment:

The origin of this seems to be r68835 and Issue 5008. Victor mentioned 
imitating the Gnu C library. Maybe there is a better way that also supports 
non-seekable files better, perhaps handle ESPIPE without failing.

This also affects Python 2, if you consider io.open() or io.FileIO directly.

--
nosy: +haypo, martin.panter
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27805] In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE

[Shane Hathaway]

New submission from Shane Hathaway:

With Python 2, the following call worked:

open('/dev/stdout', 'a')

Users of Supervisor have been depending on that for a long time. 

With Python 3.5, this is what happens:

>>> open('/dev/stdout', 'a')
Traceback (most recent call last):
  File "", line 1, in 
OSError: [Errno 29] Illegal seek

Presumably, this happens because Python 3 opens the file in 'w' mode and seeks 
to the end, while Python 2 passed the 'a' flag directly to the underlying C 
library; the underlying library is apparently smart enough to treat 'a' and 'w' 
identically when opening character device files and FIFOs.

It's a nasty little surprise for those upgrading from Python 2.

--
components: IO
messages: 273158
nosy: hathawsh
priority: normal
severity: normal
status: open
title: In Python 3, open('/dev/stdout', 'a') raises OSError with errno=ESPIPE
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com