[issue22709] restore accepting detached stdin in fileinput binary mode

[R. David Murray]

R. David Murray added the comment:

Hopefully 'better late than never' applies to this.  Sigh.

--
resolution:  -> fixed
stage: commit review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22709] restore accepting detached stdin in fileinput binary mode

[R. David Murray]

Changes by R. David Murray :


--
stage:  -> commit review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22709] restore accepting detached stdin in fileinput binary mode

[Akira Li]

New submission from Akira Li:

The patch for Issue #21075: fileinput.FileInput now reads bytes from standard 
stream if binary mode is specified broke code that used
sys.stdin = sys.stdin.detach() with FileInput(mode='rb') in Python 3.3

I've attached the patch that makes FileInput to accept detached sys.stdin 
(without 'buffer' attribute) in binary mode.

--
components: Library (Lib)
files: fileinput-detached-stdin.diff
keywords: patch
messages: 229859
nosy: akira
priority: normal
severity: normal
status: open
title: restore accepting detached stdin in fileinput binary mode
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6
Added file: http://bugs.python.org/file36997/fileinput-detached-stdin.diff

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22709] restore accepting detached stdin in fileinput binary mode

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The code

sys.stdin = sys.stdin.detach()

is incorrect because sys.stdin should be text stream, but detach() returns 
binary stream.

--
nosy: +serhiy.storchaka

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22709] restore accepting detached stdin in fileinput binary mode

[Akira Li]

Akira Li added the comment:

It is incorrect that sys.stdin is *always* a text stream. It often is,
but not always.

There are cases when it is not e.g., 

   $ tar zcf - stuff | gpg -e | ssh user@server 'cat -  stuff.tar.gz.gpg'

tar's stdout is *not* a text stream.
gpg's stdin/stdout are *not* text streams.
ssh's stdin is *not* a text stream.
etc.

If any of the steps are implemented in Python then it is useful to
consider sys.stdin as a binary stream.

Any script written before Python 3.4.1 (#21075) that used FileInput binary mode
*had to* use sys.stdin = sys.stdin.detach()

A bugfix release should not break working code.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22709] restore accepting detached stdin in fileinput binary mode

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

 It is incorrect that sys.stdin is *always* a text stream. It often is,
 but not always.
 
 There are cases when it is not e.g.,
 
$ tar zcf - stuff | gpg -e | ssh user@server 'cat -  stuff.tar.gz.gpg'
 
 tar's stdout is *not* a text stream.
 gpg's stdin/stdout are *not* text streams.
 ssh's stdin is *not* a text stream.
 etc.

This is not related to Python. Terms character, string, text, file can 
have different meaning in different domains. In Python we use Python 
terminology. There is no such thing as sys.stdin in Posix-compatible shell, 
because Posix-compatible shell has no the sys module and doesn't use a dot to 
access attributes.

 Any script written before Python 3.4.1 (#21075) that used FileInput binary
 mode *had to* use sys.stdin = sys.stdin.detach()
 
 A bugfix release should not break working code.

Correct solution in this case would be to use the workaround sys.stdin = 
sys.stdin.detach() conditionally, only in Python versions which have a bug.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22709] restore accepting detached stdin in fileinput binary mode

[Akira Li]

Akira Li added the comment:

 This is not related to Python. Terms character, string, text, file 
 can have different meaning in different domains. In Python we use Python 
 terminology. There is no such thing as sys.stdin in Posix-compatible shell, 
 because Posix-compatible shell has no the sys module and doesn't use a dot to 
 access attributes.

I use Python terminology (text - Unicode string, binary data - bytes).

Though text vs. binary data distinction is language independent (
it doesn't matter how Unicode type is called in a particular language).

Python can be used to implement `tar`, `gpg`, `ssh`, `7z`, etc. I don't
see what POSIX has anything to do with that fact.

It is very simple actually: 

  text - encode character encoding - bytes
  bytes - decode character encoding - text

In most cases text should be human readable.

It doesn't make sense to encode/decode input/output of gpg-like utilities using 
a character encoding. *Therefore* the notion of 
sys.stdin being a bytes stream (io.BufferedReader) can be useful
in this case.

The lines produced by FileInput are often (after optional processing)
written to sys.stdout. If binary mode is used then FileInput(mode='rb') 
yields bytes therefore it is also useful to consider sys.stdout
a binary stream (io.BufferedWriter) in this case.

It introduces a nice symmetry:

  text FileInput mode - text streams
  binary FileInput mode - binary streams

By design, FileInput treats stdin as any other file. It
even supports a special name for it: '-'. A file may be in
binary mode; stdin should be able too.

sys.stdout is used outside of FileInput therefore no changes in 
FileInput itself are necessary but sys.stdin is used inside FileInput
that is why the change is needed.

 Correct solution in this case would be to use the workaround sys.stdin = 
sys.stdin.detach() conditionally, only in Python versions which have a bug.

Do you mean every Python 3 version before Python 3.4.1?

Correct solution is to avoid blaming users 
(your fault - you change your programs) for our mistakes  
and fix the bug in Python itself. The patch is attached.

--

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22709] restore accepting detached stdin in fileinput binary mode

[R. David Murray]

R. David Murray added the comment:

I actually agree that this should be applied not only for backward 
compatibility reasons, but because it is better duck typing.  It unfortunately 
leaves code still having to potentially deal with if python version is 3.4.1 
or 3.4.2, but there is nothing that can be done about that.

--
nosy: +r.david.murray
versions:  -Python 3.6

___
Python tracker rep...@bugs.python.org
http://bugs.python.org/issue22709
___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com