[issue28647] python --help: -u is misdocumented as binary mode

2017-10-13 Thread Berker Peksag 

Berker Peksag  added the comment:

Modules/main.c and Python.man is same in 3.6 branch. We could backport the 
change in Doc/library/sys.rst from 7f580970836b0f6bc9c5db868d95bea81a3e1558 but 
I didn't do it yet since it needs be manually backported.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-13 Thread STINNER Victor 

STINNER Victor  added the comment:

Thanks Berker for this nice documentation enhancement! It was required.

Do we need to update Python 3.6 documentation using the commit 
5f908005ce16b06d5af7b413264009c4b062f33c, or are we good? (sorry, I didn't 
check)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-13 Thread Berker Peksag 

Berker Peksag  added the comment:

Thank you for reviews, Serhiy and Victor.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-13 Thread Berker Peksag 

Berker Peksag  added the comment:


New changeset 7f580970836b0f6bc9c5db868d95bea81a3e1558 by Berker Peksag in 
branch 'master':
bpo-28647: Update -u documentation after bpo-30404 (GH-3961)
https://github.com/python/cpython/commit/7f580970836b0f6bc9c5db868d95bea81a3e1558


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread Serhiy Storchaka 

Serhiy Storchaka  added the comment:

> Hum, I propose to mention stdin in -u documentation as: "The option has no 
> effect on stdin." What do you think?

This LGTM too. More precise, it has no effect on stdin buffering. It has effect 
on the line_buffering attribute, but this attribute has no effect on reading.

> The last sentence is wrong and should be removed from sys.stdin 
> documentation, no?

Or correct it, making it related only to stdout and stderr.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread STINNER Victor 

STINNER Victor  added the comment:

Serhiy: "I think there is no need to mention stdin in the context of the -u 
option at all. -u doesn't affect stdin buffering, whatever that would mean. 
Period."

Hum, I propose to mention stdin in -u documentation as: "The option has no 
effect on stdin." What do you think?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread STINNER Victor 

STINNER Victor  added the comment:

> I think we need to document behavior of stdin somewhere, because current the 
> sys.stdin documentation states:
>
>> When interactive, standard streams are line-buffered. Otherwise, they
>> are block-buffered like regular text files. You can override this value
>> with the -u command-line option.

The last sentence is wrong and should be removed from sys.stdin documentation, 
no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread STINNER Victor 

STINNER Victor  added the comment:

Interesting comment in create_stdio() of Python/pylifecycle.c:
---
/* stdin is always opened in buffered mode, first because it shouldn't
   make a difference in common use cases, second because TextIOWrapper
   depends on the presence of a read1() method which only exists on
   buffered streams.
*/
if (Py_UnbufferedStdioFlag && write_mode)
buffering = 0;
else
buffering = -1;
---

stdin is always buffered ;-)

I created bpo-31775: "Support unbuffered TextIOWrapper".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread Berker Peksag 

Berker Peksag  added the comment:

> -u doesn't affect stdin buffering, whatever that would mean.

I think we need to document behavior of stdin somewhere, because current the 
sys.stdin documentation states:

> When interactive, standard streams are line-buffered. Otherwise, they
> are block-buffered like regular text files. You can override this value
> with the -u command-line option.

https://docs.python.org/3/library/sys.html#sys.stdin

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread Serhiy Storchaka 

Serhiy Storchaka  added the comment:

stdin is mentioned in the documentation of the -u option only due to weird 
internal buffering in Python 2, because user can expect that -u disables it. It 
is documented what methods use internal buffering and how get rid of it. No 
other buffering is mentioned.

This no longer actual in Python 3. I think there is no need to mention stdin in 
the context of the -u option at all. -u doesn't affect stdin buffering, 
whatever that would mean. Period.

Alternatively you can include a lecture about different kinds of buffering and 
how -u doesn't affect them.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread STINNER Victor 

STINNER Victor  added the comment:

Serhiy Storchaka:
https://github.com/python/cpython/pull/3961#issuecomment-336136160

"I suggest to continue the discussion on the tracker."

Ok, let's continue here.

"We are fixing the outdated documentation inherited from Python 2. First than 
keep some statement we should consider what it means in the context of Python 2 
and what it means in the context of Python 3."

stdin buffering is a complex thing.

When running the UNIX command "producer | consumer", many users are confused by 
the buffering on the *producer* side.

When running a program in a TTY, the TTY does line buffering for you, you 
cannot get immediately a single character (without changing the default TTY 
configuration).

I don't think that we need to say too much. I just suggest to say "stdin is 
always buffered". That's all.

See my previous messages for the my definition of "buffered" versus 
"unbuffered" read.

Note: Today I learned the UNIX "stdbuf" command, useful to configure the stdin, 
stdout and stderr buffering of C applications using .

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread STINNER Victor 

STINNER Victor  added the comment:

Serhiy: "(...) I think it is more correct to say that stdin is always 
unbuffered in Python 3."

I disagree. Technically, sys.stdin.read(1) reads up to 1024 bytes from the file 
descriptor 0. For me, "unbuffered read" means that read(1) reads a single byte.

Expected behaviour of an fully unbuffered stdin:

assert sys.stdin.read(1) == 'a'
assert os.read(0, 1) == b'b'

The program should not fail with an assertion error nor block if you write 'ab' 
characters into stdin.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread Serhiy Storchaka 

Serhiy Storchaka  added the comment:

In Python 2 there is an internal buffering in xreadlines(), readlines() and 
file-object iterators. You need to enter many lines first that the program get 
the first of them. And -u doesn't help.

But in Python 3 the program gets the input right as it becomes available. 
Reading is not blocked if the input is available. There are internal buffers, 
but they affect only performance, not the behavior. If you can edit a line 
before pressing Enter, this is because your terminal buffers a line before 
sending it to the program. I think it is more correct to say that stdin is 
always unbuffered in Python 3.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread Berker Peksag 

Berker Peksag  added the comment:

I think it's better to be explicit and mention 'stdin' in this case. Plus, 
"stdin is always buffered" is quite short so potential of creating noise is 
quite minimal IMO.

(FYI, I asked this question on GitHub: 
https://github.com/python/cpython/pull/3961#issuecomment-336035358)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread Serhiy Storchaka 

Serhiy Storchaka  added the comment:

The -u option doesn't affect the buffering of stdin. Is it worth to document 
this explicitly? Or it just adds a noise? Maybe remove this from the help and 
manpage but add in the RST documentation?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread Serhiy Storchaka 

Serhiy Storchaka  added the comment:

Don't forget to update Misc/python.man.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread Berker Peksag 

Berker Peksag  added the comment:

Good catch, I thought it was already fixed in master after bpo-30404. I've 
opened PR 3961.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-12 Thread Berker Peksag 

Change by Berker Peksag :


--
pull_requests: +3938
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-11 Thread STINNER Victor 

STINNER Victor  added the comment:

I just checked the master branch:

-u : unbuffered binary stdout and stderr, stdin always buffered;
 also PYTHONUNBUFFERED=x
 see man page for details on internal buffering relating to '-u'

The doc is wrong. stdout and stderr are fully unbuferred since Serhiy changed 
them: commit 77732be801c18013cfbc86e27fcc50194ca22c8e, bpo-30404.

--
resolution: fixed -> 
status: closed -> open
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-11 Thread Gareth Rees 

Gareth Rees  added the comment:

You're welcome.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-11 Thread Berker Peksag 

Berker Peksag  added the comment:

Thank you for the patch, Gareth.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-11 Thread Berker Peksag 

Berker Peksag  added the comment:


New changeset 5f908005ce16b06d5af7b413264009c4b062f33c by Berker Peksag in 
branch '3.6':
bpo-28647: Update -u documentation (GH-3954)
https://github.com/python/cpython/commit/5f908005ce16b06d5af7b413264009c4b062f33c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-11 Thread Berker Peksag 

Berker Peksag  added the comment:

Pull request for issue 30404 has been merged so we only need the documentation 
patch for the 3.6 branch (unfortunately 3.5 is now in security-fix-only mode) 
I've opened PR 3954.

--
versions:  -Python 3.5, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-10-11 Thread Berker Peksag 

Change by Berker Peksag :


--
pull_requests: +3929

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-05-24 Thread Armin Rigo 

Changes by Armin Rigo :


--
nosy:  -arigo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-05-24 Thread Stéphane Wirtel 

Changes by Stéphane Wirtel :


--
pull_requests: +1880

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-05-24 Thread Stéphane Wirtel 

Changes by Stéphane Wirtel :


--
pull_requests: +1879

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-05-19 Thread Serhiy Storchaka 

Serhiy Storchaka added the comment:

Issue30404 makes stdout and stderr truly unbuffered when run with -u.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-05-18 Thread Berker Peksag 

Changes by Berker Peksag :


--
pull_requests: +1749

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-03-15 Thread Nick Coghlan 

Nick Coghlan added the comment:

I just ran into this discrepancy working on the test cases for PEP 538 - +1 for 
Gareth's suggested approach of just aligning the `--help` output with the man 
page.

--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2017-01-06 Thread Berker Peksag 

Changes by Berker Peksag :


--
nosy: +berker.peksag
stage:  -> patch review
type:  -> behavior
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2016-11-12 Thread Gareth Rees 

Gareth Rees added the comment:

Here's a patch that copies the text for the -u option from the man page to the 
--help output.

--
keywords: +patch
Added file: http://bugs.python.org/file45463/issue28647.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2016-11-12 Thread Gareth Rees 

Gareth Rees added the comment:

The output of "python3.5 --help" says:

-u : unbuffered binary stdout and stderr, stdin always buffered;
 also PYTHONUNBUFFERED=x
 see man page for details on internal buffering relating to '-u'

If you look at the man page as instructed then you'll see a clearer
explanation:

-u   Force  the  binary  I/O  layers  of  stdout  and  stderr  to  be
 unbuffered.  stdin is always buffered.  The text I/O layer  will
 still be line-buffered.

For example, if you try this:

python3.5 -uc 'import 
sys,time;w=sys.stdout.buffer.write;w(b"a");time.sleep(1);w(b"b");'

then you'll see that the binary output is indeed unbuffered as
documented.

The output of --help is trying to abbreviate this explanation, but I
think it's abbreviated too much. The explanation from the man page
seems clear to me, and is only a little longer, so I suggest changing
the --help output to match the man page.

--
nosy: +Gareth.Rees

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2016-11-09 Thread STINNER Victor 

STINNER Victor added the comment:

Would it make sense to have two modes: line buferred and unbuffered,
as the C function setvbuf() for stdout?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2016-11-09 Thread Serhiy Storchaka 

Serhiy Storchaka added the comment:

The write_through argument was added only in 3.3. I think this is a good idea 
to pass `write_through = Py_True` when Py_UnbufferedStdioFlag is set. Using -u 
means that performance is not important (that is why this is not default 
behavior).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2016-11-09 Thread STINNER Victor 

STINNER Victor added the comment:

I don't recall, I would have to search in old issues for the rationale. But
the explanation is probably performance, reduce the number of syscalls.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2016-11-09 Thread Eryk Sun 

Eryk Sun added the comment:

> It's not (currently) possible to have a fully unbuffered stdout.

Why doesn't create_stdio also pass `write_through = Py_True` when 
Py_UnbufferedStdioFlag is set? This would immediately pass writes through to 
the FileIO object, even without containing a newline (i.e. it sets 
text_needflush in _io_TextIOWrapper_write_impl).

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2016-11-09 Thread STINNER Victor 

STINNER Victor added the comment:

Right. Moreover, "unbuffered" is wrong. It's line buffered: sys.stdout.buffer 
is directly a io.FileIO object, but TextIOWrapper only calls buffer.write() 
when the message contains a newline character. It's not (currently) possible to 
have a fully unbuffered stdout.

--
nosy: +haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28647] python --help: -u is misdocumented as binary mode

2016-11-09 Thread Armin Rigo 

New submission from Armin Rigo:

``python3.5 --help`` gives this information:

-u : unbuffered binary stdout and stderr, stdin always buffered

However, stdout and stderr are actually always opened in text mode, and print() 
always expects a string and never a bytes object.  This usage of "binary" in 
the --help is in contradiction with the usage of "binary" in the description of 
files (e.g. ``help(open)``).

--
components: Interpreter Core
messages: 280390
nosy: arigo
priority: normal
severity: normal
status: open
title: python --help: -u is misdocumented as binary mode
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com