[issue29779] New environment variable PYTHONHISTORY

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

$HISTFILE is the name of the file in which Bash command history is saved. 
Shouldn't the name of similar environment variable for Python be PYTHONHISTFILE?

Bash uses several environment variables for control its command history: 
HISTCONTROL, HISTFILE, HISTFILESIZE, HISTIGNORE, HISTSIZE, HISTTIMEFORMAT. Does 
Python need corresponding environment variables?

Wouldn't be better to control all these details by Python code in .pythonrc.py 
rather than by environment variables? For example I have the following code in 
my .pythonrc.py:

if sys.version_info < (3,):
try:
import readline, os, atexit
histfile = os.path.join(os.path.expanduser("~"), ".python2_history")
try:
readline.read_history_file(histfile)
except IOError:
pass
atexit.register(readline.write_history_file, histfile)
del os, histfile
except ImportError:
pass

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23894] lib2to3 doesn't recognize rb'...' and f'...' in Python 3.6

[Łukasz Langa]

Changes by Łukasz Langa :


--
pull_requests: +489

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28298] can't set big int-like objects to items in array 'Q', 'L' and 'I'

[Oren Milman]

Oren Milman added the comment:

Thanks for the reviews :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23894] lib2to3 doesn't recognize rb'...' and f'...' in Python 3.6

[Łukasz Langa]

Łukasz Langa added the comment:

I don't think we'll fix it for 3.5 anymore but it's definitely worth doing so 
for 3.6.1.

Ned, I have a patch, will upload momentarily. I'd like this to go in 3.6.1 
since without it fixed tools depending on lib2to3 will crash seeing f-strings. 
This includes YAPF, a pretty widely used auto-formatter for Python code. The 
fix is relatively trivial. What do you think?

--
nosy: +lukasz.langa, ned.deily
title: lib2to3 doesn't recognize rb'...' as a raw byte string in Python 3 -> 
lib2to3 doesn't recognize rb'...' and f'...' in Python 3.6
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23894] lib2to3 doesn't recognize rb'...' and f'...' in Python 3.6

[Łukasz Langa]

Changes by Łukasz Langa :


--
assignee:  -> lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23894] lib2to3 doesn't recognize rb'...' and f'...' in Python 3.6

[Łukasz Langa]

Changes by Łukasz Langa :


--
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29780] Interpreter hang on self._epoll.poll(timeout, max_ev)

[Keyvan Hedayati]

New submission from Keyvan Hedayati:

Hello
We have an issue with our application, it randomly hangs and doesn't respond to 
new requests, at first we thought the problem lies within our code but after 
attaching to hanged process using gdb I couldn't find any code related to our 
application so I thought it might be python bug.
Here is the info extracted from gdb:
https://gist.github.com/k1-hedayati/96e28bf590c4392840650902cb5eceda
Python 3.5.2

We run multiple instances of our application and they are fine for a couple of 
days/hours but suddenly one of them starts hanging and others follow and 
unfortunately we can't reproduce the problem.
I'll be glad to receive any advice on how to solve or debug this issue.

--
components: asyncio
messages: 289344
nosy: gvanrossum, k1.hedayati, yselivanov
priority: normal
severity: normal
status: open
title: Interpreter hang on self._epoll.poll(timeout, max_ev)
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29781] SSLObject.version returns incorrect value before handshake.

[Cory Benfield]

Cory Benfield added the comment:

I updated the test script to try with a file-descriptor set and OpenSSL returns 
TLSv1.2 for that one as well. This strongly suggests that OpenSSL's 
SSL_get_version documentation is somewhat misleading, and that an SSL object 
will return a version even when it's not connected.

If Python wants to consider this a bug, it will need to track connections state 
for the SSLObject like it does for the SSLSocket. Otherwise, Python can 
redocument version for SSLObject to say that it will always return a value.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29782] Use __builtin_clzl for bits_in_digit if available

[Niklas Fiekas]

New submission from Niklas Fiekas:

Baseline performance (9e6ac83acae):

$ ./python -m timeit "12345678 == 12345678.0"
500 loops, best of 5: 40 nsec per loop
$ ./python -m timeit "1 == 1.0"
1000 loops, best of 5: 38.8 nsec per loop
$ ./python -m timeit "(1234578987654321).bit_length()"
1000 loops, best of 5: 39.4 nsec per loop

Upcoming PR:

$ ./python -m timeit "12345678 == 12345678.0"
1000 loops, best of 5: 34.3 nsec per loop
$ ./python -m timeit "1 == 1.0"
1000 loops, best of 5: 34.4 nsec per loop
$ ./python -m timeit "(1234578987654321).bit_length()"
1000 loops, best of 5: 36.4 nsec per loop

--
components: Interpreter Core
messages: 289353
nosy: niklasf
priority: normal
severity: normal
status: open
title: Use __builtin_clzl for bits_in_digit if available
type: performance
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29782] Use __builtin_clzl for bits_in_digit if available

[Niklas Fiekas]

Changes by Niklas Fiekas :


--
pull_requests: +490

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29258] __init__.py required for pkgutil.walk_packages in python3

[Wolfgang Maier]

Wolfgang Maier added the comment:

While it is rather trivial to implement the proposed functionality - all that's 
required here is to eliminate the check for __init__.py from 
pkgutil._iter_file_finder_modules - this would have undesired impacts on, e.g., 
pydoc.apropos:
This function would then recursively report *any* directory/subdirectory on 
sys.path, which is quite surely not what people want.

I think this is a fundamental problem with namespace packages: they are nice 
and flexible for specific imports, but they make it impossible to know whether 
a directory found on the filesystem is *intended* as a Python package or not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29781] SSLObject.version returns incorrect value before handshake.

[Cory Benfield]

New submission from Cory Benfield:

The SSLObject object from the ssl module has a version() method that is 
undocumented. A reasonable assumption for the behaviour of that method is that 
it would follow the behaviour of the same method on SSLSocket(), which has the 
following documentation:

> Return the actual SSL protocol version negotiated by the connection as
> a string, or None is no secure connection is established. As of this
> writing, possible return values include "SSLv2", "SSLv3", "TLSv1",
> "TLSv1.1" and "TLSv1.2". Recent OpenSSL versions may define more return
> values.

However, SSLObject does not follow that behaviour:

Python 3.6.0 (default, Jan 18 2017, 18:08:34) 
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import ssl
>>> ctx = ssl.create_default_context()
>>> in_bio = ssl.MemoryBIO()
>>> out_bio = ssl.MemoryBIO()
>>> buffers = ctx.wrap_bio(in_bio, out_bio)
>>> buffers.version()
'TLSv1.2'

That is, a SSLObject that does not have a TLS session established will 
incorrectly report that it is using a TLS version. This method should return 
None in this case.

--
assignee: christian.heimes
components: SSL
messages: 289346
nosy: Lukasa, christian.heimes
priority: normal
severity: normal
status: open
title: SSLObject.version returns incorrect value before handshake.
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29319] Embedded 3.6.0 distribution cannot run pyz files

[Paul Moore]

Paul Moore added the comment:

Nick (or Steve) can you explain how I'd do that?

I have a git checkout of the 3.6 branch that I can build. But, how do I merge 
in the changes from https://github.com/python/cpython/pull/575? The PR seems to 
be against master, so I'm not sure how to apply it to another branch (I'm not 
even sure how to apply it to my copy of master, TBH!) Whatever I do need to do 
doesn't seem to be in the devguide, although I may be missing it. Maybe there 
should be a section in there on how to do common tasks like this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29258] __init__.py required for pkgutil.walk_packages in python3

[Wolfgang Maier]

Wolfgang Maier added the comment:

> all that's required here is to eliminate the check for __init__.py from 
> pkgutil._iter_file_finder_modules

Ok, I was exaggerating here. To do it right would require a more complex 
change, but that's all that's needed to get an estimate of the effect the real 
thing would have.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29781] SSLObject.version returns incorrect value before handshake.

[Cory Benfield]

Cory Benfield added the comment:

A quick test reveals that Python 3.5 is also affected.

--
versions: +Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29781] SSLObject.version returns incorrect value before handshake.

[Cory Benfield]

Cory Benfield added the comment:

This actually appears to be an outcome of OpenSSL's logic. I've attached a 
smallish C file that, when run against OpenSSL 1.0.2 on my machine, prints 
"TLSv1.2".

This seems like a behaviour we'll have to work around in Python to get the 
outcome we want here.

--
Added file: http://bugs.python.org/file46715/test.c

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29767] build python failed on test_socket due to unused_port is actually used.

[Shuo Li]

Shuo Li added the comment:

The error message is just a "Port already used".

Possible cause 1: find_port default interface is HOST, which is 127.0.0.1. And 
in most test cases, they use 0.0.0.0. So they are on different interface. 

2: system reuse the port, since I build python on a busy server, that is 
possible.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29612] TarFile.extract() suffers from hard links inside tarball

[Thomas Guettler]

Thomas Guettler added the comment:

I have the same issue on Python 2.7.12 (Ubuntu 16.04)

I tried to execute tartest.py. But I could not find a way how to create the tar 
which is needed for tartest.py.

--
nosy: +guettli

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29785] Registration link sent via email by the tracker is http

[Maxime Buquet]

New submission from Maxime Buquet:

The link[1] sent via email by the tracker for registration confirmation is 
http, whereas https is already setup on the tracker itself.

Would it be possible to change it to https.

[1] http://bugs.python.org/?@action=confrego=TOKEN

--
components: Demos and Tools
messages: 289372
nosy: pep.
priority: normal
severity: normal
status: open
title: Registration link sent via email by the tracker is http

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29319] Embedded 3.6.0 distribution cannot run pyz files

[Steve Dower]

Steve Dower added the comment:

That will work fine. Thanks for checking

The process for pulling someone's PR into your own repo is roughly branch then 
pull from the repo sending the PR. Github should show instructions for this 
under m hidden behind a "merge manually" button (though you want to skip the 
final push that would complete the merge, obviously).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29319] Embedded 3.6.0 distribution cannot run pyz files

[Paul Moore]

Paul Moore added the comment:

Thanks for that Steve. I had a recollection that there's a way of referencing 
the PR itself as a branch within the main repo (I guess it must *be* a branch, 
as how otherwise would github be able to do things like get Travis to build 
it?) but I don't recall the details.

Hmm, a bit of googling later I found 
https://help.github.com/articles/checking-out-pull-requests-locally/

git fetch origin pull/ID/head:BRANCHNAME

And indeed that works - I checked out Nick's patch like that. I'd still need to 
merge it into the 3.6 branch, which is another set of git commands I don't yet 
know (cherry-pick, maybe?)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower added the comment:

I thought we'd documented that if you set the path when embedding you should 
also set the program name, but perhaps not (didn't check just now). If not, we 
should do that.

We shouldn't be loading python3.dll anywhere. Are you sure that's in CPython? 
Do you have a reference to the source file?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29777] argparse arguments in main parser hide an argument in subparser

[Eric V. Smith]

Changes by Eric V. Smith :


--
resolution: rejected -> 
stage: resolved -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Alex Gaynor]

Alex Gaynor added the comment:

An FYI for the future, it would have been very helpful if this had been 
documented in the whats-changed file for 3.5.

--
nosy: +alex

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29478] email.policy.Compat32(max_line_length=None) not as documented

[R. David Murray]

R. David Murray added the comment:

Thanks for the PR.  However, rereading this: since compat32 is providing 
backward compatibility with the behavior of the python 3.2 email package, we 
need to check what it would do in this situation before changing the behavior.  
What we may need instead is a doc fix, unfortunately :(.  But a test for this 
is needed either way.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +492

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +493

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29741] BytesIO methods don't accept integer types, while StringIO counterparts do

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I would accept changes to Modules/_io/ because I consider them as code cleanup 
(with little side effect). But I'm not sure about changes to Python 
implementation and tests. Could you create more narrow PR for Modules/_io/ 
changes and left other changes for the consideration of the io module 
maintainers?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[STINNER Victor]

New submission from STINNER Victor:

The codecs.StreamReaderWriter() class still has old unfixed issues like the 
issue #12508 (open since 2011). This issue is even seen as a security 
vulnerability by the owasp-pysec project:
https://github.com/ebranca/owasp-pysec/wiki/Unicode-string-silently-truncated

I propose to modify codecs.open() to reuse the io module: call io.open() with 
newline=''. The io module is now battle-tested and handles well many corner 
cases of incremental codecs with multibyte encodings.

With this change, codecs.open() cannot be used with non-text encodings... but 
I'm not sure that this feature ever worked in Python 3:

$ ./python -bb
Python 3.7.0a0
>>> import codecs
>>> f = codecs.open('test', 'w', encoding='rot13')
>>> f.write('hello')
TypeError: a bytes-like object is required, not 'str'
>>> f.write(b'hello')
TypeError: a bytes-like object is required, not 'dict'

The next step would be to deprecate the codecs.StreamReaderWriter class and the 
codecs.open(). But my latest attempt to deprecate them was the PEP 400 and it 
wasn't a full success, so I now prefer to move step by step :-)

Attached PR:

* Modify codecs.open() to use io.open()
* Remove "; use codecs.open() to handle arbitrary codecs" from io.open() and 
_pyio.open() error messages
* Replace codecs.open() with open() at various places

--
components: Unicode
messages: 289362
nosy: ezio.melotti, haypo, lemburg, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Modify codecs.open() to use the io module instead of 
codecs.StreamReaderWriter()
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 10.03.2017 15:17, STINNER Victor wrote:
> 
> The codecs.StreamReaderWriter() class still has old unfixed issues like the 
> issue #12508 (open since 2011). This issue is even seen as a security 
> vulnerability by the owasp-pysec project:
> https://github.com/ebranca/owasp-pysec/wiki/Unicode-string-silently-truncated

The issue should be fixed. Patches welcome :-)

The reason for the problem is the UTF-8 decoder (and other
decoders) expecting an extension to the codec decoder API,
which are not implemented in its StreamReader class (it simply
uses the base class). It's not a problem of the base class, but
that of the codec.

And no: it doesn't have anything to do with codec.open()
or the StreamReaderWriter class.

> I propose to modify codecs.open() to reuse the io module: call io.open() with 
> newline=''. The io module is now battle-tested and handles well many corner 
> cases of incremental codecs with multibyte encodings.

-1. People who want to use the io module should use it directly.

> With this change, codecs.open() cannot be used with non-text encodings... but 
> I'm not sure that this feature ever worked in Python 3:
> 
> $ ./python -bb
> Python 3.7.0a0
 import codecs
 f = codecs.open('test', 'w', encoding='rot13')
 f.write('hello')
> TypeError: a bytes-like object is required, not 'str'
 f.write(b'hello')
> TypeError: a bytes-like object is required, not 'dict'

That's a bug in the rot13 codec, not a feature. codec.open()
works just find with 'hex' and 'base64'.

> The next step would be to deprecate the codecs.StreamReaderWriter class and 
> the codecs.open(). But my latest attempt to deprecate them was the PEP 400 
> and it wasn't a full success, so I now prefer to move step by step :-)

I'm still -1 on the deprecations in PEP 400. You are essentially
suggesting to replace the complete codecs subsystem with the
io module, but forgetting that all codecs use StreamWriter and
StreamReader as base classes.

StreamReaderWriter is just an amalgamation of the two
classes StreamReader and StreamWriter, nothing more. It's
a completely harmless class in the codecs.py.

The codecs sub system has a clean design. If used correctly
and maintained with more care, it works really well. Trying
to rip things out won't make it better. Fixing implementations,
where the appropriate care was not applied, is a much better
strategy.

I'm tired of having to fight these fights every few years.
Can't we just stop having them, please ?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29786] asyncio.wrap_future() is not documented

[STINNER Victor]

New submission from STINNER Victor:

The following asyncio function is not documented. Is it deliberate? The 
function is exported in the asyncio module.

def wrap_future(future, *, loop=None):
"""Wrap concurrent.futures.Future object."""

--
assignee: docs@python
components: Documentation, asyncio
messages: 289376
nosy: docs@python, gvanrossum, haypo, yselivanov
priority: normal
severity: normal
status: open
title: asyncio.wrap_future() is not documented
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Alex Gaynor]

Alex Gaynor added the comment:

Yeah, this got me (happy to explain what I was trying to do in more detail, if 
it'd be helpful), took me longer to understand why my tests passed on 
{26,27,33,34} but failed on 35 since the public "what's changed" docs page is 
where I went to.

Ultimately I discovered the root cause when I started reading the 
find_library() source code, and found this issue :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

codecs.open() works with bytes-to-bytes codecs.

>>> f = codecs.open('test1', 'w', encoding='hex_codec')
>>> f.write(b'hello')
>>> f.close()
>>> open('test1', 'rb').read()
b'68656c6c6f'

In additional the interface of StreamReaderWriter is not fully compatible with 
the interface of io classes. This would be compatible-breaking change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29784] Erroneous link in shutil.copy description

[Maxime Buquet]

New submission from Maxime Buquet:

https://docs.python.org/3/library/shutil.html#shutil.copy

The link to "copy()" in the description seems to be pointing to the copy 
module, but I suppose it was meant to point at shutil.copy.

--
assignee: docs@python
components: Documentation
messages: 289370
nosy: docs@python, pep.
priority: normal
severity: normal
status: open
title: Erroneous link in shutil.copy description

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[STINNER Victor]

STINNER Victor added the comment:

> codecs.open() works with bytes-to-bytes codecs.

Oh ok. What are non-text encodings? I found:

* base64
* bz2
* hex
* quopri
* rot13
* uu
* zlib

It seems like all these codecs can be used with codecs.open() to write bytes 
strings, except of rot13.

Last time I used these codecs was at least 5 years ago. When I ported code to 
Python 3, I used directly the module implementing the codecs (like base64 
module for base64 codec). It's easy to write code working on Python 2 and 
Python 3 when using directly the module.


> In additional the interface of StreamReaderWriter is not fully compatible 
> with the interface of io classes.

StreamReaderWriter has 3 addition attributes: reader, writer and stream. Do you 
expect that these attributes are used in the wild?

I expect that the most common uses of codecs.open() are to read or write text 
files.

The question is if it is easy to update the code for the rare cases using 
codecs.open() for other purposes. And if it is possible to write code working 
on Python 2.7 and 3.7.


> This would be compatible-breaking change.

It's deliberate, this issue breaks the backward compatibility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I agree that it is better to use directly the module implementing the codecs. 
But old third-party code still can use non-text codecs.

There should be good reasons for breaking backward compatibility. Wouldn't be 
better to deprecate codecs.open()?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23606] ctypes.util.find_library("c") no longer makes sense

[Steve Dower]

Steve Dower added the comment:

Noted.

Did this bite you somehow? Is there something else that should be added/changed?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29765] 2.7.12 compile error from ssl related

[ada]

ada added the comment:

The output is: 

[root@root local]# rpm -qf /usr/local/include/openssl/asn1.h 
file /usr/local/include/openssl/asn1.h is not owned by any package

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28739] PEP 498: docstrings as f-strings

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
pull_requests: +494

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29784] Erroneous link in shutil.copy description

[Xiang Zhang]

Xiang Zhang added the comment:

Thanks for your report, Maxime! I open PRs to fix it.

--
nosy: +xiang.zhang
versions: +Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29784] Erroneous link in shutil.copy description

[Xiang Zhang]

Xiang Zhang added the comment:

Assigned to Mariatta :-).

--
assignee: docs@python -> Mariatta
nosy: +Mariatta

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29784] Erroneous link in shutil.copy description

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

See also issue21818. I suspect there are other incorrect references.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29779] New environment variable PYTHONHISTORY

[Barry A. Warsaw]

Changes by Barry A. Warsaw :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29779] New environment variable PYTHONHISTORY

[Barry A. Warsaw]

Barry A. Warsaw added the comment:

I don't think the Python envar has to follow the contraction from bash.  
$PYTHONHISTORY reads very nicely.

I have similar code in my $PYTHONSTARTUP, but it would be nice to be able to 
get rid of it and just let Python do the common thing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29758] Previously-working SWIG code fails in Python 3.6

[Brett Cannon]

Brett Cannon added the comment:

Please keep this issue closed until you hear back from the SWIG team. Just 
because your code worked under Python 3.5 doesn't mean SWIG didn't accidentally 
emit something that breaks under Python 3.6 because we started being more 
stringent about something. Basically unless we have C code (and not SWIG or C++ 
code) to reproduce this then we have to go on the assumption it's a problem on 
SWIG's side.

--
resolution:  -> third party
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29258] __init__.py required for pkgutil.walk_packages in python3

[Brett Cannon]

Changes by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28739] PEP 498: docstrings as f-strings

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

Thanks for reviewing, Serhiy and Eric. Documentation has been updated and 
backported to 3.6.

OK to close this issue?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28739] PEP 498: docstrings as f-strings

[Eric V. Smith]

Eric V. Smith added the comment:

Yes, I think it can be closed. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29782] Use __builtin_clzl for bits_in_digit if available

[Mark Dickinson]

Mark Dickinson added the comment:

Thanks for the idea, and the PR!

To be useful, this would need a bit of tweaking: we can't assume that a digit 
is always `unsigned long` (in fact, I'd expect that to be rare on 64-bit 
non-Windows systems, where `unsigned long` typically has 64 bits, and `digit` 
should be using a 32-bit type), so we'd need to identify and use the most 
appropriate variant of clz/clzl/clzll somehow.

I think it would also make sense to move the detection of the existence of 
`__builtin_clz` and friends into the configuration machinery: these builtins 
aren't just restricted to GCC (clang supports them as well), and that would 
allow other platforms to provide their own substitutes by modifying `pyport.h`. 
Ideally, the configuration machinery #defines a HAVE_CLZ variable, and then in 
longobject.c we do an #ifdef HAVE_CLZ ...

We also need to trade-off the additional complication in the implementation 
against the benefits: though we do certainly care about the speed, speed at all 
costs isn't the target here; readability, portability and maintainability of 
the source counts, too. It'll probably be easier to weigh those factors once 
there's an implementation that addresses the above points.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29782] Use __builtin_clzl for bits_in_digit if available

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think we can assume that digit is no larger than unsigned long, otherwise 
PyLong_AsLong() and like wouldn't work.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28739] PEP 498: docstrings as f-strings

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[STINNER Victor]

STINNER Victor added the comment:

About the issue #12508, would it be possible to modify StreamReader to use an 
incremental decoder? Or my idea doesn't make sense?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[STINNER Victor]

STINNER Victor added the comment:

Serhiy Storchaka added the comment:
> I agree that it is better to use directly the module implementing the codecs. 
> But old third-party code still can use non-text codecs.
>
> There should be good reasons for breaking backward compatibility. Wouldn't be 
> better to deprecate codecs.open()?

I'm not sure that I understood. Do you consider that using
codecs.open() with a non-text codecs is a "legit" use case or not?

I understood that you suggest a smoother transition using a
deprecation to give more time to developers to update their code.

But what do you want to deprecate? The whole codecs.open() function?
Or using non-text codecs with codecs.open()? Or using text codecs with
codecs.open()?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29786] asyncio.wrap_future() is not documented

[Guido van Rossum]

Changes by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20087] Mismatch between glibc and X11 locale.alias

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I'm feeling there is something wrong with the current locale design. See issues 
issue504219, issue10466, issue20088, issue25191, issue29571.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29788] Add absolute_path option to tarfile, disabled by default

[STINNER Victor]

New submission from STINNER Victor:

I noticed that "python3 -m tarfile -x archive.tar" uses absolute paths by 
default, whereas the UNIX tar command doesn't by default. The UNIX tar command 
requires to add explicitly --absolute-paths (-P) option.

I suggest to add a boolean absolute_path option to tarfile, disabled by default.

Example to create such archive. See that tar also removes "/" by default and 
requires to pass explicitly -P:

$ cd $HOME
# /home/haypo
$ echo TEST > test
$ tar -cf test.tar /home/haypo/test
tar: Removing leading `/' from member names

$ rm -f test.tar
$ tar -P -cf test.tar /home/haypo/test
$ rm -f test


Extracting such archive using tar is safe *by default*:

$ mkdir z
$ cd z
$ tar -xf ~/test.tar
tar: Removing leading `/' from member names
$ find
.
./home
./home/haypo
./home/haypo/test


Extracting such archive using Python is unsafe:

$ python3 -m tarfile -e ~/test.tar
$ cat ~/test
TEST
$ pwd
/home/haypo/z

Python creates files outside the current directory which is unsafe, wheras tar 
doesn't.

--
messages: 289388
nosy: haypo
priority: normal
severity: normal
status: open
title: Add absolute_path option to tarfile, disabled by default
type: security
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29789] zipfile: Add absolute_path option, disabled by default

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Is this issue exists on zipfile?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17062] An os.walk inspired replacement for pkgutil.walk_packages

[Mark Lawrence]

Changes by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29784] Erroneous link in shutil.copy description

[Xiang Zhang]

Changes by Xiang Zhang :


--
pull_requests: +495

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29780] Interpreter hang on self._epoll.poll(timeout, max_ev)

[Guido van Rossum]

Changes by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29723] 3.6.1rc1 adds the current directory to sys.path when running a subdirectory's __main__.py; previous versions did not

[Nick Coghlan]

Nick Coghlan added the comment:

Paul Moore reporting on trying out the new PR in 
http://bugs.python.org/issue29319#msg289357 and confirmed that it still solves 
the originally reported problem in issue 29319.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29478] email.policy.Compat32(max_line_length=None) not as documented

[Mircea Cosbuc]

Mircea Cosbuc added the comment:

Just to be sure, I performed the same operations with my changes in place, 
there's no change in behaviour. I think it's expected since I only modified how 
the Compat32 policy passes `max_line_length` to the header class.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20087] Mismatch between glibc and X11 locale.alias

[Marc-Andre Lemburg]

Marc-Andre Lemburg added the comment:

On 10.03.2017 08:37, Benjamin Peterson wrote:
> 
> Do you believe this program should work?
> 
> import locale, os
> for l in open("/usr/share/i18n/SUPPORTED"):
> alias, encoding = l.strip().split()
> locale.setlocale(locale.LC_ALL, alias)
> try:
> enc = locale.getlocale()[1]
> except ValueError:
> continue # not in table
> normalized = enc.replace("ISO", "ISO-"). \
>  replace("_", "-"). \
>  replace("euc", "EUC-"). \
>  replace("big5", "big5-").upper()
> assert normalized == locale.nl_langinfo(locale.CODESET)
> 
> After my change it does—the encoding returned from getlocale() is the one 
> actually being used by glibc. It fails dramatically on earlier versions of 
> Python (for example on the en_IN example from #29571.) I don't understand why 
> Python needs to editorialize whatever choices libc or the system 
> administrator has made.

Your program essentially tests what alias is configured
on your particular system. It will fail on older systems
(with a different or no version of SUPPORTED), it will fail on
systems that do not have all locales installed, it will
fail on systems that use the X.org aliases table as basis
rather than some list of supported locales of glibc, or
custom alias tables.

What we want in Python is a consistent mapping of aliases to locales
across all (Unix based) Python installations, just like what we
have for encoding aliases and those mappings should be taken
from a support alias database, not a list of default installations
on some glibc version.

Also note that a lot of these discussions are really academic,
since locales should always be specified with encoding.

While Unix gravitates to UTF-8 for all system related things,
users still use other encodings a lot for their daily operations,
as you can see in the X.org aliases file.

This is why defaulting to UTF-8 for locales (as e.g.
is done for many locales in the glibc default installs) is not
a good idea. Locales affect user work products. What's fine for
command line interfacing or piping, is not necessarily for
fine for e.g. documents created by users.

So to answer your question: No, I don't believe that SUPPORTED
has any authority for our purposes and thus don't think that
the program can be considered a valid test case.

The SUPPORTED file can server as extra resource for fixing bugs
in the table, but nothing more.

> Is getlocale() expected to return something different from the underlying C 
> locale?

getlocale() will return whatever is currently configured via
setlocale().

Of course, it can return something different from what some glibc
SUPPORTED lists as default installation encoding, if you don't provide
the encoding when using setlocale(), but it will always default
to the same locale and encoding on all platforms where you
run Python.

> In fact, why have this table at all instead of using nl_langinfo to return 
> the encoding for the current locale?

The table is meant to normalize locale names and enrich
them with default encodings from a well known database of
such aliases, where necessary. As mentioned above the locale setting
should ideally include the encoding as well, so that any such
guesses are not necessary.

Regarding nl_langinfo():

nl_langinfo() will only work if you have called
setlocale() already, since a process always starts up in
the C locale without this call.

If you don't have a problem with calling setlocale() for
testing the default locale settings (e.g. Python is not
embedded, you don't have other threads running, no
APIs which use locale information called yet, setlocale()
was already called to setup the locale, etc.),
you can use the approach taken by getpreferredencoding(),
which is to temporarily set the locale to the default.

Going forward, I think that the following changes make
sense:

* from ISO8859-1 to ISO8859-15 (the -15 version adds
  the Euro sign)

* casing changes e.g. 'zh_CN.gb2312' to 'zh_CN.GB2312'

* fixes which undo removal of modifiers such as
  'uz_uz@cyrillic' -> 'uz_UZ.UTF-8' to 'uz_UZ.UTF-8@cyrillic'

As for the other changes: please undo them and also
revert the unconditional use of glibc mappings overriding
the X.org ones, as mentioned earlier in the thread.

We can readd some of the modifications later on if there's
evidence that they actually do make sense.

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29787] Internal importlib frames visible when module imported by import_module throws exception

[Ulrich Petri]

New submission from Ulrich Petri:

Importing a module that raises an exception on import trough 
`importlib.import_module()` causes importlib to not strip it's internal frames 
from the traceback.


Minimal example:

--a.py--
import importlib

importlib.import_module("b")
--a.py--


--b.py--
raise Exception()
--b.py--

#~ python3.6 a.py
Traceback (most recent call last):
  File "a.py", line 3, in 
importlib.import_module("b")
  File 
"/Users/ulo/.pythonz/pythons/CPython-3.6.0/lib/python3.6/importlib/__init__.py",
 line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
  File "", line 978, in _gcd_import
  File "", line 961, in _find_and_load
  File "", line 950, in _find_and_load_unlocked
  File "", line 655, in _load_unlocked
  File "", line 678, in exec_module
  File "", line 205, in _call_with_frames_removed
  File "/Users/ulo/t/b.py", line 1, in 
raise Exception()
Exception

--
messages: 289381
nosy: ulope
priority: normal
severity: normal
status: open
title: Internal importlib frames visible when module imported by import_module 
throws exception
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16258] test_local.TestEnUSCollection failures on Solaris 10

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

May be issue15954 is related to this issue. Is this issue still reproduced?

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29478] email.policy.Compat32(max_line_length=None) not as documented

[R. David Murray]

R. David Murray added the comment:

So what happens when you do that same operation in 3.5/6 with your change in 
place?  Does the behavior change?  (I haven't looked back at the code to see if 
I think it will :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29788] tarfile: Add absolute_path option to tarfile, disabled by default

[STINNER Victor]

Changes by STINNER Victor :


--
components: +Library (Lib)
title: Add absolute_path option to tarfile, disabled by default -> tarfile: Add 
absolute_path option to tarfile, disabled by default

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[STINNER Victor]

STINNER Victor added the comment:

> The reason for the problem is the UTF-8 decoder (and other
> decoders) expecting an extension to the codec decoder API,
> which are not implemented in its StreamReader class (it simply
> uses the base class). It's not a problem of the base class, but
> that of the codec.
>
> And no: it doesn't have anything to do with codec.open()
> or the StreamReaderWriter class.

open("document.txt", encoding="utf-8") uses IncrementalDecoder of
encodings.utf_8. This object doesn't seem to have the discussed issue.

IMHO the issue is that StreamReader doesn't use an incremental
decoder. I don't see how it could support multibyte encodings and
error handlers without an incremental decoder.

I like TextIOWrapper design between it only handles codecs and text
buffering. Bytes buffering is done at lower-level in a different
object.

I'm not confortable to modify StreamReader because it combines
TextIOWrapper with BufferedReader and so is more complex.

>> I propose to modify codecs.open() to reuse the io module: call io.open() 
>> with newline=''. The io module is now battle-tested and handles well many 
>> corner cases of incremental codecs with multibyte encodings.
>
> -1. People who want to use the io module should use it directly.

When porting code to Python 3, many people chose to use codecs.open()
to get text files using a single code base for Python 2 and Python 3.
Once the code is ported, I don't expect that anyone will replace
codecs.open() with io.open(). You know, nobody cares of the technical
debt...

>> The next step would be to deprecate the codecs.StreamReaderWriter class and 
>> the codecs.open(). But my latest attempt to deprecate them was the PEP 400 
>> and it wasn't a full success, so I now prefer to move step by step :-)
>
> I'm still -1 on the deprecations in PEP 400. You are essentially
> suggesting to replace the complete codecs subsystem with the
> io module, but forgetting that all codecs use StreamWriter and
> StreamReader as base classes.

You can elaborate on "all codecs use StreamWriter and StreamReader as
base classes". Only codecs.open() uses StreamReader and StreamWriter,
no?

All codecs implement a StreamReader and StreamWriter class, but my
question is how use these classes?

> The codecs sub system has a clean design. If used correctly
> and maintained with more care, it works really well.

It seems like we lack such maintainer, since I wrote the PEP, many
issues are still open:

http://bugs.python.org/issue7262
http://bugs.python.org/issue8630
http://bugs.python.org/issue10344
http://bugs.python.org/issue12508
http://bugs.python.org/issue12512

See also issue #5445 (wontfix, whereas TextIOWrapper.writeslines()
uses "for line in lines") and issue #12513 (this one is not fair, io
also has the same bug: issue #12215 :-)).

> I'm tired of having to fight these fights every few years.
> Can't we just stop having them, please ?

The status quo is to do nothing, but as a consequence, bugs are still
not fixed yet, and users are still affected by these bugs :-( I'm
trying to find a solution.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29789] zipfile: Add absolute_path option, disabled by default

[STINNER Victor]

New submission from STINNER Victor:

Same issue than tarfile issue #29788, but on zipfile.

I suggest to add a boolean absolute_path option to zipfile, disabled by default.

--
components: Library (Lib)
messages: 289389
nosy: haypo
priority: normal
severity: normal
status: open
title: zipfile: Add absolute_path option, disabled by default
type: security
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29789] zipfile: Add absolute_path option, disabled by default

[STINNER Victor]

STINNER Victor added the comment:

Oh, it seems like zipfile is not affected by the issue, only tarfile. The 
documentation is correct:
https://docs.python.org/dev/library/zipfile.html#zipfile.ZipFile.extract

You can test using attached test2.zip.

haypo@selma$ python3 -m zipfile -l ~/test2.zip 
File Name Modified Size
/home/haypo/test   2017-03-10 17:29:065

haypo@selma$ cd $HOME
haypo@selma$ rm -rf z/
haypo@selma$ mkdir z
haypo@selma$ cd z
haypo@selma$ python3 -m zipfile -e ~/test2.zip .
haypo@selma$ find
.
./home
./home/haypo
./home/haypo/test
haypo@selma$ cat ~/test
cat: /home/haypo/test: No such file or directory

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
Added file: http://bugs.python.org/file46716/test2.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28685] Optimizing list.sort() by performing safety checks in advance

[Elliot Gorokhovsky]

Elliot Gorokhovsky added the comment:

On Fri, Mar 10, 2017 at 12:26 AM Serhiy Storchaka 
wrote:

>
> Serhiy Storchaka added the comment:
>
> The issue shouldn't be closed until it resolved or rejected.
>

Ya, sorry about that. This is my first time contributing.

>
>
I like the idea, and benchmarking results for randomized lists look nice.
> But could you please run benchmarks for already sorted lists?
>

David Mertz asked for the same thing on python-ideas. Here's what I replied
(you can also find these numbers in my pull request description):

***
You are entirely correct, as the benchmarks below demonstrate. I used the
benchmark lists from Objects/listsort.txt, which are:

  \sort: descending data
  /sort: ascending data
  3sort: ascending, then 3 random exchanges
  +sort: ascending, then 10 random at the end
  %sort: ascending, then randomly replace 1% of elements w/ random
values
  ~sort: many duplicates
  =sort: all equal

My results are below (the script can be found at
https://github.com/embg/python-fastsort-benchmark/blob/master/bench-structured.py
):
Homogeneous ([int]):
\sort: 54.6%
/sort: 56.5%
3sort: 53.5%
+sort: 55.3%
%sort: 52.4%
~sort: 48.0%
=sort: 45.2%

Heterogeneous ([int]*n + [0.0]):
\sort: -17.2%
/sort: -19.8%
3sort: -18.0%
+sort: -18.8%
%sort: -10.0%
~sort: -2.1%
=sort: -21.3%

As you can see, because there's a lot less non-comparison overhead in the
structured lists, the impact of the optimization is much greater, both in
performance gain and in worst-case cost. However, I would argue that these
data do not invalidate the utility of my patch: the probability of
encountering a type-heterogeneous list is certainly less than 5% in
practice. So the expected savings, even for structured lists, is something
like (5%)(-20%) + (95%)(50%) = 46.5%. And, of course, not *all* the lists
one encounters in practice are structured; certainly not *this* structured.
So, overall, I would say the numbers above are extremely encouraging.
Thanks for pointing out the need for this benchmark, though!
***

Thanks for the feedback!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29784] Erroneous link in shutil.copy description

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
pull_requests: +496

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29478] email.policy.Compat32(max_line_length=None) not as documented

[Mircea Cosbuc]

Mircea Cosbuc added the comment:

Thanks for the prompt feedback. In Python 3.2, the closest equivalent for the 
illustrated issue I could find is:

>>> from email.message import Message
>>> from email.generator import Generator
>>> from sys import stdout
>>> m = Message()
>>> m["Field"] = "x" * 100
>>> g0 = Generator(stdout, maxheaderlen=0)
>>> gn = Generator(stdout, maxheaderlen=None)

>>> g0.flatten(m)
Field: 


>>> gn.flatten(m)
Field: 
 



It may be the case that a documentation change is all that is needed. I'm not 
sure that this change would break compatibility since `max_line_length=None` 
for the policy is not the default value. Please advise further if I should just 
update the documentation and modify the test to guard for future changes.

--
nosy: +mcosbuc

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29783] Modify codecs.open() to use the io module instead of codecs.StreamReaderWriter()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think using codecs.open() with a non-text codecs is a legit use case, but is 
not the best way. I suggest:

1) Encourage of using io.open() rather than codecs.open() for text encodings. 
codecs.open() was recommended way since it worked in all Python versions, 
including <2.6, but now we can ignore this advantage.

2) Discourage of using non-text codecs.

3) Deprecate codecs.open() (may be after EOL for Python 2.7).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29786] asyncio.wrap_future() is not documented

[Martin Panter]

Changes by Martin Panter :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> asyncio.wrap_future undocumented

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29788] tarfile: Add absolute_path option to tarfile, disabled by default

[Ned Deily]

Changes by Ned Deily :


--
nosy: +lars.gustaebel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23407] os.walk always follows Windows junctions

[Craig Holmquist]

Changes by Craig Holmquist :


Added file: http://bugs.python.org/file46718/issue23407-5.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28810] Document bytecode changes in 3.6

[Brett Cannon]

Changes by Brett Cannon :


--
pull_requests: +500

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29741] BytesIO methods don't accept integer types, while StringIO counterparts do

[Oren Milman]

Oren Milman added the comment:

done

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23894] lib2to3 doesn't recognize rb'...' and f'...' in Python 3.6

[Ned Deily]

Ned Deily added the comment:

Sorry, the announced deadline for bugfixes for 3.6.1 has already passed; at 
this point, only showstopper problems with 3.6.1rc1 are on the table for the 
final.  Trivial impact isn't a criterion for post-rc1 changes and there's a 
good reason for that: we ask everyone in the community to test a release 
candidate with the expectation that this is the final release.  I don't think 
it's fair to add more unrelated changes and risk that it will break something 
and/or cause additional release candidates to be produced.  The original 
problem has been open for nearly two years now and, while "f" string support 
adds to the importance of the tokenizer getting updated, this can wait 3 more 
months for 3.6.2 with a proper review cycle and with tests.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29746] Update marshal docs to Python 3

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +Mariatta, martin.panter, r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Steve Dower added the comment:

Ah, I see. We force load it in PC/getpathp.c to ensure that it's ours and not 
another version's python3.dll.

We should probably refactor the GetModuleFileNameW call into its own function 
so we can call it from anywhere we need.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29688] Document Path.absolute

[Brett Cannon]

Brett Cannon added the comment:

I'm still thinking about this but I'm still leaning towards deprecating 
pathlib.absolute().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29741] BytesIO methods don't accept integer types, while StringIO counterparts do

[Oren Milman]

Changes by Oren Milman :


--
pull_requests: +501

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29741] BytesIO methods don't accept integer types, while StringIO counterparts do

[Oren Milman]

Oren Milman added the comment:

thanks :)
I would update the original PR soon.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28810] Document bytecode changes in 3.6

[Brett Cannon]

Changes by Brett Cannon :


--
pull_requests: +502

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29760] tarfile chokes on reading .tar file with no entries (but does fine if the same file is bzip2'ed)

[Matt Bogosian]

Changes by Matt Bogosian :


Removed file: http://bugs.python.org/file46717/tarfile.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24755] asyncio.wrap_future undocumented

[Martin Panter]

Changes by Martin Panter :


--
components: +asyncio
stage:  -> patch review
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29778] _Py_CheckPython3 uses uninitialized dllpath when embedder sets module path with Py_SetPath

[Steve Dower]

Changes by Steve Dower :


--
stage:  -> needs patch
type:  -> behavior
versions: +Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29760] tarfile chokes on reading .tar file with no entries (but does fine if the same file is bzip2'ed)

[Matt Bogosian]

Matt Bogosian added the comment:

After some consideration, I think this is probably more correct:

```
--- /Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/tarfile.py  
2016-12-17 12:41:21.0 -0800
+++ tarfile.py  2017-03-10 14:57:15.0 -0800
@@ -2347,9 +2347,10 @@

 # Advance the file pointer.
 if self.offset != self.fileobj.tell():
-self.fileobj.seek(self.offset - 1)
+self.fileobj.seek(max(self.offset - 1, 0))
 if not self.fileobj.read(1):
 raise ReadError("unexpected end of data")
+self.fileobj.seek(self.offset)

 # Read the next block.
 tarinfo = None
```

But again, I'm no expert here.

--
Added file: http://bugs.python.org/file46719/tarfile.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29790] Optional use of /dev/random on linux

[Steven D'Aprano]

Changes by Steven D'Aprano :


--
nosy: +haypo, ncoghlan, steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29786] asyncio.wrap_future() is not documented

[Adam Höse]

Adam Höse added the comment:

While fixing this issue I found that it's a duplicate of issue 24755.

--
nosy: +adisbladis
pull_requests: +498
type:  -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29782] Use __builtin_clzl for bits_in_digit if available

[Mark Dickinson]

Mark Dickinson added the comment:

True enough. It looks as though someone (*cough*) already documented that 
restriction, too: 
https://github.com/mdickinson/cpython/blob/v3.6.0/Include/longintrepr.h#L28-L30

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29765] 2.7.12 compile error from ssl related

[Christian Heimes]

Christian Heimes added the comment:

You have some custom, unsupported version of OpenSSL installed in /usr/local. 
That custom version is not compatible with Python 2.7.12.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29722] heapq.merge docs are misleading with the "reversed" flag

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
versions:  -Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29760] tarfile chokes on reading .tar file with no entries (but does fine if the same file is bzip2'ed)

[Matt Bogosian]

Matt Bogosian added the comment:

This patch (also attached) seems to address this particular use case:

```
--- a/Lib/tarfile.py2016-12-17 12:41:21.0 -0800
+++ b/Lib/tarfile.py2017-03-10 12:23:34.0 -0800
@@ -2347,7 +2347,7 @@
 
 # Advance the file pointer.
 if self.offset != self.fileobj.tell():
-self.fileobj.seek(self.offset - 1)
+self.fileobj.seek(max(self.offset - 1, 0))
 if not self.fileobj.read(1):
 raise ReadError("unexpected end of data")
 
```

However, I am unfamiliar with the code, especially in light of #24259, and 
haven't tested it thoroughly. Oversight is needed.

--
keywords: +patch
Added file: http://bugs.python.org/file46717/tarfile.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29790] Optional use of /dev/random on linux

[Ivan Anishchuk]

New submission from Ivan Anishchuk:

Right now secrets module uses SystemRandom which is hardcoded to use 
os.urandom() which is fine for most users but some have good hardware sources 
of entropy (or otherwise replenish entropy pool) in which case it would be much 
better to use getrandom() with GRND_RANDOM flag i.e. to read from /dev/random 
pool.

Simply subclassing SystemRandom is not enough, the idea is to make it possible 
for every library and program to use the big entropy pool if it's available. So 
I'm thinking it would be best to configure it with an environment variable, 
something like PYTHONTRUERANDOM or PYTHONDEVRANDOM.

Admittedly, only a small subset of users would benefit from this but changes 
required are also small and I'm willing to do all the work here. Are there any 
reason this patch won't be accepted? Any preferences regarding variable name?

--
components: Library (Lib)
messages: 289410
nosy: IvanAnishchuk
priority: normal
severity: normal
status: open
title: Optional use of /dev/random on linux
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29791] print documentation: flush is also a keyword argument

[Lucio Ricardo Montero Valenzuela]

New submission from Lucio Ricardo Montero Valenzuela:

In the print() function documentation 
(https://docs.python.org/3/library/functions.html#print), the first line says 
"Print objects to the text stream file, separated by sep and followed by end. 
sep, end and file, if present, must be given as keyword arguments.", but the 
function definition is said to be "print(*objects, sep=' ', end='\n', 
file=sys.stdout, flush=False)". Based on the Python user function definition 
grammar, the only way of passing an value to a non-star parameters that appear 
after a star-parameter is with the keyword (so the interpreter knows not to 
push the value into the star-parameter list/mapping). So the flush parameter 
can be set only naming explicitly the keyword 'flush' ¿Isn't it?. So the first 
line of the print() function documentation should say "Print objects to the 
text stream file, separated by sep and followed by end. sep, end, file and 
flush, if present, must be given as keyword arguments.". Flush is a new 
parameter, so maybe you forgot to update this line of the documentation to 
include it.
Best regards.

--
assignee: docs@python
components: Documentation
messages: 289411
nosy: Lucio Ricardo Montero Valenzuela, docs@python
priority: normal
severity: normal
status: open
title: print documentation: flush is also a keyword argument
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29782] Use __builtin_clzl for bits_in_digit if available

[Niklas Fiekas]

Niklas Fiekas added the comment:

Oops. Actually clz should commonly be enough. And platforms where clz and clzl 
are different (<=> unsigned int and unsigned long are different) should be rare.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24755] asyncio.wrap_future undocumented

[Adam Höse]

Changes by Adam Höse :


--
pull_requests: +497

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com