[issue24928] mock.patch.dict spoils order of items in collections.OrderedDict

2018-12-20 Thread Karthikeyan Singaravelan


Karthikeyan Singaravelan  added the comment:

Thinking about it further the attached test is based on the ordering of dict 
and hence works only on 3.6 and above. But this test will be backported to mock 
on PyPI where this will fail on 2.7 and 3.4, 3.5. Is it okay to apply the fix 
that makes backporting easy though it has no effect on current CPython 
implementation or to only apply the test since it's fixed in CPython due to 
dict implementation detail and make sure the cabal backporting is done with 
this detail taken into account to run test only on 3.6+. But the latter makes 
the issue still prevalent in cabal's mock repo.

I think it's better to apply the fix along with the test that will make it easy 
on both ends. I am not aware of the backport process for mock on PyPI so 
feedback will be helpful on this.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21478] mock calls don't propagate to parent (autospec)

2018-12-20 Thread Karthikeyan Singaravelan


Change by Karthikeyan Singaravelan :


--
keywords: +patch
pull_requests: +10508
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22703] Idle Code Context menu entrie(s)

2018-12-20 Thread Terry J. Reedy


Terry J. Reedy  added the comment:

This is a dependency of #33610.

--
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35259] Py_FinalizeEx unconditionally exists in Py_LIMITED_API

2018-12-20 Thread Serhiy Storchaka


Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35548] memoryview needlessly (?) requires represented object to be hashable

2018-12-20 Thread Ilya Kulakov

New submission from Ilya Kulakov :

Implementation of memoryview's hashing method [1] imposes the following 
constraints in order to be hashable (per documentation):

> One-dimensional memoryviews of hashable (read-only) types with formats ‘B’, 
> ‘b’ or ‘c’ are also hashable. The hash is defined as hash(m) == 
> hash(m.tobytes()):

However it's not clear why original type needs to be hashable given that 
memoryview deals with 1-dimensional read-only bytes representation of an 
object. Not only it requires the developer to make an extra copy of C-bytes, 
but also calls __hash__ of a represented object without using the result other 
than to detect an error.

My particular use case involves a memory view of a readonly numpy's ndarray. My 
view satisfies the following constraints:

>>> print(data.format, data.readonly, data.shape, data.c_contiguous)
b True (25,) True

But nevertheless the hashing fails because ndarray itself is not hashable.

Stefan Krah wrote [2]:

> Note that memory_hash() raises an error if the exporter *itself* is
not hashable, so it only hashes immutable objects by design.

But while __hash__ indeed tells that object is (supposed to be) immutable, 
there is no requirement for all immutable objects to have __hash__.

Both threads I have found ([3], [4]) are quite lengthy and show certain discord 
in opinions regarding the issue. Perhaps after 6 years since the release of the 
feature the view on the problem has changed?

1: 
https://github.com/python/cpython/blob/d1e717588728a23d576c4ead775f7dbd68149696/Objects/memoryobject.c#L2829-L2876
2: https://bugs.python.org/issue15814#msg169510
3: https://bugs.python.org/issue15573
4: https://bugs.python.org/issue15573

--
components: Library (Lib)
messages: 332280
nosy: Kentzo, skrah
priority: normal
severity: normal
status: open
title: memoryview needlessly (?) requires represented object to be hashable
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35547] email.parser / email.policy does correctly handle multiple RFC2047 encoded-word tokens across RFC5322 folded headers

2018-12-20 Thread era


era  added the comment:

I don't think this is a bug. My impression is that encoded words should be 
decodable in isolation.

--
nosy: +era

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35208] IDLE: Squeezed lines count ignores window width

2018-12-20 Thread Terry J. Reedy


Terry J. Reedy  added the comment:

Tal, trying to understand your confused description of what behavior you want 
to fix required me to experiment and think.  There are at least 2 separate 
issues: triggering of auto-squeeze and lines reported (regardless of what 
triggers squeezing).  The following pair of experiments exhibits inconsistency 
in both respects.

>>> print('a'*3920) # Fills 49 80-char lines, correctly not squeezed.
...
>>> print('a'*3921)  # Wrapped to 50 lines, correctly auto squeezed.
[Squeezed text (50 lines).]  # Correct number when reporting wrapped lines.
>>> print('a'*3921+'\n')  # Ditto, but not auto-squeezed.
...
# Squeeze manually
[Squeezed text (1 line).]  # Different line count -- of output lines.
>>> print('a'*3920+'\na')  # Not initially squeezed, '2 lines'.

>From msg331784 it appears that you are more concerned here with auto squeeze 
>triggering than with line count.  Now that I think I know what you are trying 
>to fix, I can review the code change.

I agree to consider the ambiguity between output lines and display lines, and 
the effect on line count, later.

Part of my thinking with the simple auto-squeeze formula, besides just 
simplifying the code, it this.  Raymond claimed that squeezing slows down 
printing.  If measurably true, one way to avoid a slow down would be to use a 
simple heuristic formula to estimate the number of wrapped lines instead of 
exactly counting.  This would be a separate issue, if needed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35547] email.parser / email.policy does correctly handle multiple RFC2047 encoded-word tokens across RFC5322 folded headers

2018-12-20 Thread Martijn Pieters


Martijn Pieters  added the comment:

That regex is incorrect, I should not post untested code from a mobile phone. 
Corrected workaround with more context:

import re
from email.policy import EmailPolicy

class UnfoldingEncodedStringHeaderPolicy(EmailPolicy):
def header_fetch_parse(self, name, value):
# remove any leading whitespace from header lines
# that separates apparent encoded-word token before further processing 
# using somewhat crude CRLF-FWS-between-encoded-word matching
value = re.sub(r'(?<=\?=)((?:\r\n|[\r\n])[\t ]+)(?==\?)', '', value)
return super().header_fetch_parse(name, value)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35547] email.parser / email.policy does correctly handle multiple RFC2047 encoded-word tokens across RFC5322 folded headers

2018-12-20 Thread Martijn Pieters


Martijn Pieters  added the comment:

Right, re-educating myself on the MIME RFCs, and found 
https://bugs.python.org/issue1372770 where the same issue is being discussed 
for previous incarnations of the email library.

Removing the FWS after CRLF is the wrong thing to do, **unless** RFC2047 
separating encoded-word tokens. The work-around regex is a bit more 
complicated, but ideally the EW handling should use a specialist FWS token to 
delimit encoded-word sections that renders to '' as is done in unstructured 
headers, but everywhere. Because in practice, there are email clients out there 
that use EW in structured headers, regardless. 

Regex to work around this 

# crude CRLF-FWS-between-encoded-word matching
value = re.sub(r'(?<=\?=(\r\n|\n|\r))([\t ]+)(?==\?)', '', value)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35547] email.parser / email.policy does correctly handle multiple RFC2047 encoded-word tokens across RFC5322 folded headers

2018-12-20 Thread Martijn Pieters


Change by Martijn Pieters :


--
components: +email
nosy: +barry, r.david.murray
type:  -> behavior
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35545] asyncio.base_events.create_connection doesn't handle scoped IPv6 addresses

2018-12-20 Thread Erwan Le Pape


Change by Erwan Le Pape :


--
keywords: +patch
pull_requests: +10507
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35545] asyncio.base_events.create_connection doesn't handle scoped IPv6 addresses

2018-12-20 Thread Erwan Le Pape


Erwan Le Pape  added the comment:

While the 3.7+ getaddrinfo isn't the best human representation of an IPv6 
address, I believe it does make the most sense to keep it that way.
In any case, this is a regression and changing return values of getaddrinfo for 
3.7 isn't something that should be considered.

The issue stems from the refactoring of the underlying socketmodule.c handling 
of IPv4/IPv6 addresses with dedicated make_ipv4_addr and make_ipv6_addr 
functions which returns proper tuples of:
  str/int for IPv4: 
https://github.com/python/cpython/blob/master/Modules/socketmodule.c#L1270
  str/int/int/int for IPv6: 
https://github.com/python/cpython/blob/master/Modules/socketmodule.c#L1325

The actual issue is that _ensure_resolved naively assumes IPv4 and truncates 
the address to its first 2 members
https://github.com/python/cpython/blob/3.7/Lib/asyncio/base_events.py#L1269
and never redefines them again in the case where they were set.

I'd suggest passing the remaining elements of address in a packed *args or an 
optional flowinfo=0, scopeid=0 to _ipaddr_info since fundamentally, that's the 
method stripping valuable information.

--
nosy: +lepaperwan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35485] Mac: tkinter windows turn black while resized

2018-12-20 Thread Kevin Walzer


Kevin Walzer  added the comment:

http://core.tcl.tk/tk/tktview?name=ef9c3730e3 has some useful information on 
this from the Tk side.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34745] asyncio ssl memory leak

2018-12-20 Thread Alexander Mohr


Change by Alexander Mohr :


--
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34745] asyncio ssl memory leak

2018-12-20 Thread Chase N Peterson


Chase N Peterson  added the comment:

I ran the code snippet below using uvloop/master in a docker container. As it 
ran, the container continually leaked memory. I included a graph with the 
memory usage.

Environment:
# cat /etc/*-release
PRETTY_NAME="Debian GNU/Linux 9 (stretch)"
NAME="Debian GNU/Linux"
VERSION_ID="9"
VERSION="9 (stretch)"
ID=debian
HOME_URL="https://www.debian.org/;
SUPPORT_URL="https://www.debian.org/support;
BUG_REPORT_URL="https://bugs.debian.org/;

# uname -r
4.18.16-300.fc29.x86_64

# python -V
Python 3.7.1

# pip freeze
asyncio==3.4.3
Cython==0.29.2
idna==2.8
multidict==4.5.2
uvloop==0.12.0rc2
yarl==1.3.0

I had to tweak the code a bit to run in a docker container successfully, but 
here is the code I used:

import asyncio
import logging
import ssl
import socket
import sys
import yarl
import uvloop

CREDENTIAL = ''

URLS = {
'https://s3.us-west-2.amazonaws.com/archpi.dabase.com/style.css': {
'method': 'get',
'headers': {'User-Agent': 'Botocore/1.8.21 Python/3.6.4 Darwin/17.5.0', 
'X-Amz-Date': '20180518T025044Z', 'X-Amz-Content-SHA256': 
'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 
'Authorization': f'AWS4-HMAC-SHA256 
Credential={CREDENTIAL}/20180518/us-west-2/s3/aws4_request, 
SignedHeaders=host;x-amz-content-sha256;x-amz-date, 
Signature=ae552641b9aa9a7a267fcb4e36960cd5863e55d91c9b45fd39b30fdcd2e81489', 
'Accept-Encoding': 'identity'}
},

'https://s3.ap-southeast-1.amazonaws.com/archpi.dabase.com/doesnotexist': {
'method': 'GET' if sys.argv[1] == 'get_object' else 'HEAD',
'headers': {'User-Agent': 'Botocore/1.8.21 Python/3.6.4 Darwin/17.5.0', 
'X-Amz-Date': '20180518T025221Z', 'X-Amz-Content-SHA256': 
'e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855', 
'Authorization': f'AWS4-HMAC-SHA256 
Credential={CREDENTIAL}/20180518/ap-southeast-1/s3/aws4_request, 
SignedHeaders=host;x-amz-content-sha256;x-amz-date, 
Signature=7a7675ef6d70cb647ed59e02d532ffa80d437fb03976d8246ea9ef102d118794', 
'Accept-Encoding': 'identity'}
}
}

asyncio.set_event_loop_policy(uvloop.EventLoopPolicy())

class HttpClient(asyncio.streams.FlowControlMixin):
transport = None

def __init__(self, *args, **kwargs):
self.__url = kwargs.pop('url')
self.__logger = logging.getLogger()
super().__init__()

def connection_made(self, transport):
self.transport = transport

url_parts = yarl.URL(self.__url)
entry = URLS[self.__url]

body = f'{entry["method"]} {url_parts.path} HTTP/1.1\r\nAccept: 
*/*\r\nHost: {url_parts.host}\r\n'
for name, value in entry['headers'].items():
body += f'{name}: {value}\r\n'

body += '\r\n'
self.transport.write(body.encode('ascii'))
self.__logger.info(f'data sent: {body}')

def data_received(self, data):
self.__logger.info(f'data received: {data}')

self.transport.close()
# asyncio.get_event_loop().call_later(1.0, )

def eof_received(self):
self.__logger.info('eof_received')

def connection_lost(self, exc):
self.__logger.info(f'connection lost: {exc}')
super().connection_lost(exc)

@classmethod
def create_factory(cls, url: str):
def factory(*args, **kwargs):
return cls(*args, url=url, **kwargs)

return factory


async def test_asyncio(ssl_context):
loop = asyncio.get_event_loop()

url = 
'https://s3.ap-southeast-1.amazonaws.com/archpi.dabase.com/doesnotexist'
url_parts = yarl.URL(url)
port = url_parts.port or (80 if url_parts.scheme == 'http' else 443)
infos = await loop.getaddrinfo(url_parts.host, port, family=socket.AF_INET)
family, type, proto, canonname, sockaddr = infos[0]
await loop.create_connection(HttpClient.create_factory(url), sockaddr[0], 
port, ssl=ssl_context, family=family, proto=proto, flags=socket.AI_NUMERICHOST, 
server_hostname=url_parts.host, local_addr=None)


async def asyncio_test():
ssl_context = ssl.create_default_context()

while True:
await test_asyncio(ssl_context)


def main():
print('running')
loop = asyncio.get_event_loop()
loop.run_until_complete(asyncio_test())


main()

--
nosy: +cnpeterson
Added file: https://bugs.python.org/file48012/memory_usage.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

> I am going to merge the PR tomorrow, except if someone sees a good reason to 
> not merge it.

I changed my mind and I decided to wait until I'm back from holiday instead :-)
https://github.com/python/cpython/pull/11242#issuecomment-449151524

The function is more tricky than what I expected!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27772] Refer to actual format string when creating “zero padding” error message

2018-12-20 Thread Serhiy Storchaka


Change by Serhiy Storchaka :


--
components: +Interpreter Core
versions: +Python 3.8 -Python 2.7, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27772] Refer to actual format string when creating “zero padding” error message

2018-12-20 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:

PR 11270 fixes this issue by making such format valid. Preceding the width 
field by '0' no longer affects the default alignment for strings, i.e. no 
longer sets the alignment to invalid '=' if it is not specified explicitly.

>>> format('abc', '<8')
'abc '
>>> format('abc', '8')
'abc '
>>> format('abc', '<08')
'abc0'
>>> format('abc', '08')
'abc0'

This does not contradict the documentation, which specifies this case for 
numeric types, but not for strings.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue27772] Refer to actual format string when creating “zero padding” error message

2018-12-20 Thread Serhiy Storchaka


Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +10506
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35485] Mac: tkinter windows turn black while resized

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:

> The point of the minimal code snippet is that the behavior is tk/tkinter 
> generic

Ah!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

Options that should be easy to support later:

* relative executable: use shutil.which() or expose C posix_spawnp() function 
as os.posix_spawnp()

* env: just pass it as the 3rd argument of os.posix_spawn() (trivial patch)

* restore_signals: use setsigdef (Joannah has a patch)


I don't see how to support following options:

* preexec_fn: I really hate this feature...

* cwd

* start_new_session

* close_fds: there is posix_spawn_file_actions_addclose(), but I'm not sure 
that we can create a list of file descriptor in a reliable way, since a 
different thread can open a FD in the meanwhile... Currently, _posixsubprocess 
iterates on /proc/self/fd/ *after* the fork, when there is a single thread.

* pass_fds: there is not API to mark a fd as inheritable (clear O_CLOEXEC 
flag). We cannot just change the O_CLOEXEC temporarily since a different thread 
can spawn a child process "at the same time". The GIL doesn't protect C threads 
which don't use the GIL.


For pipes like stdout=PIPE or stdout=fd, I didn't look yet how to implement 
that properly.


Maybe one way to work around the pass_fds limitation in an applicaton is to 
mark the FDs to pass as inheritable (os.set_inheritable(fd, True)) and use 
close_fds=False:

fd = ...
subprocess.Popen(..., pass_fds={fd})
os.close(fd)

would become:

fd = ...
os.set_inheritable(fd, True)
subprocess.Popen(..., close_fds=False)
os.close(fd)

But this pattern is not thread if the application has other threads. So that 
should be the responsibility of the developer, not of Python, to write "unsafe" 
code" which requires the knownledge of the application behavior.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35485] Mac: tkinter windows turn black while resized

2018-12-20 Thread Terry J. Reedy


Terry J. Reedy  added the comment:

The point of the minimal code snippet is that the behavior is tk/tkinter 
generic and not specific to Texts or IDLE or particular non-default parameters 
or geometry.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35259] Py_FinalizeEx unconditionally exists in Py_LIMITED_API

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:


New changeset 5241ecff161ccf34083b6a824d1ae6d79917d889 by Ned Deily (Miss 
Islington (bot)) in branch '3.6':
bpo-35259: Limit `Py_FinalizeEx()` to `Py_LIMITED_API >= 0x0306`. 
(GH-10620) (GH-11269)
https://github.com/python/cpython/commit/5241ecff161ccf34083b6a824d1ae6d79917d889


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

I wasn't sure how posix_spawn() handles file descriptors of standard streams 
with O_CLOEXEC flag set.

I wrote a test. Result: _posixsubprocess and os.posix_spawn() have the same 
behavior! If fd 2 (stderr) is marked with O_CLOEXEC, the fd 2 is closed in the 
child process, as expected. (There is no black magic to keep it open.)

$ cat x.py 
import subprocess, sys, os
args = [sys.executable, 'y.py']
os.set_inheritable(2, False)
subprocess.run(args, close_fds=False, restore_signals=False)

$ cat y.py 
import os
def fd_valid(fd):
try:
os.fstat(fd)
return True
except OSError:
return False
for fd in (0, 1, 2):
print("fd %s valid? %s" % (fd, fd_valid(fd)))

$ python3 x.py  # unpatched
fd 0 valid? True
fd 1 valid? True
fd 2 valid? False

$ ./python x.py # patched, use posix_spawn()
fd 0 valid? True
fd 1 valid? True
fd 2 valid? False

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35546] String formatting produces incorrect result with left-aligned zero-padded format

2018-12-20 Thread Eric V. Smith


Eric V. Smith  added the comment:

I think this falls in to the consenting adults category. You can also do things 
like:
>>> format(42, '->3')
'-42'

But why bother preventing this?

It is unfortunate that %-formatting and .__format__() are different in this 
regard, but at this point I wouldn't recommend making any changes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35259] Py_FinalizeEx unconditionally exists in Py_LIMITED_API

2018-12-20 Thread miss-islington


miss-islington  added the comment:


New changeset d1e717588728a23d576c4ead775f7dbd68149696 by Miss Islington (bot) 
in branch '3.7':
bpo-35259: Limit `Py_FinalizeEx()` to `Py_LIMITED_API >= 0x0306`. (GH-10620)
https://github.com/python/cpython/commit/d1e717588728a23d576c4ead775f7dbd68149696


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue34686] Add `-r`, as opposed to `-R` to Python core interpreter

2018-12-20 Thread Erwan Le Pape


Erwan Le Pape  added the comment:

As mentioned by Benjamin Peterson, this is a duplicate of #34722, which already 
has an implementation proposal by Peter Ebden.

While implementation specifics change, the aim is the same and #34722 is the 
better solution (again, credit to Benjamin Peterson).

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35461] Document C API functions which swallow exceptions

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:


New changeset ff740db42326082fac7d415ae9aff148628a83ed by Ned Deily (Miss 
Islington (bot)) in branch '3.6':
bpo-35461: Document C API functions which suppress exceptions. (GH-9) 
(GH-11210)
https://github.com/python/cpython/commit/ff740db42326082fac7d415ae9aff148628a83ed


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35360] Update SQLite to 3.26 in Windows and macOS installer builds

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:

We should look at doing this for the next set of maintenance updates.  This 
doesn't appear to be critical enough to delay current releases unless someone 
can show how this exploit could be used in a typical Python application.

--
components: +macOS
nosy: +ned.deily, ronaldoussoren
priority: normal -> high
title: [Windows] Update SQLite dependency -> Update SQLite to 3.26 in Windows 
and macOS installer builds
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35259] Py_FinalizeEx unconditionally exists in Py_LIMITED_API

2018-12-20 Thread miss-islington


Change by miss-islington :


--
pull_requests: +10504

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35259] Py_FinalizeEx unconditionally exists in Py_LIMITED_API

2018-12-20 Thread miss-islington


Change by miss-islington :


--
pull_requests: +10505

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35259] Py_FinalizeEx unconditionally exists in Py_LIMITED_API

2018-12-20 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:


New changeset 3e8f962e63c2f929604443531a9a3aced242f3e8 by Serhiy Storchaka 
(Arthur Neufeld) in branch 'master':
bpo-35259: Limit `Py_FinalizeEx()` to `Py_LIMITED_API >= 0x0306`. (GH-10620)
https://github.com/python/cpython/commit/3e8f962e63c2f929604443531a9a3aced242f3e8


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

Just in case, I checked again 3.6 and 3.7 branches with "./configure CC=clang 
--with-lto --enable-shared": they still pass my manual tests ;-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35475] Docs do not show PyImport_AddModuleObject() returns a borrowed reference.

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:


New changeset 75f187478603de33c15f501a947207bfe8ba833f by Ned Deily (Miss 
Islington (bot)) in branch '3.6':
bpo-35475: Add more PyImport* functions in refcounts.dat. (GH-11142) (GH-11199)
https://github.com/python/cpython/commit/75f187478603de33c15f501a947207bfe8ba833f


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35482] can't open python368rc1.chm and python372rc1.chm

2018-12-20 Thread Steve Dower


Change by Steve Dower :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:

Also cherry-picked to 3.6 in commit a21bedf9edeacce6f0de3b2df0783e2ad6aa3b18
[3.6] bpo-35257: Avoid leaking LTO linker flags into distutils (GH-10900) 
(GH-11265)

--
priority: release blocker -> 
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35499] "make profile-opt" overrides CFLAGS_NODIST

2018-12-20 Thread Ned Deily


Change by Ned Deily :


--
pull_requests:  -10503

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35499] "make profile-opt" overrides CFLAGS_NODIST

2018-12-20 Thread STINNER Victor


Change by STINNER Victor :


--
pull_requests: +10503

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31715] Add mimetype for extension .mjs

2018-12-20 Thread Ned Deily


Change by Ned Deily :


--
versions: +Python 2.7, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31715] Add mimetype for extension .mjs

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:


New changeset d9a2aca839b1326e011ecca17ba7b45123c47a64 by Ned Deily (Myles 
Borins) in branch '2.7':
[2.7] bpo-31715 Add mimetype for extension .mjs (GH-3908) (GH-10978)
https://github.com/python/cpython/commit/d9a2aca839b1326e011ecca17ba7b45123c47a64


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31715] Add mimetype for extension .mjs

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:


New changeset 70db385944ecf2ceed10ed7d91fce68399f9ca8a by Ned Deily (Myles 
Borins) in branch '3.6':
[3.6] bpo-31715 Add mimetype for extension .mjs (GH-3908) (GH-10976)
https://github.com/python/cpython/commit/70db385944ecf2ceed10ed7d91fce68399f9ca8a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue31715] Add mimetype for extension .mjs

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:


New changeset c7b7a43cd8964a90669bb37735cdafc5b0ec60cc by Ned Deily (Myles 
Borins) in branch '3.7':
[3.7] bpo-31715 Add mimetype for extension .mjs (GH-3908) (GH-10977)
https://github.com/python/cpython/commit/c7b7a43cd8964a90669bb37735cdafc5b0ec60cc


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22166] test_codecs leaks references

2018-12-20 Thread Brett Cannon


Change by Brett Cannon :


--
nosy:  -brett.cannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35499] "make profile-opt" overrides CFLAGS_NODIST

2018-12-20 Thread Ned Deily


Change by Ned Deily :


--
pull_requests:  -10497

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue21478] mock calls don't propagate to parent (autospec)

2018-12-20 Thread Karthikeyan Singaravelan


Karthikeyan Singaravelan  added the comment:

I think this should be handled in _check_and_set_parent where if value's type 
is FunctionType then value.mock should be used against which parent and name 
should be set since create_autospec returns function with mock attached to 
'mock' attribute and all helper methods attached. There could be a case where a 
function is directly attached to mock where value will be of FunctionType but 
value.mock will not be set since it's not created with create_autospec and the 
AttributeError has to silenced. I think the below patch will handle both 
scenarios. This doesn't cause any test failure and hence it will be good to 
convert the original report as a unit test. Feedback welcome on the approach. I 
will raise a PR with tests and I am updating the relevant versions where this 
fix can be applied.

Sample program

from unittest import mock

def foo(a, b):
pass

parent = mock.Mock()
a = mock.create_autospec(foo)
parent.child = a # 'a' is FunctionType and has a.mock attribute set 
(create_autospec -> _set_signature -> _setup_func)

parent.child_func = foo # 'foo' is FunctionType with no mock attribute set that 
could cause AttributeError

parent.child(1, 2) # Recorded
parent.child_func(2, 3) # Not recorded since it's actual call to child_func and 
has no parent set due to AttributeError
print(parent.method_calls) # [call.child(1, 2)]

Patch : 

diff --git a/Lib/unittest/mock.py b/Lib/unittest/mock.py
index 38189c9aec..143263722d 100644
--- a/Lib/unittest/mock.py
+++ b/Lib/unittest/mock.py
@@ -321,6 +321,12 @@ class _CallList(list):


 def _check_and_set_parent(parent, value, name, new_name):
+if isinstance(value, FunctionTypes):
+try:
+value = value.mock
+except AttributeError:
+pass
+
 if not _is_instance_mock(value):
 return False
 if ((value._mock_name or value._mock_new_name) or

--
nosy: +cjw296, mariocj89, xtreak
versions: +Python 3.7, Python 3.8 -Python 3.3, Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35499] "make profile-opt" overrides CFLAGS_NODIST

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:


New changeset 782e1d537778d93eb4cba1343f71bfc51e7e3c00 by Ned Deily (Victor 
Stinner) in branch '3.6':
bpo-35499: make profile-opt don't override CFLAGS_NODIST (GH-11164) (GH-11267)
https://github.com/python/cpython/commit/782e1d537778d93eb4cba1343f71bfc51e7e3c00


--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35424] multiprocessing.Pool: emit ResourceWarning

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

The first implementation of my PR also emitted ResourceWarning when .join() 
hasn't been called, but the merged PR only emit a warning if .close() and 
.terminated() have not been called: if the pool is still running.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35424] multiprocessing.Pool: emit ResourceWarning

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:


New changeset 9a8d1d7562b11969034b92217fe66aab7a951fb6 by Victor Stinner in 
branch 'master':
bpo-35424: emit ResourceWarning at multiprocessing.Pool destruction (GH-10974)
https://github.com/python/cpython/commit/9a8d1d7562b11969034b92217fe66aab7a951fb6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35485] Mac: tkinter windows turn black while resized

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:

Also, to be clear, I'm seeing any IDLE window turn completely black while 
resizing including IDLE's shell window; the "import tkinter" ... commands 
aren't necessary to see the problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35485] Mac: tkinter windows turn black while resized

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:

I'm seeing the same behavior, Terry. The installer's version was built from the 
Tk 8.6.9.1 release. I'll try building Tk from the core-8-6-branch unless Kevin 
has a better idea.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35482] can't open python368rc1.chm and python372rc1.chm

2018-12-20 Thread Ned Deily


Ned Deily  added the comment:


New changeset 789b0ee023f14385a2fd635272768c3b55a99773 by Ned Deily (Steve 
Dower) in branch '3.6':
bpo-35482: Fixes HTML escaping in CHM index and build location of NEWS file 
(GH-11224) (GH-11251)
https://github.com/python/cpython/commit/789b0ee023f14385a2fd635272768c3b55a99773


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35482] can't open python368rc1.chm and python372rc1.chm

2018-12-20 Thread Steve Dower


Steve Dower  added the comment:

The 3.6 PR is blocked from merging, so looking to Ned to resolve.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35482] can't open python368rc1.chm and python372rc1.chm

2018-12-20 Thread Steve Dower


Change by Steve Dower :


--
pull_requests: +10501

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35546] String formatting produces incorrect result with left-aligned zero-padded format

2018-12-20 Thread Matthew Barnett


Matthew Barnett  added the comment:

A similar issue exists with centring:

>>> format(42, '^020')
'0420'

--
nosy: +mrabarnett

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35547] email.parser / email.policy does correctly handle multiple RFC2047 encoded-word tokens across RFC5322 folded headers

2018-12-20 Thread Martijn Pieters

New submission from Martijn Pieters :

The From header in the following email headers is not correctly decoded; both 
the subject and from headers contain UTF-8 encoded data encoded with RFC2047 
encoded-words, in both cases a multi-byte UTF-8 codepoint has been split 
between the two encoded-word tokens:

>>> msgdata = '''\
From: =?utf-8?b?4ZuX4Zqr4Zqx4ZuP4ZuB4ZuD4Zq+4ZuI4ZuB4ZuW4ZuP4ZuW4Zo=?=
 =?utf-8?b?seGbiw==?= 
Subject: =?utf-8?b?c8qHdcSxb2THnXBvyZQgOC3ihLLiiqXiiKkgx53Kh8qOcS3E?=
 
=?utf-8?b?scqHyoNuya8gyaXKh8Sxyo0gx53Gg8mQc3PHncmvIMqHc8edyocgybnHncaDdW/Kgw==?=
'''
>>> from io import StringIO
>>> from email.parser import Parser
>>> from email import policy
>>> msg = Parser(policy=policy.default).parse(StringIO(msgdata))
>>> print(msg['Subject'])  # correct
sʇuıodǝpoɔ 8-Ⅎ⊥∩ ǝʇʎq-ıʇʃnɯ ɥʇıʍ ǝƃɐssǝɯ ʇsǝʇ ɹǝƃuoʃ
>>> print(msg['From'])  # incorrect
ᛗᚫᚱᛏᛁᛃᚾᛈᛁᛖᛏᛖ� �ᛋ 

Note the two FFFD placeholders in the From line.

The issue is that the raw value of the From and Subject contain the folding 
space at the start of the continuation lines:

>>> for name, value in msg.raw_items():
... if name in {'Subject', 'From'}:
... print(name, repr(value))
...
>From '=?utf-8?b?4ZuX4Zqr4Zqx4ZuP4ZuB4ZuD4Zq+4ZuI4ZuB4ZuW4ZuP4ZuW4Zo=?=\n 
>=?utf-8?b?seGbiw==?= '
Subject '=?utf-8?b?c8qHdcSxb2THnXBvyZQgOC3ihLLiiqXiiKkgx53Kh8qOcS3E?=\n 
=?utf-8?b?scqHyoNuya8gyaXKh8Sxyo0gx53Gg8mQc3PHncmvIMqHc8edyocgybnHncaDdW/Kgw==?='

For the Subject header, _header_value_parser.get_unstructured is used, which 
*expects* there to be spaces between encoded words; it inserts 
EWWhiteSpaceTerminal tokens in between which are turned into empty strings. But 
for the From header,  AddressHeader parser does not, the space at the start of 
the line is retained, and the surrogate escapes at the end of one encoded-word 
and the start start of the next encoded-word never ajoin, so the later handling 
of turning surrogates back into proper data fails.

Since unstructured header parsing doesn't mind if a space is missing between 
encoded-word atoms, the work-around is to explicitly remove the space at the 
start of every line; this can be done in a custom policy:

import re
from email.policy import EmailPolicy

class UnfoldingHeaderEmailPolicy(EmailPolicy):
def header_fetch_parse(self, name, value):
# remove any leading whitespace from header lines
# before further processing
value = re.sub(r'(?<=[\n\r])([\t ])', '', value)
return super().header_fetch_parse(name, value)

custom_policy = UnfoldingHeaderEmailPolicy()

after which the From header comes out without placeholders:

>>> msg = Parser(policy=custom_policy).parse(StringIO(msgdata))
>>> msg['from']
'ᛗᚫᚱᛏᛁᛃᚾᛈᛁᛖᛏᛖᚱᛋ '
>>> msg['subject']
'sʇuıodǝpoɔ 8-Ⅎ⊥∩ ǝʇʎq-ıʇʃnɯ ɥʇıʍ ǝƃɐssǝɯ ʇsǝʇ ɹǝƃuoʃ'

This issue was found by way of https://stackoverflow.com/q/53868584/100297

--
messages: 332243
nosy: mjpieters
priority: normal
severity: normal
status: open
title: email.parser / email.policy does correctly handle multiple RFC2047 
encoded-word tokens across RFC5322 folded headers

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35504] `del OSError().characters_written` raises `SystemError`

2018-12-20 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:


New changeset f347c6eb75ca46990cd7ad3efbe02002603d8460 by Serhiy Storchaka 
(Zackery Spytz) in branch '2.7':
bpo-35504: Fix segfaults and SystemErrors when deleting certain attrs. 
(GH-11175) (GH-11249)
https://github.com/python/cpython/commit/f347c6eb75ca46990cd7ad3efbe02002603d8460


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35504] `del OSError().characters_written` raises `SystemError`

2018-12-20 Thread Zackery Spytz


Change by Zackery Spytz :


--
pull_requests: +10500

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35482] can't open python368rc1.chm and python372rc1.chm

2018-12-20 Thread miss-islington


miss-islington  added the comment:


New changeset aae2e85af772a409bf8904bddc17efe9bf809174 by Miss Islington (bot) 
in branch '3.7':
bpo-35482: Fixes HTML escaping in CHM index and build location of NEWS file 
(GH-11224)
https://github.com/python/cpython/commit/aae2e85af772a409bf8904bddc17efe9bf809174


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35482] can't open python368rc1.chm and python372rc1.chm

2018-12-20 Thread miss-islington


Change by miss-islington :


--
pull_requests: +10499

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue22831] Use "with" to avoid possible fd leaks

2018-12-20 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:


New changeset c5d5dfdb223efb0e668e3f317d31b8b70ae96aa6 by Serhiy Storchaka in 
branch 'master':
bpo-22831: Use "with" to avoid possible fd leaks in distutils. (GH-10921)
https://github.com/python/cpython/commit/c5d5dfdb223efb0e668e3f317d31b8b70ae96aa6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35521] IDLE: Add doc section for Code Context and ref links.

2018-12-20 Thread Terry J. Reedy


Change by Terry J. Reedy :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

Serhiy Storchaka, Alexey Izbyshev: I read and understood your valid concerns.

restore_signals=True will be quickly implemented, and so posix_spawn() code 
path will be tested by more tests of test_subprocess. If not, I will ensure 
that missing tests will be added.

Enhance _posixsubprocess to use vfork() is an interesting project, but IMHO 
it's complementary and doesn't replace all advantages of posix_spawn().

I am going to merge the PR tomorrow, except if someone sees a good reason to 
not merge it. I prefer to merge the PR early in the 3.8 development cycle to 
have more time to handle any issue if someone notice bugs. If something goes 
badly, we will be able to easily revert the change. Don't worry, I like to 
revert changes ;-)

Again, I'm mentoring Joannah who is learning Python, so I prefer to move step 
by step (with small steps :-)). We will support more and more subprocess.Popen 
options.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

> Are you going to do experiments? If not, I can try to do some in early 
> January.

I'm only interested to use posix_spawn(). If you want to experiment vfork() in 
_posixsubprocess: please go ahead!


> I've studied that, and that's what I referred to as 
> "quality-of-implementation" problem. After glibc devs removed heap 
> allocations and tweaked some other things, they could use vfork() in all 
> cases. "musl" libc never had those problems and used vfork() from the 
> beginning.

_posixsubprocess shouldn't allocate memory on the heap *after* fork(), only 
before. If it does, it's a bug. Last time I checked _posixsubprocess, it looked 
to be written properly.

But the subprocess module has the crazy 'preexec_fn' option which makes things 
complicated... Maybe we should deprecate (and then remove) it, it's dangerous 
and it can be avoided in many cases. Using a launcher program is safer than 
relying on preexec_fn. In OpenStack, I reimplemented prlimit in pure Python. 
It's an example of launcher to set an option in the child process:
https://github.com/openstack/oslo.concurrency/blob/master/oslo_concurrency/prlimit.py

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread Alexey Izbyshev


Alexey Izbyshev  added the comment:

> I'm open to experiment to use vfork() in _posixsubprocess
Are you going to do experiments? If not, I can try to do some in early January.

> Using vfork() can cause new issues: that's why there is a 
> POSIX_SPAWN_USE_VFORK flag (the caller had to explicitly enable it). See also 
> bpo-34663 the history of vfork in posix_spawn() in the glibc.

I've studied that, and that's what I referred to as "quality-of-implementation" 
problem. After glibc devs removed heap allocations and tweaked some other 
things, they could use vfork() in all cases. "musl" libc never had those 
problems and used vfork() from the beginning.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35499] "make profile-opt" overrides CFLAGS_NODIST

2018-12-20 Thread STINNER Victor


Change by STINNER Victor :


--
pull_requests: +10498

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

> In the end, I think that migrating subprocess to vfork-exec would have more 
> impact for users than adding "fast paths" and have consistent performance 
> regardless of subprocess.Popen arguments (with few exceptions). Please 
> consider it. Thanks!

I'm open to experiment to use vfork() in _posixsubprocess, but I still want to 
use posix_spawn():

* it's standard
* it's safer
* it can be more efficient

On macOS, posix_spawn() is implemented as a syscall.

Using vfork() can cause new issues: that's why there is a POSIX_SPAWN_USE_VFORK 
flag (the caller had to explicitly enable it). See also bpo-34663 the history 
of vfork in posix_spawn() in the glibc. I'm not against using it, but I'm not 
sure that vfork() can be used in all cases, when all subprocess options are 
used. Maybe you will end up with the same state: vfork() used or not depending 
on subprocess.Popen options...


> For example, close_fds is True by default in subprocess, and I don't think 
> there is a reliable way to use posix_spawn() in this case.

Since PEP 446 has been implemented in Python 3.4, I would be interested to 
revisit close_fds default value: don't close by default... But I'm not sure if 
it's *safe* to change the default... At least, we may promote close_fds=False 
in the doc explaining that it's faster and should be safer in the common case.


> Another example is "cwd" parameter: there is no standard posix_spawn file 
> action to change the working directory in the child, so such a seemingly 
> trivial operation would trigger the "slow" fork-exec path.

I don't think that it's a bug that subprocess have different performances 
depending on the flags. It would be a bug if the behavior would be diffrent. I 
don't think that it's the case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread Alexey Izbyshev


Alexey Izbyshev  added the comment:

Victor and Joannah, thanks for working on adding vfork() support to subprocess. 
Regarding speedups in the real world, I can share a personal anecdote. Back at 
the time when AOSP was built with make (I think it was AOSP 5) I've observed 
~2x slowdown (or something close to that) if fork() is used instead of vfork() 
in make. This is the slowdown of the *whole* build (not just process creation 
time), and it's dramatic given the amount of pure computation (compilation of 
C++/Java code) involved in building AOSP. The underlying reason was that make 
merged all AOSP subproject makefiles into a single gigantic one, so make 
consumed more than 1 GB of RAM, and each shell invocation in recipes resulted 
in copying a large page table.

That said, I'm not sure that the chosen approach of adding posix_spawn() "fast 
paths" for particular combinations of arguments is an optimal way to expose 
vfork() benefits to most users. For example, close_fds is True by default in 
subprocess, and I don't think there is a reliable way to use posix_spawn() in 
this case. So users would need to use "magic" combinations of arguments to 
subprocess.Popen to benefit from vfork(). Another example is "cwd" parameter: 
there is no standard posix_spawn file action to change the working directory in 
the child, so such a seemingly trivial operation would trigger the "slow" 
fork-exec path.

Another approach would be to use vfork-exec instead fork-exec in 
_posixsubprocess. I know that previous discussions were resolved against 
vfork(), but I'm still not convinced and suggest to reevaluate those decisions 
again. Some arguments:

1. While POSIX contain scare-wording forbidding to do pretty anything in a 
vfork-spawned child, real systems where posix_spawn() is not a system call call 
vfork() and then execute all specified actions in the child. Cases where 
vfork() is avoided seem to be a quality-of-implementation issue, not a 
fundamental issue (for example, until recently glibc used fork() in some cases 
because of heap memory allocations, but they could be avoided). In practice, 
there is no problem with calling at least async-signal-safe functions in 
vfork-children, and there is no technical reason why there would be any problem.

2. _posixsubprocess is already very careful and calls only async-signal-safe 
functions in almost all cases (an obvious exception is preexec_fn, which is 
discouraged anyway).

3. Our use case for vfork() is restricted, so some concerns don't apply to us. 
For example, the setuid() problem outlined in Rich Felker's post 
(https://ewontfix.com/7) doesn't seem to apply. The problem with signal 
handlers from the same post should be possible to avoid except for preexec_fn, 
but we'd have to fallback to fork() in this case anyway due to memory 
allocation.

4. In the standard example of fork() unsafety in multithreaded processes (state 
of memory-based locks is copied to the child, and there could be nobody to 
unlock them), vfork() is *safer* than fork() because it still shares memory 
with the parent, and all threads other than the parent one are running. In 
particular, it should be perfectly safe to use any memory allocator that 
protects its state with something like futexes and atomics in a vfork-child 
even if other threads of the parent are using it concurrently.

5. Other language runtimes are already using vfork(). Java has been doing it 
for ages, and Victor referenced Go.

Some possible counter-arguments:

1. We seem to be reimplementing posix_spawn(). In fact, due to (2) above, we 
can fully reuse the existing fork-exec code, with additional tweaks that 
doesn't seem hard at this point.

2. My points above are based on Linux, but I don't know much about other 
Unix-likes, so in theory additional complications could exist. I can't, 
however, imagine any fundamental complication.

In the end, I think that migrating subprocess to vfork-exec would have more 
impact for users than adding "fast paths" and have consistent performance 
regardless of subprocess.Popen arguments (with few exceptions). Please consider 
it. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

I repeated the same tests for Python 3.6 on PR 11265: all tests are ok!

I replaced "38" with "36" and "3.8" and 3.6" and my 3 scripts :-) I modified 
step1.sh for PGO+LTO and to compile with -O0 (just to make tests faster): 
similar script than in my previous message.

./configure CC=clang --with-lto --prefix /opt/py36

(1) 4 => OK!
(2) 0, False => OK!
(3) 0 => OK!

./configure CC=clang --with-lto --prefix /opt/py36 --enable-shared

(1) 4 => OK!
(2) 0, False => OK!
(3) 0 => OK!

./configure CC=clang --with-lto --prefix /opt/py36 --enable-optimizations

(1) 8 => OK!
(2) 0, False => OK!
(3) 0 => OK!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:


New changeset 0198f52ea2328fd932622ad2299381f617a041f2 by Victor Stinner in 
branch '3.7':
bpo-35257: Avoid leaking LTO linker flags into distutils (GH-10900) (GH-11264)
https://github.com/python/cpython/commit/0198f52ea2328fd932622ad2299381f617a041f2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

I repeated the same tests for Python 3.7 on PR 11264.

I replaced "38" with "37" and "3.8" and 3.7" and my 3 scripts :-) I modified 
step1.sh for PGO+LTO with -O0:

set -x -e
git clean -fdx
./configure CC=clang --with-lto --prefix /opt/py37 --enable-optimizations
sed -i -e 's/^PROFILE_TASK=.*/PROFILE_TASK=-c pass/' Makefile
sed -i -e 's/-O3/-O0/' Makefile
make 2>&1|tee log
grep -E -- '-o python|-o Python/bltinmodule.o|Modules/_asynciomodule.o' 
log|grep -c lto
# 4 expected: -flto passed to compiler *and* linker


Test results:

./configure CC=clang --with-lto --prefix /opt/py38

(1) 4
(2) 0, False => OK!
(3) 0 => OK!

./configure CC=clang --with-lto --enable-shared

(1) 4
(2) 0, False => OK!
(3) 0 => OK!

./configure CC=clang --with-lto --enable-optimizations

(1) 8 => OK!
(2) 0, False => OK!
(3) 0 => OK!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35546] String formatting produces incorrect result with left-aligned zero-padded format

2018-12-20 Thread Serhiy Storchaka


New submission from Serhiy Storchaka :

Compare printf-style string formatting and new-style string formatting.

>>> '%-020d' % 42
'42  '
>>> format(42, '<020')
'4200'
>>> format(42, '<020d')
'4200'
>>> '%-020x' % 42
'2a  '
>>> format(42, '<020x')
'2a00'
>>> '%-020g' % 1.2e-8
'1.2e-08 '
>>> format(1.2e-8, '<020')
'1.2e-080'
>>> format(1.2e-8, '<020g')
'1.2e-080'
>>> format(1.2e-8, '<020e')
'1.20e-08'

New-style string formatting produces the result that looks like a correctly 
formatted number, but it represents incorrect number.

I think that zero padding should not be allowed for left-aligned format for 
numbers (except the 'f' format). Zero padding is already disallowed for complex 
numbers.

--
components: Interpreter Core
messages: 332231
nosy: eric.smith, serhiy.storchaka
priority: normal
severity: normal
status: open
title: String formatting produces incorrect result with left-aligned 
zero-padded format

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread Charalampos Stratakis


Charalampos Stratakis  added the comment:

Small correction. This regression has been on 3.7 for some time now (since it 
was the master branch then), but then I requested to have the buggy commit 
backported to 3.6 to fix the --with-lto flag there. Which unfortunately 
introduced the issue to 3.6 anyway. Master branch is fixed now and backports 
are open for 3.7 and 3.6 if those are to be accepted by the release manager.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

Ned: We have to do something with this issue before 3.6 and 3.7 releases. 
Either revert or backport fixes (merge PR 11264 and PR 11265).

--
priority: critical -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread STINNER Victor


Change by STINNER Victor :


--
pull_requests: +10496

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35499] "make profile-opt" overrides CFLAGS_NODIST

2018-12-20 Thread STINNER Victor


Change by STINNER Victor :


--
pull_requests: +10497

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread Charalampos Stratakis


Charalampos Stratakis  added the comment:

And also 3.7.2rc1

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread Charalampos Stratakis


Charalampos Stratakis  added the comment:

This change fixes a regression introduced in 3.6.8rc1 with 
https://bugs.python.org/issue31354

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35257] Avoid leaking linker flags into distutils: add PY_LDFLAGS_NODIST

2018-12-20 Thread STINNER Victor


Change by STINNER Victor :


--
pull_requests: +10495

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35105] Document that CPython accepts "invalid" identifiers

2018-12-20 Thread Windson Yang


Change by Windson Yang :


--
keywords: +patch
pull_requests: +10494
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35324] ssl: FileNotFoundError when do handshake

2018-12-20 Thread Windson Yang


Windson Yang  added the comment:

>From the docs

Changed in version 3.7: Hostname or IP address is matched by OpenSSL during 
handshake. The function match_hostname() is no longer used. In case OpenSSL 
refuses a hostname or IP address, the handshake is aborted early and a TLS 
alert message is send to the peer.

I guess OpenSSL refuse your IP address and port.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35544] unicode.encode docstring says return value can be unicode

2018-12-20 Thread radiocane


radiocane  added the comment:

>> I'll keep on finding "-> string or unicode" misleading.
> How is it misleading when its true?

[I promise this is the last reply: I won't waste more of your time]

me: How fast does this car go?
docstring: 100 km/h or 300 km/h

me: actually most people use it to do 100km/h and I don't know how to do 300 
km/h
people: no physical law forbids 300 km/h so it's true
people2: if you remove the seats, the lights, the windshield etc and basically 
end up with a chassis with four wheels and a motor, it can do 300 km/h

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35302] create_connection with local_addr misses valid socket bindings

2018-12-20 Thread twisteroid ambassador

twisteroid ambassador  added the comment:

I don't have a Mac, so I have not tested Ronald's workaround. Assuming it 
works, we will have to either i) implement platform-specific behavior and only 
apply IPV6_V6ONLY on macOS for each AF_INET6 socket created, or ii) apply it to 
all AF_INET6 sockets on all platforms, ideally after testing the option on all 
these platforms to make sure it doesn't have any undesirable side effect.

Linux's man page of ipv6 (http://man7.org/linux/man-pages/man7/ipv6.7.html ) 
has this to say about IPV6_V6ONLY:


If this flag is set to true (nonzero), then the socket is re‐
stricted to sending and receiving IPv6 packets only.  In this
case, an IPv4 and an IPv6 application can bind to a single
port at the same time.

If this flag is set to false (zero), then the socket can be
used to send and receive packets to and from an IPv6 address
or an IPv4-mapped IPv6 address.


So setting IPV6_V6ONLY might break some use cases? I have no idea how prevalent 
that may be.

The upside of this solution, as well as the second suggestion in Neil's OP 
(filter out local addrinfos with mismatching family), is that they should not 
increase connect time for normal cases. My solution (for which I have already 
submitted a PR) probably has a negligible increase in connection time and 
resource usage, because a fresh socket object is created for each pair of 
remote and local addrinfo.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:

Oh, nice! So this is not an obstacle for using os.posix_spawn().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread Alexey Izbyshev


Alexey Izbyshev  added the comment:

Serhiy, PyOS_* functions are called only if preexec_fn != None. But it will 
never be possible to implement support for preexec_fn (and some other 
subprocess features, e.g. close_fds) if processes are run via posix_spawn, so I 
don't see why anything should be done with those functions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25184] "python -m pydoc -w" fails in nondecodable directory

2018-12-20 Thread Serhiy Storchaka


Change by Serhiy Storchaka :


--
versions: +Python 3.7, Python 3.8 -Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue25184] "python -m pydoc -w" fails in nondecodable directory

2018-12-20 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:

Could you please create a PR for your path Martin?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35544] unicode.encode docstring says return value can be unicode

2018-12-20 Thread Steven D'Aprano


Steven D'Aprano  added the comment:

> I'll keep on finding "-> string or unicode" misleading.

How is it misleading when its true?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:

> Do you mean running tests twice, one with posix_spawn(), one without?

I mean that after writing tests they can be tested manually by disabling 
conditions for posix_spawn one by one. I.e. some tests should fail if remove 
"stdout is None" and some tests should fail if remove "not close_fds", etc.

> Can you please elaborate? posix_spawn() may or may not use fork() internally.

_posixsubprocess.fork_exec() calls 
PyOS_BeforeFork/PyOS_AfterFork_Child/PyOS_AfterFork_Parent. If use 
os.posix_spawn(), these calls will be omitted. This is a behavior change. We 
should either call these functions manually before/after os.posix_spawn() (but 
I do not know what to do with PyOS_AfterFork_Child), or disable using 
os.posix_spawn() if non-trivial callbacks were added, or use other methods for 
calling registered callbacks. But the stdlib already adds callbacks in the 
threading and random modules.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35521] IDLE: Add doc section for Code Context and ref links.

2018-12-20 Thread miss-islington


miss-islington  added the comment:


New changeset 5d0498a6967d45042b7a9345f6d871047edbaa25 by Miss Islington (bot) 
in branch '3.7':
bpo-35521: Add more cross-refs to IDLE docs (GH-11257)
https://github.com/python/cpython/commit/5d0498a6967d45042b7a9345f6d871047edbaa25


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

> We need to add tests for all corner cases. I.e. disabling any of conditions 
> for posix_spawn should cause the failure.

Do you mean running tests twice, one with posix_spawn(), one without?


> We need to do also something with 
> PyOS_BeforeFork/PyOS_AfterFork_Child/PyOS_AfterFork_Parent.

Can you please elaborate? posix_spawn() may or may not use fork() internally.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread Serhiy Storchaka


Serhiy Storchaka  added the comment:

We need to add tests for all corner cases. I.e. disabling any of conditions for 
posix_spawn should cause the failure.

We need to do also something with 
PyOS_BeforeFork/PyOS_AfterFork_Child/PyOS_AfterFork_Parent.

--
nosy: +pablogsal, pitrou, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35544] unicode.encode docstring says return value can be unicode

2018-12-20 Thread Marc-Andre Lemburg


Marc-Andre Lemburg  added the comment:

You can install any codec you like and those essentially decide
on what to return as type. However, the unicode methods only
allow strings or unicode to be returned in Python 2.
In Python 3, .encode() only allows bytes.

You can still get the full codec encode/decode functionality
via the codecs encode/decode methods in Python 3.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Experts (#1, Dec 20 2018)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> Python Database Interfaces ...   http://products.egenix.com/
>>> Plone/Zope Database Interfaces ...   http://zope.egenix.com/


::: We implement business ideas - efficiently in both time and costs :::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
   Registered at Amtsgericht Duesseldorf: HRB 46611
   http://www.egenix.com/company/contact/
  http://www.malemburg.com/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35521] IDLE: Add doc section for Code Context and ref links.

2018-12-20 Thread miss-islington


Change by miss-islington :


--
pull_requests: +10493

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35521] IDLE: Add doc section for Code Context and ref links.

2018-12-20 Thread Terry J. Reedy


Terry J. Reedy  added the comment:


New changeset 292cd6e33104d9f458232a14998fe5c62f7f7e81 by Terry Jan Reedy in 
branch 'master':
bpo-35521: Add more cross-refs to IDLE docs (#11257)
https://github.com/python/cpython/commit/292cd6e33104d9f458232a14998fe5c62f7f7e81


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

See also:

"How a fix in Go 1.9 sped up our Gitaly service by 30x" (hint: posix_spawn)
https://about.gitlab.com/2018/01/23/how-a-fix-in-go-19-sped-up-our-gitaly-service-by-30x/

fork/exec vs. posix_spawn benchmarks:
https://github.com/rtomayko/posix-spawn#benchmarks

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

Benchmark on FreeBSD:

$ env/bin/python -m perf compare_to fork_exec.json posix_spawn.json 
Mean +- std dev: [fork_exec] 52.8 ms +- 4.7 ms -> [posix_spawn] 499 us +- 45 
us: 105.91x faster (-99%)

Wow! 106x faster!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35545] asyncio.base_events.create_connection doesn't handle scoped IPv6 addresses

2018-12-20 Thread Максим Аристов

New submission from Максим Аристов :

loop.create_connection doesn't handle ipv6 RFC4007 addresses right since 3.7 


TEST CASE
# Set up listener on link-local address fe80::1%lo
sudo ip a add dev lo fe80::1

# 3.6 handles everything fine
socat file:/dev/null tcp6-listen:12345,REUSEADDR &
python3.6 -c 'import 
asyncio;loop=asyncio.get_event_loop();loop.run_until_complete(loop.create_connection(lambda:asyncio.Protocol(),host="fe80::1%lo",port="12345"))'

# 3.7 and later fails
socat file:/dev/null tcp6-listen:12345,REUSEADDR &
python3.7 -c 'import 
asyncio;loop=asyncio.get_event_loop();loop.run_until_complete(loop.create_connection(lambda:asyncio.Protocol(),host="fe80::1%lo",port="12345"))'

Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.7/asyncio/base_events.py", line 576, in 
run_until_complete
return future.result()
  File "/usr/lib/python3.7/asyncio/base_events.py", line 951, in 
create_connection
raise exceptions[0]
  File "/usr/lib/python3.7/asyncio/base_events.py", line 938, in 
create_connection
await self.sock_connect(sock, address)
  File "/usr/lib/python3.7/asyncio/selector_events.py", line 475, in 
sock_connect
return await fut
  File "/usr/lib/python3.7/asyncio/selector_events.py", line 480, in 
_sock_connect
sock.connect(address)
OSError: [Errno 22] Invalid argument


CAUSE

Upon asyncio.base_events.create_connection _ensure_resolved is called twice, 
first time here:
https://github.com/python/cpython/blob/3.7/Lib/asyncio/base_events.py#L908
then here through sock_connect:
https://github.com/python/cpython/blob/3.7/Lib/asyncio/base_events.py#L946
https://github.com/python/cpython/blob/3.7/Lib/asyncio/selector_events.py#L458

_ensure_resolved calls getaddrinfo, but in 3.7 implementation changed:

% python3.6 -c 'import 
socket;print(socket.getaddrinfo("fe80::1%lo",12345)[0][4])'
('fe80::1%lo', 12345, 0, 1)

% python3.7 -c 'import 
socket;print(socket.getaddrinfo("fe80::1%lo",12345)[0][4])'
('fe80::1', 12345, 0, 1)

_ensure_connect only considers host and port parts of the address tuple:
https://github.com/python/cpython/blob/3.7/Lib/asyncio/base_events.py#L1272

In case of 3.7 first call to _ensure_resolved returns
('fe80::1', 12345, 0, 1)
then second call returns
('fe80::1', 12345, 0, 0)
Notice that scope is now completely lost and is set to 0, thus actual call to 
socket.connect is wrong

In case of 3.6 both first and second call to _ensure_resolved return
('fe80::1%lo', 12345, 0, 1)
because in 3.6 case scope info is preserved in address and second call can 
derive correct address tuple

--
components: asyncio
messages: 332211
nosy: asvetlov, yselivanov, Максим Аристов
priority: normal
severity: normal
status: open
title: asyncio.base_events.create_connection doesn't handle scoped IPv6 
addresses
type: behavior
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35537] use os.posix_spawn in subprocess

2018-12-20 Thread STINNER Victor


STINNER Victor  added the comment:

Benchmark on macOS Mojave (10.14.2), x86_64:

$ ./python.exe -m perf compare_to fork_exec.json posix_spawn.json 
Mean +- std dev: [fork_exec] 2.79 ms +- 0.06 ms -> [posix_spawn] 1.64 ms +- 
0.03 ms: 1.70x faster (-41%)

The speedup is "only" 1.7x faster, it's less impressive. Maybe fork()+exec() is 
more efficient on macOS than on Linux.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35544] unicode.encode docstring says return value can be unicode

2018-12-20 Thread radiocane


radiocane  added the comment:

Given that:
1) No standard codec returns unicode
2) I consider as "most common scenario" the case where a user wants to encode a 
unicode object using some character encoding and get back an str-like object

I'll keep on finding "-> string or unicode" misleading. I'd rather have the 
same as str.encode i.e. "-> object".
Anyway thanks for your time and attention :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



[issue35544] unicode.encode docstring says return value can be unicode

2018-12-20 Thread Serhiy Storchaka


Change by Serhiy Storchaka :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com



  1   2   >