[issue34736] Confusing base64.b64decode output

[miss-islington]

miss-islington  added the comment:


New changeset 1fba2ffc37da52c08db51fe4360459990b0311c9 by Miss Islington (bot) 
(Tal Einat) in branch 'master':
bpo-34736: improve error message for invalid length b64decode inputs (GH-9563)
https://github.com/python/cpython/commit/1fba2ffc37da52c08db51fe4360459990b0311c9


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34827] Make argparse.NameSpace iterable

[Micheal Taylor]

New submission from Micheal Taylor :

There was an issue to make argparse.Namespace iterable with a specific intent 
on being able to convert it to a dictionary 
(https://bugs.python.org/issue8982).  Additionally there was another 
improvement request to make it accessible like a dictionary.  
(https://bugs.python.org/issue8979)

While vars(args) and args.thing do accomplish the needs, I'm really lazy, and 
would like the object to be more accessible - for instance, if I will always 
need to access every attribute (because I make a small namespace that fits a 
simple script, making it iterable would be convenient.  It's also more 
convenient to use the typical **thing syntax for passing it into functions if 
all the attributes are required for different things.  This keeps code within 
the functions that would use it simpler, because we no longer need to reference 
args.thing, we can reference thing directly within the function it's been 
passed to.  vars(args) does accomplish this, but it is arguably not as familiar 
for folks as the more "familiar" syntax of **args.

--
components: Library (Lib)
messages: 326605
nosy: bubthegreat
priority: normal
severity: normal
status: open
title: Make argparse.NameSpace iterable
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34826] io.BufferedReader crashes in 2.7 on memoryview attribute access

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27195] Crash when RawIOBase.write(b) evaluates b.format

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34777] urllib.request accepts anything as a header parameter for some URLs

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Thanks for the details. Each server behaves differently for these headers which 
depends on the server configuration and using other client like curl will also 
return the same result as Python does. So I would propose closing it as not a 
bug since there is no bug with Python and it behaves like other clients do.

Thanks again for the report!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

Also worth noting:  other projects need to combine hashes too.  Here's a 64-bit 
version of the highly regarded C++ Boost[1] library's hash_combine() function 
(I replaced its 32-bit int literal with "a random" 64-bit one):

x ^= (Py_uhash_t)y + 0x94ae1875aa5647f1UL + (x << 6) + (x >> 2);

It's very cheap.  It also sucks horribly if used as the guts Python's tuple 
hash loop ;-)

This isn't a paradox.  Best I can tell, Python may be unique in trying _not_ to 
make all hashes "look random".  If you have only randomish hashes to combine, 
the Boost approach works fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

Perhaps worth noting that FNV-1a works great if used as _intended_:  on a 
stream of unsigned bytes.  All tests except the new tuple hash test suffer no 
collisions; the new test suffers 14.  Nothing is needed to try to worm around 
nested tuple catastrophes, or to worm around mixing integers of similar 
magnitude but different signs.  The obvious downside:  on a 64-bit box, it's 8 
times as many multiplies :-(  Here's a 64-bit version:

Py_uhash_t t = (Py_uhash_t)y;
for (int i = 0; i < sizeof(t); ++i) {
x = (x ^ (t & 0xff)) * (Py_uhash_t)1099511628211ULL;
t >>= 8;
}

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] Multiple tests (test_socket, test_multiprocessing_*) fail due to incorrect recvmsg(2) buffer lengths, causing failures on FreeBSD CURRENT

[Kubilay Kocak]

Kubilay Kocak  added the comment:

To clarify affected tests/versions:

Python 3.x - test_socket test_multiprocessing_spawn
Python 2.7 - test_multiprocessing_spawn

All of the issues are related to incorrect recvmsg(2) buffer length use, so 
I've amended the issue summary to describe the complete root problem.

The be explicit,

- test_socket fix requires backporting to all supported 3.x branches
- test_multiprocessing_* test fixes (via multiprocessing.reduction fix) 
requires backporting to all supported 3.x branches *and* 2.7

--
title: test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD 
CURRENT -> Multiple tests (test_socket, test_multiprocessing_*) fail due to 
incorrect recvmsg(2) buffer lengths, causing failures on FreeBSD CURRENT

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18578] Rename and document test.bytecode_helper as test.support.bytecode_helper

[Cheryl Sabella]

Cheryl Sabella  added the comment:

@seydou, would you be interested in making a GitHub pull request for your patch?

--
assignee:  -> docs@python
components: +Documentation, Tests
nosy: +cheryl.sabella, docs@python
stage:  -> needs patch
type:  -> enhancement
versions: +Python 3.8 -Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34818] test.test_ssl.ThreadedTests.test_tls1_3 fails in 2.7 with AttributeError: __exit__

[Dimitri John Ledkov]

Dimitri John Ledkov  added the comment:

Similar client sockets are used in that file, but they are wrapped in extra 
`with closing(...) as s:`

Is closing() wrapper missing in this test case?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +9012

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests: +9012, 9013

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34826] io.BufferedReader crashes in 2.7 on memoryview attribute access

[Zackery Spytz]

Zackery Spytz  added the comment:

See #27195 and PR 8415.

--
nosy: +ZackerySpytz

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34826] io.BufferedReader crashes in 2.7 on memoryview attribute access

[Gregory P. Smith]

New submission from Gregory P. Smith :

The following test code segfaults on Python 2.7:

```python
import io
class X(io.RawIOBase):
  def readable(self):
return True
  def readinto(self, b):
if isinstance(b, memoryview):
  print(b.format)  # XXX boom, crashes here.
return 0

io.BufferedReader(X()).read(8)
```

The crash happens on the b.format attribute access.

This does not happen on 3.6.

--
components: IO, Library (Lib)
messages: 326597
nosy: gregory.p.smith
priority: normal
severity: normal
stage: needs patch
status: open
title: io.BufferedReader crashes in 2.7 on memoryview attribute access
type: crash
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32552] Improve text for file arguments in argparse.ArgumentDefaultsHelpFormatter class

[paul j3]

paul j3  added the comment:

An alternative to customizing a HelpFormatter is to write your own utility 
`add_argument` function, e.g.

def my_add_argument(parser, *args, add_default=True, **kwargs):
if add_default:
help = kwargs.get('help','')
help += ' (default: %(default)s)'
kwargs['help'] = help
return parser.add_argument(*args, **kwargs)

which could be used as

my_add_argument(parser, '-g', help='bar help', default='other', 
add_default=False)

There are some refinements to the _get_help_string() that I showed earlier, 
such as only adding the '%s' to actions where default makes sense (optionals 
and a subset positionals).  One could also skip it if the default is the 
default default None, etc.

One way or other the user can already control whether the help line shows the 
default.  ArgumentDefaultsHelpFormatter just automates this for a straight 
forward parser.  

I'm going to close this issue since it isn't really needed (and no one has 
proposed a clever patch).

--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34709] Suggestion: make getuser.getpass() also look at SUDO_USER environment variable

[Amos S]

Amos S  added the comment:

The use of environment variables like USER and LOGNAME instead of getuid() etc 
is done in order to try to get "who really initiated this process?" rather than 
"who this process belongs to?". This is hidden today when SUDO_USER is ignored.

This also fits my interpretation of the function's documentation: 'Return the 
"login name" of the user.'

For instance, in a system I use I have to sudo to a system user other than root 
in order to execute certain scripts and this patch would reveal the original 
user who executed the sudo command.

I updated the test case already in the existing PR.

I update to the documentation to list SUDO_USER.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32956] python 3 round bug

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> the silent and subtle change in behaviour from Python 2 to Python 3 was a bit 
> unpleasant, and a possible source of late-discovered (or undiscovered) bugs. 

This change was advertised in the "What’s New In Python 3.0" document.

https://docs.python.org/3/whatsnew/3.0.html#builtins

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34777] urllib.request accepts anything as a header parameter for some URLs

[Jose Gama]

Jose Gama  added the comment:

Thank you for the quick reply. You are correct about the difficulties of using 
a universally accepted list.This is one example that generates errors on the 
server side. Just for the record.

#!/usr/bin/env python3
from urllib.request import Request, urlopenfrom urllib.error import URLError
# process SSB dataurl1 = 
'https://raw.githubusercontent.com/mapnik/test-data/master/csv/points.csv'url2 
= 
'https://gitlab.cncf.ci/kubernetes/kubernetes/raw/c69582dffba33e9f1c08ff2fc67924ea90f1448c/test/test_owners.csv'url3
 = 
'http://data.ssb.no/api/klass/v1/classifications/131/changes?from=2016-01-01=-12-31'headers1
 = {'Accept': 'text/csv'}headers2 = {'Akcept': 'text/csv'}headers3 = {'Accept': 
'tekst/cxv'}headers4 = {'Accept': '1234'}req = Request(url3, 
headers=headers4)resp = urlopen(req)content =  
resp.read().decode(resp.headers.get_content_charset()) # get the character 
encoding from the server responseprint(content)
'''req = Request(url3, headers=headers3)
urllib.error.HTTPError: HTTP Error 500: Internal Server Error

req = Request(url3, headers=headers4)
urllib.error.HTTPError: HTTP Error 406: Not Acceptable'''

On Tuesday, September 25, 2018, 8:38:26 AM GMT+2, Karthikeyan Singaravelan 
 wrote:  

Karthikeyan Singaravelan  added the comment:

Thanks for the report. I tried similar requests and it works this way for other 
tools like curl since Akcept could be a custom header in some use cases though 
it could be a  typo in this context. There is no predefined set of media types 
that we need to validate as far as I can see from 
https://tools.ietf.org/html/rfc2616#section-14.1 and it depends on the server 
configuration to do validation. It's hard for Python to maintain a list of 
acceptable MIME types for validation across releases. A list of registered MIME 
types that is updated periodically : 
https://www.iana.org/assignments/media-types/media-types.xhtml and RFC for 
registration : https://tools.ietf.org/html/rfc6838

Some sample requests from curl with invalid headers.

curl -X GET https://httpbin.org/get -H 'Authorization: Token 
bc23f14356c114a8ffa319773583426878b7b37f' -H 'Cache-Control: no-cache' -H 
'Content-Type: application/json' -H 'Akcept: tekst/csv'
{
  "args": {},
  "headers": {
    "Accept": "*/*",
    "Akcept": "tekst/csv",
    "Authorization": "Token bc23f14356c114a8ffa319773583426878b7b37f",
    "Cache-Control": "no-cache",
    "Connection": "close",
    "Content-Type": "application/json",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.37.1"
  },
  "origin": "182.73.135.26",
  "url": "https://httpbin.org/get;
}

curl -X GET https://httpbin.org/get -H 'Authorization: Token 
bc23f14356c114a8ffa319773583426878b7b37f' -H 'Cache-Control: no-cache' -H 
'Content-Type: application/json' -H 'Accept: tekst'
{
  "args": {},
  "headers": {
    "Accept": "tekst",
    "Authorization": "Token bc23f14356c114a8ffa319773583426878b7b37f",
    "Cache-Control": "no-cache",
    "Connection": "close",
    "Content-Type": "application/json",
    "Host": "httpbin.org",
    "User-Agent": "curl/7.37.1"
  },
  "origin": "182.73.135.26",
  "url": "https://httpbin.org/get;
}

Feel free to add in if I am missing something here but I think it's hard for 
Python to maintain the updated list and adding warning/error might break 
someone's code.

Thanks

--
nosy: +xtreak

___
Python tracker 

___

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34802] asyncio.iscoroutine() documentation is wrong

[Gefn]

Gefn  added the comment:

I think it's much clearer

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34802] asyncio.iscoroutine() documentation is wrong

[Yury Selivanov]

Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34248] dbm errors should contain file names

[Berker Peksag]

Change by Berker Peksag :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34248] dbm errors should contain file names

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 9df346bf98069a87de14a3c2f69009d800994c63 by Berker Peksag (Zsolt 
Cserna) in branch 'master':
bpo-34248: Add filename to error raised in {gnu,ndbm}.open() (GH-8590)
https://github.com/python/cpython/commit/9df346bf98069a87de14a3c2f69009d800994c63


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34802] asyncio.iscoroutine() documentation is wrong

[miss-islington]

miss-islington  added the comment:


New changeset 85ccedc5b57ddda198e7176ba787e3896435c504 by Miss Islington (bot) 
in branch '3.7':
bpo-34802: Fix asyncio.iscoroutine() docs (GH-9611)
https://github.com/python/cpython/commit/85ccedc5b57ddda198e7176ba787e3896435c504


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34802] asyncio.iscoroutine() documentation is wrong

[miss-islington]

Change by miss-islington :


--
pull_requests: +9010

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34802] asyncio.iscoroutine() documentation is wrong

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 59ee5b12938efbf534f2a19300a847bf6b23a77d by Yury Selivanov in 
branch 'master':
bpo-34802: Fix asyncio.iscoroutine() docs (GH-9611)
https://github.com/python/cpython/commit/59ee5b12938efbf534f2a19300a847bf6b23a77d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue7674] select.select() corner cases: duplicate fds, out-of-range fds

[Berker Peksag]

Change by Berker Peksag :


--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34802] asyncio.iscoroutine() documentation is wrong

[Yury Selivanov]

Yury Selivanov  added the comment:

I'll make a simple fix for the asyncio.coroutine decorator docs.

> I would suggest below for #coroutines :  

>> Coroutines declared with async/await syntax is the preferred way of writing 
>> asyncio applications but asyncio also supports legacy generator-based 
>> coroutines.

I don't want to further emphasize generator-based coroutines beyond how they 
are covered right now.  We'll drop support for them in a just a couple of 
versions anyways.

--
assignee:  -> docs@python
components: +Documentation, asyncio
nosy: +docs@python
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34802] asyncio.iscoroutine() documentation is wrong

[Yury Selivanov]

Change by Yury Selivanov :


--
keywords: +patch
pull_requests: +9009
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34762] Change contextvars C API to use PyObject

[Yury Selivanov]

Yury Selivanov  added the comment:

Thank you Serhiy, for re-opening this!  I've pushed fixes to 3.7 and master 
branches.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34802] asyncio.iscoroutine() documentation is wrong

[Brett Cannon]

Change by Brett Cannon :


--
nosy: +asvetlov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34762] Change contextvars C API to use PyObject

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 24cb7de15d3a5979425b281ab4f600f7c2b401f2 by Yury Selivanov in 
branch '3.7':
[3.7] bpo-34762: Update PyContext* refs to PyObject* in asyncio and decimal 
(GH-9610)
https://github.com/python/cpython/commit/24cb7de15d3a5979425b281ab4f600f7c2b401f2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34533] Apply PEP384 to _csv module

[Berker Peksag]

Berker Peksag  added the comment:

This is a duplicate of issue 14935. Could you please retarget PR 8977 against 
issue 14935?

--
nosy: +berker.peksag
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed
superseder:  -> PEP 384 Refactoring applied to _csv module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34762] Change contextvars C API to use PyObject

[Yury Selivanov]

Change by Yury Selivanov :


--
pull_requests: +9008

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34751] Hash collisions for tuples

[Tim Peters]

Tim Peters  added the comment:

I should have spelled this out before:  these are all permutations, so in 
general permuting the result space of `x * mult + y` (or any other permutation 
involving x and y) is exactly the same as not permuting it but applying a 
different permutation to y instead.

Specifically, the sequence:

x = x * mult + y  (mod 2**N)
x = P(x)  # P is any permutation of N-bit integers

is the same as (and this isn't "deep" - it's trivial):

x = x * mult + Q(y, x)  (mod 2**N)

where Q(y, x) = P(x * mult + y) - x * mult  (mod 2**N)

Q is a "richer" class of permutation though, because it's a different 
permutation for each fixed value of `x`, and uses multiplication to help 
disperse bits.

While it would take a lot of work to quantify this, in my experiments the tuple 
hash is significantly less touchy when permuting x after than when permuting y 
before, presumably because Q is richer.

The two tuple hash tests have many inputs whose tuple component hashes have 
very similar (even identical) bit patterns.  There's only so much dirt-simple 
permutations (like "y ^= y << 1") can do to break the patterns.  So, e.g., 
change a shift count, change the multiplier, ..., and at least one of those two 
tests fails depressingly often.  Not spectacularly, but over the limit they 
allow.  Q(y, x) does a far better job of magnifying small differences.

Something I haven't tried:  use a richer permutation of y that _doesn't_ depend 
on x.  For example, the frozenset hash has much the same underlying challenge, 
and uses this permutation to "blow up" small input differences:

static Py_uhash_t
_shuffle_bits(Py_uhash_t h)
{
return ((h ^ 89869747UL) ^ (h << 16)) * 3644798167UL;
}

But that's a lot more expensive than a single shift-xor, and the speed of tuple 
hashing is more important than of frozenset hashing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34762] Change contextvars C API to use PyObject

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 994269ccee5574f03cda6b018399347fc52bf330 by Yury Selivanov in 
branch 'master':
bpo-34762: Update PyContext* to PyObject* in asyncio and decimal (GH-9609)
https://github.com/python/cpython/commit/994269ccee5574f03cda6b018399347fc52bf330


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32956] python 3 round bug

[Mark Dickinson]

Mark Dickinson  added the comment:

[Joshua]

> 1. Update the round() docs to make the documentation of this behavior less 
> buried,

Sounds reasonable to me; I'm definitely open to documentation improvements. 
Though it doesn't seem all that buried to me: the round-ties-to-even behaviour 
is described in the third sentence in the first place I'd look for round 
documentation (https://docs.python.org/3/library/functions.html#round). It 
would be misleading to move the information earlier, because the use of 
round-ties-to-even is specific to the builtin types: user-defined types can do 
whatever they like via the __round__ magic method.

> 2. include a (brief) justification (possibly even just a link to 
> http://wiki.c2.com/?BankersRounding or some more-authoritative document), and

Sure, a link to a source on bankers rounding could work.

> 3. link to where else this change in Python 3 was discussed more, if 
> anywhere, or else confirm this change was made based on no additional 
> analysis that we can find written down.

I'm not aware of much discussion beyond the thread that Serhiy already pointed 
to. There's a little bit more (but not much) on rounding the py3k mailing list 
(try a Google search for "site:mail.python.org/pipermail/python-3000 rounding").

> It'd also be interesting to hear if this is something we wish we'd done 
> differently now, but that shouldn't distract from 1, 2, and 3.

I can't speak for anyone else, but it's certainly not something I think should 
have been done differently, with one caveat: the silent and subtle change in 
behaviour from Python 2 to Python 3 was a bit unpleasant, and a possible source 
of late-discovered (or undiscovered) bugs.

> so maybe changing from round_half_up to round_half_even was necessary for the 
> other improvements [...]

No. The change was independent of other fixes and changes. There _is_ quite a 
history of round changes: fixes for the single-argument round function in odd 
corner cases (earlier versions of Python used the simple add-half-and-chop 
algorithm, with gives the wrong answer for 0.4999 and for 
4503599627370497.0 thanks to FPU-level rounding in the add-half step); making 
two-argument round correctly-rounded in all cases in Python 2.7 and 3.1 via the 
same dtoa.c machinery used for str<->float conversions; changing the return 
type of single-argument round in Python 3; making round generic via the 
__round__ magic method, etc. But none of these required the change in rounding 
mode.

We need to recognise that there are various different contexts where the idea 
of "rounding" comes into play in a general-purpose language. Some examples:

1. FPU-level rounding for basic floating-point operations (addition, 
multiplication, sqrt, etc.)
2. Conversion of source-code decimal numeric literals (e.g., in "bad_pi = 
3.14") to the _nearest_ exactly representable binary float/double; the notion 
of _nearest_ needs some way to break ties.
3. Formatting a float for output as a string (format(my_float, ".2f"))
4. Rounding a float to the nearest integer (Python's single-argument "round")
5. Rounding a binary float to some number of decimal places (two-argument 
round), which is a rather more subtle operation than it might seem at first 
sight

For 1., there's decades of numerical evidence that round-ties-to-even is what 
you want to do, and that's why IEEE 754 makes it the default rounding mode, and 
why it's the rounding mode you're likely to be using for numeric work out of 
the box in any mainstream language. [For one demonstration of where the 
unbiasedness of round-ties-to-even can matter, see 
https://stackoverflow.com/a/45245802/270986. Apologies for linking to my own 
answer here, but it was easily accessible. I'm sure there are many better 
demonstrations out there.]

Case 2 is really a special case of 1. Though not (usually) FPU-supported: you 
can think of conversion from decimal string to binary floating-point as another 
primitive floating-point operation, and it's one that's covered by IEEE 754; 
round-ties-to-even (or at least, some precision- or algorithm-limited 
_approximation_ to round-ties-to-even) is again a common default across 
languages and operating systems.

Case 3 is also covered by IEEE 754, and I believe that "most" languages use 
round-ties-to-even here, too. C's fprintf (for example) specifies that e-style, 
f-style, and g-style formatting should be "correctly rounded" (C99 
7.19.6.1p13), where "correctly rounded" means "[...] nearest in value, subject 
to the current rounding mode [...]" (C99 3.9); in practice, that's usually 
round-ties-to-even. Java's DecimalFormat uses round-ties-to-even by default 
(source:  
https://docs.oracle.com/javase/7/docs/api/java/text/DecimalFormat.html). I 
haven't checked other languages, but I expect that many of them do something 
similar.

Cases 4 and 5 are mostly what we're arguing about in this issue. It's much less 
clear to me that the numerical benefits are 

[issue34762] Change contextvars C API to use PyObject

[Yury Selivanov]

Change by Yury Selivanov :


--
pull_requests: +9007
stage: resolved -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34820] binascii.c:1578:1: error: the control flow of function ‘binascii_crc32’ does not match its profile data (counter ‘arcs’)

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Thanks for the report. 

* Can you please add the OS, GCC details, full traceback and command that 
triggers the error? 
* Is it consistent?
* Are you adding a flag to treat all the warnings as errors? 

I assume you are running PGO. As I can see on the bots for PGO they are green 
on Debian for the commit : 
https://buildbot.python.org/all/#/builders/47/builds/1609/steps/2/logs/stdio .  
There is a issue related to PGO that causes internal compiler error but I don't 
think it's related here and it suggests doing make clean since previous build 
artifacts might cause an issue : https://bugs.python.org/issue31963#msg305708

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34820] binascii.c:1578:1: error: the control flow of function ‘binascii_crc32’ does not match its profile data (counter ‘arcs’)

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34821] Crash after run Python interpreter from removed directory

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Just installed Python 3.4 and can confirm this is fixed in 3.5 but exists on 
Python 3.4.9 though I don't receive any segfaults though as in the picture with 
3.4.

karthi@ubuntu-s-1vcpu-1gb-blr1-01:~$ mkdir /tmp/foo
karthi@ubuntu-s-1vcpu-1gb-blr1-01:~$ cd /tmp/foo
karthi@ubuntu-s-1vcpu-1gb-blr1-01:/tmp/foo$ rm -rf /tmp/foo
karthi@ubuntu-s-1vcpu-1gb-blr1-01:/tmp/foo$ python3.4
Python 3.4.9 (default, Aug  3 2018, 23:38:40)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
Failed calling sys.__interactivehook__
Traceback (most recent call last):
  File "/usr/lib/python3.4/site.py", line 419, in register_readline
import readline
  File "", line 2237, in _find_and_load
  File "", line , in _find_and_load_unlocked
  File "", line 2164, in _find_spec
  File "", line 1940, in find_spec
  File "", line 1911, in _get_spec
  File "", line 1879, in _path_importer_cache
FileNotFoundError: [Errno 2] No such file or directory
>>>
karthi@ubuntu-s-1vcpu-1gb-blr1-01:/tmp/foo$ python3.5
Python 3.5.2 (default, Nov 23 2017, 16:37:01)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

Python 3.4.9 is the latest release and Python 3.4.10 will be released in March 
2019 according to https://www.python.org/dev/peps/pep-0429/#release-schedule . 
Looking at the patch for this issue and since 3.4 is in security fixes mode I 
think this fix has a less probability of getting backported which I think is an 
explicit decision as I can see from the issue.


Thanks

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34825] Add more entries to os module to pathlib reference table

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
keywords: +patch
pull_requests: +9006
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34022] 6 tests fail using SOURCE_DATE_EPOCH env var

[Elvis Pranskevichus]

Elvis Pranskevichus  added the comment:

https://github.com/python/cpython/pull/9607 turns SOURCE_DATE_EPOCH into a 
*default* rather than absolute override.

test_cmd_line_script and test_runpy fail due to the lack of support for 
hash-based .pycs in zipimport, which needs fixing also.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34825] Add more entries to os module to pathlib reference table

[Karthikeyan Singaravelan]

New submission from Karthikeyan Singaravelan :

I found issue33194 where unlink from pathlib can be used to delete a file or 
symlink similar to os.unlink or os.remove but the visibility for the function 
was low. I found there is a reference table with os module functions and their 
equivalent pathlib methods. I think this can be improved by adding an entry for 
Path.unlink as an equivalent for os.remove and os.unlink. I would suggest 
adding some more functions to this table. The added functions are as below : 

:func:`os.chmod`   :meth:`Path.chmod`
:func:`os.mkdir`   :meth:`Path.mkdir`
:func:`os.rename`  :meth:`Path.rename`
:func:`os.replace` :meth:`Path.replace`
:func:`os.rmdir`   :meth:`Path.rmdir`
:func:`os.remove`, :func:`os.unlink`   :meth:`Path.unlink` 
:func:`os.path.samefile`   :meth:`Path.samefile`

The only issue is that some functions have a dir_fd=None in os module and I 
think this table is used for functions that do equivalent actions and not as 
drop-in replacements with equal type signatures. Feedback welcome. Since the 
table is present only on 3.7 and master I am leaving 3.6 for versions. I will 
push a PR shortly for this.

Thanks

--
assignee: docs@python
components: Documentation
messages: 326577
nosy: docs@python, xtreak
priority: normal
severity: normal
status: open
title: Add more entries to os module to pathlib reference table
type: enhancement
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34821] Crash after run Python interpreter from removed directory

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Thanks for the report and steps. Can you try this on Python 3.5 ? I think this 
is the related issue issue22834 that was not fixed in Python 3.4 and has the 
same line numbers. I can't reproduce this on my Ubuntu machine with Python 
3.6.5 from Anaconda though in the path

Steps to reproduce as below : 

karthi@ubuntu-s-1vcpu-1gb-blr1-01:~$ ls /tmp/foo
ls: cannot access '/tmp/foo': No such file or directory

# Create the directory
karthi@ubuntu-s-1vcpu-1gb-blr1-01:~$ mkdir /tmp/foo
karthi@ubuntu-s-1vcpu-1gb-blr1-01:~$ cd /tmp/foo

# Load a REPL

karthi@ubuntu-s-1vcpu-1gb-blr1-01:/tmp/foo$ python3
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>

# Delete the directory
karthi@ubuntu-s-1vcpu-1gb-blr1-01:/tmp/foo$ rm -rf /tmp/foo/
karthi@ubuntu-s-1vcpu-1gb-blr1-01:/tmp/foo$ rm -rf /tmp/foo

# Load the REPL

karthi@ubuntu-s-1vcpu-1gb-blr1-01:/tmp/foo$ python3
Python 3.6.5 |Anaconda, Inc.| (default, Apr 29 2018, 16:14:56)
[GCC 7.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>>
karthi@ubuntu-s-1vcpu-1gb-blr1-01:/tmp/foo$ lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:Ubuntu 16.04.5 LTS
Release:16.04
Codename:   xenial
karthi@ubuntu-s-1vcpu-1gb-blr1-01:/tmp/foo$ uname -a
Linux ubuntu-s-1vcpu-1gb-blr1-01 4.4.0-127-generic #153-Ubuntu SMP Sat May 19 
10:58:46 UTC 2018 x86_64 x86_64 x86_64 GNU/Linux

# Python 2

karthi@ubuntu-s-1vcpu-1gb-blr1-01:/tmp/foo$ python2
Python 2.7.12 (default, Dec  4 2017, 14:50:18)
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>


Hope it helps! Unfortunately I don't have Python 3.4 on my machine to give this 
a try. If you are not able to reproduce this with Python 3.5 then I propose 
closing this as duplicate linking to the other issue. Thanks again for the 
report!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34821] Crash after run Python interpreter from removed directory

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34762] Change contextvars C API to use PyObject

[Yury Selivanov]

Yury Selivanov  added the comment:

Right, I'll make a PR.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34762] Change contextvars C API to use PyObject

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Perhaps this change caused new compiler warnings:

In file included from ./Include/pytime.h:6:0,
 from ./Include/Python.h:68,
 from /home/serhiy/py/cpython/Modules/_asynciomodule.c:1:
/home/serhiy/py/cpython/Modules/_asynciomodule.c: In function 
‘_asyncio_Task___init___impl’:
./Include/object.h:895:14: warning: assignment from incompatible pointer type 
[-Wincompatible-pointer-types]
 (op) = (op2);   \
  ^
/home/serhiy/py/cpython/Modules/_asynciomodule.c:1954:5: note: in expansion of 
macro ‘Py_XSETREF’
 Py_XSETREF(self->task_context, PyContext_CopyCurrent());
 ^~
/home/serhiy/py/cpython/Modules/_decimal/_decimal.c: In function 
‘init_current_context’:
/home/serhiy/py/cpython/Modules/_decimal/_decimal.c:1503:44: warning: passing 
argument 1 of ‘PyContextVar_Set’ from incompatible pointer type 
[-Wincompatible-pointer-types]
 PyContextToken *tok = PyContextVar_Set(current_context_var, tl_context);
^~~
In file included from ./Include/Python.h:116:0,
 from /home/serhiy/py/cpython/Modules/_decimal/_decimal.c:29:
./Include/context.h:63:24: note: expected ‘PyObject * {aka struct _object *}’ 
but argument is of type ‘PyContextVar * {aka struct _pycontextvarobject *}’
 PyAPI_FUNC(PyObject *) PyContextVar_Set(PyObject *var, PyObject *value);
^~~~
/home/serhiy/py/cpython/Modules/_decimal/_decimal.c:1503:27: warning: 
initialization from incompatible pointer type [-Wincompatible-pointer-types]
 PyContextToken *tok = PyContextVar_Set(current_context_var, tl_context);
   ^~~~
/home/serhiy/py/cpython/Modules/_decimal/_decimal.c: In function 
‘current_context’:
/home/serhiy/py/cpython/Modules/_decimal/_decimal.c:1517:26: warning: passing 
argument 1 of ‘PyContextVar_Get’ from incompatible pointer type 
[-Wincompatible-pointer-types]
 if (PyContextVar_Get(current_context_var, NULL, _context) < 0) {
  ^~~
In file included from ./Include/Python.h:116:0,
 from /home/serhiy/py/cpython/Modules/_decimal/_decimal.c:29:
./Include/context.h:56:17: note: expected ‘PyObject * {aka struct _object *}’ 
but argument is of type ‘PyContextVar * {aka struct _pycontextvarobject *}’
 PyAPI_FUNC(int) PyContextVar_Get(
 ^~~~
/home/serhiy/py/cpython/Modules/_decimal/_decimal.c: In function 
‘PyDec_SetCurrentContext’:
/home/serhiy/py/cpython/Modules/_decimal/_decimal.c:1564:44: warning: passing 
argument 1 of ‘PyContextVar_Set’ from incompatible pointer type 
[-Wincompatible-pointer-types]
 PyContextToken *tok = PyContextVar_Set(current_context_var, v);
^~~
In file included from ./Include/Python.h:116:0,
 from /home/serhiy/py/cpython/Modules/_decimal/_decimal.c:29:
./Include/context.h:63:24: note: expected ‘PyObject * {aka struct _object *}’ 
but argument is of type ‘PyContextVar * {aka struct _pycontextvarobject *}’
 PyAPI_FUNC(PyObject *) PyContextVar_Set(PyObject *var, PyObject *value);
^~~~
/home/serhiy/py/cpython/Modules/_decimal/_decimal.c:1564:27: warning: 
initialization from incompatible pointer type [-Wincompatible-pointer-types]
 PyContextToken *tok = PyContextVar_Set(current_context_var, v);
   ^~~~
/home/serhiy/py/cpython/Modules/_decimal/_decimal.c: In function 
‘PyInit__decimal’:
/home/serhiy/py/cpython/Modules/_decimal/_decimal.c:5542:25: warning: 
assignment from incompatible pointer type [-Wincompatible-pointer-types]
 current_context_var = PyContextVar_New("decimal_context", NULL);
 ^

--
nosy: +serhiy.storchaka
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29708] support reproducible Python builds

[Elvis Pranskevichus]

Change by Elvis Pranskevichus :


--
pull_requests: +9005

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34022] 6 tests fail using SOURCE_DATE_EPOCH env var

[Elvis Pranskevichus]

Change by Elvis Pranskevichus :


--
pull_requests: +9004

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34824] _ssl.c: Possible null pointer dereference

[Zackery Spytz]

Change by Zackery Spytz :


--
keywords: +patch
pull_requests: +9003
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34822] Simplify AST for slices

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +9002
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34824] _ssl.c: Possible null pointer dereference

[Yury Selivanov]

Change by Yury Selivanov :


--
nosy: +yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34824] _ssl.c: Possible null pointer dereference

[Zackery Spytz]

New submission from Zackery Spytz :

If _PyBytes_Resize() fails in _ssl_MemoryBIO_read_impl(), Py_DECREF() will be 
called on a null pointer.

--
assignee: christian.heimes
components: Extension Modules, SSL
messages: 326573
nosy: ZackerySpytz, christian.heimes
priority: normal
severity: normal
status: open
title: _ssl.c: Possible null pointer dereference
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34776] Postponed annotations break inspection of dataclasses

[INADA Naoki]

Change by INADA Naoki :


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34823] libffi detection doesn’t work in my setup

[Michael Stapelberg]

New submission from Michael Stapelberg :

(Tested with Python 3.7, but AFAICT, the situation hasn’t changed in Python 
3.8.)

Python’s configure tries to fill the LIBFFI_INCLUDEDIR system variable with the 
value of -I as returned by pkg-config libffi --cflags-only-I.

Python’s setup.py tries to determine whether libffi is present by searching a 
couple of well-known library directories.

Both of these checks fail in my environment (think Linux From Scratch). I’m 
setting the following environment variables:

CPATH=/home/michael/libffi-3.2.1/include
LIBRARY_PATH=/home/michael/libffi-3.2.1/lib64

gcc can build against libffi just fine:

% echo echo '#include   
> int main() {}' > foo.c
% gcc -o foo foo.c -lffi
% ldd foo
linux-vdso.so.1 (0x7ffe4450a000)
libffi.so.6 => /home/michael/libffi-3.2.1/lib64/libffi.so.6 
(0x7f3935aba000)
libc.so.6 => /home/michael/glibc-2.27/lib/libc.so.6 (0x7f3935902000)
/lib64/ld-linux-x86-64.so.2 => 
/home/michael/glibc-2.27/lib/ld-linux-x86-64.so.2 (0x7f3935aca000)

However, when compiling Python, I’m getting the following error message, 
resulting in the _ctypes module not being built:

INFO: Could not locate ffi libs and/or headers

Now, one issue is that pkg-config understands CPATH, so calling pkg-config 
--cflags-only-I is not sufficient to obtain the include directory, one also 
needs to clear CPATH:

% pkg-config libffi --cflags-only-I  

% CPATH= pkg-config libffi --cflags-only-I
-I/home/michael/libffi-3.2.1/include

After patching this in configure, LIBFFI_INCLUDEDIR is set correctly, but the 
build still fails, as the libffi shared object is located in my non-standard 
path, which setup.py doesn’t check.

Without knowing much about the Python internals, it seems to me that both of 
these issues would be fixed if Python stopped trying to find the libffi 
include/lib locations and just used pkg-config to detect the required CFLAGS 
and LDFLAGS, just like Python currently does with openssl.

Is there any reason we can’t use pkg-config for libffi like that?

Thanks!

--
components: Installation
messages: 326572
nosy: stapelberg
priority: normal
severity: normal
status: open
title: libffi detection doesn’t work in my setup
type: compile error
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34803] argparse int type does not accept scientific notation

[paul j3]

Change by paul j3 :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34776] Postponed annotations break inspection of dataclasses

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
nosy: +levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34815] Change Py_Ellipse __str__ behavior.

[tom.r]

tom.r  added the comment:

That's understandable. Thanks for considering it, closing the patch.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34822] Simplify AST for slices

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

Currently an AST for expressions can have non-terminal nodes of two types: expr 
and slice. Every of them can be a one of several kinds. A slice can be of kind 
Index (just an expression), Slice (optional expressions separated by ":") or 
ExtSlice (slices separated by ",").

For example, the expression "d[a, ..., b:c]" is represented as:

Subscript(
Name('d', Load()),
ExtSlice(
[
Index(Name('a', Load())),
Index(Constant(Ellipsis)),
Slice(Name('b', Load()), Name('c', Load()), None)
]
),
Load()
)

and the expression "d[a, ..., b]" is represented as:

Subscript(
Name('d', Load()),
Index(
Tuple(
[
Name('a', Load()),
Constant(Ellipsis),
Name('b', Load())
],
Load()
)
),
Load()
)

(note that ExtSlice is used only if there are slices in subexpressions).

I suggest to get rid of the separate slice type. The Slice kind will be a kind 
of the expr type instead of the slice type. The ExtSlice kind will be always 
replaced with a Tuple, even if subexpressions contain slices. Nodes of the 
Index kind will be replaced with expr nodes to which they are referred. For 
example, the expression "d[a, ..., b:c]" will be represented as:

Subscript(
Name('d', Load()),
Tuple(
[
Name('a', Load()),
Constant(Ellipsis),
Slice(Name('b', Load()), Name('c', Load()), None)
],
Load()
),
Load()
)

This will simplify the code for handling AST, especially the C code. The 
following PR removes around 400 lines of code (a half of them are generated, 
but others are handwritten). I hope that this regularization will help to write 
a general code for walking AST for expressions and remove more duplicated code 
in ast_opt.c, ast_unparse.c, and symtable.c. This even can help to solve a 
problem with crashes in handling too deep AST if implement the recursion 
without using the C stack (but this is dreams).

This change is more breaking than a change in issue32892. What will continue to 
work:

* The code for creating an AST when pass values as arguments: `Index(value)` 
will return just `value`, `ExtSlice(slices)` will return `Tuple(slices, 
Load())`.

* NodeVisitor based processing. Methods visit_Index() and visit_ExtSlice() will 
be never invoked. The semantic of visit_Slice() will be not changed. 
visit_Tuple() will be invoked instead of visit_ExtSlice() for extended slices.

* isinstance() and issubclass() checks for Slice. Subclassing of Slice.

What will no longer work (with the current implementation):

* The code that creates empty AST nodes and sets their attributes. `node = 
Index(); node.value = value` will no longer work.

* The code that reads attributes of just created Index and ExtSlice nodes. 
`Index(value)` will return just `value` instead of a new object whose attribute 
"value" is a specified value. A list of subexpressions of ExtSlice(slices) will 
be accessible as the "elts" attribute instead of "dims" (because it is a Tuple).

* isinstance() and issubclass() checks for Index and ExtSlice will always 
return False.

* Subclassing of Index and ExtSlice. Instantiating subclasses of Index and 
ExtSlice will return the same result as for Index and ExtSlice, i.e. not 
instance of these classes.

* The code that produces a Python code from AST will need to handle indexing 
with tuples specially (see Tools/parser/unparse.py) because d[(a, b)] is valid 
syntax (although parenthesis are redundant), but d[(a, b:c)] is not.

Some limitations are caused by the desire for simplicity and can be removed. 
For example it is possible to add the "dims" alias of "elts" to Tuple, and make 
subclasses working as before. It is more hard and inefficient to keep 
isinstance() checks and attribute access for Index. If some breakage for Index 
is not avoidable, I'm not sure that it is worth to spent efforts for imitating 
the old behavior for ExtSlice.

--
components: Interpreter Core, Library (Lib)
messages: 326570
nosy: benjamin.peterson, brett.cannon, gvanrossum, nascheme, ncoghlan, 
serhiy.storchaka, thautwarm, vstinner, yselivanov
priority: normal
severity: normal
status: open
title: Simplify AST for slices
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34700] typing.get_type_hints doesn't know about typeshed

[Ivan Levkivskyi]

Change by Ivan Levkivskyi :


--
assignee:  -> levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[STINNER Victor]

STINNER Victor  added the comment:

Is test_socket of Python 2.7 affected by the issue? If not, I suggest to remove 
2.7 from this issue and close it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32892] Remove specific constant AST types in favor of ast.Constant

[STINNER Victor]

STINNER Victor  added the comment:

Thank you Serhiy :-D I really love ast.Constant, it simplify many things when 
writing code modifying the AST.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34821] Crash after run Python interpreter from removed directory

[Игорь Никитин]

New submission from Игорь Никитин :

How to repeat: see screen.

For example, command "lsb_release" is also not found removed directory, but 
there is no crash.

Python2, in this case, run without problem.

--
files: python_bug.png
messages: 326567
nosy: Игорь Никитин
priority: normal
severity: normal
status: open
title: Crash after run Python interpreter from removed directory
type: crash
versions: Python 3.4
Added file: https://bugs.python.org/file47836/python_bug.png

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32892] Remove specific constant AST types in favor of ast.Constant

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset 3f22811fef73aec848d961593d95fa877f77ecbf by Serhiy Storchaka in 
branch 'master':
bpo-32892: Use ast.Constant instead of specific constant AST types. (GH-9445)
https://github.com/python/cpython/commit/3f22811fef73aec848d961593d95fa877f77ecbf


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32892] Remove specific constant AST types in favor of ast.Constant

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> Since Python 3.6 ;-)

Thank you for correction Victor. It is what I meant. Why I had written 3.8? :-?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34815] Change Py_Ellipse __str__ behavior.

[Guido van Rossum]

Guido van Rossum  added the comment:

I'm against this. The str() and repr() of Ellipsis are 'Ellipsis' for a reason: 
to remind the user that this is "just" an object, not a piece of special syntax.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34820] binascii.c:1578:1: error: the control flow of function ‘binascii_crc32’ does not match its profile data (counter ‘arcs’)

[wencan]

New submission from wencan :

git version: a94ee12c26aa8dd7dce01373779df8055aff765b

/home/wencan/Projects/github.com/python/cpython/Modules/binascii.c: In function 
‘binascii_crc32’:
/home/wencan/Projects/github.com/python/cpython/Modules/binascii.c:1578:1: 
error: the control flow of function ‘binascii_crc32’ does not match its profile 
data (counter ‘arcs’) [-Werror=coverage-mismatch]
 }
 ^
/home/wencan/Projects/github.com/python/cpython/Modules/binascii.c:1578:1: 
error: the control flow of function ‘binascii_crc32’ does not match its profile 
data (counter ‘time_profiler’) [-Werror=coverage-mismatch]

--
components: Build
messages: 326563
nosy: wencan
priority: normal
severity: normal
status: open
title: binascii.c:1578:1: error: the control flow of function ‘binascii_crc32’ 
does not match its profile data (counter ‘arcs’)
type: compile error
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

It looks like the test_socket is fixed but some related issues are still left 
in multiprocessing.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[STINNER Victor]

STINNER Victor  added the comment:

Commit merged into master:

commit 7291108d88ea31d205da4db19d202d6cbffc6d93
Author: Pablo Galindo 
Date:   Thu Sep 27 10:25:03 2018 +0100

Fix tests in test_socket to use correctly CMSG_LEN (GH-9594)

After some failures in AMD64 FreeBSD CURRENT Debug 3.x buildbots
regarding tests in test_socket that are using
testFDPassSeparateMinSpace(), FreeBDS revision 337423 was pointed
out to be the reason the test started to fail.

A close examination of the manpage for cmsg_space(3) reveals that
the number of file descriptors needs to be taken into account when
using CMSG_LEN().

This commit fixes tests in test_socket to use correctly CMSG_LEN, taking
into account the number of FDs.

3.7:

commit addef07ca7d7b6971d59c062c3229e91a99e5f5e (upstream/3.7, 3.7)
Author: Miss Islington (bot) <31488909+miss-isling...@users.noreply.github.com>
Date:   Thu Sep 27 06:30:47 2018 -0700

3.6:

commit fe48b6df101aac10dc846fa6fd1a41f877e77025 (upstream/3.6, 3.6)
Author: Miss Islington (bot) <31488909+miss-isling...@users.noreply.github.com>
Date:   Thu Sep 27 06:30:55 2018 -0700

(the commit message doesn't contain bpo-34521, so bots didn't added comments 
here about the commits)

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[Matthias Klose]

Matthias Klose  added the comment:

Debian/Ubuntu doesn't link against the library because it would add 
dependencies on all supported Python versions. Normally this is just during 
transition times, but e.g. for the upcoming Ubuntu 18.10 release we didn't 
finish the transition and so ship two Python3 versions. The packaging tools 
would add package dependencies on both 3.6 and 3.7 what you don't want.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34370] Tkinter scroll issues on macOS

[Kevin Walzer]

Kevin Walzer  added the comment:

Ned, please hold off a bit on this--another developer is doing some final 
fine-tuning of the scrolling code so it fully passes Tk's test suite. I'm 
waiting for the final commit of this code any day now.

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34763] Python lacks 0x4E17

[Steven D'Aprano]

Change by Steven D'Aprano :


--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34709] Suggestion: make getuser.getpass() also look at SUDO_USER environment variable

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Versions 3.7 and below are all in feature-freeze, so this change could only 
apply to 3.8 and above.

I don't know if this feature is desirable or not.

If it is (sometimes?) desirable, my guess is that it would be undesirable to 
use SUDO_USER *unless* the effective user ID was 0. (Don't check for the name 
"root", that's only a convention.) In pseudocode:

names = ('LOGNAME', 'USER', 'LNAME', 'USERNAME')
if effective user ID == 0:
names = ('SUDO_USER',) + names
for name in names:
...


Also needs documentation and tests.

--
nosy: +steven.daprano
versions:  -Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34816] ctypes + hasattr

[Lars Friedrich]

Lars Friedrich  added the comment:

Thank you for your reply.

I am not sure if I understood correctly:
Do you suggest to modify ctypes.__init__.py so that the __getattr__ method of 
LibraryLoader catches the OSError and raises an AttributeError instead, as in 
your example?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34816] ctypes + hasattr

[Eryk Sun]

Eryk Sun  added the comment:

ctypes.windll is an instance of ctypes.LibraryLoader, which has a __getattr__ 
method that calls ctypes.WinDLL(name) and caches the result as an instance 
attribute. I suppose with chained exceptions it's reasonable to handle OSError 
in __getattr__ by raising AttributeError. For example:

class A:
def __init__(self, name):
raise OSError

class B:
def __getattr__(self, name):
try:
A(name)
except OSError:
raise AttributeError

Demo:

>>> b = B()
>>> b.test
Traceback (most recent call last):
  File "", line 4, in __getattr__
  File "", line 3, in __init__
OSError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "", line 1, in 
  File "", line 6, in __getattr__
AttributeError

>>> hasattr(b, 'test')
False

FYI, I recommend avoiding the cdll and windll LibraryLoader instances. I wish 
they were deprecated because globally caching CDLL and WinDLL instances leads 
to conflicts between projects that use the same shared libraries.

--
nosy: +eryksun
stage:  -> test needed
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34576] [EASY doc] http.server, SimpleHTTPServer: warn users on security

[Felipe Rodrigues]

Felipe Rodrigues  added the comment:

Well, even if we do fix some security issues in SimpleHTTPServer, it doesn't 
change the fact that it shouldn't really be used for sensitive applications. I 
like how Django docs handles a similar issue regarding their development server 
(https://docs.djangoproject.com/en/2.1/ref/django-admin/#runserver)

> DO NOT USE THIS SERVER IN A PRODUCTION SETTING. It has not gone through 
> security audits or performance tests. (And that’s how it’s gonna stay. We’re 
> in the business of making Web frameworks, not Web servers, so improving this 
> server to be able to handle a production environment is outside the scope of 
> Django.)

I think that the same philosophy applies to SimpleHTTPServer. If the warning 
should be add to the docs, I'll be glad to issue an PR fixing it!

--
nosy: +fbidu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[miss-islington]

miss-islington  added the comment:


New changeset 2b01121fd4200f1c27873422f7f72d02eec08630 by Miss Islington (bot) 
in branch '3.7':
bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures 
(GH-9599)
https://github.com/python/cpython/commit/2b01121fd4200f1c27873422f7f72d02eec08630


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[Antoine Pitrou]

Change by Antoine Pitrou :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[orlnub123]

orlnub123  added the comment:

Happy to help! I'm surprised it got merged so quickly, amazing response times 
all-around.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[miss-islington]

miss-islington  added the comment:


New changeset 3a4aa6ac55e04c42757443d5b5854b6d893e0461 by Miss Islington (bot) 
in branch '3.6':
bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures 
(GH-9599)
https://github.com/python/cpython/commit/3a4aa6ac55e04c42757443d5b5854b6d893e0461


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[Antoine Pitrou]

Antoine Pitrou  added the comment:

But as you said, we need consistency: either *never* link to libpython, or 
*always* link to libpython.  You are proposing to always link, I'm arguing we 
should never link.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[STINNER Victor]

STINNER Victor  added the comment:

Oh, I missed the Lib/concurrent/futures/ directory when I tried to patch the 
whole stdlib to use monotonic clocks for timeouts :-( Thanks for the fix!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[miss-islington]

Change by miss-islington :


--
pull_requests: +9000

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[miss-islington]

Change by miss-islington :


--
pull_requests: +9001

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset a94ee12c26aa8dd7dce01373779df8055aff765b by Antoine Pitrou 
(orlnub123) in branch 'master':
bpo-34819: Use a monotonic clock to compute timeouts in concurrent.futures 
(GH-9599)
https://github.com/python/cpython/commit/a94ee12c26aa8dd7dce01373779df8055aff765b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[STINNER Victor]

STINNER Victor  added the comment:

> I'm not talking about the standard library obviously.  I don't remember my 
> original use case exactly, but I must have been compiling a C extension on a 
> system and expected it to work on another.

It seems like we are talking about two different things:

* My issue is restricted to the C extensions compiled by Makefile and setup.py: 
C extensions of the standard libraries
* You are talking about third party extensions: that's out of the scope of this 
issue, since my issue is about Modules/makesetup.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> Is it a real use case? Why would anyone use a RHEL binary on Debian? Debian 
> already provides the full standard library.

I'm not talking about the standard library obviously.  I don't remember my 
original use case exactly, but I must have been compiling a C extension on a 
system and expected it to work on another.

> C extensions of the standard library are tidily coupled to CPython. For 
> example, it may be dangerous to use a C extension of Python 2.7.5 on Python 
> 2.7.15.

I don't believe that.  Binary wheels uploaded to PyPI seem to work fine 
regardless of the exact bugfix version.

> Third party C extensions distributed as portable wheel packages using the 
> stable ABI is different use case.

Most wheel packages don't use the stable ABI.  They are tied to a Python 
version such as 2.7, but they don't differentiate between e.g. 2.7.5 and 
2.7.15.  We don't break the ABI between bugfix releases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[STINNER Victor]

STINNER Victor  added the comment:

> It means a C extension compiled with a shared-library Python cannot be 
> imported on a monolithic Python (which doesn't have libpython.so).  It's a 
> real problem when you want to redistribute compiled C extensions: if you 
> compile it on RedHat/CentOS, it won't work on Ubuntu/Debian (the reverse 
> works).

Is it a real use case? Why would anyone use a RHEL binary on Debian? Debian 
already provides the full standard library.

C extensions of the standard library are tidily coupled to CPython. For 
example, it may be dangerous to use a C extension of Python 2.7.5 on Python 
2.7.15.

I'm talking about the very specific case of C extensions which are part of the 
stdlib.

Third party C extensions distributed as portable wheel packages using the 
stable ABI is different use case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[STINNER Victor]

STINNER Victor  added the comment:

> * FreeBSD 12 (alpha): Python 2.7 and Python 3.6 are linked to libpython 
> (--enable-shared)

Note: _ctypes, _hashlib and _struct are all linked to libpython, on Python 2 
and Python 3.

Antoine:
> Do you realize libpython.so doesn't exist on Debian / Ubuntu?

No, I didn't :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[STINNER Victor]

STINNER Victor  added the comment:

Antoine:
> Why do you call this a bug?

"./configure --enable-shared && make" links C extensions to libpython. It's 
surprising that C extensions compiled by Makefile behave differently (not 
linked to libpython). We need consistency: either *never* link to libpython, or 
*always* link to libpython.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[STINNER Victor]

STINNER Victor  added the comment:

Extract of Antoine's comment on bpo-21536:
> (AFAIK, systems notorious for providing shared library Pythons are 
> RedHat-alike systems, while Debian/Ubuntu provide statically linked Pythons)

Oh. I didn't notice this major difference...

* Ubuntu 16.04: Python 2.7.12 and Python 3.5.2 are not linked to libpython
* Fedora 28: Python 2.7.15 and Python 3.6.6 are linked to libpython 
(--enable-shared)
* FreeBSD 12 (alpha): Python 2.7 and Python 3.6 are linked to libpython 
(--enable-shared)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Sorry, please ignore the comment reference to 
https://bugs.python.org/issue29733#msg289116 . I misunderstood that it was 
about concurrent.futures using time.time but it turns out it was about the 
program using time.time as verifying the PR against the issue still has the 
timeout error.

Thanks

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 27/09/2018 à 12:49, STINNER Victor a écrit :
> 
> I search if C extensions of the Python standard libraries are always linked 
> or not to libpython... it's complicated. I tested _ctypes, _hashlib and 
> _struct modules:
> 
> * Debian and Ubuntu: NOT linked to libpython

Do you realize libpython.so doesn't exist on Debian / Ubuntu?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[STINNER Victor]

STINNER Victor  added the comment:

Ah, it seems like the bpo-832799 (reported in 2003) is similar to the RHEL bug:
https://bugzilla.redhat.com/show_bug.cgi?id=1585201

Extract of the RHEL bug report:
---
pythontest.c:
#include 

int main(int argc, char *argv[])
{
void *pylib = dlopen("libpython2.7.so.1.0", RTLD_LOCAL | RTLD_NOW);
void (*Py_Initialize)(void) = dlsym(pylib, "Py_Initialize");
Py_Initialize();
int (*PyRun_SimpleStringFlags)(const char *, void *) = dlsym(pylib, 
"PyRun_SimpleStringFlags");
PyRun_SimpleStringFlags("import json\n", 0);
return 0;
}

2. Compile with "gcc -Wall -o pythontest pythontest.c -ldl -g"

3. Run ./pythontest -

Actual results:

it will fail with ImportError: /usr/lib64/python2.7/lib-dynload/_struct.so: 
undefined symbol: PyFloat_Type
---

The reporter is already aware of the fallback on RTLD_GLOBAL: "(optionally) 
change RTLD_LOCAL to RTLD_GLOBAL and see that it works".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34814] makesetup: must link C extensions to libpython when compiled in shared mode

[STINNER Victor]

STINNER Victor  added the comment:

I search if C extensions of the Python standard libraries are always linked or 
not to libpython... it's complicated. I tested _ctypes, _hashlib and _struct 
modules:

* Debian and Ubuntu: NOT linked to libpython
* Conda: LINKED to libpython
* Mageia 7: LINKED to libpython
* Fedora 28, RHEL 7: LINKED to libpython on Python 2.7 and 3.6, except _struct 
which is NOT linked to libpython on Python 2.7

It means that using dlopen("libpython2.7.so.1.0", RTLD_LOCAL | RTLD_NOW) may or 
may not work depending on the Linux distribution and depending on the imported 
C extensions...

If we use the example of Fedora: some C extensions are compiled using Makefile 
(the Fedora package modifies Modules/Setup, as I showed previously), but others 
are compiled by setup.py. For example, _ctypes and _hashlib are compiled by 
setup.py.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[miss-islington]

Change by miss-islington :


--
pull_requests: +8999

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[miss-islington]

Change by miss-islington :


--
pull_requests: +8998
stage: backport needed -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Thanks for the report and PR. I can find a very similar issue where time.time 
was changed to time.monotonic across multiprocessing module by Victor for the 
same reason : issue34054. I don't know why it wasn't changed in 
concurrent.futures . Seems this was reported here : 
https://bugs.python.org/issue29733#msg289116. I am adding Victor and Antoine 
who might have thoughts on this. Feel free to remove yourself if it's not 
relevant.

Thanks

--
nosy: +pitrou, vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34521] test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on AMD64 FreeBSD CURRENT

[Kubilay Kocak]

Kubilay Kocak  added the comment:

Copying in my original email to Pablo that elucidated the test failure cause 
and resolution, including upstream commit/review/issue references for the 
recvmsg(2) changes IN CURRENT.

Full buildbot worker log attached

While I'm here, update list of versions (branches) affected.

--

The test suite fails:

  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/multiprocessing/reduction.py",
line 164, in recvfds
raise RuntimeError('received %d items of ancdata' %
RuntimeError: received 0 items of ancdata

For these two tests (further up in the log [1]) ...

test_fd_transfer
(test.test_multiprocessing_forkserver.WithProcessesTestConnection) ... FAIL
test_large_fd_transfer
(test.test_multiprocessing_forkserver.WithProcessesTestConnection) ... FAIL

Further up we see (in the initial test run) ...

4 tests failed:
test_multiprocessing_fork test_multiprocessing_forkserver
test_multiprocessing_spawn test_socket

Then further up we see:

==
FAIL: testFDPassSeparateMinSpace
(test.test_socket.RecvmsgSCMRightsStreamTest)
--
Traceback (most recent call last):
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3186, in testFDPassSeparateMinSpace
self.checkRecvmsgFDs(2,
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3107, in checkRecvmsgFDs
self.assertEqual(len(fds), numfds)
AssertionError: 1 != 2
==
FAIL: testFDPassSeparateMinSpace
(test.test_socket.RecvmsgIntoSCMRightsStreamTest)
--
Traceback (most recent call last):
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3186, in testFDPassSeparateMinSpace
self.checkRecvmsgFDs(2,
  File
"/usr/home/buildbot/python/3.x.koobs-freebsd-current.nondebug/build/Lib/test/test_socket.py",
line 3107, in checkRecvmsgFDs
self.assertEqual(len(fds), numfds)
AssertionError: 1 != 2
--
Ran 554 tests in 27.636s
FAILED (failures=2, skipped=57)

I looked for recvmsg() changes in FreeBSD recently, and found ...

Commit: https://svnweb.freebsd.org/base?view=revision=337423
Review: https://reviews.freebsd.org/D16561
Bug ID: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=131876
Bug Desc:[socket] FD leak by receiving SCM_RIGHTS by recvmsg with small
control message buffer

The commit/bug/review details above point to recvmsg specific changes,
specifically, but maybe not *just*, for SCM_RIGHTS messages, which (in
my opinion) is way too similar to the test case specifics not to be related.

Also, the commit date of Aug 7 16:36:48 2018  seems right around the
time I updated the machine and the tests began to fail.

I can't say yet whether or not a bug crept in with the change, *but*,
the commit does say that it *fixes* bugs, which the python tests, or the
socket code in Python may be have been relying on.

It may also be that FreeBSD specific-behaviour isn't being
handled by the code and/or tests (truncated now where it wasn't before?)

It could also be the buffer sizes the socket code/tests use.

--

--
keywords: +buildbot
stage: patch review -> backport needed
title: test_socket.RecvmsgIntoSCMRightsStreamTest fails on AMD64 FreeBSD 
CURRENT Debug 3.x -> test_socket.Recvmsg[Into]SCMRightsStreamTest tests fail on 
AMD64 FreeBSD CURRENT
versions: +Python 2.7, Python 3.6, Python 3.7, Python 3.8
Added file: 
https://bugs.python.org/file47835/freebsd-current.python.3.x.build.301.stdio.log

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34819] Executor.map and as_completed timeouts are able to deviate

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com