[issue34155] email.utils.parseaddr mistakenly parse an email

[Larry Hastings]

Larry Hastings  added the comment:

All PRs merged.  Thanks, everybody!

--
resolution:  -> fixed
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37835] typing.get_type_hints wrong namespace for forward-declaration of inner class

[Netzeband]

Netzeband  added the comment:

Thanks for your explanations. 

It sounds, that there is no way to fix this issue officially. Only the 
meta-class workaround is left, which is - as you highlight - really just a 
workaround.

I anyway always wonder, why functions, which are methods, do not hold a 
reference to the class, which they belong to. It could help here and second, 
this would be a good way to differentiate if a function object is a method or 
not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34155] email.utils.parseaddr mistakenly parse an email

[Larry Hastings]

Larry Hastings  added the comment:


New changeset 063eba280a11d3c9a5dd9ee5abe4de640907951b by larryhastings 
(Abhilash Raj) in branch '3.5':
[3.5] bpo-34155: Dont parse domains containing @ (GH-13079) (#15317)
https://github.com/python/cpython/commit/063eba280a11d3c9a5dd9ee5abe4de640907951b


--
nosy: +larry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24416] Have date.isocalendar() return a structseq instance

[Tim Peters]

Tim Peters  added the comment:

I favor making this a structseq, primarily based on Paul's attempt to find 
actual use cases, which showed that named member access would be most useful 
most often.  I have no intuition for that myself, because while I wrote the 
original functions here, I've never used them and never will ;-)  But if I ever 
did, I'd probably want the ISO week number, and would appreciate not having to 
remember which meaningless index means "week".

Don't care about moving pickles to older versions.

Don't care about the putative speed hit - it's still measured in nanoseconds, 
so can still create over a million per second.  In apps with a large number of 
dates, they're typically not _calculated_, but read up from a relatively 
snail-like database or network connection.  I doubt anyone would notice if 
`.isocalendar()` got 10x slower.

Note:  I'm unassigning myself, because I have nothing else to say :-)

--
assignee: tim.peters -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15382] os.utime() mishandles some arguments

[Dong-hee Na]

Dong-hee Na  added the comment:

@vstinner
This issue is not solved yet.
Can I finalize this issue?

--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38037] reference counter issue in signal module

[Ma Lin]

Change by Ma Lin :


--
title: Assertion failed: object has negative ref count -> reference counter 
issue in signal module

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38015] inline function generates slightly inefficient machine code

[Ma Lin]

Ma Lin  added the comment:

This range has not been changed since "preallocated small integer pool" was 
introduced:

#define NSMALLPOSINTS   257
#define NSMALLNEGINTS   5

The commit (Jan 2007):
https://github.com/python/cpython/commit/ddefaf31b366ea84250fc5090837c2b764a04102


Is it worth increase the range?
FYI, build with MSVC 2017, the `small_ints` size:

32-bit build:
sizeof(PyLongObject)16 bytes
sizeof(small_ints)4192 bytes

64-bit build:
sizeof(PyLongObject)32 bytes
sizeof(small_ints)8384 bytes

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38003] Change 2to3 to replace 'basestring' with '(str,bytes)'

[Terry J. Reedy]

Change by Terry J. Reedy :


--
nosy: +benjamin.peterson
title: Incorrect "fixing" of isinstance tests for basestring -> Change 2to3 to 
replace 'basestring' with '(str,bytes)'
versions: +Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38003] Incorrect "fixing" of isinstance tests for basestring

[Josh Rosenberg]

Josh Rosenberg  added the comment:

basestring in Python 2 means "thing that is logically text", because in Python 
2, str can mean *either* logical text *or* binary data, and unicode is always 
logical text. str and unicode can kinda sorta interoperate on Python 2, so it 
can make sense to test for basestring if you're planning to use it as logical 
text; if you do 'foo' + u'bar', that's fine in Python 2. In Python 3, only str 
is logically text; b'foo' + 'bar' is completely illegal, so it doesn't make 
sense to convert it to recognize both bytes and str.

Your problem is that you're using basestring incorrectly in Python 2, and it 
happens to work only because Python 2 did a bad job of separating text and 
binary data. Your original example code should actually have been written in 
Python 2 as:


if isinstance(value, bytes):  # bytes is an alias of str, and only str, on 2.7
value = value.decode(encoding)
elif not isinstance(value, unicode):
some other code

which 2to3 would convert correctly (changing unicode to str, and leaving 
everything else untouched) because you actually tested what you meant to test 
to control the actions taken:

1. If it was binary data (which you interpret all Py2 strs to be), then it is 
decoded to text (Py2 unicode/Py3 str)
2. If it wasn't binary data and it wasn't text, you did something else

Point is, the converter is doing the right thing. You misunderstood the logical 
meaning of basestring, and wrote code that depended on your misinterpretation, 
that's all.

Your try/except to try to detect Python 3-ness was doomed from the start; you 
referenced basestring, and 2to3 (reasonably) converts that to str, which breaks 
your logic. You wrote cross-version code that can't be 2to3-ed because it's 
*already* Python 3 code; Python 3 code should never be subjected to 2to3, 
because it'll do dumb things (e.g. change print(1, 2) to print((1, 2))); it's 
2to3, not 2or3to3 after all.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38047] multiarch headers are added when cross compiling

[Haibo Huang]

Change by Haibo Huang :


--
components: Cross-Build
nosy: Alex.Willmer, hhb
priority: normal
pull_requests: 15375
severity: normal
status: open
title: multiarch headers are added when cross compiling
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38040] Typo: "Writeable" Should be "Writable" in IO Library Documentation

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Wiktionary and dictionary.com see 'writeable' as an alternate spelling of 
'writable', while merriam-webster.com does not.  Neither of the former say 
anything about American versus British usage.  I wonder if the first two are 
rubber-stamping what used to be a misspelling.  Or maybe the unabridged 
merriam-webster behind a paywall has it as a rare variant (but if so, I would 
expect mention in the 'writable' entry.  If it is British, it should be 
changed.  If it is misleading, ditto.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38007] Regression: name of PyType_Spec::slots conflicts with Qt macro

[Terry J. Reedy]

Terry J. Reedy  added the comment:

My initial thought is similar to that of Michael Hudson in msg23757, that 
Python should not be hostage to foreign code processors.  I also note that the 
2010 patch is by Martin Loewis, who was part of the 2006 discussion, and that 
the 2010 patch was about the stable public ABI, which should not be changed.  
But I am not familiar with C details at all.

--
nosy: +terry.reedy, vstinner
versions:  -Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38005] Coercing strings and non-integer numbers to interpreter ID and channel ID

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Regardless of the intention of the _xx... author, intransitive equality is 
objectionable as it violates the assumption of sets and dicts.  We went through 
this before with decimal.Decimal when the original implementation had 1.0 == 1 
== Decimal(1) but 1.0 != Decimal(1).  The latter was fixed.  Here, I think the 
fix should be to not make the id be equal to everything it can be derived from.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38003] Incorrect "fixing" of isinstance tests for basestring

[Bob Kline]

Bob Kline  added the comment:

> Unless you have a specific proposal, ...

I _do_ have a specific proposal: replace `basestring` with `(str, bytes)`, 
which preserves the behavior of the original code. So, 

if isinstance(value, basestring)

becomes

if isinstance(value, (str, bytes))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38003] Incorrect "fixing" of isinstance tests for basestring

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Bob, this issue tracker is for managing patches to the cpython repository.  not 
for 'help me understand' requests,  The latter belong on, for instance, 
python-list.  Unless you have a specific proposal, other than leaving 
'basestring' alone(1), that we could reject or accept and implement, please 
close this(2) as 'not a bug' or whatever.

(1) Rejected as leaving code broken for 3.x.
(2) Discussion elsewhere *might* result in a concrete suggestion appropriate 
for this or a new issue.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37995] Multiline ast.dump()

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Much better.  I prefer the first version with closers on the last item.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37992] Change datetime.MINYEAR to allow for negative years

[Terry J. Reedy]

Change by Terry J. Reedy :


--
resolution:  -> rejected
stage:  -> resolved
status: open -> closed
versions: +Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38046] Can't use sort_keys in json.dumps with mismatched types

[Josh Rosenberg]

Josh Rosenberg  added the comment:

This is an exact duplicate of #25457.

--
nosy: +josh.r
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> json dump fails for mixed-type keys when sort_keys is specified

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38046] Can't use sort_keys in json.dumps with mismatched types

[Nick Timkovich]

Change by Nick Timkovich :


--
components: +Library (Lib)
title: JSON sorting type error -> Can't use sort_keys in json.dumps with 
mismatched types
versions: +Python 3.9 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38046] JSON sorting type error

[Nick Timkovich]

Nick Timkovich  added the comment:

It's not clear what you suggest, but it is likely better to alert the user that 
their keys have mismatched types than to suppress it by default.

Perhaps alongside the `sort_keys` argument, you would like a parameter that 
gets passed into `sorted()` when the items are sorted?

Perhaps an additional argument, or if sort_keys is a callable, use that as the 
`key` argument to sorted?

``` 
strange = {"1":"one", 2:"ii"}
json.dumps(strange, sort_keys=True, key=str)
json.dumps(strange, sort_keys=str)
# {"1": "one", 2: "ii"}
```

--
nosy: +nicktimko

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38046] JSON sorting type error

[guillard]

New submission from guillard :

If the dict included both str and int keys, sorted returns this error :
TypeError: '<' not supported between instances of 'int' and 'str'

It is a little bit problematic when used with default library behavior, as 
jsonify in Flask.

possible solution :
try:
items = sorted(dct.items())
except(TypeError):
items = sorted(dct.items(), key=lambda kv: str(kv[0]))

--
files: json_bug_exemple.py
messages: 351268
nosy: yoann934
priority: normal
severity: normal
status: open
title: JSON sorting type error
type: behavior
versions: Python 3.7
Added file: https://bugs.python.org/file48596/json_bug_exemple.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38041] IDLE Shell: Refine restart line

[miss-islington]

miss-islington  added the comment:


New changeset 084ba337a93facac4694459b460cf329d58d2b62 by Miss Islington (bot) 
in branch '3.7':
bpo-38041: Refine IDLE Shell restart lines.  (GH-15709)
https://github.com/python/cpython/commit/084ba337a93facac4694459b460cf329d58d2b62


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38041] IDLE Shell: Refine restart line

[miss-islington]

miss-islington  added the comment:


New changeset 4009a8522d774c36918005527dfb0975f389c8c2 by Miss Islington (bot) 
in branch '3.8':
bpo-38041: Refine IDLE Shell restart lines.  (GH-15709)
https://github.com/python/cpython/commit/4009a8522d774c36918005527dfb0975f389c8c2


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38045] enum.Flag instance creation is slow

[Antony Lee]

Antony Lee  added the comment:

Actually, microoptimizing _power_of_two is a red herring and the problem is the 
quadratic complexity of _decompose (_value2member_map_ becomes bigger and 
bigger at each iteration).  Replacing the implementation of _decompose by the 
following fixes the performance issue (the original snippet executes in ~150ms, 
which may be even more optimizable but already a 30x improvement :)), and still 
passes test_enum.py (from cpython 3.7.4):


def _decompose(flag, value):
"""Extract all members from the value."""
# _decompose is only called if the value is not named
not_covered = value
members = []
for member in flag:
member_value = member.value
if member_value and member_value & value == member_value:
members.append(member)
not_covered &= ~member_value
if issubclass(flag, IntFlag) and value > 0:
while not_covered:
new_value = 2 ** _high_bit(not_covered)
if new_value not in flag._value2member_map_:
# construct singleton pseudo-members
pseudo_member = int.__new__(flag, value)
pseudo_member._name_ = None
pseudo_member._value_ = value
flag._value2member_map_.setdefault(value, pseudo_member)
members.append(flag(new_value))
not_covered &= ~new_value
if not members and value in flag._value2member_map_:
members.append(flag._value2member_map_[value])
members.sort(key=lambda m: m._value_, reverse=True)
if len(members) > 1 and members[0].value == value:
# we have the breakdown, don't need the value member itself
members.pop(0)
return members, not_covered


Checking for issubclass(flag, IntFlag) is a bit ugly but that's because Flag 
and IntFlag really need two separate implementations of _decompose -- the 
former wants to error out on uncovered values, the latter should create them 
on-the-fly.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38041] IDLE Shell: Refine restart line

[Terry J. Reedy]

Terry J. Reedy  added the comment:


New changeset 38da805d563422cf1bb9cd9be24c73806840fe30 by Terry Jan Reedy in 
branch 'master':
 bpo-38041: Refine IDLE Shell restart lines.  (GH-15709)
https://github.com/python/cpython/commit/38da805d563422cf1bb9cd9be24c73806840fe30


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38041] IDLE Shell: Refine restart line

[miss-islington]

Change by miss-islington :


--
pull_requests: +15374
pull_request: https://github.com/python/cpython/pull/15720

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38041] IDLE Shell: Refine restart line

[miss-islington]

Change by miss-islington :


--
pull_requests: +15373
stage: commit review -> patch review
pull_request: https://github.com/python/cpython/pull/15719

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38039] Segfault when pickling dictionary with large pandas dataframes

[Ilya Valmianski]

Ilya Valmianski  added the comment:

As a sizing clarification, timed_dfs ~ 150GB, control_features ~30 GB, 
notime_dfs ~ 2GB.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38039] Segfault when pickling dictionary with large pandas dataframes

[Ilya Valmianski]

Ilya Valmianski  added the comment:

Below is the code. It segfaults with either dill or pickle on 3.6 and 3.7.

with open(output_path,'wb') as fout:

 dill.dump({
'timed_dfs': timed_dfs,   #large pandas 
dataframe with all but one columns being strings, one column is datetime
'notime_dfs'   : notime_dfs,  #large pandas 
dataframe with all elements being strings
'control_features' : control_features,#dictionary of 
lists of 3 element tuples of strings
'config'   : config,  #small-ish 
dictionary with ints, strings, and lists of int/str
'version'  : __VERSION__  #string
  },
  fout,
  protocol=4)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37380] subprocess.Popen._cleanup() "The handle is invalid" error when some old process is gone

[Eryk Sun]

Eryk Sun  added the comment:

> When a Popen instance is finalized by the garbage collector, the 
> internal handle is also finalized and closed despite the instance 
> being put on the active list. This results in _cleanup throwing 
> because the handle can no longer be used.

Thanks. I hadn't considered that. It can lead to unusual behavior like this 
when __del__ keeps an object alive. The object can't assume that referenced 
objects haven't already been finalized. All of the subsequently required 
instance methods should take this into account. For example, 
Popen._internal_poll could set the return code to -1 if self._handle.closed is 
true. That's at a higher level, and much more reasonable, than trying to handle 
ERROR_INVALID_HANDLE, which could mask bugs.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26868] Document PyModule_AddObject's behavior on error

[Brandt Bucher]

Brandt Bucher  added the comment:

It looks like the idiom of calling PyModule_AddObject without Py_DECREF'ing on 
the error condition (or even checking for it at all) has spread quite a bit 
since this first reported. I'm preparing a PR to fix the other call sites.

--
nosy: +brandtbucher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15088] Remove unused and undocumented PyGen_NeedsFinalizing() function

[STINNER Victor]

STINNER Victor  added the comment:

I merged Joannah's change: thanks.

If anyone uses the removed function, please complain before Python 3.9.0 
release :-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
title: PyGen_NeedsFinalizing is public, but undocumented -> Remove unused and 
undocumented PyGen_NeedsFinalizing() function
versions: +Python 3.9 -Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue15088] PyGen_NeedsFinalizing is public, but undocumented

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 74b662cf202753d224d82d5503974cce881f7436 by Victor Stinner 
(Joannah Nanjekye) in branch 'master':
bpo-15088 : Remove PyGen_NeedsFinalizing() (GH-15702)
https://github.com/python/cpython/commit/74b662cf202753d224d82d5503974cce881f7436


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22454] Adding the opposite function of shlex.split()

[Daniel Himmelstein]

Daniel Himmelstein  added the comment:

I am interested in shlex.join as a way to log subprocess.CompletedProcess.args 
as a string that users could run in their terminals. I initially assumed that 
this was also the scope of shlex.join. However, it seems that shlex.join does 
not accept all types of command arguments supported by subprocess, such as 
pathlib.Path objects. Should shlex.join support an split_command list that 
includes pathlib.Path objects and any other types supported by subprocess?

--
nosy: +dhimmel

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18049] Re-enable threading test on macOS

[Ronald Oussoren]

Ronald Oussoren  added the comment:

I think it is unlikely that the timeouts on Azure CI are related to this change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38015] inline function generates slightly inefficient machine code

[Sergey Fedoseev]

Sergey Fedoseev  added the comment:

I added similar patch that replaces get_small_int() with macro version, since 
it also induces unnecessary casts and makes machine code less efficient.

Example assembly can be checked at https://godbolt.org/z/1SjG3E.

This change produces tiny, but measurable speed-up for handling small ints:

$ python -m pyperf timeit -s "from collections import deque; consume = 
deque(maxlen=0).extend; r = range(256)" "consume(r)" 
--compare-to=../cpython-master/venv/bin/python --duplicate=1000
/home/sergey/tmp/cpython-master/venv/bin/python: . 1.03 us 
+- 0.08 us
/home/sergey/tmp/cpython-dev/venv/bin/python: . 973 ns +- 
18 ns

Mean +- std dev: [/home/sergey/tmp/cpython-master/venv/bin/python] 1.03 us +- 
0.08 us -> [/home/sergey/tmp/cpython-dev/venv/bin/python] 973 ns +- 18 ns: 
1.05x faster (-5%)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38033] Use After Free: PyObject_Free (valgrind)

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

Another thing also, is to be sure to utilize the python's suppression file by 
adding the --suppressions= to valgrind's command line invocation.

--
nosy: +cstratak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38015] inline function generates slightly inefficient machine code

[Sergey Fedoseev]

Change by Sergey Fedoseev :


--
pull_requests: +15372
pull_request: https://github.com/python/cpython/pull/15718

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37064] Feature request: option to keep/add flags to pathfix.

[Patrik Kopkan]

Change by Patrik Kopkan :


--
pull_requests: +15371
pull_request: https://github.com/python/cpython/pull/15717

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38033] Use After Free: PyObject_Free (valgrind)

[STINNER Victor]

STINNER Victor  added the comment:

By default, Python is optimized for performance but its pymalloc memory 
allocator causes false alarms in Valgrind.

> https://github.com/python/cpython/blob/master/Misc/README.valgrind

This or you can disable pymalloc at runtime using:

PYTHONMALLOC=malloc python3 ...

or:

PYTHONMALLOC=malloc_debug python3 ...

See:
https://docs.python.org/dev/c-api/memory.html
and
https://docs.python.org/dev/using/cmdline.html#envvar-PYTHONMALLOC

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24416] Have date.isocalendar() return a structseq instance

[STINNER Victor]

STINNER Victor  added the comment:

> It seems to affect performance.

Oh ok. So this change makes .isocalendar() slower and breaks the backward 
compatibility on the serialization. I'm not longer sure that it's worth it.

I defer the decision the datetime module maintainers: Alexander Beloposky and 
Paul Ganssle.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38037] Assertion failed: object has negative ref count

[Joannah Nanjekye]

Joannah Nanjekye  added the comment:

I will look after my next meeting which is in 10 minutes. In the meantime have 
you perused this PR : https://github.com/python/cpython/pull/15701 If it can 
solve this?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24011] Add error checks to PyInit_signal()

[STINNER Victor]

STINNER Victor  added the comment:

There is a regression related to reference count: see bpo-38037.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38037] Assertion failed: object has negative ref count

[STINNER Victor]

STINNER Victor  added the comment:

This issue is a Python 3.8 regression.

Joannah: Would you mind to have a look?

x = DefaultHandler = PyLong_FromVoidPtr((void *)SIG_DFL);
if (PyModule_AddObject(m, "SIG_DFL", x))
goto finally;

This change is not easy to read.

DefaultHandler must be a strong reference: finisignal() calls 
Py_CLEAR(DefaultHandler);

Previously, the code uses PyDict_SetItemString(d, "SIG_DFL", x): 
PyDict_SetItemString increases the reference counter, whereas 
PyModule_AddObject leaves the reference counter unchanged (yeah, it's a 
strange/bad C API).

I guess than an INCREF() is needed somewhere.

Compare it to:

int
PyModule_AddIntConstant(PyObject *m, const char *name, long value)
{
PyObject *o = PyLong_FromLong(value);
if (!o)
return -1;
if (PyModule_AddObject(m, name, o) == 0)
return 0;
Py_DECREF(o);
return -1;
}

--
nosy: +lukasz.langa, vstinner
priority: normal -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14689] make PYTHONWARNINGS variable work in libpython

[STINNER Victor]

STINNER Victor  added the comment:

I close the issue since it's now fixed ;-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions: +Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14689] make PYTHONWARNINGS variable work in libpython

[STINNER Victor]

STINNER Victor  added the comment:

I refactored deeply Modules/main.c and moved the code to Python/initconfig.c 
and Python/preconfig.c. There is now a public C API to access to all 
configuration parameters documented at:
https://docs.python.org/dev/c-api/init_config.html

The design document is the PEP 587 which has been implemented in Python 3.8.

If you use the "Python Configuration", you now get the same behaviour than 
system Python program ("python"). Or you can use "Isolated Configuration" but 
opt-in for environment variables. Or you can modify PyConfig.warnoptions:
https://docs.python.org/dev/c-api/init_config.html#c.PyConfig.warnoptions

So you know have plently of choices to use PYTHONWARNINGS env var when you 
embed Python.

Good news: Python 3.8 beta4 is now available for testing!

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38045] enum.Flag instance creation is slow

[STINNER Victor]

Change by STINNER Victor :


--
title: Flag instance creation is slow -> enum.Flag instance creation is slow

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38045] Flag instance creation is slow

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The fast method to check if the value is a power of two:

not value & (value - 1)

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38042] Crash on double list(lxml.etree.iterparse(f))

[STINNER Victor]

STINNER Victor  added the comment:

https://github.com/python/cpython/commit/8cbffc4d96d1da0fbc38da6f34f2da30c5ffd601
 has been merged after 3.7.4 has been released, correct. I will be part of the 
next 3.7.5 release.

About the workflow: we close a bug once the change is merged into the 
development branch, not when a release is published.

For the current 3.7.5 schedule, see:
https://www.python.org/dev/peps/pep-0537/#id4

In the meanwhile, avoid calling lxml.etree.iterparse(f) twice :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38045] Flag instance creation is slow

[Antony Lee]

New submission from Antony Lee :

Consider the following example

from enum import Flag
F = Flag("F", list("abcdefghijklm"))
for idx in range(2**len(F) - 1):
F(idx)

creating all possible combos of 13 flags, so 8192 instances (yes, I know the 
instances are cached later, but the initial cost is still significant).  
Locally, this takes over 4.5s to execute; profiling shows that ~30% of that 
time is spent executing enum._is_power_of_two(!):

def _power_of_two(value):
if value < 1:
return False
return value == 2 ** _high_bit(value)

Indeed, replacing the implementation of _power_of_two by

import math
_powers_of_two = {
2 ** i for i in range(math.ceil(math.log2(sys.maxsize)) + 1)}
def _power_of_two(value):
return (value in _powers_of_two if value <= sys.maxsize
else value == 2 ** _high_bit(value))

shaves off ~30% of the runtime (obviously this won't help for huge 
(>sys.maxsize) flag values, but these are rarer I would think).

An even better fix, I think, would be for Flag to cache `flags_to_check` 
(updating it at the same time as `_value2member_map_`) instead of recomputing 
it each time in _decompose

flags_to_check = [
(m, v)
for v, m in list(flag._value2member_map_.items())
if m.name is not None or _power_of_two(v)
]

as this actually needlessly introduces quadratic complexity when iterating over 
all possible combos (because the size of _value2member_map_ is growing).  
(Also, it's not actually clear to me how one can end up with unnamed 
power-of-two instances in _value2member_map?)

--
messages: 351244
nosy: Antony.Lee
priority: normal
severity: normal
status: open
title: Flag instance creation is slow
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38044] unittest causing segfaults with string malloc in c++ module

[Jeremy]

New submission from Jeremy :

If a unittest is written which accesses a module written in C++ (I used 
Pybind11 to allow Python access) which uses malloc for a string, a segmentation 
fault is caused. This is not replicated when malloc is used for some other data 
types such as int, char or char*

C++ code:

#include 
#include 
#include 
using namespace std;
void t(){
string * data = (string*)malloc(100*sizeof(string));
data[0] = "this is a test";
cout<

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38042] Crash on double list(lxml.etree.iterparse(f))

[Alexander Kurakin]

Alexander Kurakin  added the comment:

Ok, thanks!

Good that I was too lazy to open bug in June because our colleague did it 
better then me :)

Seems like the fix doesn't present in current 3.7 version (3.7.4). Will try in 
3.7.5 and will close

UPD: Oh, closed? Ok. But *no*, it's not fixed in current 3.7 version (in master 
- yes). I wrote: 3.7.4. Let's check each other instead of... Sorry. Thanks.

--
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38042] Crash on double list(lxml.etree.iterparse(f))

[STINNER Victor]

STINNER Victor  added the comment:

I understand that this bug is a duplicate of bpo-37467 that I fixed recently in 
3.7, 3.8 and master branches:

commit f9b7457bd7f438263e0d2dd1f70589ad56a2585e
Author: Victor Stinner 
Date:   Mon Jul 1 16:51:18 2019 +0200

bpo-37467: Fix PyErr_Display() for bytes filename (GH-14504)


> It has been fixed in 3.7+ by issue37467, but the fix was not backported to 
> 3.6.

Python 3.6 no longer accept bug fixes, only security fixes:
https://devguide.python.org/#status-of-python-branches

And this bug doesn't look like a security issue.

Python 2.7 doesn't seem to be affected by this issue.

In short, nothing can be one on the Python side: please upgrade to Python 3.7.

I close this issue as a duplicate of bpo-37467.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> sys.excepthook (PyErr_Display) does crash with SyntaxError 
which has a bytes filename

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37380] subprocess.Popen._cleanup() "The handle is invalid" error when some old process is gone

[STINNER Victor]

STINNER Victor  added the comment:

Steve Dower: "I triggered automatic backports to see how they go. Doesn't 
necessarily mean we'll merge them without looking more closely, but may as well 
check out what our automated tools suggest."

Thanks, I merged the backports.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27145] long_add and long_sub might return a new int where _ints[x] could be returned

[hongweipeng]

Change by hongweipeng :


--
pull_requests: +15370
pull_request: https://github.com/python/cpython/pull/15716

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38042] Crash on double list(lxml.etree.iterparse(f))

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Indeed, there is a bug in Python. It can be reproduced without lxml:

$ ./python -c "raise SyntaxError('error', (b'file', 1, 2, 'text'))"
Traceback (most recent call last):
  File "", line 1, in 
python: Objects/unicodeobject.c:397: _PyUnicode_CheckConsistency: Assertion 
`PyUnicode_Check(op)' failed.
Aborted (core dumped)

It has been fixed in 3.7+ by issue37467, but the fix was not backported to 3.6.

--
components: +Interpreter Core
nosy: +vstinner
resolution: third party -> 
stage: resolved -> 
status: closed -> open
versions:  -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37380] subprocess.Popen._cleanup() "The handle is invalid" error when some old process is gone

[STINNER Victor]

STINNER Victor  added the comment:

Thanks Ruslan Kuprieiev for the bug report and the fix. Thanks Eryk Sun (as 
usual!) for the great analysis of this issue.

Ok, the bug should now be fixed in 3.7, 3.8 and master branches.

I chose to not fix Python 2.7: see my previous comment.

See bpo-37410 for the follow-up: "[Windows] subprocess: close the handle when 
the process completes".

--
resolution:  -> fixed
stage: backport needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37380] subprocess.Popen._cleanup() "The handle is invalid" error when some old process is gone

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 1e2707d7e82aedf73c59772bc7aa228286323c3c by Victor Stinner (Miss 
Islington (bot)) in branch '3.7':
bpo-37380: subprocess: don't use _active on win (GH-14360) (GH-15706)
https://github.com/python/cpython/commit/1e2707d7e82aedf73c59772bc7aa228286323c3c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37380] subprocess.Popen._cleanup() "The handle is invalid" error when some old process is gone

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 4d1abedce9422473af2ac78047e55cde73208208 by Victor Stinner (Miss 
Islington (bot)) in branch '3.8':
bpo-37380: subprocess: don't use _active on win (GH-14360) (GH-15707)
https://github.com/python/cpython/commit/4d1abedce9422473af2ac78047e55cde73208208


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37380] subprocess.Popen._cleanup() "The handle is invalid" error when some old process is gone

[STINNER Victor]

STINNER Victor  added the comment:

> When a Popen instance is finalized by the garbage collector, the internal 
> handle is also finalized and closed despite the instance being put on the 
> active list. This results in _cleanup throwing because the handle can no 
> longer be used.

Right, that's a more practical case where this bug still occurs on Python 3.8 
and older. I would suggest to fix applications which should get a 
ResourceWarning warning in this case.

Chip Lynch wrote me an email saying that his team is impacted by the bug on 
Windows with Python 3.7.

I tried to write a simpler patch ignoring ERROR_INVALID_HANDLE, but then I read 
again this issue discussion: Eryk Sun and me agree that ignorning 
ERROR_INVALID_HANDLE is a bad idea, it can be worse: silence a real bug.

So I now agree to backport the fix from master to 3.7 and 3.8 branches.

I prefer to leave 2.7 unchanged even if it's affected. I don't want to take the 
risk of introducing another regression in 2.7. Moreover, subprocess.Popen has a 
__del__() method. Python 2.7 handles reference cycles differently than Python 
3: it doesn't implement finalizers (PEP 442). While Python 2.7 might be 
affected, it should be affected differently.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38042] Crash on double list(lxml.etree.iterparse(f))

[Alexander Kurakin]

Alexander Kurakin  added the comment:

Yes, I do. Please read all.

1) According to https://bugs.launchpad.net/lxml/+bug/1833050 (by lxml author's 
opinion) it's not a lxml bug.

2) I wouldn't said that but according to backtrace we have crash at Python-time.

3) Moreover crash existence depends on Python version!

Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38042] Crash on double list(lxml.etree.iterparse(f))

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

This tracker is for bugs in CPython (the C implementation of Python). lxml is 
not a part of the standard library. Please use a proper tracker for reporting a 
bug in lxml.

--
nosy: +serhiy.storchaka
resolution:  -> third party
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

[Kyle Stanley]

Kyle Stanley  added the comment:

> Any plan to reapply my change, or fix it?

I can try to help with this. I'm not the most familiar with the internals of 
asyncio, but I think it would provide a good learning experience.

--
nosy: +aeros167

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34037] asyncio: BaseEventLoop.close() shutdowns the executor without waiting causing leak of dangling threads

[STINNER Victor]

STINNER Victor  added the comment:

This bug still occurs randomly on the CI. Example on AppVeyor:
https://ci.appveyor.com/project/python/cpython/builds/27209898

test_reader_callback (test.test_asyncio.test_events.SelectEventLoopTests) ... ok
test_remove_fds_after_closing 
(test.test_asyncio.test_events.SelectEventLoopTests) ... ok
test_run_in_executor (test.test_asyncio.test_events.SelectEventLoopTests) ... ok
test_run_in_executor_cancel 
(test.test_asyncio.test_events.SelectEventLoopTests) ... Warning -- 
threading_cleanup() failed to cleanup 1 threads (count: 1, dangling: 2)
Dangling thread: 
Dangling thread: <_MainThread(MainThread, started 1916)>
ok

Any plan to reapply my change, or fix it?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com