[issue12915] Add inspect.locate and inspect.resolve

[Michael Felt]

Michael Felt  added the comment:

I am very busy with normal work this week. However I’ll attempt to add a pr 
with your (Victor”s)  suggestion. 

Sent from my iPhone

> On 29 Feb 2020, at 23:36, STINNER Victor  wrote:
> 
> 
> STINNER Victor  added the comment:
> 
>> File 
>> "/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/Lib/test/test_pkgutil.py",
>>  line 249, in test_name_resolution
>>   os.makedirs(d, exist_ok=True)
> (...)
>> UnicodeEncodeError: 'latin-1' codec can't encode characters in position 
>> 17-19: ordinal not in range(256)
> 
> The test should be modified to skip the current uw value in the loop 
> ("continue") if the filesystem encoding cannot encode the Unicode string 
> (catch UnicodeEncodeError on the makedirs() call).
> 
> --
> 
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39800] Inconsistent/incomplete disassembly of methods vs method source code

[S Murthy]

S Murthy  added the comment:

Yes, I know that a method body may contain more than just the return statement 
- I was referrring to the byte code for the method def f(x): return x**2.

I don't think the output of dis.dis is correct here for the source string of f 
- it doesn't make sense for example to iterate over Bytecode(f) and get only 
the instruction for the return statement, but then iterate over 
Bytecode(inspect.getsource(f)) to get only the byte code for the def - the 
documentation for dis.dis and Bytecode indicate that all the bytecode for a 
piece of code, whether specified as a method, callable, generator, async. 
generator, coroutine, class, or a source string, will be generated.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39800] Inconsistent/incomplete disassembly of methods vs method source code

[Steven D'Aprano]

Steven D'Aprano  added the comment:

Ah, I see now. I was using an older version of Python and the output of 
dis was different. It didn't recurse in to show the disassembly of the 
code object as well.

> The first block of instructions here are for the def statement, and 
> the second block for the return statement.

The first block of byte code is for the def statement, and the second 
block is for the code object, which may be more than just a return 
statement.

What would you expect the disassembly of this to show?

dis.dis("""
import y
def f(x):
a = 2*x - 1
return a**2

print('hello')
""")

Would you expect the disassembled code object to show up in that as 
well? I'm not sure what I would expect.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39805] Copying functions doesn't actually copy them

[Steven D'Aprano]

Steven D'Aprano  added the comment:

I think this is sufficient for a shallow copy.

import copy
import types

def copyfunction(func):
new = types.FunctionType(
func.__code__,
func.__globals__,
func.__name__,
func.__defaults__,
func.__closure__
)
vars(new).update(vars(func))
new.__annotations__.update(func.__annotations__)
if func.__kwdefaults__ is not None:
new.__kwdefaults__ = func.__kwdefaults__.copy()
return new

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39800] Inconsistent/incomplete disassembly of methods vs method source code

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> BTW how else are methods/functions are created in Python except via def?

Functions are objects like everything else in Python, so they have a 
type, which has a constructor:

from types import FunctionType

The documentation for FunctionType is a bit thin, so you may need to 
experiment a bit to get the details right, but it can be done.

Unfortunately copy.copy doesn't actually copy functions, which is in my 
opinion a bug (see #39805) but if you search the internet, you will find 
code that makes independent copies of function objects.

Methods are different from functions, and like functions, they 
too have a type with a constructor:

from types import MethodType

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39805] Copying functions doesn't actually copy them

[Steven D'Aprano]

New submission from Steven D'Aprano :

Function objects are mutable, so I expected that a copy of a function should be 
an actual independent copy. But it isn't.

py> from copy import copy
py> a = lambda: 1
py> b = copy(a)
py> a is b
True

This burned me when I modified the copy and the original changed too:

py> a.attr = 27  # add extra data
py> b.attr = 42
py> a.attr
42


`deepcopy` doesn't copy the function either.

--
components: Library (Lib)
messages: 363039
nosy: steven.daprano
priority: normal
severity: normal
status: open
title: Copying functions doesn't actually copy them
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39763] distutils.spawn should use subprocess (hang in parallel builds on QNX)

[Paul Ganssle]

Change by Paul Ganssle :


--
nosy:  -p-ganssle

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39763] distutils.spawn should use subprocess (hang in parallel builds on QNX)

[Elad Lahav]

Elad Lahav  added the comment:

"setup.py doesn't use multiprocessing. multiprocessing is super complex. Would 
it be possible to write a reproducer which doesn't use multiprocessing?"

But the problem is with the handling of fork() by Python modules, and 
specifically with multi-threaded fork()s. Without multiple processes there is 
no issue.

And setup.py does use use multiple processes via a thread pool, where each 
thread calls fork(), if it detects that "make" was invoked with the parallel 
(-j) option.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39804] timezone constants in time module inaccurate with negative DST (e.g. Ireland)

[Paul Ganssle]

New submission from Paul Ganssle :

>From a report on the dateutil tracker today, I found that `time.timezone` and 
>`time.altzone` are not accurate in Ireland (at least on Linux, not tested on 
>other platforms): https://github.com/dateutil/dateutil/issues/1009

Europe/Dublin in the modern era has the exact same rules as Europe/London, but 
the values for `isdst` are switched, so for Ireland GMT is the "DST" zone with 
a DST offset of -1H, and IST is the standard zone, while London has GMT as the 
standard zone and BST as a DST zone of +1h.

The documentation for the timezone constants here pretty clearly say that the 
DST zone should be the *second* value in tzname, and should be the offset for 
altzone: https://docs.python.org/3/library/time.html#timezone-constants

But when setting my TZ variable to Europe/Dublin I get the same thing as for 
Europe/London:

$ TZ=Europe/Dublin python -c \
  "from time import *; print(timezone); print(altzone); print(tzname)"

0
-3600
('GMT', 'IST')
$ TZ=Europe/London python -c \
  "from time import *; print(timezone); print(altzone); print(tzname)"
0
-3600
('GMT', 'BST')

This would be less of a problem if localtime() were *also* getting isdst wrong 
in the same way, but it's not:


$ TZ=Europe/London python -c \
  "from time import *; print(localtime())"
time.struct_time(tm_year=2020, tm_mon=3, tm_mday=1, tm_hour=2, tm_min=5, 
tm_sec=6, tm_wday=6, tm_yday=61, tm_isdst=0)

$ TZ=Europe/Dublin python -c \
  "from time import *; print(localtime())"
time.struct_time(tm_year=2020, tm_mon=3, tm_mday=1, tm_hour=2, tm_min=5, 
tm_sec=18, tm_wday=6, tm_yday=61, tm_isdst=1)


So now it seems that there's no way to determine what the correct timezone 
offset and name are based on isdst. I'm not entirely sure if this is an issue 
in our code or a problem with the system APIs we're calling. This code looks 
like a *very* dicey heuristic (I expect it would also have some problems with 
Morocco in 2017, even before they were using a type of negative DST, since they 
used DST but turned it off from May 21st to July 2nd): 
https://github.com/python/cpython/blob/0b0d29fce568e61e0d7d9f4a362e6dbf1e7fb80a/Modules/timemodule.c#L1612

One option might be to deprecate these things as sort of very leaky 
abstractions *anyway* and be done with it, but it might be nice to fix it if we 
can.

--
messages: 363037
nosy: belopolsky, lemburg, p-ganssle
priority: normal
severity: normal
status: open
title: timezone constants in time module inaccurate with negative DST (e.g. 
Ireland)
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Stefan Pochmann]

Stefan Pochmann  added the comment:

I misspoke. I do the insertions at -1 and -500 not on the same list but each on 
a fresh list (of length 2**20+1).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Stefan Pochmann]

Stefan Pochmann  added the comment:

I think I have a decent way to isolate the memmove vs for-loop times, in Python 
code. Here are example results, five rounds of inserting with list.insert or 
with slice assignment. The times for each of the two ways are pretty stable:

  insertslice
  0.024221  0.006754
  0.024157  0.006710
  0.024015  0.006734
  0.024206  0.006731
  0.024123  0.006769

For each run, I build a list of length 2**20 and append one value. Then I can 
insert 2**17 more values without the list internally resizing.

Then I measure inserting at index -1 and consider that the baseline, as it 
includes all the overhead costs like function calls and other differences 
between the two insertion methods.

Then I measure inserting at index -500 and subtract the baseline time. This 
difference should reflect how long the memmove or the for-loop took, as that's 
the difference between inserting at -1 and at -500. It's what I showed in those 
results above.

I chose -500 here because the use case I have in mind is 
sortedcontainers.SortedList, which I think mostly involves lists of length 1000 
or more (up to 2000), and insertions somewhere in the middle.

Results for index -50 instead:

  insertslice
  0.002479  0.002217
  0.002546  0.002113
  0.002566  0.002007
  0.002425  0.002081
  0.002555  0.002261

I'm btw running Python 3.8.1 32-bit on Windows 10 64-bit on an Intel i5-7200U.

Rest of this message is the benchmark code:

from timeit import repeat

statements = (
  'a.insert(-1,0)',
  'a.insert(-50,0)',
  'a[-1:0] = 0,',
  'a[-50:0] = 0,',
)

# Verify that the statements work and don't cause internal list resizes.
for stmt in statements:
a = [0] * 2**20
a.append(0)
size_before = a.__sizeof__()
stmt = compile(stmt, '', 'single')
for _ in range(2**17):
exec(stmt)
assert len(a) == 2**20 + 1 + 2**17
assert a.__sizeof__() == size_before

# Run the benchmark.
print('  insertslice')
for _ in range(5):
times = []
for stmt in statements:
t = min(repeat(stmt, 'a=[0]*2**20; a.append(0)', number=2**17, 
repeat=20))
times.append(t)
print('%10.6f' * 2 % (times[1] - times[0], times[3] - times[2]))

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Stefan Pochmann]

Change by Stefan Pochmann :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Stefan Pochmann]

Stefan Pochmann  added the comment:

I have better benchmarks now and am trying some more, though I guess we roughly 
agree about when memmove is faster and when it's slower but that the real 
question is how large the common case is. 

Do you know where I can find those past investigations? Was memmove *faster* 
for sizes *over* 16? What is the common case, i.e., how do people use 
list.insert?

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39599] ABI breakage between Python 3.7.4 and 3.7.5: change in PyGC_Head structure

[STINNER Victor]

STINNER Victor  added the comment:

> This happens on a custom C module (built via Cython) when using including 
>  with -DPy_BUILD_CORE. I'm not sure it'd happen otherwise.

If you opt-in for the internal C API, you should be prepared to have such 
issues.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35569] OSX: Enable IPV6_RECVPKTINFO

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

Proposed patch attached.

--
keywords: +patch
Added file: 
https://bugs.python.org/file48937/0001-bpo-35569-Expose-RFC-3542-socket-options-on-macOS.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39599] ABI breakage between Python 3.7.4 and 3.7.5: change in PyGC_Head structure

[Ned Deily]

Ned Deily  added the comment:

Mark:
> Or am I misunderstanding?

No, you are not misunderstanding :)

> ABI breakage shouldn't happen *at all* in bugfix releases, whether stable ABI 
> or not.

Absolutely.

Victor:
> Julien: You cannot use the same wheel package for all Python 3.7.x versions: 
> you need to either always compile at the installation, or ship a different 
> wheel package per micro Python 3.7.x version.

Just to be absolutely clear here, in general you *are* supposed to be able to 
use the same wheel package for all 3.7.x releases (or one built for any other 
3.x.x release series) as long as the extension modules in the wheel follow 
certain rules and the rules include don't use any Include/internal headers. If 
the extension module violates that restriction (as is apparently the case 
here), ABI compatibility across bugfix releases cannot be assured.

--
nosy: +ned.deily

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30097] Command-line option to suppress "from None" for debugging

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39712] Doc for `-X dev` option should mention PYTHONDEVMODE

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

> I wrote a whole new page for -X dev: 
> https://docs.python.org/dev/library/devmode.html#devmode

That's nice, but references for PYTHONFAULTHANDLER and PYTHONTRACEMALLOC are 
still missing. If you think the cross refs for -X dev and PYTHONDEVMODE 
clutters cmdline.html, I'll remove them from the PR.

Is it worth it adding https://docs.python.org/dev/library/devmode.html to 3.8 
and possibly 3.7 as well?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39599] ABI breakage between Python 3.7.4 and 3.7.5: change in PyGC_Head structure

[STINNER Victor]

STINNER Victor  added the comment:

> Understood; thanks. I agree that there shouldn't be a guarantee for private 
> undocumented calls.

I concur. That's a good summary. We don't provide ABI stability for Julien's 
use case.

Julien: You cannot use the same wheel package for all Python 3.7.x versions: 
you need to either always compile at the installation, or ship a different 
wheel package per micro Python 3.7.x version.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39786] Have the heaps library support max heap

[signing_agreement]

signing_agreement  added the comment:

Do you a suggestion for non-numeric types?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39697] Failed to build with --with-cxx-main=g++-9.2.0

[STINNER Victor]

STINNER Victor  added the comment:

> See also https://bugs.python.org/issue23644

This issue is unrelated and has been fixed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39697] Failed to build with --with-cxx-main=g++-9.2.0

[STINNER Victor]

STINNER Victor  added the comment:

_testembed is build by Makefile:

Programs/_testembed: Programs/_testembed.o $(LIBRARY) $(LDLIBRARY) $(PY3LIBRARY)
$(LINKCC) $(PY_CORE_LDFLAGS) $(LINKFORSHARED) -o $@ 
Programs/_testembed.o $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS)

with:

MAINCC= $(CC)
LINKCC= $(PURIFY) $(MAINCC)

I'm not sure how g++ landed in LINKCC or MAINCC.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35569] OSX: Enable IPV6_RECVPKTINFO

[Erlend Egeberg Aasland]

Change by Erlend Egeberg Aasland :


--
nosy: +erlendaasland

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39712] Doc for `-X dev` option should mention PYTHONDEVMODE

[STINNER Victor]

STINNER Victor  added the comment:

> An exception to that is `-X dev`, which can also be triggered by the 
> PYTHONDEVMODE variable, but that isn't mentioned in the CLI doc.

I wrote a whole new page for -X dev: 
https://docs.python.org/dev/library/devmode.html#devmode

This page starts with:

"It can be enabled using the -X dev command line option or by setting the 
PYTHONDEVMODE environment variable to 1."

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37266] Daemon threads must be forbidden in subinterpreters

[STINNER Victor]

STINNER Victor  added the comment:

> There will be a problem with `concurrent.futures.ProcessPoolExecutor`, which 
> currently launches its management thread as a daemon thread.

Please don't discuss in closed issues.

If you want to support concurrent.futures in subinterpreters, please open a 
separated RFE issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12915] Add inspect.locate and inspect.resolve

[STINNER Victor]

STINNER Victor  added the comment:

> File 
> "/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/Lib/test/test_pkgutil.py",
>  line 249, in test_name_resolution
>os.makedirs(d, exist_ok=True)
(...)
> UnicodeEncodeError: 'latin-1' codec can't encode characters in position 
> 17-19: ordinal not in range(256)

The test should be modified to skip the current uw value in the loop 
("continue") if the filesystem encoding cannot encode the Unicode string (catch 
UnicodeEncodeError on the makedirs() call).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39653] test_posix.TestPosixSpawnP.test_no_such_executable() fails with NotADirectoryError on Debian

[STINNER Victor]

Change by STINNER Victor :


--
title: test_posix fails during make test -> 
test_posix.TestPosixSpawnP.test_no_such_executable() fails with 
NotADirectoryError on Debian

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39653] test_posix fails during make test

[STINNER Victor]

STINNER Victor  added the comment:

And for this one?

import os
args=['no_such_executable']
os.execvpe(args[0], args, os.environ)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39653] test_posix fails during make test

[STINNER Victor]

STINNER Victor  added the comment:

Which exception do you get for the following code?

import os
args=['no_such_executable']
os.execve(args[0], args, os.environ)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39794] Add --without-decimal-contextvar option to use just threads in decimal

[Stefan Krah]

Change by Stefan Krah :


--
nosy: +boytsovea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39766] unittest's assertRaises removes locals from tracebacks

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39379] sys.path[0] is already absolute path

[STINNER Victor]

STINNER Victor  added the comment:

> Is sys.path[0] always absolute, or is it just a side-effect of the site 
> module (i.e. is it absolute even with -S)?

The absolute path is computed way before the site module is imported.

In Python 3.8, _PyPathConfig_ComputeSysPath0() computes sys.path[0] from 
sys.argv[0]. If the command line contains a script filename, this function uses 
realpath() on Unix and GetFullPathNameW() on Windows to get the absolute path. 
If -m is used, getcwd() is called.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39379] sys.path[0] is already absolute path

[STINNER Victor]

Change by STINNER Victor :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39794] Add --without-decimal-contextvar option to use just threads in decimal

[Stefan Krah]

Stefan Krah  added the comment:


New changeset c4ca1f8f24118dc5c29e16118fb35a13963af290 by Stefan Krah in branch 
'3.7':
[3.7] bpo-39794: Add --without-decimal-contextvar (GH-18702)
https://github.com/python/cpython/commit/c4ca1f8f24118dc5c29e16118fb35a13963af290


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39769] compileall.compile_dir(..., ddir="") omits the intermediate package paths when prepending the prefix

[Gregory P. Smith]

Change by Gregory P. Smith :


--
pull_requests: +18075
stage: commit review -> patch review
pull_request: https://github.com/python/cpython/pull/18718

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24518] json.dumps should accept key function for ``sort_keys``

[Erlend Egeberg Aasland]

Erlend Egeberg Aasland  added the comment:

I've rebased Catherine Devlin's patch onto 3.9 (and adjusted the docs a tiny 
bit). If anyone still is interested, I can create a PR (for 3.9) for this.

--
nosy: +erlendaasland
Added file: 
https://bugs.python.org/file48936/0001-bpo-24518-json.dumps-should-accept-key-function-for-.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39763] distutils.spawn should use subprocess (hang in parallel builds on QNX)

[STINNER Victor]

STINNER Victor  added the comment:

> Note the original problem stems when you using "setup.py -j ".

setup.py doesn't use multiprocessing. multiprocessing is super complex. Would 
it be possible to write a reproducer which doesn't use multiprocessing?

> Stack trace: (...)

Can you try to get the Python traceback? Try py-bt command in gdb, or attempt 
to use faulthandler.dump_traceback_later().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39548] Request fails when 'WWW-Authenticate' header for Digest Authentication does not contain 'qop'

[Senthil Kumaran]

Change by Senthil Kumaran :


--
assignee:  -> orsenthil
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39794] Add --without-decimal-contextvar option to use just threads in decimal

[Stefan Krah]

Stefan Krah  added the comment:


New changeset 4d7012410cf4f91cbca4c406f4747289c2802333 by Stefan Krah in branch 
'3.8':
[3.8] bpo-39794: Add --without-decimal-contextvar (GH-18702)
https://github.com/python/cpython/commit/4d7012410cf4f91cbca4c406f4747289c2802333


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39794] Add --without-decimal-contextvar option to use just threads in decimal

[Stefan Krah]

Change by Stefan Krah :


--
pull_requests: +18074
pull_request: https://github.com/python/cpython/pull/18714

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39548] Request fails when 'WWW-Authenticate' header for Digest Authentication does not contain 'qop'

[miss-islington]

miss-islington  added the comment:


New changeset e4686b79798f7a492dcbaa62cf51f4d07fd5ae78 by Miss Islington (bot) 
in branch '3.8':
bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest Auth (GH-18338)
https://github.com/python/cpython/commit/e4686b79798f7a492dcbaa62cf51f4d07fd5ae78


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39794] Add --without-decimal-contextvar option to use just threads in decimal

[Stefan Krah]

Change by Stefan Krah :


--
pull_requests: +18072
pull_request: https://github.com/python/cpython/pull/18713

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39548] Request fails when 'WWW-Authenticate' header for Digest Authentication does not contain 'qop'

[miss-islington]

miss-islington  added the comment:


New changeset cf347f3089631c3c2467e46ed609bfe67e539487 by Miss Islington (bot) 
in branch '3.7':
bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest Auth (GH-18338)
https://github.com/python/cpython/commit/cf347f3089631c3c2467e46ed609bfe67e539487


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39548] Request fails when 'WWW-Authenticate' header for Digest Authentication does not contain 'qop'

[miss-islington]

Change by miss-islington :


--
pull_requests: +18071
pull_request: https://github.com/python/cpython/pull/18712

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37266] Daemon threads must be forbidden in subinterpreters

[Kyle Stanley]

Kyle Stanley  added the comment:

> To me, yes.

If Victor is also on-board with the idea, I can look into writing a patch for 
it (after testing to verify that it allows us to change the daemon threads to 
normal threads in concurrent.futures without issues).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39548] Request fails when 'WWW-Authenticate' header for Digest Authentication does not contain 'qop'

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 2.0 -> 3.0
pull_requests: +18070
pull_request: https://github.com/python/cpython/pull/18711

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39548] Request fails when 'WWW-Authenticate' header for Digest Authentication does not contain 'qop'

[Senthil Kumaran]

Senthil Kumaran  added the comment:


New changeset 5e260e0fde211829fcb67060cfd602f4b679f802 by Stephen Balousek in 
branch 'master':
bpo-39548: Fix handling of 'WWW-Authenticate' header for Digest Auth (GH-18338)
https://github.com/python/cpython/commit/5e260e0fde211829fcb67060cfd602f4b679f802


--
nosy: +orsenthil

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39799] Never return base's fragment from urljoin (urllib.parse)

[Open Close]

Change by Open Close :


--
pull_requests: +18069
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18710

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39788] Exponential notation should return an int if it can

[Tim Peters]

Tim Peters  added the comment:

Then please take it to python-ideas?  The issue tracker isn't a right place for 
endless argument.  python-ideas is.  This is my last response here.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39788] Exponential notation should return an int if it can

[Marco Sulla]

Marco Sulla  added the comment:

> >>> int(1e100)
> 1159028911097599180468360808563945281389781327557747838772170381060813469985856815104

.
Oh my God... I'm just more convinced than before :-D

> Ya, this change will never be made - give up gracefully :-)

Well, even if it's Tim Peters himself that ask it to me :-) I can't.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39788] Exponential notation should return an int if it can

[Tim Peters]

Tim Peters  added the comment:

Ya, this change will never be made - give up gracefully :-)

1e100 and 10**100 aren't just of different types, they have different 
mathematical _values_ now:

>>> 1e100 == 10**100
False
>>> int(1e100)
1159028911097599180468360808563945281389781327557747838772170381060813469985856815104

Of course this has visible consequences, like:

>>> 1e100 % 1000
104.0
>>> int(1e100) % 1000
104
>>> 10**100 % 1000
0

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39788] Exponential notation should return an int if it can

[Marco Sulla]

Marco Sulla  added the comment:

All the examples you mentioned seems to me to fix code, instead of breaking it.

About 1e300**1, it's not a bug at all. No one can stop you to full your RAM 
in many other ways :-D

About conventions, it does not seems to me that Python cares about other 
languages very much, if it's more natural for normal people to expect a result 
instead of a consolidated one among devs. See `1 / 2 == 0.5`, for example.

> But by your own feature request, this would return an int and your 
"feature" would bite you

You're citing the *optional* extra to the original idea. We can agree it is not 
a good addition at all.

I continue to think that nE+m, where n and m are integers, should return an 
integer. If this can break old code, I'm the first to think it should not be 
implemented, but I don't see any problem (yet).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39803] _PyLong_FormatAdvancedWriter has an unnecessary str

[Andy Lester]

Change by Andy Lester :


--
pull_requests: +18068
pull_request: https://github.com/python/cpython/pull/18709

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39803] _PyLong_FormatAdvancedWriter has an unnecessary str

[Andy Lester]

Change by Andy Lester :


--
pull_requests:  -18067

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39803] _PyLong_FormatAdvancedWriter has an unnecessary str

[Andy Lester]

Change by Andy Lester :


--
pull_requests: +18067
pull_request: https://github.com/python/cpython/pull/18708

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34271] Please support logging of SSL master secret by env variable SSLKEYLOGFILE

[Johannes Frank]

Johannes Frank  added the comment:

Yes, I didn't revisit the issue since, but Malcolm is right. Implemented in
python 3.8.

Thanks to all the contributors.

On Sat, Feb 29, 2020 at 8:58 AM Malcolm Smith 
wrote:

>
> Malcolm Smith  added the comment:
>
> It looks like this has now been done and released. Can the issue be closed?
>
> --
> nosy: +Malcolm Smith
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39803] _PyLong_FormatAdvancedWriter has an unnecessary str

[Andy Lester]

Change by Andy Lester :


--
pull_requests:  -18066

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39803] _PyLong_FormatAdvancedWriter has an unnecessary str

[Andy Lester]

Change by Andy Lester :


--
keywords: +patch
pull_requests: +18066
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18707

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39803] _PyLong_FormatAdvancedWriter has an unnecessary str

[Andy Lester]

New submission from Andy Lester :

_PyLong_FormatAdvancedWriter has a PyObject *str that is never used.  Remove it.

--
components: Interpreter Core
messages: 363006
nosy: petdance
priority: normal
severity: normal
status: open
title: _PyLong_FormatAdvancedWriter has an unnecessary str

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39802] Ensure {get, set}_escdelay and {get, set}_tabsize only implemented when the extensions are activated

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
nosy: +jcea
versions: +Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39802] Ensure {get, set}_escdelay and {get, set}_tabsize only implemented when the extensions are activated

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
keywords: +patch
pull_requests: +18065
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18705

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39802] Ensure {get, set}_escdelay and {get, set}_tabsize only implemented when the extensions are activated

[Batuhan Taskaya]

New submission from Batuhan Taskaya :

Python can't build curses on Solaris because of extensions aren't activated

/export/home/isidentical/cpython/Modules/_cursesmodule.c: In function 
‘_curses_get_escdelay_impl’:
/export/home/isidentical/cpython/Modules/_cursesmodule.c:3272:28: error: 
‘ESCDELAY’ undeclared (first use in this function)
 return PyLong_FromLong(ESCDELAY);
^
/export/home/isidentical/cpython/Modules/_cursesmodule.c:3272:28: note: each 
undeclared identifier is reported only once for each function it appears in
/export/home/isidentical/cpython/Modules/_cursesmodule.c: In function 
‘_curses_set_escdelay_impl’:
/export/home/isidentical/cpython/Modules/_cursesmodule.c:3296:29: error: 
implicit declaration of function ‘set_escdelay’ 
[-Werror=implicit-function-declaration]
 return PyCursesCheckERR(set_escdelay(ms), "set_escdelay");
 ^
/export/home/isidentical/cpython/Modules/_cursesmodule.c: In function 
‘_curses_set_tabsize_impl’:
/export/home/isidentical/cpython/Modules/_cursesmodule.c:3335:29: error: 
implicit declaration of function ‘set_tabsize’ 
[-Werror=implicit-function-declaration]
 return PyCursesCheckERR(set_tabsize(size), "set_tabsize");

--
components: Build
messages: 363005
nosy: BTaskaya
priority: normal
severity: normal
status: open
title: Ensure {get,set}_escdelay and {get,set}_tabsize only implemented when 
the extensions are activated
type: compile error

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Marking this as "pending".  It is more of a personal research project than a 
bug report, feature request, or known optimization.

The underlying hypothesis is that compilers aren't smart enough to generate 
good code for a simple for-loop that moves data.

The evidence is weak as involves timing unusual cases, with significantly 
different calling patterns, and that make repeated calls to realloc() which can 
throw the results off wildly.

--
priority: normal -> low
status: open -> pending
versions: +Python 3.9 -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39794] Add --without-decimal-contextvar option to use just threads in decimal

[Stefan Krah]

Stefan Krah  added the comment:


New changeset 815280eb160af637e1347213659f9236adf78f80 by Stefan Krah in branch 
'master':
bpo-39794: Add --without-decimal-contextvar (#18702)
https://github.com/python/cpython/commit/815280eb160af637e1347213659f9236adf78f80


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39788] Exponential notation should return an int if it can

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> Integers are implicitly converted to floats in operations with floats. 
> How can this change break old code?

# Status quo
print(1e60 + 1)
=> prints 1e+60

# In the future:
=> prints 10001

# Status quo
1e300*1e300
=> returns inf (a float)

# In the future
=> returns 100...00 (600 zeroes, an int)

# Status quo:
1e300**1
=> instantly raises overflow error

# In the future
=> locks up the machine until you run out of memory

etc.

> > if you are worried about the performance
> 
> No, I'm worried about the expectations of coders.

Oh that's good then, because exponentional notation like 1e10 has been 
well known to return a float in Python for over a quarter of a century, 
and in dozens or hundreds of other languages too. It is, as far as I can 
tell, universally true that coders can distinguish floats from ints (in 
languages that distinguish them) using a simple test:

- if it contains a ".", "e" or "E", it's a float
- otherwise its an int

I don't know of any languages where numbers containing a dot or an "E" 
are ints. Do you?

> If I wanted a float, I'd wrote 1.0E2 

But by your own feature request, this would return an int and your 
"feature" would bite you:

if the "base number" has a number of decimal places <= the
exponent, the result should be an integer.

You would need to write 1.000E2 for it to work. Luckily for you, we're 
rejecting this idea, before it can fool you into writing buggy code :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Stefan Pochmann]

Stefan Pochmann  added the comment:

Good point, Tim. Although binarysort really moves very few slots (I think at 
most 62, and average like 13). That might not be representative for how people 
use list.insert, either. I think I mainly use it for the mentioned case of a 
"self-sorting list", either naively via bisect.insort with a single list or via 
sortedcontainers' SortedList using multiple lists (used it only once so far). 
If I'm not mistaken, SortedList has a default load factor of 1000 and splits at 
double that size.

I might do more reasonable benchmarks later (haven't read Raymond's reply yet).

In any case, binarysort does its own inserting, so at least it wouldn't be 
affected if list.insert were changed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39740] Select module fails to build on Solaris 11.4

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
nosy: +BTaskaya, jcea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Here's a diff you can try out:

diff --git a/Objects/listobject.c b/Objects/listobject.c
index 3c39c6444b..ac6c9cc07a 100644
--- a/Objects/listobject.c
+++ b/Objects/listobject.c
@@ -272,8 +272,7 @@ ins1(PyListObject *self, Py_ssize_t where, PyObject *v)
 if (where > n)
 where = n;
 items = self->ob_item;
-for (i = n; --i >= where; )
-items[i+1] = items[i];
+memmove(items+where+1, items+where+0, (n - where) * sizeof(PyObject *));
 Py_INCREF(v);
 items[where] = v;
 return 0;

For me, that patch makes little difference in the timings.

Another thing that could be tried is to have ins1() call list_ass_slice() to do 
the work.  That way, both paths will use the same underlying insertion code.  
It would be interesting to see if this makes any difference.

A few other thoughts:

* We usually prefer deques to lists when prepending because the former are O(1) 
and the latter are O(n).

* Modern compilers are very good at optimizing data movements in a simple 
for-loop.

* It is very difficult to get good timings for these kind of operations.  
Internally, both make a call to list_resize() which in turn uses realloc().  
The latter function can be very fast or very slow depending solely on whether 
it can resize in-place or has to move the data.  Accordingly, timings are 
easily confounded by minor changes to memory useage.  To get a clearcut timing, 
you would need to isolate just the for-loop or memmove operation and not 
include Python calling overhead, realloc operations, and everything else that 
is going on in the given timings.

* Another source of confounding is the compiler itself.  Changes that makes 
small improvements on Clang may hurt the GCC build or vice-versa.  The 
comparison can also be thrown-off by whether you're timing a PGO build (which 
tends to do an amazing job of optimizing for-foops).  The results may also 
differ between 32-bit builds and 64-builds.

* The timing isn't representative of how people use insert(x, 0).  It would be 
a mistake to optimize an uncommon case if it comes at the expense of the common 
case (insertions into small lists).  To guard against this, you would need to 
measure inserts into a small list to find-out whether memmove() has more setup 
time than the for-loop.  When we've done such investigations in the past, 
for-loops were found to beat memmove() for sizes under 16.  Almost certainly 
this changed as compilers and processors have changed over the years.

* For smaller list sizes, the calling overhead dominates the insertion time.  
Disassembling the two different calls show a good deal of overhead for calls.

>>> from dis import dis
>>> dis('a.insert(0,0)')
  1   0 LOAD_NAME0 (a)
  2 LOAD_METHOD  1 (insert)
  4 LOAD_CONST   0 (0)
  6 LOAD_CONST   0 (0)
  8 CALL_METHOD  2
 10 RETURN_VALUE
>>> dis('a[0:0]=[0]')
  1   0 LOAD_CONST   0 (0)
  2 BUILD_LIST   1
  4 LOAD_NAME0 (a)
  6 LOAD_CONST   0 (0)
  8 LOAD_CONST   0 (0)
 10 BUILD_SLICE  2
 12 STORE_SUBSCR
 14 LOAD_CONST   1 (None)
 16 RETURN_VALUE

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39788] Exponential notation should return an int if it can

[Marco Sulla]

Marco Sulla  added the comment:

Sorry, but I can't figure out what code can break this change. Integers are 
implicitly converted to floats in operations with floats. How can this change 
break old code?

> if you are worried about the performance

No, I'm worried about the expectations of coders.
Personally, I expected that 1E2 returned a integer. And this is not true.
If I wanted a float, I'd wrote 1.0E2 . The fact the exponential notation 
returns always a float is really misleading, IMHO.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Tim Peters]

Tim Peters  added the comment:

Be cautious about this.  Many years ago I looked into this, mostly in the 
context of the list sort's binary insertion sort, and results were all over the 
map, depending on the compiler in use, the optimization level, and the range at 
which (if any!) memmove() was actually faster than a simple loop.  A fancy 
memmove() will (at least) arrange to copy (say) 8 bytes at a time, but the 
overhead of setting that up (+ the function call) made it a loser when the 
number of bytes to be moved was "too small".

Don't know whether the comment in listobject.c's binarysort() still applies:

   Caution: using memmove is much slower under MSVC 5;
   we're not usually moving many slots. */

Optimizing to move 100,000 elements isn't really sane - cutting the constant 
factor on an inherently quadratic-time too-simplistic basic approach isn't 
really doing you a favor ;-)

Also note that sortedcontainers does not use unbounded-length Python lists 
under the covers.  It puts an upper bound on the length of the Python lists it 
uses, precisely to avoid visibly quadratic-time behavior.

--
nosy: +tim.peters

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39800] Inconsistent/incomplete disassembly of methods vs method source code

[S Murthy]

S Murthy  added the comment:

BTW how else are methods/functions are created in Python except via def?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Stefan Pochmann]

Stefan Pochmann  added the comment:

Benchmarking with two *Python* versions of bisect.insort, the "insert" version 
takes 1.08 seconds to insort the shuffled range(10**5) while the slice 
assignment version only takes 0.46 seconds:

from timeit import timeit
import random
from bisect import bisect_right

def insort1(a, x):
lo = bisect_right(a, x)
a.insert(lo, x)

def insort2(a, x):
lo = bisect_right(a, x)
a[lo:lo] = [x]

for _ in range(3):
a = list(range(10**5))
random.shuffle(a)
for insort in insort1, insort2:
it = iter(a)
s = []
print(timeit(lambda: insort(s, next(it)), number=len(a)))
print()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39800] Inconsistent/incomplete disassembly of methods vs method source code

[S Murthy]

S Murthy  added the comment:

@steven.daprano In this case, the method f was created by via def. And calling 
dis.dis(s) where s is the source code of f (say s = inspect.getsource(f)) shows 
the bytecode both for the header and the body, as is clear enough from the 
example I first posted.

>>> dis.dis('def f(x): return x**2')
  1 0 LOAD_CONST   0 (", 
line 1>)
  2 LOAD_CONST   1 ('f')
  4 MAKE_FUNCTION0
  6 STORE_NAME   0 (f)
  8 LOAD_CONST   2 (None)
 10 RETURN_VALUE

Disassembly of ", line 1>:
  1 0 LOAD_FAST  0 (x)
  2 LOAD_CONST   1 (2)
  4 BINARY_POWER
  6 RETURN_VALUE

The first block of instructions here are for the def statement, and the second 
block for the return statement.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39793] make_msgid fail on FreeBSD 12.1-RELEASE-p1 with different domains

[Batuhan Taskaya]

Batuhan Taskaya  added the comment:

> but that sure looks like a broken system.

These are all aliases to the same name.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Stefan Pochmann]

Stefan Pochmann  added the comment:

I believe it also affects bisect.insort, which I occasionally use when I need a 
"self-sorting list" (I can't easily test it, as I'm not set up to modify the C 
version of bisect.insort).

And also the popular sortedcontainers package, which relies on such list 
operations to be fast: http://www.grantjenks.com/docs/sortedcontainers/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Raymond Hettinger]

Change by Raymond Hettinger :


--
assignee:  -> rhettinger
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39794] Add --without-decimal-contextvar option to use just threads in decimal

[Stefan Krah]

Change by Stefan Krah :


--
keywords: +patch
pull_requests: +18064
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/18702

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34271] Please support logging of SSL master secret by env variable SSLKEYLOGFILE

[Malcolm Smith]

Malcolm Smith  added the comment:

It looks like this has now been done and released. Can the issue be closed?

--
nosy: +Malcolm Smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39771] EmailMessage may need to support RFC-non-compliant MIME parameter encoding (encoded words in quotes) for output.

[R. David Murray]

R. David Murray  added the comment:

I actually agree: if most (by market share) MUAs handle the RFC-incorrect 
parameter encoding style, and a significant portion does not handle the RFC 
correct style, then we should support the de-facto standard rather than the 
official standard as the default.  I just wish Microsoft would write better 
software :)  If on the other hand it is only microsoft out of the big market 
share players that is broken, I'm not sure I'd want it to be the default.  But 
we could still support it optionally.

So yeah, we could have a policy control that governs which one is actually used.

So this is a feature request, and ideally should be supported by an 
investigation of what MUAs support what, by market share.  And there's another 
question: does this only affect the filename parameter, or is it all MIME 
parameters?  I would expect it to be the latter, but someone should check at 
least a few examples of that to be sure.

--
stage:  -> needs patch
title: EmailMessage.add_header doesn't work -> EmailMessage may need to support 
RFC-non-compliant MIME parameter encoding (encoded words in quotes) for output.
type: behavior -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39793] make_msgid fail on FreeBSD 12.1-RELEASE-p1 with different domains

[R. David Murray]

R. David Murray  added the comment:

I don't object to this patch, but that sure looks like a broken system.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39800] Inconsistent/incomplete disassembly of methods vs method source code

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> would it not be better to for dis.dis to behave consistently for methods and 
> source strings of methods

I don't think so. The inputs are different. One is a string containing a `def` 
statement, which is an executable statement. The other is a function or method 
object, which may or may not have been created by a `def` statement.

(You can, although not easily, assemble a function object yourself without 
using either `def` or `lambda` directly.)

The `def` statement assembles a function object out of a pre-compiled body, and 
that's what dis shows when you give it a source code string that happens to 
contain a `def`. It's just another statement, like an import or loop or 
assignment. The contents of the body (the code object) isn't shown because the 
byte-code generated for a `def` knows nothing about the contents of the body.

If you want to know the contents of the body, you have to look at the body (the 
code object) itself.

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38527] configure script fails to detect "float word ordering" on Solaris

[Batuhan Taskaya]

Change by Batuhan Taskaya :


--
nosy: +BTaskaya, jcea

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39667] Update zipfile.Path with zipp 3.0

[Jason R. Coombs]

Jason R. Coombs  added the comment:


New changeset 0aeab5c4381f0cc11479362af2533b3a391312ac by Jason R. Coombs in 
branch 'master':
bpo-39667: Sync zipp 3.0 (GH-18540)
https://github.com/python/cpython/commit/0aeab5c4381f0cc11479362af2533b3a391312ac


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39667] Update zipfile.Path with zipp 3.0

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 1.0 -> 2.0
pull_requests: +18063
pull_request: https://github.com/python/cpython/pull/18701

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue12915] Add inspect.locate and inspect.resolve

[Michael Felt]

Michael Felt  added the comment:

PR18517 has, likely, a utf-8 dependency. AIX, default latin-1 does not accept 
the new test.

Starting with this merge AIX bot fails with:
==
ERROR: test_name_resolution (test.test_pkgutil.PkgutilTests)
--
Traceback (most recent call last):
  File 
"/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/Lib/test/test_pkgutil.py",
 line 249, in test_name_resolution
os.makedirs(d, exist_ok=True)
  File "/home/buildbot/buildarea/3.x.aixtools-aix-power6/build/Lib/os.py", line 
223, in makedirs
mkdir(name, mode)
UnicodeEncodeError: 'latin-1' codec can't encode characters in position 17-19: 
ordinal not in range(256)

Not completely resolved.

--
nosy: +Michael.Felt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow, likely due to manual memmove

[Stefan Pochmann]

Change by Stefan Pochmann :


--
title: list.insert is slow due to manual memmove -> list.insert is slow, likely 
due to manual memmove

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39801] list.insert is slow due to manual memmove

[Stefan Pochmann]

New submission from Stefan Pochmann :

Using a list's insert function is much slower than using slice assignment:

> python -m timeit -n 10 -s "a=[]" "a.insert(0,0)"
10 loops, best of 5: 19.2 usec per loop

> python -m timeit -n 10 -s "a=[]" "a[0:0]=[0]"
10 loops, best of 5: 6.78 usec per loop

(Note that the list starts empty but grows to 100,000 elements.)

At first I thought maybe it's the attribute lookup or function call overhead or 
so, but inserting near the end shows that that's negligible:

> python -m timeit -n 10 -s "a=[]" "a.insert(-1,0)"
10 loops, best of 5: 79.1 nsec per loop

I asked at StackOverflow and someone pointed out that list.insert uses a manual 
loop instead of memmove: https://stackoverflow.com/a/60466572/12671057

--
components: Interpreter Core
messages: 362986
nosy: Stefan Pochmann
priority: normal
severity: normal
status: open
title: list.insert is slow due to manual memmove
type: performance
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39800] Inconsistent/incomplete disassembly of methods vs method source code

[S Murthy]

New submission from S Murthy :

I am using the dis module to look at source (and logical) lines of code vs 
corresponding bytecode instructions. I am bit confused by the output of dis.dis 
when disassembling a given method vs the corresponding source string, e.g.

>>> def f(x): return x**2
>>> dis.dis(f)
  1 0 LOAD_FAST  0 (x)
  2 LOAD_CONST   1 (2)
  4 BINARY_POWER
  6 RETURN_VALUE

This is the bytecode instruction block for the body only (not the method 
header), but dis.dis('def f(x): return x**2') produces the instructions for the 
header and body:

>>> dis.dis('def f(x): return x**2')
  1 0 LOAD_CONST   0 (", 
line 1>)
  2 LOAD_CONST   1 ('f')
  4 MAKE_FUNCTION0
  6 STORE_NAME   0 (f)
  8 LOAD_CONST   2 (None)
 10 RETURN_VALUE

Disassembly of ", line 1>:
  1 0 LOAD_FAST  0 (x)
  2 LOAD_CONST   1 (2)
  4 BINARY_POWER
  6 RETURN_VALUE

I have traced this difference to the different behaviour of dis.dis for methods 
vs source code strings:

def dis(x=None, *, file=None, depth=None):
...
...
if hasattr(x, '__code__'):
x = x.__code__
...
# Perform the disassembly
...
elif hasattr(x, 'co_code'): # Code object
_disassemble_recursive(x, file=file, depth=depth)
...
elif isinstance(x, str):# Source code
_disassemble_str(x, file=file, depth=depth)
...

It appears as if the method body is contained in the code object produced from 
compiling the source (_try_compile(source, '', ...)) but not if the code 
object was obtained from f.__code__.

Why is this the case, and would it not be better to for dis.dis to behave 
consistently for methods and source strings of methods, and to generate/produce 
the complete instruction set, including for any headers? The current behaviour 
of dis.dis means that Bytecode(x) is also affected, as iterating over the 
instructions gives you different instructions depending on whether x is a 
method or a source string of x:

>>> for instr in dis.Bytecode(f): 
... print(instr) 
...
Instruction(opname='LOAD_FAST', opcode=124, arg=0, argval='x', argrepr='x', 
offset=0, starts_line=1, is_jump_target=False)
Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval=2, argrepr='2', 
offset=2, starts_line=None, is_jump_target=False)
Instruction(opname='BINARY_POWER', opcode=19, arg=None, argval=None, 
argrepr='', offset=4, starts_line=None, is_jump_target=False)
Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, 
argrepr='', offset=6, starts_line=None, is_jump_target=False

>>> for instr in dis.Bytecode(inspect.getsource(f)): 
... print(instr) 
...
Instruction(opname='LOAD_CONST', opcode=100, arg=0, argval=", line 1>, argrepr='", line 1>', offset=0, starts_line=1, 
is_jump_target=False)
Instruction(opname='LOAD_CONST', opcode=100, arg=1, argval='f', argrepr="'f'", 
offset=2, starts_line=None, is_jump_target=False)
Instruction(opname='MAKE_FUNCTION', opcode=132, arg=0, argval=0, argrepr='', 
offset=4, starts_line=None, is_jump_target=False)
Instruction(opname='STORE_NAME', opcode=90, arg=0, argval='f', argrepr='f', 
offset=6, starts_line=None, is_jump_target=False)
Instruction(opname='LOAD_CONST', opcode=100, arg=2, argval=None, 
argrepr='None', offset=8, starts_line=None, is_jump_target=False)
Instruction(opname='RETURN_VALUE', opcode=83, arg=None, argval=None, 
argrepr='', offset=10, starts_line=None, is_jump_target=False)

--
components: Library (Lib)
messages: 362985
nosy: smurthy
priority: normal
severity: normal
status: open
title: Inconsistent/incomplete disassembly of methods vs method source code
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39799] Never return base's fragment from urljoin (urllib.parse)

[Open Close]

Open Close  added the comment:

Uploaded the patch (1.patch).

--
keywords: +patch
Added file: https://bugs.python.org/file48935/1.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39799] Never return base's fragment from urljoin (urllib.parse)

[Open Close]

New submission from Open Close :

According to RFC3986 5.2.2.,
target fragment is always reference fragment
(T.fragment = R.fragment;).

This is different from RFC1808 (4. and 5.2.).
And it is not mentioned
in Modifications section in RFC2396 and RFC3986.

Current:
>>> import urllib.parse
>>> urllib.parse.urljoin('http://a/b#f', '')
'http://a/b#f'

Should return:
'http://a/b'

---

https://tools.ietf.org/html/rfc3986#section-5.2.2
https://tools.ietf.org/html/rfc1808.html#section-4
https://tools.ietf.org/html/rfc1808.html#section-5.2

--
components: Library (Lib)
messages: 362983
nosy: op368
priority: normal
severity: normal
status: open
title: Never return base's fragment from urljoin (urllib.parse)
type: behavior
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39798] Update and Improve README.AIX

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +Michael.Felt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39798] Update and Improve README.AIX

[Batuhan Taskaya]

New submission from Batuhan Taskaya :

I was building python on AIX but the old README.AIX file didn't help much. It 
would be super cool to someone who is familiar with AIX update and improve that 
file with all new additions and current issues about AIX.

--
components: Build
messages: 362982
nosy: BTaskaya, David.Edelsohn
priority: normal
severity: normal
status: open
title: Update and Improve README.AIX

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39796] warning extension module inited twice in python3.9

[hai shi]

Change by hai shi :


--
keywords: +patch
pull_requests: +18062
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/18700

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39796] warning extension module inited twice in python3.9

[hai shi]

New submission from hai shi :

In master branch, `_PyWarnings_Init()` have been called twice.
```
(gdb) bt
#0  _PyWarnings_Init () at Python/_warnings.c:1338
#1  0x00525df3 in pycore_init_import_warnings 
(tstate=tstate@entry=0x9a19c0, sysmod=0x77f7e5f0) at 
Python/pylifecycle.c:680
...
Breakpoint 1, _PyWarnings_Init () at Python/_warnings.c:1338
1338{
(gdb) bt
#0  _PyWarnings_Init () at Python/_warnings.c:1338
#1  0x00511aac in _imp_create_builtin (module=, 
spec=0x77f2e7d0) at Python/import.c:1293
```

but in 2.7 branch, '_PyWarnings_Init()':
```
Breakpoint 1, _PyWarnings_Init() at Python/_warnings.c:886
886 m = Py_InitModule3(MODULE_NAME, warnings_functions, 
warnings__doc__);
Missing separate debuginfos, use: debuginfo-install 
glibc-2.17-260.el7_6.4.x86_64
(gdb) bt
#0  _PyWarnings_Init () at Python/_warnings.c:886
#1  0x004fc4db in Py_InitializeEx (install_sigs=1) at 
Python/pythonrun.c:242
#2  0x004fcb03 in Py_Initialize () at Python/pythonrun.c:370
#3  0x004154fd in Py_Main (argc=1, argv=0x7fffe428) at 
Modules/main.c:505
#4  0x004145f0 in main (argc=1, argv=0x7fffe428) at 
./Modules/python.c:23
```

Why? because pylifecycle.c and _imp extension module will call 
`_PyWarnings_Init()`.

isn't it?

--
components: Extension Modules
messages: 362980
nosy: brett.cannon, shihai1991
priority: normal
severity: normal
status: open
title: warning extension module inited twice in python3.9
type: enhancement
versions: Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39797] shutdown() in socketserver.BaseServer should be in a different thread from serve_forever()

[Ama Aje My Fren]

New submission from Ama Aje My Fren :

When a subclass of socketserver.BaseServer is running after calling 
serve_forever() and needs to be shutdown, it may be shut down by sending a 
[shutdown()](https://docs.python.org/3/library/socketserver.html#socketserver.BaseServer.shutdown).
 The thing is though that the shutdown() call must be run in a different thread 
than the one where the serve_forever() was called otherwise it will deadlock. 
This is documented in the 
[code](https://github.com/python/cpython/blob/3.8/Lib/socketserver.py#L244) but 
not in the documentation. It should be in the documentation as well as it is 
not obvious.

--
assignee: docs@python
components: Documentation
messages: 362981
nosy: amaajemyfren, docs@python
priority: normal
severity: normal
status: open
title: shutdown() in socketserver.BaseServer should be in a different thread 
from serve_forever()
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30082] hide command prompt when using subprocess.Popen with shell=False on Windows

[Steve Dower]

Steve Dower  added the comment:

If you use subprocess to launch any process that is marked SUBSYSTEM_CONSOLE, 
Windows will create a new console for it. For example, "python.exe" is such a 
process.

Eryk's answer in the very first StackOverflow link shows how to create a 
STARTUPINFO parameter with the correct settings. We just want to add a Boolean 
argument to subprocess.Popen.__init__() that can add that setting automatically.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30082] hide command prompt when using subprocess.Popen with shell=False on Windows

[swgmma]

swgmma  added the comment:

I would like to pick up work on this (have started at link below), however I 
would like some help:

For testing purposes, could someone please provide a minimum working example of 
using subprocess that causes a command prompt to pop up?

I have not been able to reproduce it (using Python 3.7), though I vaguely 
remember encountering it in the past.

https://github.com/ammgws/cpython/commit/03b9dac9205b9090dd67170427d887a9952345d3

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39379] sys.path[0] is already absolute path

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I have merged the doc change since it's minor. I will leave this open for 
clarification regarding Brett's question.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39379] sys.path[0] is already absolute path

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:


New changeset 1f0cd3c61a5ae3aac5ebaccc75ae9828ca4f96c4 by Ananthakrishnan in 
branch 'master':
bpo-39379: Remove reference to sys.path[0] being absolute path in whatsnew 
(GH-18561)
https://github.com/python/cpython/commit/1f0cd3c61a5ae3aac5ebaccc75ae9828ca4f96c4


--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37266] Daemon threads must be forbidden in subinterpreters

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> To me, this seems best implemented as a new public function for the threading 
> module, perhaps something like `threading.register_atexit()`. Would this be 
> reasonable?

To me, yes.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com