[issue26791] shutil.move fails to move symlink (Invalid cross-device link)

[Jeffrey Kintscher]

Jeffrey Kintscher  added the comment:

SilentGhost's analysis is correct.  The provided example code causes the error 
because it is trying to move the symlink into its target when the target is a 
directory. Any cross-device moving issues are unrelated to this example code.  
Here is the relevant code in the master branch:

if os.path.isdir(dst):
if _samefile(src, dst):
# We might be on a case insensitive filesystem,
# perform the rename anyway.
os.rename(src, dst)
return

shutil._samefile() considers the example link and its target to be the same.  
When _samefile() returns False, this code gets executed:

real_dst = os.path.join(dst, _basename(src))

if os.path.exists(real_dst):
raise Error("Destination path '%s' already exists" % real_dst)
try:
os.rename(src, real_dst)
except OSError:
if os.path.islink(src):
linkto = os.readlink(src)
os.symlink(linkto, real_dst)
os.unlink(src)

A simple fix is to check whether src is a symlink when _samefile() returns 
True.  The "Destination path...already exists" error isn't a problem for our 
symlink case because the shell mv command also returns an error.

$ ls -l /tmp/tmpdir/
total 0
lrwxr-xr-x  1 jeff  staff  11 Aug  5 23:36 test_dir -> /tmp/tmpdir
$ mv test_dir /tmp/tmpdir
mv: test_dir and /tmp/tmpdir/test_dir are identical

I will generate a pull request.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41456] loop.run_in_executor creat thread but not destory it after the task run over

[Ronald Oussoren]

Ronald Oussoren  added the comment:

This is expected behaviour. Run_in_executor uses a 
concurrent.futures.ThreadPoolExecutor to manage the worker threads. Those 
threads will get reused when more work is scheduled, and will exit when the 
runloop is shut down.

--
nosy: +ronaldoussoren

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41491] plistlib can't load macOS BigSur system LaunchAgent

[Ronald Oussoren]

Ronald Oussoren  added the comment:

Please also report this at apple using the feedback assistent.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26791] shutil.move fails to move symlink (Invalid cross-device link)

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
versions: +Python 3.10, Python 3.7, Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41493] Refactoring dictresize to accept only new keysize

[Inada Naoki]

New submission from Inada Naoki :

dictresize accept `minsize` and calculate `newsize` which is `newsize >= 
minsize`.
Some caller pass the exact dk_size so calculating `newsize` is not necessary.

Split calculating `newsize` from `minused` into new function.
Additonally, use _Py_bit_length for the calculation when possible.

--
components: Interpreter Core
messages: 374919
nosy: inada.naoki
priority: normal
severity: normal
status: open
title: Refactoring dictresize to accept only new keysize
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41493] Refactoring dictresize to accept only new keysize

[Inada Naoki]

Change by Inada Naoki :


--
keywords: +patch
pull_requests: +20894
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21751

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41456] loop.run_in_executor creat thread but not destory it after the task run over

[KevinGuo]

KevinGuo  added the comment:

OK, thanks for your reply

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41456] loop.run_in_executor creat thread but not destory it after the task run over

[KevinGuo]

Change by KevinGuo :


--
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Vedran Čačić]

Vedran Čačić  added the comment:

> s.append(x) if side == s_side else t.append(x)

To me, (s if side == s_side else t).append(x) seems much better. Not only more 
is factored, but .append is viewed as a procedure (returning None, changing its 
object), and if-expression is really used for its value, not the side-effect.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41465] io.TextIOWrapper.errors not writable

[Walter Dörwald]

Walter Dörwald  added the comment:

I guess that is good enough. "Being changeable" does not necessarily mean mean 
"being changeable via attribute assignment".

Thanks for your research. Closing the issue as "not a bug".

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[STINNER Victor]

STINNER Victor  added the comment:

> On macOS CLOCK_UPTIME_RAW is the same as mach_absolute_time (according to the 
> manpage).

https://developer.apple.com/documentation/kernel/1462446-mach_absolute_time 
says:

"mach_absolute_time: Returns current value of a clock that increments 
monotonically in tick units (starting at an arbitrary point), this clock does 
not increment while the system is asleep."

"Discussion: Prefer to use the equivalent 
clock_gettime_nsec_np(CLOCK_UPTIME_RAW) in nanoseconds."

where clock_gettime_nsec_np() is: "As a non-portable extension, the 
clock_gettime_nsec_np() function will return the clock value in 64-bit 
nanoseconds."

It doesn't say anything about NTP, whereas CLOCK_MONOTONIC_RAW seems to a clock 
which is not adjusted by NTP:

"""
CLOCK_MONOTONIC_RAW

clock that increments monotonically, tracking the time
since an arbitrary point like CLOCK_MONOTONIC.  How-
ever, this clock is unaffected by frequency or time
adjustments.  It should not be compared to other
system time sources.
"""

So right, it sounds like mach_absolute_time() returns *seconds*.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41273] asyncio: proactor read transport: use recv_into instead of recv

[STINNER Victor]

STINNER Victor  added the comment:

Tony: "asyncio: proactor read transport: use recv_into instead of recv" seems 
to be implemented, but PR 21446 "bpo-41279: Add StreamReaderBufferedProtocol" 
is a new feature. I suggest to open a new issue for this feature, and close 
this one.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Ronald Oussoren]

Ronald Oussoren  added the comment:

mach_absolute_time returns time in ticks, there's a separate API that returns 
the resolution of this clock (which is already used).

The manpage explicitly says that mach_absolute_time and CLOCK_UPTIME_RAW are 
the same clock:

 CLOCK_UPTIME_RAW   clock that increments monotonically, in the same manner 
as CLOCK_MONOTONIC_RAW, but that does
not increment while the system is asleep.  The returned 
value is identical to the result of
mach_absolute_time() after the appropriate 
mach_timebase conversion is applied.

Switching from mach_absolute_time to CLOCK_UPTIME_RAW would therefore bring us 
no improvements, and would complicate the code base because clock_gettime is 
only available starting from macOS 10.12.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41494] Add window resizing support [ SIGWINCH ] to Lib/pty

[Soumendra Ganguly]

New submission from Soumendra Ganguly :

This was tested using Python 3.7 after commenting out the sys.audit lines.

https://docs.python.org/3/library/pty.html presents us with an example usage of 
pty.spawn. This example mimics script(1). However, the script(1) from 
util-linux has fantastic signal handing that pty.spawn does not directly 
provide. In fact, Lib/pty.py says "Bugs: No signal handling. Doesn't set slave 
termios and window size."

xterm(1) on Debian 10 GNU/Linux was used to test the pty.spawn example 
mentioned above; upon resizing the xterm(1) window, the output of programs such 
as ls(1) became scattered and hard to visually parse.

This patch does not modify any of the functions that are already present in 
Lib/pty. Instead, it exposes a new function called "wspawn" [ pty.wspawn ]. 
This is like pty.spawn + the following differences.

1. Window size is set at the beginning.
2. A SIGWINCH handler is registered. The old handler is saved and restored 
later.
3. If the above two steps fail, then cleanup is done, and an exception is 
raised, so that the programmer can catch the exception and use pty.spawn 
instead.
4. Unlike pty.spawn, this does not depend on OSError to break out of the parent 
mainloop. Instead, the main loop calls select with an adjustable  timeout, so 
that waitpid with WNOHANG can be called periodically to check if the spawned 
child process has undergone an alteration of state.
5. While the return value is same as that of pty.spawn, this accepts an extra 
optional "timeout" argument for the select call.

The aforementioned pty.spawn example now works well with window resizing if 
pty.wspawn is used in place of pty.spawn.

Signed-off-by: Soumendra Ganguly 

--
components: Library (Lib)
files: pty.diff
keywords: patch
messages: 374926
nosy: soumendra
priority: normal
severity: normal
status: open
title: Add window resizing support [ SIGWINCH ] to Lib/pty
type: enhancement
versions: Python 3.10, Python 3.7, Python 3.8, Python 3.9
Added file: https://bugs.python.org/file49372/pty.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Mark Dickinson]

Mark Dickinson  added the comment:

If we want to do this (and I'm still not convinced that we do), I think there's 
a simpler way: use `frexp` to decompose each float into a fraction and an 
exponent, multiply the fractions (which barring zeros will all be in [0.5, 
1.0)), and keep track of the accumulated exponents separately. Then combine 
everything at the end.

There's a possibility of the accumulated product of the fractions underflowing, 
but only every 1000 floats or so, so it's enough to check every 500 floats 
(say) whether the product is smaller than 2**-500 or not, and scale by 2**1000 
(adjusting the exponent correspondingly) if not.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40275] test.support has way too many imports

[hai shi]

Change by hai shi :


--
pull_requests: +20895
pull_request: https://github.com/python/cpython/pull/21743

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Mark Dickinson]

Mark Dickinson  added the comment:

Here's code to illustrate the idea. It doesn't yet handle zeros, infinities or 
nans; that support would need to be added.

import math

def fprod(numbers):
# Product of numbers, avoiding intermediate underflow and overflow.
# Does not handle zeros, infinities or nans

# Running product is acc_m * 2**acc_e
acc_m, acc_e = float.fromhex("1p1000"), -1000

count = 0
for number in numbers:
m, e = math.frexp(number)
acc_m *= m
acc_e += e
if count == 1000:
if acc_m < 1.0:
acc_m = math.ldexp(acc_m, 1000)
acc_e -= 1000
count = 0

return math.ldexp(acc_m, acc_e)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Mark Dickinson]

Mark Dickinson  added the comment:

Whoops. There's a missing `count += 1` in there, of course.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37625] Class variable is still accessible after class instance has been overwritten out

[john mathew]

john mathew  added the comment:

I am Grateful that you shared this informational post. Your website has 
everything that I have been looking for so long. Your knowledge about this 
topic is quite impressive. I am amazed by the content you shared on this 
website.
I am John Mathew a technical advisor, for more information visit my websites.
http://tplinklogins.com
http://linksysextendersetups.com
http://tplinkloginn.com
http://tplinkextendersetupp.com

--
nosy: +john745

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue40275] test.support has way too many imports

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 79bb2c93f2d81702fdf1f93720369e18a76b7d1a by Hai Shi in branch 
'master':
bpo-40275: Use new test.support helper submodules in tests (GH-21743)
https://github.com/python/cpython/commit/79bb2c93f2d81702fdf1f93720369e18a76b7d1a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37625] Class variable is still accessible after class instance has been overwritten out

[Eric V. Smith]

Change by Eric V. Smith :


--
Removed message: https://bugs.python.org/msg374930

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41494] Add window resizing support [ SIGWINCH ] to Lib/pty

[Soumendra Ganguly]

Change by Soumendra Ganguly :


--
pull_requests: +20896
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21752

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41495] Technical advise

[john mathew]

New submission from john mathew :

We provide Technical Help to our users by a diagnosis of their computer and 
other devices. And if there is an issue to be solved, we give out the solution. 
This helps the user to avoid any existing issues.
http://tplinklogins.com
http://linksysextendersetups.com
http://tplinkloginn.com
http://tplinkextendersetupp.com

--
messages: 374932
nosy: john745
priority: normal
severity: normal
status: open
title: Technical advise

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41495] Technical advise

[Steven D'Aprano]

Change by Steven D'Aprano :


--
Removed message: https://bugs.python.org/msg374932

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41495] SPAM

[Steven D'Aprano]

Change by Steven D'Aprano :


--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
title: Technical advise -> SPAM

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41496] Create public API for typing._eval_type

[Dominik V.]

New submission from Dominik V. :

In this [python-ideas 
thread](https://mail.python.org/archives/list/python-id...@python.org/thread/T6K4DWENPM7LYXSDVYQYDVFEVBMA5K3L/)
 it was suggested to create a public API for `typing._eval_type` in order to be 
able to create custom versions of `get_type_hints`. Specifically a version that 
allows to specify an upper boundary in the MRO when retrieving type hints for a 
class object.

The public API should use `None` as defaults for `globalns` and `localns` and 
not expose the `recursive_guard` parameter.

--
messages: 374933
nosy: Dominik V.
priority: normal
severity: normal
status: open
title: Create public API for typing._eval_type
type: enhancement
versions: Python 3.10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41496] Create public API for typing._eval_type

[Karthikeyan Singaravelan]

Change by Karthikeyan Singaravelan :


--
nosy: +gvanrossum, levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41478] Empty representation of AssertionError

[Ilya Kamenshchikov]

Ilya Kamenshchikov  added the comment:

That's a solution, except you must know ahead of time this issue exists.

Best Regards,
--
Ilya Kamen

On Tue, Aug 4, 2020 at 6:59 PM Rémi Lapeyre  wrote:

>
> Rémi Lapeyre  added the comment:
>
> Hi, can you not use its repr:
>
>
> >>> try: raise ValueError
> ... except Exception as e: print(f"Following happened: {e!r}")
> ...
> Following happened: ValueError()
>
> ?
>
> --
> nosy: +remi.lapeyre
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22107] tempfile module misinterprets access denied error on Windows

[Václav Dvořák]

Change by Václav Dvořák :


--
nosy: +Václav Dvořák

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[nooB]

nooB  added the comment:

> perf_counter documentation says "It does include time elapsed during sleep 
> and is system-wide." where "sleep" here means time.sleep()

Apologies for misinterpreting the documentation. A clock function that includes 
system suspend time can be useful. Consider this as a feature request.

I noticed that on both Linux and Mac OS, the functions time.monotonic and 
time.perf_counter are exactly same. They do not count time elapsed during 
suspend.

--- CODE -

import time
for clock in "monotonic", "perf_counter":
print(f"{clock}: {time.get_clock_info(clock)}")

--- MAC OUTPUT 

monotonic: namespace(adjustable=False, implementation='mach_absolute_time()', 
monotonic=True, resolution=1e-09)
perf_counter: namespace(adjustable=False, 
implementation='mach_absolute_time()', monotonic=True, resolution=1e-09)

--- LINUX OUTPUT --

monotonic: namespace(adjustable=False, 
implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, 
resolution=1e-09)
perf_counter: namespace(adjustable=False, 
implementation='clock_gettime(CLOCK_MONOTONIC)', monotonic=True, 
resolution=1e-09)

--

In Windows, perf_counter uses "QueryPerformanceCounter()" which includes "the 
time when the machine was in a sleep state such as standby, hibernate, or 
connected standby"
refer: 
https://docs.microsoft.com/en-us/windows/win32/sysinfo/acquiring-high-resolution-time-stamps

I learnt that "mach_continuous_time()" in Mac OS, 
"clock_gettime(CLOCK_BOOTTIME)" in Linux and "QueryPerformanceCounter()" in 
Windows all include system suspend time. It would be nice if perf_counter can 
be tweaked to provide similar behaviour across platforms. If it is not 
recommended, the mentioned functions can be exposed as a separate clock 
function.

Thanks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41496] Create public API for typing._eval_type

[Dominik V.]

Change by Dominik V. :


--
keywords: +patch
pull_requests: +20898
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21753

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Rishav Kundu]

Rishav Kundu  added the comment:

While I agree that the behavior of perf_counter should be consistent across 
macOS/Linux and Windows wrt system suspend, you can already access those clocks 
by using time.clock_gettime [1] with appropriate clock IDs (CLOCK_MONOTONIC_RAW 
on macOS and CLOCK_BOOTTIME on Linux)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[STINNER Victor]

STINNER Victor  added the comment:

> While I agree that the behavior of perf_counter should be consistent across 
> macOS/Linux and Windows wrt system suspend,

perf_counter behavior during system suspend is undefined.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Rishav Kundu]

Rishav Kundu  added the comment:

[1]: https://docs.python.org/3.8/library/time.html#time.clock_gettime

[2]: 
https://developer.apple.com/documentation/kernel/1646199-mach_continuous_time 
effectively uses CLOCK_MONOTONIC_RAW on macOS.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38119] resource tracker destroys shared memory segments when other processes should still have valid access

[Damian Barabonkov]

Damian Barabonkov  added the comment:

As per Guido's comment 
(https://github.com/python/cpython/pull/21516#issuecomment-668110711), I'm 
going to use this space to discuss ways to go forward with resource tracking 
and SharedMemory.

Taking inspiration from Vinay (https://bugs.python.org/issue37754#msg351445), I 
think the simplest and best way forward is to use a small section of the shared 
memory at the start as a reference counter. 

Every time a process latches onto a shared memory block, it does and atomic 
increment to the reference counter. And if it detaches, it does an atomic 
decrement. This atomic operations are available in C via hardware specific 
instructions. This would require modifying the Python C code posixshmem.c. It 
should not be a difficult change.

This would then change the SharedMemory API such that a call to `close()` could 
check the reference count at the end, and aromatically unlink if it reaches 0. 
Basically, the purpose of the explicit `unlink()` call is dissolved.

I think this would also play nice with the current implementation of the 
`resource_tracker`. A small change would need to take place such that it calls 
`close()` instead of `unlink()` as the clean up function. Nonetheless, it would 
keep track if all attachments of shared memory call `close()` at the end, which 
they should, and issue a warning if they do not. It would do this with the 
current code, no need to change anything.

--
nosy: +damian.barabonkov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Rishav Kundu]

Rishav Kundu  added the comment:

> perf_counter behavior during system suspend is undefined

Does the same apply for time.monotonic? I would argue that the difference in 
behavior between Linux/macOS and Windows is unreasonable; given that 
time.monotonic exists for measuring time intervals (which are not necessarily 
required to be of short duration – unlike perf_counter).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41478] Empty representation of AssertionError

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

> That's a solution, except you must know ahead of time this issue exists.

If we changed str(e) to make it the same as repr(e), there would be no way to 
get only the message. str(e) returns the message so if the message given was 
empty (or no message was given) you get an empty string. This cannot be changed 
without breaking a lot of code.

--
versions: +Python 3.10, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41492] Fix signing description for Windows release builds

[Steve Dower]

Change by Steve Dower :


--
keywords: +patch
pull_requests: +20899
stage: needs patch -> patch review
pull_request: https://github.com/python/cpython/pull/21754

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41304] [CVE-2020-15801] python 38 embed ignore python38._pth file on windows

[Steve Dower]

Steve Dower  added the comment:

Yes, it only affects Windows OS.

On all other platforms, the python38._pth file is _always_ ignored. We have not 
implemented this support for those platforms.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Tim Peters]

Tim Peters  added the comment:

I'm skeptical of the need for - and wisdom of - this. Where does it come up? I 
can't think of any context where this would have been useful, or of any other 
language or package that does something like this. Long chains of mults are 
unusual outside of integer combinatorics to begin with.

Closest I can recall came up in the Spambayes project. There we needed to 
compute the sum of the logs of a potentially large number of probabilities.  
But log (lack of!) speed proved to be a bottleneck, so I changed it to compute 
the log of the product of the probabilities. That product was all but certain 
to underflow to 0.0. But "for real" - no rearrangement of terms would have made 
any difference to that. Instead, akin to what Mark spelled out, every now & 
again frexp() was used to push the product bits back into [0.5, 1.0), and the 
power-of-2 exponent was tracked apart from that.

fsum() is primarily aimed at a very different problem: improving the low-order 
bits.

How to explain what this change to prod() would do? It may or may not stop 
spurious overflows or underflows, and the result depends on the order of the 
multiplicands. But in what way? If we can't/won't define what it does, how can 
other implementations of Python reproduce CPython's result?

While I don't (yet?) see a real need for it, one thing that could work:  
compute the product left to right. Full speed - no checks at all. If the result 
is a 0, a NaN, or an infinity (which is bound to be rare in real life), do it 
again left to right with the frexp() approach. Then it can be explained: the 
result is what you'd get from left-to-right multiplication if the hardware had 
an unbounded exponent field, suffering overflow/underflow at the end only if 
the true exponent has magnitude too large for the hardware.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38119] resource tracker destroys shared memory segments when other processes should still have valid access

[Guido van Rossum]

Guido van Rossum  added the comment:

I recommend bringing this new proposal up on python-dev or python-ideas, to get 
more eyeballs on the ideas before attempting implementation. One immediate 
worry I have is that if the reference counter is maintained in the shared 
memory segment, every process has to participate, and if a process crashes 
(segfaults) the shared refcount will be forever wrong and the segment will 
leak. Distributed GC is hard!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41492] Fix signing description for Windows release builds

[miss-islington]

Change by miss-islington :


--
pull_requests: +20901
pull_request: https://github.com/python/cpython/pull/21756

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41492] Fix signing description for Windows release builds

[Steve Dower]

Steve Dower  added the comment:


New changeset 777b611c8c5676b80898a429f71d28e59bddc49d by Steve Dower in branch 
'master':
bpo-41492: Fixes the description appearing in UAC prompts on Windows (GH-21754)
https://github.com/python/cpython/commit/777b611c8c5676b80898a429f71d28e59bddc49d


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41492] Fix signing description for Windows release builds

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +20900
pull_request: https://github.com/python/cpython/pull/21755

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41473] test_gdb fails on AMD64 Fedora Rawhide 3.x

[STINNER Victor]

STINNER Victor  added the comment:

I looked at the aarch64 failure on Fedora Rawhide using GCC and the master 
branch.

When I run "./python -m test -v test_gdb  -m test_pycfunction" multiple times, 
I get a different error at each run, and sometimes the test pass, whereas 
nothing changes between the runs.

* gcc (GCC) 10.2.1 20200723 (Red Hat 10.2.1-1) [gcc-10.2.1-1.fc33.aarch64]
* GNU gdb (GDB) Fedora 9.2-2.fc33 [gdb-9.2-2.fc33.aarch64]

When gdb fails to rebuild the stack file, it logs the message: "opening 
file=(...)/_testcapi.cpython-310d-aarch64-linux-gnu.so [0]; direct_opencount=1".

It seems to be an issue in gdb, not in Python.

I modified test_gdb.py to get subprocess arguments and output (out/err).


== Logs of a success ==

args=('gdb', '--batch', '-nx', '-iex', 'add-auto-load-safe-path 
/home/vstinner/python/master/python-gdb.py', '--eval-command=set breakpoint 
pending yes', '--eval-command=break meth_fastcall', '--eval-command=set print 
address off', '--eva
l-command=run', '--eval-command=set print entry-values no', 
'--eval-command=bt', '--eval-command=py-bt', '--args', 
'/home/vstinner/python/master/python', '-S', '-c', '\nimport _testcapi\ndef 
foo():\n_testcapi.meth_fastcall()\ndef bar(
):\nfoo()\nbar()\n')

  
--out-- 

  
Breakpoint 1 (meth_fastcall) pending.   

  
[Thread debugging using libthread_db enabled]   

  
Using host libthread_db library "/lib64/libthread_db.so.1". 

  


  
Breakpoint 1, meth_fastcall (self=, args=, 
nargs=0) at /home/vstinner/python/master/Modules/_testcapimodule.c:5255 

  
5255{   

  
#0  meth_fastcall (self=, args=, nargs=0) at 
/home/vstinner/python/master/Modules/_testcapimodule.c:5255 

#1  cfunction_vectorcall_FASTCALL (func=, args=, nargsf=, 
kwnames=) at Objects/methodobject.c:424  

#2  _PyObject_VectorcallTstate (tstate=, callable=, args=, 
nargsf=9223372036854775808, kwnames=0x0) at ./Include/cpython/abstract.h:114
  
#3  PyObject_Vectorcall (kwnames=0x0, nargsf=9223372036854775808, args=, 
callable=) at ./Include/cpython/abstract.h:123
  
#4  call_function (tstate=, pp_stack=, oparg=0, kwnames=0x0) at 
Python/ceval.c:5121 

  
#5  _PyEval_EvalFrameDefault (tstate=, f=Frame 0x883070, for file , 
line 4, in foo (), throwflag=) at Python/ceval.c:3516

  
#6  _PyEval_EvalFrame (throwflag=0, f=Frame 0x883070, for file , line 
4, in foo (), tstate=) at ./Include/internal/pycore_ceval.h:40  

#7  function_code_fastcall (tstate=, co=, args=, nargs=0, 
globals=) at Objects/call.c:329  

 
#8  _PyFunction_Vectorcall (func=, stack=, 
nargsf=, kwnames=) at Objects/call.c:366  

 
#9  _PyObject_VectorcallTstate (tstate=, callable=, args=, nargsf=9223372036854775808, kwnames=0x0

[issue41492] Fix signing description for Windows release builds

[miss-islington]

miss-islington  added the comment:


New changeset 713ba03276c4ddc33c9debc51b03164ea18eead6 by Miss Islington (bot) 
in branch '3.8':
bpo-41492: Fixes the description appearing in UAC prompts on Windows (GH-21754)
https://github.com/python/cpython/commit/713ba03276c4ddc33c9debc51b03164ea18eead6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41492] Fix signing description for Windows release builds

[miss-islington]

miss-islington  added the comment:


New changeset 1437bb06a9cfbf7dd2d2aa14f6a784bb020bdb84 by Miss Islington (bot) 
in branch '3.9':
bpo-41492: Fixes the description appearing in UAC prompts on Windows (GH-21754)
https://github.com/python/cpython/commit/1437bb06a9cfbf7dd2d2aa14f6a784bb020bdb84


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Raymond Hettinger]

Change by Raymond Hettinger :


Added file: https://bugs.python.org/file49373/sum_and_prod.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41473] test_gdb fails on AMD64 Fedora Rawhide 3.x

[STINNER Victor]

STINNER Victor  added the comment:

More logs on the AArch64 issue.

It seems like *sometimes*, gdb fails to parse debug symbols of 
_testcapi.cpython-310d-aarch64-linux-gnu.so, whereas most of the time, "bt" 
shows symbols as expected.

stdout+stderr when the bug happens.

###
Function "meth_fastcall" not defined.
Breakpoint 1 (meth_fastcall) pending.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".
   1069559: opening 
file=/home/vstinner/python/master/build/lib.linux-aarch64-3.10-pydebug/_testcapi.cpython-310d-aarch64-linux-gnu.so
 [0]; direct_opencount=1
   1069559: 

Program received signal SIGTRAP, Trace/breakpoint trap.
?? () from 
/home/vstinner/python/master/build/lib.linux-aarch64-3.10-pydebug/_testcapi.cpython-310d-aarch64-linux-gnu.so
#0  ?? () from 
/home/vstinner/python/master/build/lib.linux-aarch64-3.10-pydebug/_testcapi.cpython-310d-aarch64-linux-gnu.so
#1  ?? ()
Backtrace stopped: previous frame identical to this frame (corrupt stack?)
Unable to locate python frame
###

Extract of a successful run:
###
Function "meth_fastcall" not defined.
Breakpoint 1 (meth_fastcall) pending.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib64/libthread_db.so.1".

Breakpoint 1, meth_fastcall (self=, args=, 
nargs=0) at /home/vstinner/python/master/Modules/_testcapimodule.c:5255
5255{
#0  meth_fastcall (self=, args=, nargs=0) at 
/home/vstinner/python/master/Modules/_testcapimodule.c:5255
#1  cfunction_vectorcall_FASTCALL (func=, args=, nargsf=, 
kwnames=) at Objects/methodobject.c:424
#2  _PyObject_VectorcallTstate (tstate=, callable=, args=, 
nargsf=9223372036854775808, kwnames=0x0) at ./Include/cpython/abstract.h:114
###


I can reproduce the issue with command "./script.sh" using the following files.


script.sh:
---
$ cat script.sh 
OUT=$(mktemp)
while true; do
  gdb --batch -x cmds -args ./python -S x.py < /dev/null 2>&1 | tee $OUT 
  grep -q -F '

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41473] test_gdb fails on AMD64 Fedora Rawhide 3.x

[STINNER Victor]

STINNER Victor  added the comment:

I reported https://bugzilla.redhat.com/show_bug.cgi?id=1866884 to gdb: "Arch64: 
sometimes, gdb fails to load symbols of a dynamic library with a pending 
breakpoint".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38119] resource tracker destroys shared memory segments when other processes should still have valid access

[Vinay Sharma]

Vinay Sharma  added the comment:

That's a valid point Guido. But, I guess this can be easily handled by resource 
tracker. At this current moment resource tracker unlinks shared memory if the 
process which created it dies without unliking it.

Therefore, resource tracker handles cleaning up resources which the respective 
processes couldn't or didn't do.

So, instead of unlinking the shared memory segment, resource tracker can 
instead decrement the reference count, if the process failed to do so, and if 
the reference count becomes 0, then unlink the shared memory segment.

This approach will ensure that even if the respective processes died 
unexpectedly, there are no leaks.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35805] email package folds msg-id identifiers using RFC2047 encoded words where it must not

[Olivier Dony]

Olivier Dony  added the comment:

With regard to msg349895, is there any chance this fix could be considered for 
backport?

I imagine you could view it as a new feature, but it seems to be the only 
official fix we have for the fact that Python 3 generates invalid SMTP 
messages. And that's not a minor problem because many popular MTAs (GMail, 
Outlook, etc.) will rewrite non-RFC-conformant Message IDs, causing the 
original ID to be lost and missing in subsequent replies. This breaks an 
important mechanism to support email threads.

To this day, several Linux distributions still ship 3.6 or 3.7, even in their 
latest LTS, and users and vendors are stuck with supporting those for a while.

Thanks!

--
nosy: +odo2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35805] email package folds msg-id identifiers using RFC2047 encoded words where it must not

[Olivier Dony]

Olivier Dony  added the comment:

Further, under Python 3.8 the issue is not fully solved, as other 
identification headers are still being folded in a non-RFC-conformant manner 
(see OP for RFC references). This was indicated on the original PR by the 
author: https://github.com/python/cpython/pull/13397#issuecomment-493618544

It is less severe of a problem than for Message-ID, but still means that 
MTA/MUA may fail to recognize the threading structure because identifiers are 
lost.

Is it better to open a new issue for this?


# Example on 3.8.2: the `In-Reply-To` header is RFC2047-folded.

Python 3.8.2 (default, Jul 16 2020, 14:00:26) 
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import email.message
>>> import email.policy
>>> msg = email.message.EmailMessage(policy=email.policy.SMTP)
>>> msg['Message-Id'] = 
>>> '<929227342217024.1596730490.324691772460938-example-30661-some.refere...@test-123.example.com>'
>>> msg['In-Reply-To'] = 
>>> '<92922734221723.1596730568.324691772460444-another-30661-parent.refere...@test-123.example.com>'
>>> print(msg.as_string())
Message-Id: 
<929227342217024.1596730490.324691772460938-example-30661-some.refere...@test-123.example.com>
In-Reply-To: =?utf-8?q?=3C92922734221723=2E1596730568=2E324691772460444-anot?=
 =?utf-8?q?her-30661-parent=2Ereference=40test-123=2Eexample=2Ecom=3E?=

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38119] resource tracker destroys shared memory segments when other processes should still have valid access

[Damian Barabonkov]

Damian Barabonkov  added the comment:

Unless the resource_tracker also dies along with the process. In which case, 
I'm not sure what there is there to do.

I believe the resource_tracker actually spawns a process alongside the process 
that uses it. So if the parent process seg-faults, the resource_tracker should 
still be alive.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35805] email package folds msg-id identifiers using RFC2047 encoded words where it must not

[Olivier Dony]

Olivier Dony  added the comment:

Somehow the message identifiers in the code sample got messed up in previous 
comment, here's the actual code, for what it's worth ;-) 
https://gist.github.com/odony/0323eab303dad2077c1277076ecc3733

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38119] resource tracker destroys shared memory segments when other processes should still have valid access

[Vinay Sharma]

Vinay Sharma  added the comment:

Well, the chances of resource tracker dying abruptly are very less because it's 
thoroughly tested, and there are mechanisms to re-spawn resource_tracker 
process if you see the code. There is a function called `def ensure_running`.

Resource tracker is still alive even if the process for which it was created 
dies. It also handles cleaning shared semaphores. So, I guess this is something 
we can rely on for cleaning up things, because at the end of the day that's 
what it was made for.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41478] Empty representation of AssertionError

[Ilya Kamenshchikov]

Ilya Kamenshchikov  added the comment:

Changing behavior and it's impact on existing code is, without a doubt, a
big deal here. Maybe it's a reason not to do anything about it.

Just to understand guiding design principle, what is expected from __str__
in more general case? I thought about it as "user-friendly string
representation", therefore expected an Exception without a message to
identify itself by it's class.

Best Regards,
--
Ilya Kamen

On Thu, Aug 6, 2020 at 6:05 PM Rémi Lapeyre  wrote:

>
> Rémi Lapeyre  added the comment:
>
> > That's a solution, except you must know ahead of time this issue exists.
>
> If we changed str(e) to make it the same as repr(e), there would be no way
> to get only the message. str(e) returns the message so if the message given
> was empty (or no message was given) you get an empty string. This cannot be
> changed without breaking a lot of code.
>
> --
> versions: +Python 3.10, Python 3.9
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38119] resource tracker destroys shared memory segments when other processes should still have valid access

[Damian Barabonkov]

Damian Barabonkov  added the comment:

Agreed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22107] tempfile module misinterprets access denied error on Windows

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy:  -Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41456] loop.run_in_executor creat thread but not destory it after the task run over

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy:  -Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

[Uncle Timmy]
> I'm skeptical of the need for - and wisdom of - this.

Granted, the need is not common.  On the other hand, if we can do it cheaply, 
why wouldn't we?  Unnecessary overflow or underflow is never a desirable 
outcome.  Currently, users do not have an easy and fast way to avoid that 
outcome.

This comes up a lot when I use or teach how to apply Hypothesis to test 
floating point code. That tool makes you acutely aware of how much your 
functions have to be constrained to assure a useful output.  And if you aren't 
satisfied with those constraints, it can be hard to fix unless you have robust 
primitives.  Presumably that is why numpy uses pairwise summation instead of 
straight addition for example.

To me, this is in the same category fixing the overflow error in bisect or 
using scaling in hypot() to avoid overflow or underflow.  The core idea is to 
make the primitives as robust as possible if it can be done with only a minor 
performance impact.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

FWIW, the occasions where this mattered all involved a mix of multiplications 
and divisions that mostly cancel out.   

The quadratic formula example is typical:   product([4.0, a, c, 1.0/b, 1.0/b].

Or a floating point implementation of comb(): product([1000, 999, 998, 997, 
1/4, 1/3, 1/2, 1/1])

Or terms in series expansions where both the numerator and denominator have 
many factors.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41497] Potential UnicodeDecodeError in dis

[JIanqiu Tao]

New submission from JIanqiu Tao :

A potential UnicodeDecodeError could be raised when run "python -m dis" on 
non-utf8 encoding environment.

Assume there is a file named "a.py", and contains "print('喵')", then save with 
UTF8 encoding.

Run "python -m dis ./a.py", on non-UTF8 encoding environment, for example a 
Windows PC which default language is Chinese.

A UnicodeDecodeError raised.

Traceback (most recent call last):
  File "C:\Program Files\Python38\lib\runpy.py", line 194, in 
_run_module_as_main
return _run_code(code, main_globals, None,
  File "C:\Program Files\Python38\lib\runpy.py", line 87, in _run_code
exec(code, run_globals)
  File "C:\Program Files\Python38\lib\dis.py", line 553, in 
_test()
  File "C:\Program Files\Python38\lib\dis.py", line 548, in _test
source = infile.read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0xb5 in position 9: illegal 
multibyte sequence

That because Windows' default encoding is decided by language. Chinese use 
cp936(GB2312) as default encoding and can't handle UTF8 encoding.

It just need to read in "rb" mode instead of "r".

--
components: Library (Lib)
messages: 374961
nosy: zkonge
priority: normal
severity: normal
status: open
title: Potential UnicodeDecodeError in dis
type: behavior
versions: Python 3.10, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 
3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41497] Potential UnicodeDecodeError in dis

[JIanqiu Tao]

Change by JIanqiu Tao :


--
keywords: +patch
pull_requests: +20902
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21757

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

I attached a file with latest (and more tested) recipe.  The underflow test was 
fixed: "not total and old_total and x".  Also, the final call to math.prod() 
was in-lined so that I could time it with PyPy.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41498] Undefinied _Py_Sigset_Converter function when HAVE_SIGSET_T not set

[Roman Yurchak]

New submission from Roman Yurchak :

The `_Py_Sigset_Converter` function is conditionally defined  in 
https://github.com/python/cpython/blob/777b611c8c5676b80898a429f71d28e59bddc49d/Modules/posixmodule.h#L27
 if `ifdef HAVE_SIGSET_T`

However this function is called unconditionally in 
https://github.com/python/cpython/blob/777b611c8c5676b80898a429f71d28e59bddc49d/Modules/clinic/signalmodule.c.h#L385
 (there are 4 such occurrences), which leads to a compilation error when 
`HAVE_SIGSET_T` is not set.

This is regression in 3.8+ as far as I can tell due to changes in 
https://github.com/python/cpython/pull/6720 

I imagine the solution could be to always define this function but raise an 
exception if it is called when HAVE_SIGSET_T` is undefined, as done in the 
following patch: 
https://github.com/iodide-project/pyodide/blob/fc5495ffdb54a11fd588dc60ba6b8777f90c3724/cpython/patches/0006-fix-Py_Sigset_Converter.patch
 

Would this be acceptable, or is there a better way of handling it? Thanks!

--
messages: 374963
nosy: Roman Yurchak, pablogsal, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Undefinied _Py_Sigset_Converter function when HAVE_SIGSET_T not set
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41498] Undefinied _Py_Sigset_Converter function when HAVE_SIGSET_T not set

[Roman Yurchak]

Change by Roman Yurchak :


--
type:  -> compile error

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Tim Peters]

Tim Peters  added the comment:

See "wisdom" earlier ;-) It's ad hoc trickery that seemingly can't be explained 
without showing the precise implementation in use today. As already mentioned, 
frexp() trickery _can_ be explained: exactly what you'd get if left-to-right HW 
multiplication were performed with an unbounded exponent, over- and 
under-flowing if and only if the infinite precision exponent at the end 
"doesn't fit". It completely eliminates spurious over-/under-flow. If someone 
wants that, it would be suitable for an fprod() function (which, like fsum(), 
makes strong guarantees at real cost).

Precisely which cases does this other thing protect against? Is there any way 
to characterize them beyond "well, try it and see"?

If there's an actual problem here, doesn't it deserve to be fixed? Just making 
it "less common" in some unquantifiable, unexplainable way seems unprincipled, 
and _possibly_ even harmful (e.g., by making it harder to provoke the 
underlying still-there problem without white box adversarial testing).


> Presumably that is why numpy uses pairwise summation instead of
> straight addition for example.

Loss of precision in fp summation is an extremely common real-life problem with 
real-life consequences.  That's why you can find dozens of papers, over the 
decades, on schemes to improve that. Adding a vector of a million floats is 
common; multiplying a vector of even a hundred floats is rare.

Similarly, scaled hypot implementations have been re-invented dozens of times. 
Etc.  When there's "a real" problem in fp life, it's hard _not_ to find mounds 
of prior art trying to address it.

You can find a few pieces on avoiding spurious under-/over-flow when chaining 
multiplies and divides, primarily older papers talking about 32-bit float 
formats with very limited dynamic range. The common advice: rearrange the 
handful of inputs in the source code by hand, based on your application 
knowledge of their likely magnitudes, to make left-to-right evaluation "safe".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41497] Potential UnicodeDecodeError in dis

[JIanqiu Tao]

JIanqiu Tao  added the comment:

I searched the whole Lib folder and find a lot of code uses "open(filename, 
'r')" without handling default encoding.

Should we open another issue for these problem?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[miss-islington]

miss-islington  added the comment:


New changeset 5f0769a7529fcbc28190864f098f192053a10747 by Nathan M in branch 
'master':
bpo-41371: Handle lzma lib import error in test_zoneinfo.py (GH-21734)
https://github.com/python/cpython/commit/5f0769a7529fcbc28190864f098f192053a10747


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41371] test_zoneinfo fails when lzma module is unavailable

[miss-islington]

Change by miss-islington :


--
pull_requests: +20903
pull_request: https://github.com/python/cpython/pull/21758

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

The algorithm stops all spurious overflows and underflows.  If favorable 
cancellations exist, it finds them.

Algorithm in Words
--

For every x in the sequence, multiply onto the total if possible.

If x and "total" can't be combined without overflow/underflow, then x is given 
a "side" depending on |x| > 1.0.  This indicates whether multiplying by x would 
increase the magnitude or decrease it.

Note that "total" has the same side as "x". If one would increased magnitude 
and the other would decreased it, then our "total *= x" would have succeeded.

The list "s" has the other pending multiplies.  Each of them are on the same 
side — either they all increase the magnitude or they all decrease it.  The 
direction is stored in the "s_side" variable.

If "x" is on the same side as everything else in "s", we just append it.  No 
cancels are possible.

If "x" and "t" are on the opposite side of the elements in "s", then we 
multiply the big/little pairs to get favorable cancellations.  Any t_i will 
successfully combine with any s_i.

At the end of the loop, "total" and "s" are both on the same side and no 
further favorable cancellations are possible.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26791] shutil.move fails to move symlink (Invalid cross-device link)

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
keywords: +patch
pull_requests: +20904
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/21759

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Variable name correction:
- If "x" and "t" are on the opposite side 
+ If "x" and "total" are on the opposite side 

Extreme points will successfully combine:
>>> float_info.max * float_info.min
3.9996

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Tim Peters]

Tim Peters  added the comment:

Cool!  So looks like you could also address an accuracy (not out-of-range) 
thing the frexp() method also does as well as possible: loosen the definition 
of "underflow" to include losing bits to subnormal products. For example, with 
the inputs

>>> xs = [1e-200, 1e-121, 1e308]
>>> product(xs)
9.98012604599318e-14
>>> _.hex()
'0x1.c17704f58189cp-44'
>>> math.prod(xs) # same thing
9.98012604599318e-14
>>> prod2(xs) # a tiny implementation of the frexp() method
1e-13
>>> _.hex()
'0x1.c25c268497682p-44'

product/math.prod get only a handful of the leading result bits right, because

>>> 1e-200 * 1e-121
1e-321
>>> _.hex()
'0x0.000cap-1022'

loses all but a handful of the trailing bits due to being in the subnormal 
range.  Changing the order can repair that:

>>> 1e-200 * 1e308 * 1e-121
1e-13

IOW, so far as "underflow" goes, we _start_ losing bits when the product 
becomes subnormal, well before it reaches 0.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41458] Avoid overflow/underflow in math.prod()

[Tim Peters]

Tim Peters  added the comment:

I may well have misread the code, believing it can still allow spurious 
over/underflows. On second reading of the current file, I don't know - it's 
more complicated than I thought.

If it does guarantee to prevent them, then I shift from -1 to (probably ) -0.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41419] Path.mkdir and os.mkdir don't respect setgid if its parent is g-s

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy: +Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41232] Python `functools.wraps` doesn't deal with defaults correctly

[Caleb Donovick]

Change by Caleb Donovick :


--
nosy: +donovick

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41238] Python 3 shelve.DbfilenameShelf is generating 164 times larger files than Python 2.7 when storing dicts

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy: +Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41493] Refactoring dictresize to accept only new keysize

[Inada Naoki]

Inada Naoki  added the comment:


New changeset d9323a8c6e07071a59dc4c910661db33236c01b2 by Inada Naoki in branch 
'master':
bpo-41493: Refactoring dictresize (GH-21751)
https://github.com/python/cpython/commit/d9323a8c6e07071a59dc4c910661db33236c01b2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41493] Refactoring dictresize to accept only new keysize

[Inada Naoki]

Change by Inada Naoki :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41303] perf_counter result does not count system sleep time in Mac OS

[Eryk Sun]

Eryk Sun  added the comment:

> Does the same apply for time.monotonic? I would argue that the 
> difference in behavior between Linux/macOS and Windows is 
> unreasonable; given that time.monotonic exists for measuring time 
> intervals

For time.monotonic in Windows, Python could switch to using 
QueryUnbiasedInterruptTime [1], which excludes periods in which the system is 
suspended or hibernated. In Windows 10, QueryUnbiasedInterruptTimePrecise [2] 
can be used, which directly reads the counter in the hardware timer instead of 
the value at the last timer interrupt.

For time.time in Windows, Python 3.10 should switch to using 
GetSystemTimePreciseAsFileTime [3] instead of GetSystemTimeAsFileTime. 

For time.perf_counter in Windows, QueryPerformanceCounter remains the best 
option since it uses the invariant TSC in the CPU if available.

---

Note that, unlike GetTickCount64, the resolution for QueryUnbiasedInterruptTime 
isn't lpTimeIncrement from GetSystemTimeAdjustment [4]. lpTimeIncrement is the 
initial default and upper bound for the interrupt period, which can be adjusted 
down with timeBeginPeriod or the undocumented system call NtSetTimerResolution. 
The former can lower the interrupt period from the default of about 10-16 ms 
(usually 15.625 ms, i.e. 64 interrupts/second) down to about 1 ms, while 
NtSetTimerResolution goes down to about 0.5 ms. It's not free, however, else it 
would be the default setting. More frequent timer interrupts can decrease 
system performance and increase power consumption.

---

[1] 
https://docs.microsoft.com/en-us/windows/win32/api/realtimeapiset/nf-realtimeapiset-queryunbiasedinterrupttime

[2] 
https://docs.microsoft.com/en-us/windows/win32/api/realtimeapiset/nf-realtimeapiset-queryunbiasedinterrupttimeprecise

[3] 
https://docs.microsoft.com/en-us/windows/win32/api/sysinfoapi/nf-sysinfoapi-getsystemtimepreciseasfiletime

[4] 
https://docs.microsoft.com/en-us/windows/win32/api/sysinfoapi/nf-sysinfoapi-getsystemtimeadjustment

--
nosy: +eryksun
versions: +Python 3.10, Python 3.9 -Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41440] os.cpu_count doesn't work on VxWorks RTOS

[Dong-hee Na]

Dong-hee Na  added the comment:


New changeset 3405e0542839cde94df9833b3809710a67a33c7c by pxinwr in branch 
'master':
bpo-41440: add os.cpu_count() support for VxWorks RTOS (GH-21685)
https://github.com/python/cpython/commit/3405e0542839cde94df9833b3809710a67a33c7c


--
nosy: +corona10

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41440] os.cpu_count doesn't work on VxWorks RTOS

[Dong-hee Na]

Change by Dong-hee Na :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37724] [[Errno 17] File exists: ] # Try create directories that are not part of the archive with

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy: +Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41497] Potential UnicodeDecodeError in dis

[Inada Naoki]

Change by Inada Naoki :


--
versions:  -Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue38119] resource tracker destroys shared memory segments when other processes should still have valid access

[Vinay Sharma]

Vinay Sharma  added the comment:

As suggested by Guido I have floated this solution to python-dev mailing list.
Link to archive: 
https://mail.python.org/archives/list/python-...@python.org/thread/O67CR7QWEOJ7WDAJEBKSY74NQ2C4W3AI/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39871] math.copysign raises SystemError with non-float x and custom y

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 4.0 -> 5.0
pull_requests: +20905
pull_request: https://github.com/python/cpython/pull/21760

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39871] math.copysign raises SystemError with non-float x and custom y

[Inada Naoki]

Inada Naoki  added the comment:


New changeset 54636355805dd2877bb54fbad8d967e1ddd8b553 by Zackery Spytz in 
branch 'master':
bpo-39871: Fix an error in a news entry (GH-21749)
https://github.com/python/cpython/commit/54636355805dd2877bb54fbad8d967e1ddd8b553


--
nosy: +inada.naoki

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue39871] math.copysign raises SystemError with non-float x and custom y

[miss-islington]

miss-islington  added the comment:


New changeset fc354ca51d0b6f24f2871788dfa7e35c56129f8b by Miss Islington (bot) 
in branch '3.9':
bpo-39871: Fix an error in a news entry (GH-21749)
https://github.com/python/cpython/commit/fc354ca51d0b6f24f2871788dfa7e35c56129f8b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41169] socket.inet_pton raised when pass an IPv6 address like "[::]" to it

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy: +Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41157] email.message_from_string() is unable to find the headers for the .msg files

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy: +Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41168] Lack of proper checking in PyObject_SetAttr leads to segmentation fault

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy: +Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41497] Potential UnicodeDecodeError in dis

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Good catch. Yes, when read Python source files you should either open them in 
binary mode if reading bytes is enough for use, or open them with 
tokenize.open() if we need string data, or use token.detect_encoding() and pass 
the result to open().

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41168] Lack of proper checking in PyObject_SetAttr leads to segmentation fault

[Jeffrey Kintscher]

Jeffrey Kintscher  added the comment:

Can you attach the Python source code for the PoC?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41419] Path.mkdir and os.mkdir don't respect setgid if its parent is g-s

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

os.mkdir() is a thin wrapper around syscalls mkdir or mkdirat. Path.mkdir() is 
a thin wrapper around os.mkdir(). If you think that the behavior of mkdir 
differs from the documentation please file a report in the Linux kernel.

--
nosy: +serhiy.storchaka
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue41419] Path.mkdir and os.mkdir don't respect setgid if its parent is g-s

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy:  -Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37495] socket.inet_aton parsing issue on some libc versions

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy: +Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34360] urllib.parse doesn't fully comply to RFC 3986

[Jeffrey Kintscher]

Change by Jeffrey Kintscher :


--
nosy: +Jeffrey.Kintscher

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com