[issue35730] IDLE: Fix squeezer test_reload.

[Tal Einat]

Change by Tal Einat :


--
pull_requests: +11272, 11273, 11274

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35730] IDLE: Fix squeezer test_reload.

[Tal Einat]

Change by Tal Einat :


--
pull_requests: +11272, 11273

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35730] IDLE: Fix squeezer test_reload.

[Tal Einat]

Change by Tal Einat :


--
pull_requests: +11272

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[Christian Heimes]

Christian Heimes  added the comment:

+1, /usr/bin:/bin sounds good to me.

My /usr/include/paths.h has #define _PATH_DEFPATH "/usr/bin:/bin" and #define 
_PATH_STDPATH "/usr/bin:/bin:/usr/sbin:/sbin". The file is pretty old and has 
copyright from 89 and 93, https://code.woboq.org/gcc/include/paths.h.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35750] process finished with exit code -1073740940 (0xc0000374)

[Anil kunduru]

Anil kunduru  added the comment:

Thanks guys for quick responses.  Eryk Sun  comment wa very help full.

--
resolution:  -> fixed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35750] process finished with exit code -1073740940 (0xc0000374)

[Anil kunduru]

Anil kunduru  added the comment:

thanks Eryk Sun i am able to solve the problem, it was extension modules 
problem. it was helpful great.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[Gregory P. Smith]

Gregory P. Smith  added the comment:

I'm not arguing against this change, just trying to figure out where it came 
from in the first place.  We should fix the value on all OSes.

It would be a behavior change so probably only good for 3.8+.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[Gregory P. Smith]

Gregory P. Smith  added the comment:

ntpath and macpath appear to have the same potential issue.
They've basically had this defpath value forever.

keep following it and you find a commit from 1994 
https://github.com/python/cpython/commit/2979b01ff88ac4c5b316d9bf98edbaaaffac8e24

Changing them only alters behavior of users of os.defpath or in the (odd?) 
situation when the PATH environment variable is unset, anything using of 
shutil.which() or distutils.spawn.find_executabnle(), the only stdlib users of 
os.defpath.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[Kyle Evans]

Kyle Evans  added the comment:

I'll be preparing a patch for our posix_spawn's signal handling.

My mistake in my setuid assessment was pointed out to me- it doesn't seem like 
a highly likely attack vector, but it is indeed possible.

If the child errored out, they will indeed be properly reapable at that point 
due to how vfork is implemented -- any observation to the contrary is to be 
considered a bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35756] Using `return value` in a generator function skips the returned value on for-loop iteration

[Steven D'Aprano]

Steven D'Aprano  added the comment:

> I understood the PEP to include `return expr` in the iteration values 
> as per the first bullet of the proposal.
> 
> > Any values that the iterator yields are passed directly to the caller.
> 
> This bullet doesn't have any additional formatting on the word 
> "yields" so I consider it as not directly referring to the "yield" 
> keyword.

I read it as "yields", not "yields or returns". Lack of formatting is 
irrelevant -- we shouldn't expect every use of a word with a technical 
meaning to necessarily be formatted specifically.

Read the Proposal section:

The following new expression syntax will be allowed in the body 
of a generator 
[...]
FURTHERMORE, when the iterator is another generator, the 
subgenerator is allowed to execute a return statement with a 
value, AND THAT VALUE BECOMES THE VALUE OF THE YIELD FROM 
EXPRESSION. [emphasis added]

https://legacy.python.org/dev/peps/pep-0380/#id11

It does not say "that value is yielded and then becomes the value of the 
yiueld from expression".

To me, it is clear that the intention here is that the return value is 
not yielded.

The Abstract also says:

Additionally, the subgenerator is allowed to return with a
value, and the value IS MADE AVAILABLE to the delegating
generator. [emphasis added]

The use of "is made available" suggests that the return value is treated 
differently from a yield. Values yielded from the subgenerator are 
automatically yielded from the calling generator, without any additional 
effort. The value returned is merely *made available*, for the calling 
generator to do with whatever it wishes.

And if there is still any doubt, there is specification of the behaviour 
of "result = yield from expression" which makes it clear that the return 
value carried by the StopIteration exception is not yielded, but used as 
the value of the expression (i.e. assigned to `result`).

The motivation of yield from returning a value is to allow a side- 
channel independent of the iterable values. It isn't intended as a "one 
last yield and then bail out". I don't think that your interpretation 
can be justified by reading the PEP.

> Essentially, allowing `return expr` in generator functions without 
> invoking the generator using `yield from generator` will lose the last 
> value.

No, because the return value is not intended to be used as one of the 
iteration values. Quoting one of Greg Ewing's examples:

I hope it is also clear why returning values via yield,
or having 'yield from' return any of the yielded values,
would be the wrong thing to do. The send/yield channel and
the return channel are being used for completely different
purposes, and conflating them would be disastrous.

http://www.cosc.canterbury.ac.nz/greg.ewing/python/yield-from/yf_current/Examples/Parser/parser.txt

That example is indirectly linked to from the PEP.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34691] _contextvars missing in xmaster branch Windows build?

[Tim Peters]

Tim Peters  added the comment:

Precisely _how_ are you building it?  As noted above, if you're using Visual 
Studio, try using build.bat instead for the first time.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34691] _contextvars missing in xmaster branch Windows build?

[ossdev]

ossdev  added the comment:

I am building python for windows on ARM64,
I am getting "No module named '_contextvars'" when trying to import contextvars:
>>> import contextvars
Traceback (most recent call last):
  File "", line 1, in 
  File 
"C:\Users\ps-emb\Downloads\cpython-master\cpython-master\lib\contextvars.py", 
line 1, in 
from _contextvars import Context, ContextVar, Token, copy_context
ModuleNotFoundError: No module named '_contextvars'

thanks

--
nosy: +ossdev07
versions: +Python 3.8 -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35758] Disable x87 control word for MSVC ARM compiler

[Minmin Gong]

New submission from Minmin Gong :

Msvc defines _M_ARM for arm target, but it doesn't have x87 control word. Need 
to disable it to prevent some compiling problems.

--
messages: 333821
nosy: Minmin.Gong
priority: normal
pull_requests: 11270
severity: normal
status: open
title: Disable x87 control word for MSVC ARM compiler

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[Kyle Evans]

Kyle Evans  added the comment:

On Wed, Jan 16, 2019 at 5:47 PM Alexey Izbyshev  wrote:
> > Hi,
>
> > As a disclaimer, I'm a FreeBSD developer interested in making sure we're 
> > doing the right thing here. =)
>
> > May I ask what the above assessment is based on, and specifically what we 
> > need to address?
>
> Hello, Kyle! That assessment is based on my quick and incorrect reading of 
> posix_spawn source code. I didn't notice that "error" is visible in the 
> parent due to address space sharing. Sorry for that, and thank you for the 
> correction. A proper error notification is required for posix_spawn to be 
> used in subprocess as a replacement for fork/exec, and FreeBSD does it right.

Oh, good to hear. =) koobs had pointed this thread out in hopes that we can try 
to reconcile and figure out what work needs to be done here. =)

> While we're here, would you be kind to answer several questions about 
> posix_spawn on FreeBSD?

Most definitely, if I can!

> 1) On Linux, without taking additional precautions, signals may be delivered 
> to a child process created via vfork(). If a signal handler runs in such a 
> child, it can leave traces of its work in the (shared) memory, potentially 
> surprising the parent process. To avoid this, glibc[1] and musl[2] disable 
> all signals (even signals used internally for thread cancellation) before 
> creating a child, but FreeBSD calls vfork() right away. Is this not 
> considered a problem, or is something hidden in vfork() implementation?
>

Right, after glancing over our implementation details- this is indeed a problem 
here. Our manpage indicates that we only block SIGTTOU for SIGTTIN for children 
in vfork(), and some light testing seems to indicate that's the case. Signal 
handlers are inherited from the parent, so that's less than stellar.

> 2) Another problem with vfork() is potential sharing of address space between 
> processes with different privileges (see Rich Felker's post[3] about this and 
> the previous problem). Is this a valid concern on FreeBSD?

My initial read-through of the relevant bits seem to indicate that image 
activation will cause the child process to be allocated a new process space, so 
it's looking kind of like a 'no' -- I'm outsourcing the answer to this one to 
someone more familiar with those bits, though, so I'll get back to you.


> 3) Does FreeBSD kernel guarantee that when waitpid(WNOHANG) is called at [4], 
> the child is ready for reaping? On Linux, it's not always the case[5].

I want to say vfork semantics guarantee it- we got to this point, so the child 
hit an error and _exit(127). I'm double-checking this one, though, to verify 
the child's properly marked a zombie before we could possibly reschedule the 
parent.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35757] slow subprocess.Popen(..., close_fds=True)

[Kirill Kolyshkin]

New submission from Kirill Kolyshkin :

In case close_fds=True is passed to subprocess.Popen()
or its users (subprocess.call() etc), it might spend
some considerable time closing non-opened file descriptors,
as demonstrated by the following snippet from strace:

close(3)= -1 EBADF (Bad file descriptor)
close(5)= -1 EBADF (Bad file descriptor)
close(6)= -1 EBADF (Bad file descriptor)
close(7)= -1 EBADF (Bad file descriptor)
...
close(1021) = -1 EBADF (Bad file descriptor)
close(1022) = -1 EBADF (Bad file descriptor)
close(1023) = -1 EBADF (Bad file descriptor)

This happens because the code in _close_fds() iterates from 3 up to
MAX_FDS = os.sysconf("SC_OPEN_MAX").

Now, syscalls are cheap, but SC_OPEN_MAX (also known as RLIMIT_NOFILE
or ulimit -n) can be quite high, for example:

$ docker run --rm python python3 -c \
$'import os\nprint(os.sysconf("SC_OPEN_MAX"))'
1048576

This means a million syscalls before spawning a child process, which
can result in a major delay, like 0.1s as measured on my fast and mostly
idling laptop. Here is the comparison with python3 (which does not have this 
problem):

$ docker run --rm python python3 -c $'import subprocess\nimport time\ns = 
time.time()\nsubprocess.check_call([\'/bin/true\'], 
close_fds=True)\nprint(time.time() - s)\n'
0.0009245872497558594

$ docker run --rm python python2 -c $'import subprocess\nimport time\ns = 
time.time()\nsubprocess.check_call([\'/bin/true\'], 
close_fds=True)\nprint(time.time() - s)\n'
0.0964419841766

--
components: Library (Lib)
messages: 333819
nosy: Kirill Kolyshkin
priority: normal
pull_requests: 11269
severity: normal
status: open
title: slow subprocess.Popen(..., close_fds=True)
type: performance
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35701] [uuid] 3.8 breaks weak references for UUIDs

[Josh Rosenberg]

Josh Rosenberg  added the comment:

David, the What's New note about weak references no longer being possible 
should be removed as part of this change. I'm not sure the note on arbitrary 
attributes no longer being addable is needed either (__setattr__ blocked that 
beforehand, it's just even more impossible now).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35756] Using `return value` in a generator function skips the returned value on for-loop iteration

[bryan.koch]

bryan.koch  added the comment:

I understood the PEP to include `return expr` in the iteration values as per 
the first bullet of the proposal.

> Any values that the iterator yields are passed directly to the caller.

This bullet doesn't have any additional formatting on the word "yields" so I 
consider it as not directly referring to the "yield" keyword.

With the current implementation, I have to concern myself if a generator 
function was created with the intention of being called using `last_ret = yield 
from function(); yield last_ret` or as `for ret in function(): yield ret`.  The 
first also yields the return value but would also yield an additional `None` if 
a `return` was not the terminal cause; the second will miss the last value if 
the generator uses `return`.

Essentially, allowing `return expr` in generator functions without invoking the 
generator using `yield from generator` will lose the last value.

I support either of the below resolutions:
* `return expr` being invoked from a generator that is not being iterated using 
`yield from generator` is a SyntaxError
* `return expr` being invoked from a generator that is not being iterated using 
`yield from generator` includes the final return value in the iterated set

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

The one in PR 11583 is twice as fast:

>timeit for -> 
>topsort([(2,11),(9,11),(9,8),(9,10),(10,11),(10,3),(11,7),(11,5),(8,7),(8,3)])
12.4 µs ± 59.1 ns per loop (mean ± std. dev. of 7 runs, 10 loops each)

>timeit for -> 
>tsort([(2,11),(9,11),(9,8),(9,10),(10,11),(10,3),(11,7),(11,5),(8,7),(8,3)])
29.1 µs ± 147 ns per loop (mean ± std. dev. of 7 runs, 1 loops each)

--
Removed message: https://bugs.python.org/msg333815

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

The one in PR 11583 is twice as faster:

>timeit for -> 
>topsort([(2,11),(9,11),(9,8),(9,10),(10,11),(10,3),(11,7),(11,5),(8,7),(8,3)])
12.4 µs ± 59.1 ns per loop (mean ± std. dev. of 7 runs, 10 loops each)

>timeit for -> 
>tsort([(2,11),(9,11),(9,8),(9,10),(10,11),(10,3),(11,7),(11,5),(8,7),(8,3)])
29.1 µs ± 147 ns per loop (mean ± std. dev. of 7 runs, 1 loops each)

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35754] When writing/closing a closed Popen.stdin, I get OSError vs. BrokenPipeError randomly or depending on bufsize

[Eryk Sun]

Change by Eryk Sun :


--
components: +IO, Interpreter Core, Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware
type:  -> behavior
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35754] When writing/closing a closed Popen.stdin, I get OSError vs. BrokenPipeError randomly or depending on bufsize

[Eryk Sun]

Eryk Sun  added the comment:

If I'm right, we can reduce your example down as follows:

import os
import subprocess
import time
import ctypes

RtlGetLastNtStatus = ctypes.WinDLL('ntdll').RtlGetLastNtStatus
RtlGetLastNtStatus.restype = ctypes.c_ulong

msys = os.path.normpath("C:/msys64/usr/bin")
head = os.path.join(msys, "head.exe")

p = subprocess.Popen(head, stdin=subprocess.PIPE,
stdout=subprocess.PIPE, bufsize=0)

# head.exe reads 1 KiB. It closes stdin if it finds 10 lines.
p.stdin.write(b'\n' * 1024)

# If we immediately fill up the pipe again plus 1 extra byte, 
# i.e. 4097 bytes for the default queue size, then NPFS will 
# internally queue a pending IRP. We're synchronous, so the 
# I/O manager will wait for I/O completion. Meanwhile the child
# has requested to close its end of the pipe. In this case,
# NPFS will complete the pending IRP with STATUS_PIPE_BROKEN,
# which maps to WinAPI ERROR_PIPE_BROKEN and C errno EPIPE.
#
# On the other hand, if we wait to give the child's close request
# time to complete, then NPFS will fail our 4097 byte write 
# immediately with STATUS_PIPE_CLOSING, which maps to WinAPI
# ERROR_NO_DATA and C errno EINVAL.

time.sleep(0.0) # STATUS_PIPE_BROKEN / ERROR_PIPE_BROKEN / EPIPE
#time.sleep(0.5) # STATUS_PIPE_CLOSING / ERROR_NO_DATA / EINVAL

try:
p.stdin.write(b'\n' * 4097)
except OSError:
ntstatus = RtlGetLastNtStatus()
if ntstatus == 0xC000_00B1:
   print('NT Status: STATUS_PIPE_CLOSING\n')
elif ntstatus == 0xC000_014B:
print('NT Status: STATUS_PIPE_BROKEN\n')
else:
print('NT Status: {}\n'.format(ntstatus, '#010x'))
raise

This could be addressed by improving our exception handling to look at the C 
runtime's _doserrno [1] value for EINVAL errors, in order to map ERROR_NO_DATA 
to EPIPE instead of EINVAL. Only two NT status codes are mapped to 
ERROR_NO_DATA, and both are pipe related (STATUS_PIPE_CLOSING and 
STATUS_PIPE_EMPTY), so using EPIPE should be fine.

[1]: 
https://docs.microsoft.com/en-us/cpp/c-runtime-library/errno-doserrno-sys-errlist-and-sys-nerr?view=vs-2017

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34782] Pdb crashes when code is executed in a mapping that does not define `__contains__`

[ppperry]

ppperry  added the comment:

You have to step into the eval for the error to be raised.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34782] Pdb crashes when code is executed in a mapping that does not define `__contains__`

[Henry Chen]

Henry Chen  added the comment:

Hmm, the example works for me (Python 3.6.5):

>>> import pdb
>>> class FakeContainer:
... def __getitem__(self, key):
... raise KeyError(key)
...
>>> pdb.run("eval('1+1',{},FakeContainer())")
> (1)()
(Pdb) c
>>>

As for exec/eval accepting an incomplete mapping, that strikes me as a less 
than thorough checking on the part of exec/eval, perhaps for performance 
reasons(?)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35747] Python threading event wait influenced by date change

[ido k]

ido k  added the comment:

thanks @pablogsal.

looks like duplicate of bugs.python.org/issue23428 

It seems like i cant trust the threading wait function and need to use busy 
waiting.

--
resolution:  -> duplicate

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Thanks. I have some API nits to work to make this more broadly useful.  I'll 
all those on the PR comments soonish :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests:  -11268

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
pull_requests:  -11267, 11268

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35756] Using `return value` in a generator function skips the returned value on for-loop iteration

[Steven D'Aprano]

Steven D'Aprano  added the comment:

You say:

> The PEP reads as if returning a value via StopIteration was meant to signal 
> that the generator was finished and that StopIteration.value was the final 
> value.

To me, the PEP is clear that `return expr` is equivalent to `raise 
StopIteration(expr)` which is *not* used as an iteration value. Can you point 
to the passage in the PEP that suggests something different to you?

--
nosy: +steven.daprano

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35751] traceback.clear_frames manages to deadlock a background task

[Taras Voinarovskyi]

Taras Voinarovskyi  added the comment:

For now, it seems like ``copy.copy(exception)`` before raising seems to prevent 
this behaviour. So for all exceptions originating from background tasks I raise 
a copy to the user, rather than the original exception. It prints correct 
stack, so no real impact on the library.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

Would it make sense to use os.confstr('CS_PATH') instead of a hardcoded path, 
or is identical behavior on all POSIX platforms preferred to that?

--
nosy: +izbyshev

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

[Alexander Belopolsky]

Alexander Belopolsky  added the comment:

Here is an implementation that I've used for years. It is somewhat shorter than 
the one in PR 11583:

class CycleError(LogicalError, ValueError):
"""dependencies cycle detected

"""


def tsort(pairs):
"""topological sort

Just like unix tsort(1)

>>> tsort([(1, 2), (7, 8), (8, 10), (7, 4), (2, 3), (4, 10)])
[1, 7, 2, 8, 4, 3, 10]
>>> try:
... tsort([(1,2), (2,1)])
... except CycleError as e:
... print(e)
([], Counter({1: 1, 2: 1}), {1: [2], 2: [1]})
"""
# inspired by 
http://mail.python.org/pipermail/python-list/1999-July/002831.html
successors = {}
predecessor_counts = collections.Counter()
for x, y in pairs:
successors.setdefault(x, []).append(y)
predecessor_counts.setdefault(x, 0)
predecessor_counts[y] += 1
ordered = [x for x in predecessor_counts
   if predecessor_counts[x] == 0]
for x in ordered:
del predecessor_counts[x]
for y in successors.get(x, ()):
predecessor_counts[y] -= 1
if predecessor_counts[y] == 0:
ordered.append(y)
if predecessor_counts:
raise CycleError(ordered, predecessor_counts, successors)
return ordered

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

> One of the issue that I have with using posix_spawn() is that the *exact* 
> behavior of subprocess is not properly defined by test_subprocess. Should we 
> more more tests, or document that the exact behavior is "an implementation 
> detail"?

Regarding using PATH from "env" instead of parent's environment, it may be 
considered documented because subprocess docs refer to os.execvp(), where it's 
explicitly documented:

"""
The variants which include a “p” near the end (execlp(), execlpe(), execvp(), 
and execvpe()) will use the PATH environment variable to locate the program 
file. When the environment is being replaced (using one of the exec*e variants, 
discussed in the next paragraph), the new environment is used as the source of 
the PATH variable.
"""

The problem is that it differs from execvp() in libc (and POSIX), so I would 
consider such behavior a bug if it wasn't so old to become a feature. Thanks to 
Victor for noticing that, I missed it.

So, if we can't change os.execvp() and/or current subprocess behavior, 
posix_spawnp seems to be ruled out. (I don't consider temporarily changing the 
parent environment as a solution). A naive emulation of posix_spawnp would be 
repeatedly calling posix_spawn for each PATH entry, but that's prohibitively 
expensive. Could we use a following hybrid approach instead?

Iterate over PATH entries and perform a quick check for common exec errors 
(ENOENT, ENOTDIR, EACCES) manually (e.g. by stat()). If the check fails, exec 
would also fail so just skip the entry. (It's important not to get false 
negatives here, but the child process would have the same privileges as the 
parent since we don't use POSIX_SPAWN_RESETIDS, so I can't think of one). 
Otherwise, call posix_spawn with the absolute path. If it fails, skip the entry.

Looks ugly, but are there other problems? This would seem to work just as well 
as posix_spawnp in the common case, but in the worst (contrived) case it would 
be equivalent to calling posix_spawn for each PATH entry.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch, patch
pull_requests: +11266, 11267
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch, patch, patch
pull_requests: +11266, 11267, 11268
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue17005] Add a topological sort algorithm

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +11266
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

Change by STINNER Victor :


Added file: https://bugs.python.org/file48064/subprocess_curdir.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

STINNER Victor  added the comment:

I wrote attached execv_curdir.py to check if os.execv() tries to find the 
executable in the current directory if it doesn't contain a directory: yes, it 
does.

$ python3 execv_curdir.py 
execv() searchs in the current directory

I also wrote attached subprocess_curdir.py which confirms that subprocess runs 
a program from the current directory if it exists.

$ python3 subprocess_curdir.py 
defpath = :/bin:/usr/bin
subprocess searchs in the current directory

Moreover, the current directory has the priority over /bin and /usr/bin.

--
Added file: https://bugs.python.org/file48063/execv_curdir.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[Alexey Izbyshev]

Alexey Izbyshev  added the comment:

> Hi,

> As a disclaimer, I'm a FreeBSD developer interested in making sure we're 
> doing the right thing here. =)

> May I ask what the above assessment is based on, and specifically what we 
> need to address?

Hello, Kyle! That assessment is based on my quick and incorrect reading of 
posix_spawn source code. I didn't notice that "error" is visible in the parent 
due to address space sharing. Sorry for that, and thank you for the correction. 
A proper error notification is required for posix_spawn to be used in 
subprocess as a replacement for fork/exec, and FreeBSD does it right.

While we're here, would you be kind to answer several questions about 
posix_spawn on FreeBSD?

1) On Linux, without taking additional precautions, signals may be delivered to 
a child process created via vfork(). If a signal handler runs in such a child, 
it can leave traces of its work in the (shared) memory, potentially surprising 
the parent process. To avoid this, glibc[1] and musl[2] disable all signals 
(even signals used internally for thread cancellation) before creating a child, 
but FreeBSD calls vfork() right away. Is this not considered a problem, or is 
something hidden in vfork() implementation?

2) Another problem with vfork() is potential sharing of address space between 
processes with different privileges (see Rich Felker's post[3] about this and 
the previous problem). Is this a valid concern on FreeBSD?

3) Does FreeBSD kernel guarantee that when waitpid(WNOHANG) is called at [4], 
the child is ready for reaping? On Linux, it's not always the case[5].

[1] 
https://sourceware.org/git/?p=glibc.git;a=blob;f=sysdeps/unix/sysv/linux/spawni.c;h=353bcf5b333457d191320e358d35775a2e9b319b;hb=HEAD#l372
[2] 
http://git.musl-libc.org/cgit/musl/tree/src/process/posix_spawn.c?id=5ce3737931bb411a8d167356d4d0287b53b0cbdc#n171
[3] https://ewontfix.com/7
[4] 
https://svnweb.freebsd.org/base/head/lib/libc/gen/posix_spawn.c?view=markup#l229
[5] 
https://sourceware.org/git/?p=glibc.git;a=commit;h=aa95a2414e4f664ca740ad5f4a72d9145abbd426

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35756] Using `return value` in a generator function skips the returned value on for-loop iteration

[Bryan Koch]

New submission from Bryan Koch :

Using the new "`return value` is semantically equivalent to `raise 
StopIteration(value)`" syntax created in PEP-380 
(https://legacy.python.org/dev/peps/pep-0380/#formal-semantics) causes the 
returned value to be skipped by standard methods of iteration.

The PEP reads as if returning a value via StopIteration was meant to signal 
that the generator was finished and that StopIteration.value was the final 
value.  If StopIteration.value is meant to represent the final value, then the 
built-in for-loop should not skip it and the current implementation in 3.3, 
3.4, 3.5, and 3.6 should be considered an oversight of the PEP and a bug (I 
don't have a version of 3.7 or 3.8 to test newer versions).

Reproduction code is attached with comments/annotations.

--
files: ex1.py
messages: 333802
nosy: Bryan Koch
priority: normal
severity: normal
status: open
title: Using `return value` in a generator function skips the returned value on 
for-loop iteration
type: behavior
versions: Python 3.4, Python 3.5, Python 3.6
Added file: https://bugs.python.org/file48062/ex1.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35755] Remove current directory from posixpath.defpath to enhance security

[STINNER Victor]

New submission from STINNER Victor :

Currently, posixpath.defpath is equal to:

defpath = ':/bin:/usr/bin'

It gives 3 directories:

>>> posixpath.defpath.split(posixpath.pathsep)
['', '/bin', '/usr/bin']

where the empty string means "the current directory". Trying to locate an 
executable from the current directory can be security issue when an attacker 
tries to execute arbitrary command.

The Linux exec(3) manual page contains an interesting note about the removal of 
the empty string from glibc 2.24 by accident:

http://man7.org/linux/man-pages/man3/execvp.3.html

NOTES

   The default search path (used when the environment does not contain
   the variable PATH) shows some variation across systems.  It generally
   includes /bin and /usr/bin (in that order) and may also include the
   current working directory.  On some other systems, the current
   working is included after /bin and /usr/bin, as an anti-Trojan-horse
   measure.  The glibc implementation long followed the traditional
   default where the current working directory is included at the start
   of the search path.  However, some code refactoring during the
   development of glibc 2.24 caused the current working directory to be
   dropped altogether from the default search path.  This accidental
   behavior change is considered mildly beneficial, and won't be
   reverted.

   (...)

Context of this issue: This discussion started from my PR 11579 which modifies 
the subprocess module to use posix_spawnp():
https://github.com/python/cpython/pull/11579#pullrequestreview-193261299


So I propose to replace defpath = ':/bin:/usr/bin' with defpath = 
'/bin:/usr/bin' which gives 2 directories:

>>> '/bin:/usr/bin'.split(posixpath.pathsep)
['/bin', '/usr/bin']

This change would only affect os.get_exec_path(), and so indirectly the 
subprocess module (when the executable contains no directory), *when the PATH 
environmant variable is not set*.

--
components: Library (Lib)
messages: 333801
nosy: christian.heimes, giampaolo.rodola, gregory.p.smith, vstinner
priority: normal
severity: normal
status: open
title: Remove current directory from posixpath.defpath to enhance security
type: security
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 8c349565e8a442e17f1a954d1a9996847749d778 by Victor Stinner in 
branch 'master':
Revert "bpo-35537: subprocess can now use os.posix_spawnp (GH-11579)" (GH-11582)
https://github.com/python/cpython/commit/8c349565e8a442e17f1a954d1a9996847749d778


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> One of the issue that I have with using posix_spawn() is that the *exact* 
> behavior of subprocess is not properly defined by test_subprocess. Should we 
> more more tests, or document that the exact behavior is "an implementation 
> detail"? I guess that the best for users is get the same behavior on all 
> platforms, but can we really warranty that? Python rely on the operating 
> system and the libc, but each platform has subtle behavior differneces. 
> Handling time and date is a good example: strptime() and strftime() have big 
> differences between platforms. posix_spawn() is another example where the 
> implementation is very different depending on the platform.

I don't think we can get the same behaviour in all platforms, but I would want 
to know what Gregory P. Smith thinks about this potential divergence in 
behaviour and what are the guarantees that posix_subprocess should maintain.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31267] threading.Timer object is affected by changes to system time: Python locks should use a monotonic clock if available

[STINNER Victor]

STINNER Victor  added the comment:

I'm sorrry, I read the issue too quickly and misunderstood it. I guess that 
it's a duplicate of bpo-23428: "Use the monotonic clock for thread conditions 
on POSIX platforms". This issue is blocked the libc...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35753] Importing call from unittest.mock directly causes ValueError

[ppperry]

ppperry  added the comment:

Sorry, looks like I was wrong about it being a duplicate. The actual bug 
appears to be "doctest can't run on a module that contains un-unwrappable 
objects", which there probably is an issue for but I don't know what it is.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35747] Python threading event wait influenced by date change

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I think this is becase at the end, the threading code calls

sem_timedwait(thelock, ) where  is the timespect structure. The manpage 
of sem_timedwait says:

...

The timeout shall expire when the absolute time specified by abs_timeout 
passes, as measured by the clock on which timeouts are based (that is, when the 
value of that clock equals or exceeds abs_timeout), or if the absolute time 
specified by abs_timeout has already been passed at the time of the call.

If the Timers option is supported, the timeout shall be based on the 
CLOCK_REALTIME clock. If the Timers option is not supported, the timeout shall 
be based on the system clock as returned by the time() function. The resolution 
of the timeout shall be the resolution of the clock on which it is based. The 
timespec data type is defined as a structure in the  header.

So I assume this is using the system clock, so is affected by the date.

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor  added the comment:

One of the issue that I have with using posix_spawn() is that the *exact* 
behavior of subprocess is not properly defined by test_subprocess. Should we 
more more tests, or document that the exact behavior is "an implementation 
detail"? I guess that the best for users is get the same behavior on all 
platforms, but can we really warranty that? Python rely on the operating system 
and the libc, but each platform has subtle behavior differneces. Handling time 
and date is a good example: strptime() and strftime() have big differences 
between platforms. posix_spawn() is another example where the implementation is 
very different depending on the platform.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor  added the comment:

Oh... Using directly posix_spawnp() in subprocess to locate the executable in 
the PATH introduces subtle behavior changes...
https://github.com/python/cpython/pull/11579/#pullrequestreview-193261299

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +11263, 11264, 11265

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +11263, 11264

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +11263

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35754] When writing/closing a closed Popen.stdin, I get OSError vs. BrokenPipeError randomly or depending on bufsize

[jimbo1qaz_ via Gmail]

jimbo1qaz_ via Gmail  added the comment:

Popen._stdin_write() has comments linking to https://bugs.python.org/issue19612 
and https://bugs.python.org/issue30418 .

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35754] When writing/closing a closed Popen.stdin, I get OSError vs. BrokenPipeError randomly or depending on bufsize

[jimbo1qaz_ via Gmail]

New submission from jimbo1qaz_ via Gmail :

Windows 10 1709 x64, Python 3.7.1.

Minimal example and stack traces at 
https://gist.github.com/jimbo1qaz/75d7a40cac307f8239ce011fd90c86bf

Essentially I create a subprocess.Popen, using a process (msys2 head.exe) which 
closes its stdin after some amount of input, then write nothing but b"\n"*1000 
bytes to its stdin.

If the bufsize is small (1000 bytes), I always get OSError: [Errno 22] Invalid 
argument

If the bufsize is large (1 million bytes), I always get BrokenPipeError: [Errno 
32] Broken pipe. (This happens whether I write 1 million newlines or 1000 at a 
time).

Originally I created a ffmpeg->ffplay pipeline with a massive bufsize (around 
1280*720*3 * 2 frames), then wrote 1280*720*3 bytes of video frames at a time. 
Closing ffplay's window usually created BrokenPipeError, but occasionally 
OSError. This was actually random.




It seems that this is known to some extent, although I couldn't find any 
relevant issues on the bug tracker, and "having to catch 2 separate errors" 
isn't explained on the documentation. (Is it intended though undocumented 
behavior?)

Popen._communicate() calls Popen._stdin_write(), but specifically ignores 
BrokenPipeError and OSError where exc.errno == errno.EINVAL == 22 (the 2 cases 
I encountered).

But I don't call Popen.communicate() but instead write directly to stdin, since 
I have a loop that outputs 1 video frame at a time, and rely on pipe blocking 
to stop my application from running too far ahead of ffmpeg/ffplay.



popen.stdin is a <_io.BufferedWriter name=3>.

https://docs.python.org/3/library/io.html#io.BufferedIOBase.write

>Write the given bytes-like object, b, and return the number of bytes written 
>(always equal to the length of b in bytes, since if the write fails an OSError 
>will be raised). Depending on the actual implementation, these bytes may be 
>readily written to the underlying stream, or held in a buffer for performance 
>and latency reasons.

The page doesn't mention BrokenPipeError at all (Ctrl+F). So why do I 
*sometimes* get a BrokenPipeError (subclasses ConnectionError subclasses 
OSError) instead?

--
messages: 333792
nosy: jimbo1qaz_
priority: normal
severity: normal
status: open
title: When writing/closing a closed Popen.stdin, I get OSError vs. 
BrokenPipeError randomly or depending on bufsize
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35753] Importing call from unittest.mock directly causes ValueError

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Indeed, closed as duplicate of https://bugs.python.org/issue25532

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35753] Importing call from unittest.mock directly causes ValueError

[ppperry]

ppperry  added the comment:

According to the error message, this is a duplicate of 
https://bugs.python.org/issue25532

--
nosy: +ppperry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35753] Importing call from unittest.mock directly causes ValueError

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Hi David,

Can you provide a full reproducer? In your example, mocked_print is undefined. 
Please, provide a self-contained script that can be executed to test the 
behaviour.

--
nosy: +pablogsal

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35753] Importing call from unittest.mock directly causes ValueError

[David Antonini]

David Antonini  added the comment:

Apologies for initial malformed message, I hit submit prematurely.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35753] Importing call from unittest.mock directly causes ValueError

[David Antonini]

David Antonini  added the comment:

I'm having a problem with mock.call when I import it directly from 
unittest.mock.

When I do:

from unittest.mock import patch, mock_open, call

mocked_print.assert_has_calls([
call("first print"),
call("second print"),
])

I get: 

C:\Program Files (x86)\Python37-64\lib\doctest.py:932: in find
self._find(tests, obj, name, module, source_lines, globs, {})
C:\Program Files (x86)\Python37-64\lib\doctest.py:991: in _find
if ((inspect.isroutine(inspect.unwrap(val))
C:\Program Files (x86)\Python37-64\lib\inspect.py:515: in unwrap
raise ValueError('wrapper loop when unwrapping {!r}'.format(f))
E   ValueError: wrapper loop when unwrapping call
collected 1 item / 1 errors

But when I don't import call directly my test runs as expected:

from unittest.mock import patch, mock_open

import unittest.mock

mocked_print.assert_has_calls([
mock.call(),
mock.call(),
])

I have the same issue when using:

assert mocked_print.call_args_list == [call("first print"), call("second 
print")] <- ValueError
assert mocked_print.call_args_list == [mock.call("first print"), 
mock.call("second print")] <- Works as expected.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35753] Importing call from unittest.mock directly causes ValueError

[David Antonini]

New submission from David Antonini :

Ok so that's a pretty odd bug. I already had from unittest.mock import patch, 
mock_open so I simply modified that to from  instead of doing mock.call in my 
test. Changing this to from unittest import mock and then mock.call fixed the 
error.

from unittest.mock import patch, mock_open, call

mocked_print.assert_has_calls([
call("first print"),
call("second print"),
])

I get: 

C:\Program Files (x86)\Python37-64\lib\doctest.py:932: in find
self._find(tests, obj, name, module, source_lines, globs, {})
C:\Program Files (x86)\Python37-64\lib\doctest.py:991: in _find
if ((inspect.isroutine(inspect.unwrap(val))
C:\Program Files (x86)\Python37-64\lib\inspect.py:515: in unwrap
raise ValueError('wrapper loop when unwrapping {!r}'.format(f))
E   ValueError: wrapper loop when unwrapping call
collected 1 item / 1 errors

But when I don't import call directly my test runs as expected:

from unittest.mock import patch, mock_open

import unittest.mock

mocked_print.assert_has_calls([
mock.call(),
mock.call(),
])

I have the same issue when using:

assert mocked_print.call_args_list == [call("first print"), call("second 
print")] <- ValueError
assert mocked_print.call_args_list == [mock.call("first print"), 
mock.call("second print")] <- Works as expected.

--
components: Tests
messages: 333786
nosy: toonarmycaptain
priority: normal
severity: normal
status: open
title: Importing call from unittest.mock directly causes ValueError
type: behavior
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34782] Pdb crashes when code is executed in a mapping that does not define `__contains__`

[ppperry]

ppperry  added the comment:

The thing is, though, that the same error occurs if a larger code being 
debugged calls `exec` or `eval` with such a mapping 
(`pdb.run("eval('1+1',{},FakeContainer())" also crashes with the same error), 
and the test suite contains code that evals other code in an incomplete mapping 
(test_var_annot_custom_maps in test_grammar)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35749] Rewrite asyncio signal handler

[Yury Selivanov]

Change by Yury Selivanov :


--
stage: resolved -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35749] Rewrite asyncio signal handler

[Yury Selivanov]

Yury Selivanov  added the comment:

> You are right, the issue is "theoretical", I didn't see problems with current 
> implementation.
> The self-pipe buffer is pretty big, a chance to reach its limit is relatively 
> low.

Actually, I disagree.  The problem manifests itself easily when you orchestrate 
a big number of subprocesses (think 1000s).

I've implemented a different strategy for signals handling in uvloop [1] and I 
think we can easily employ similar approach in asyncio.

Please keep this open. 

[1] 
https://github.com/MagicStack/uvloop/commit/6e03e513625eb544a5f60c9bf9922019fc3a0b29

--
resolution: rejected -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35749] Rewrite asyncio signal handler

[Andrew Svetlov]

Change by Andrew Svetlov :


--
resolution:  -> rejected
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35749] Rewrite asyncio signal handler

[Andrew Svetlov]

Andrew Svetlov  added the comment:

Victor, thanks for the message.
I forgot mentioned changes.
You are right, the issue is "theoretical", I didn't see problems with current 
implementation.
The self-pipe buffer is pretty big, a chance to reach its limit is relatively 
low.
But a change to break asyncio on some OS with the patch is much higher.
Moreover, the patch assumes that python signal handler will be called *before* 
reading from self-pipe. 
Otherwise, a signal callback will be postponed up to next writing to the pipe, 
which looks like a hidden bug.

Let's close the issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35752] test_buffer fails on ppc64le: memoryview pack_single() is miscompiled

[Charalampos Stratakis]

Charalampos Stratakis  added the comment:

Possibly relevant:

https://fedoraproject.org/wiki/Changes/PPC64LE_Float128_Transition#Detailed_Description

But the work is not complete.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34782] Pdb crashes when code is executed in a mapping that does not define `__contains__`

[Henry Chen]

Henry Chen  added the comment:

I agree that this behavior is normal, unless there is a use case for passing a 
partially implemented mapping object to pdb.run. However I cannot think of such 
a use case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13285] signal module ignores external signal changes

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

For reference, the sources for my implementation: 
https://github.com/sagemath/cysignals/blob/master/src/cysignals/pysignals.pyx

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35752] test_buffer fails on ppc64le: memoryview pack_single() is miscompiled

[Stefan Krah]

Stefan Krah  added the comment:

Does it also work with -fno-inline, just for ppc64? There are other
places in the sources where an inlined memcpy() could be miscompiled.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35752] test_buffer fails on ppc64le: memoryview pack_single() is miscompiled

[STINNER Victor]

STINNER Victor  added the comment:

> This is a performance sensitive function, so I prefer not to add
volatile.

I don't suggest to use volatile. I'm just pointing to the function causing the 
bug and I said that volatile worked for me :-)

I made my tests on:

* Red Hat Enterprise Linux release 8.0 Beta (Ootpa)
* gcc (GCC) 8.2.1 20180905 (Red Hat 8.2.1-3)
* Linux kernel 4.18.0-56.el8.ppc64le
* uname -p: ppc64le

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35752] test_buffer fails on ppc64le: memoryview pack_single() is miscompiled

[Stefan Krah]

Stefan Krah  added the comment:

This is a performance sensitive function, so I prefer not to add
volatile. MSVC also had a bug with that function, but only in PGO
mode. Microsoft has fixed the issue long ago.

Do newer gcc versions have this issue?

I'm fine with wrapping the entire macro in an #ifdef for __ppc64__.

Who is using this platform? IBM Power9?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35582] Argument Clinic: inline parsing code for functions with only positional parameters

[STINNER Victor]

STINNER Victor  added the comment:

I ran benchmarks on speed.python.org, it's the 
5bb146aaea1484bcc117ab6cb38dda39ceb5df0f dot (Jan 13, 2019). I didn't look at 
results.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35752] test_buffer fails on ppc64le: memoryview pack_single() is miscompiled

[Charalampos Stratakis]

Change by Charalampos Stratakis :


--
nosy: +cstratak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35751] traceback.clear_frames manages to deadlock a background task

[Yury Selivanov]

Yury Selivanov  added the comment:

Very interesting. Thanks for reporting this issue. I'll definitely take a look 
at this before 3.8 is released.  For 3.7 you'll likely have to find a 
workaround.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35752] test_buffer fails on ppc64le: memoryview pack_single() is miscompiled

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
Added file: https://bugs.python.org/file48061/pack_single.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35752] test_buffer fails on ppc64le: memoryview pack_single() is miscompiled

[STINNER Victor]

New submission from STINNER Victor :

The bug was first reported on Fedora:
https://bugzilla.redhat.com/show_bug.cgi?id=1540995

==
FAIL: test_memoryview_struct_module (test.test_buffer.TestBufferProtocol)
--
Traceback (most recent call last):
  File "/builddir/build/BUILD/Python-3.6.4/Lib/test/test_buffer.py", line 2540, 
in test_memoryview_struct_module
self.assertEqual(m[1], nd[1])
AssertionError: -21.08474121094 != -21.10381469727

The problem is the conversion from C double (64-bit float) and C float (32-bit 
float). There are 2 implementations:

* Objects/memoryobject.c: pack_single() and unpack_single()
* Modules/_struct.c: nu_float() and np_float()

Attached ppc64_float32_bug.py is the simplified test case to trigger the bug.

The result depends on the compiler optimization level:

* gcc -O0: -21.10381469727 == -21.10381469727, OK
* gcc -O1: -21.08474121094 != -21.10381469727, ERROR
* (I guess that higher optimization level also trigger the bug)

The problem is that the pack_single() function is "miscompiled" (or "too 
optimized").

Adding "volatile" to PACK_SINGLE() prevents the unsafe compiler optimization 
and fix the issue for me: try attached pack_single_volatile.patch.


=== -O1 assembler code with the bug ===

PACK_SINGLE(ptr, d, float);


r30 = ptr
(gdb) p $vs63.v2_double
$17 = {0, -21.101}

=> 0x100a1178 :   stxsspx vs63,0,r30

(gdb) p /x (*ptr)@4
$10 = {0xcc, 0xcc, 0xa8, 0xc1}

The first byte is 0xcc: WRONG.


=== -O1 assembler code without the bug (volatile) ===

r30 = ptr

(gdb) p $f31
$1 = -21.101

=> 0x100a11e4 :   frspf31,f31

(gdb) p $f31
$2 = -21.10381469727

   0x100a11e8 :   stfsf31,152(r1)
   0x100a11ec :   lwz r9,152(r1)

(gdb) p /x $r9
$8 = 0xc1a8cccd

   0x100a11f0 :   stw r9,0(r30)

(gdb) p /x (*ptr)@4
$9 = {0xcd, 0xcc, 0xa8, 0xc1}

   0x100a11f4 :   li  r3,0
   0x100a11f8 :   lfd f31,216(r1)
   0x100a11fc :   ld  r30,200(r1)

The first byte is 0xcd: GOOD.

--
components: Library (Lib)
files: ppc64_float32_bug.py
messages: 333774
nosy: mark.dickinson, skrah, vstinner
priority: normal
severity: normal
status: open
title: test_buffer fails on ppc64le: memoryview pack_single() is miscompiled
versions: Python 3.7, Python 3.8
Added file: https://bugs.python.org/file48060/ppc64_float32_bug.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35674] Expose os.posix_spawnp()

[STINNER Victor]

STINNER Victor  added the comment:

FYI I modified subprocess to use os.posix_spawnp() when it's available:

New changeset 07858894689047c77f9c12ddc061d30681368d19 by Victor Stinner in 
branch 'master':
bpo-35537: subprocess can now use os.posix_spawnp (GH-11579)
https://github.com/python/cpython/commit/07858894689047c77f9c12ddc061d30681368d19

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35674] Expose os.posix_spawnp()

[STINNER Victor]

STINNER Victor  added the comment:

Well done Joannah Nanjekye :-) Thanks to Joannah for this nice enhancemenet and 
to all reviewers.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 07858894689047c77f9c12ddc061d30681368d19 by Victor Stinner in 
branch 'master':
bpo-35537: subprocess can now use os.posix_spawnp (GH-11579)
https://github.com/python/cpython/commit/07858894689047c77f9c12ddc061d30681368d19


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35751] traceback.clear_frames manages to deadlock a background task

[Taras Voinarovskyi]

Taras Voinarovskyi  added the comment:

So yes, the `clear_frames` function will force a running generator to close. 
See https://github.com/python/cpython/blob/3.7/Objects/frameobject.c#L566, it 
explicitly does a Finalize. Would that be the desired behaviour for 
assertRaises is not clear. I find it strange that catching an exception is 
closing my running coroutine.

The reproduce example can be lowered to something like::

import asyncio


async def background(error_future):
try:
raise ValueError
except Exception as exc:
error_future.set_exception(exc)

await asyncio.sleep(1)


async def main():
loop = asyncio.get_event_loop()
error_future = loop.create_future()
task = asyncio.create_task(background(error_future))

await asyncio.wait([error_future])
exc = error_future.exception()
import traceback
traceback.clear_frames(exc.__traceback__)

# Will block forever, as task will never be waken up
await task


if __name__ == "__main__":
asyncio.run(main())

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27015] subprocess.CalledProcessError's repr changes based on kwargs, and doesn't unpickle

[Rémi Lapeyre]

Rémi Lapeyre  added the comment:

I tried to fix the issue, the attached PR solves the issue of saving the kwargs 
and unpickling the exception but I was not able to fix a regression I caused in 
test_memory_error_in_PyErr_PrintEx.

--
versions: +Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27015] subprocess.CalledProcessError's repr changes based on kwargs, and doesn't unpickle

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch, patch, patch
pull_requests: +11260, 11261, 11262
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27015] subprocess.CalledProcessError's repr changes based on kwargs, and doesn't unpickle

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch, patch
pull_requests: +11260, 11261
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27015] subprocess.CalledProcessError's repr changes based on kwargs, and doesn't unpickle

[Rémi Lapeyre]

Change by Rémi Lapeyre :


--
keywords: +patch
pull_requests: +11260
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35749] Rewrite asyncio signal handler

[STINNER Victor]

STINNER Victor  added the comment:

In short, attached PR reverts the following commit:

commit fe5649c7b7bf52147480d6b1124a3d8e3597aee3
Author: Victor Stinner 
Date:   Thu Jul 17 22:43:40 2014 +0200

Python issue #21645, Tulip issue 192: Rewrite signal handling

Since Python 3.3, the C signal handler writes the signal number into the 
wakeup
file descriptor and then schedules the Python call using 
Py_AddPendingCall().

asyncio uses the wakeup file descriptor to wake up the event loop, and 
relies
on Py_AddPendingCall() to schedule the final callback with call_soon().

If the C signal handler is called in a thread different than the thread of 
the
event loop, the loop is awaken but Py_AddPendingCall() was not called yet. 
In
this case, the event loop has nothing to do and go to sleep again.
Py_AddPendingCall() is called while the event loop is sleeping again and so 
the
final callback is not scheduled immediatly.

This patch changes how asyncio handles signals. Instead of relying on
Py_AddPendingCall() and the wakeup file descriptor, asyncio now only relies 
on
the wakeup file descriptor. asyncio reads signal numbers from the wakeup 
file
descriptor to call its signal handler.

=> https://github.com/python/asyncio/issues/192


Since this change, the C signal handler of Python has been reworked in 
bpo-30038 by:

commit 4ae01496971624c75080431806ed1c08e00f22c7
Author: Nathaniel J. Smith 
Date:   Tue May 16 14:12:11 2017 -0700

bpo-30038: fix race condition in signal delivery + wakeup fd (#1082)

Before, it was possible to get the following sequence of
events (especially on Windows, where the C-level signal handler for
SIGINT is run in a separate thread):

- SIGINT arrives
- trip_signal is called
- trip_signal writes to the wakeup fd
- the main thread wakes up from select()-or-equivalent
- the main thread checks for pending signals, but doesn't see any
- the main thread drains the wakeup fd
- the main thread goes back to sleep
- trip_signal sets is_tripped=1 and calls Py_AddPendingCall to notify
  the main thread the it should run the Python-level signal handler
- the main thread doesn't notice because it's asleep

This has been causing repeated failures in the Trio test suite:
  https://github.com/python-trio/trio/issues/119


I am very scared by signals. These things are very complex and it's hard to 
handle them proplery on all platforms :-( So I'm not excited by changing the 
code.


> If the pipe is full OSError is raised.

Are you able to reproduce this issue, or is it a theorical issue?

What is the pipe buffer size on Linux? See bpo-33733.

--
nosy: +njs, vstinner
title: Ignore exception if event loop wakeup pipe is full -> Rewrite asyncio 
signal handler

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +11257

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +11257, 11258

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35537] use os.posix_spawn in subprocess

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +11257, 11258, 11259

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35674] Expose os.posix_spawnp()

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 92b8322e7ea04b239cb1cb87b78d952f13ddfebb by Victor Stinner 
(Joannah Nanjekye) in branch 'master':
bpo-35674: Add os.posix_spawnp() (GH-11554)
https://github.com/python/cpython/commit/92b8322e7ea04b239cb1cb87b78d952f13ddfebb


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35748] urlparse library detecting wrong hostname leads to open redirect vulnerability

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

There are also some notes at https://tools.ietf.org/html/rfc3986#section-7.6

Because the userinfo subcomponent is rarely used and appears before
the host in the authority component, it can be used to construct a
URI intended to mislead a human user by appearing to identify one
(trusted) naming authority while actually identifying a different
authority hidden behind the noise.  For example

ftp://cnn.example.com=breaking_news@10.0.0.1/top_story.htm

might lead a human user to assume that the host is 'cnn.example.com',
whereas it is actually '10.0.0.1'.  Note that a misleading userinfo
subcomponent could be much longer than the example above.

A misleading URI, such as that above, is an attack on the user's
preconceived notions about the meaning of a URI rather than an attack
on the software itself.  User agents may be able to reduce the impact
of such attacks by distinguishing the various components of the URI
when they are rendered, such as by using a different color or tone to
render userinfo if any is present, though there is no panacea.  More
information on URI-based semantic attacks can be found in [Siedzik]

In Firefox nightly and latest chrome pasting the above URL makes a request to 
10.0.0.1/top_story.htm where in Chrome the URL in the address bar is changed to 
10.0.0.1/top_story.htm and Firefox has the same URL in the address bar. Python 
also returns '10.0.0.1' as the hostname for the above example using urlparse.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13285] signal module ignores external signal changes

[Jeroen Demeyer]

Jeroen Demeyer  added the comment:

> In Jeroen's API, I can see what the Python-level signal handler is, but 
> there's no way to find out whether that signal handler is actually in use or 
> not.

I added support for that in the latest cysignals release. Now you can do

>>> import signal
>>> from cysignals.pysignals import getossignal, python_os_handler
>>> _ = signal.signal(signal.SIGINT, signal.default_int_handler)
>>> getossignal(signal.SIGINT) == python_os_handler
True

Note that cysignals is POSIX-only for now (it assumes sigaction), but the code 
could easily be ported to other systems. Ideally it would become part of 
CPython's signal module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35746] [ssl][CVE-2019-5010] TALOS-2018-0758 Denial of Service

[Christian Heimes]

Christian Heimes  added the comment:

The bug is less critical and harder to exploit than I initially thought. td;dr 
if you have cert validation enabled and only trust public root CAs from CA/B 
forum, then you are not affected.

The bug is only exploitable under two conditions:
1) The user has disabled TLS/SSL certificate validation *and* calls 
getpeercert() in 3rd party code.
2) Or the user trusts a CA that does not properly validate end-entity 
certificates.

When cert validation is enabled, the ssl module will refuse any untrusted 
certificate during the handshake. The SSLSocket.getpeercert() and 
SSLObject.getpeercert() methods raise an exception, when the handshake was not 
successful. Python 2.7 - 3.6 hostname verification code only calls 
getpeercert() after the cert chain was validated successfully. Python 3.7+ no 
longer calls getpeercert() for hostname verification. Further more hostname 
verification can't be enabled when cert validation is disabled.

For publicly trusted CAs governed by CA/B baseline requirements, CRL DPs must 
by valid URI general names with HTTP links. From CA/Browser Forum Baseline 
Requirements Version 1.6.2, December 10, 2018, section 7.1.2.3. Subscriber 
Certificate:

b. cRLDistributionPoints
This extension MAY be present. If present, it MUST NOT be marked critical, and 
it MUST contain the HTTP URL of the CA’s CRL service.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue34782] Pdb crashes when code is executed in a mapping that does not define `__contains__`

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

There is no a crash. There is just an exception raised when you pass an 
inappropriate object as locals. This is an expected behavior, and I do not 
think that anything need to be changed.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35750] process finished with exit code -1073740940 (0xc0000374)

[Eryk Sun]

Eryk Sun  added the comment:

A crash with the code STATUS_HEAP_CORRUPTION (0xC374) is due to a bug, not 
memory exhaustion. The bug may not be the fault of the interpreter or standard 
library if you're working with extension modules or ctypes. It's easy to 
corrupt the internal structures of a heap. For example:

>>> import ctypes
>>> b = (ctypes.c_char * 10)()
>>> ctypes.POINTER(ctypes.c_int)(b)[-1] = 0
>>> del b

[crashed]

C:\>echo %errorlevel%
-1073740940

Heap corruption is best examined with a debugger (e.g. windbg or cdb) attached 
to the faulting process, with full page heap checking enabled for the 
executable in the registry (usually set via gflags). This reserves a page 
before and after each heap allocation to ensure that writes that would 
otherwise corrupt the heap will instead immediately raise an access violation.

--
components: +Windows
nosy: +eryksun, paul.moore, steve.dower, tim.golden, zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35750] process finished with exit code -1073740940 (0xc0000374)

[STINNER Victor]

STINNER Victor  added the comment:

You can enable faulthandler to get the Python traceback where the bug occurs. 
It might help a little bit. Either write the traceback into a file (file 
argument of faulthandler.enable()) or run your program in a terminal to see the 
traceback on the terminal.

http://docs.python.org/dev/library/faulthandler.html

I also suggest you to try to run your program using "python -X dev program.py" 
to enable more debug checks:
https://pythondev.readthedocs.io/debug_tools.html

PYTHONMALLOC=debug environment variable may help.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35750] process finished with exit code -1073740940 (0xc0000374)

[Eric V. Smith]

Eric V. Smith  added the comment:

I assume this is on Windows? 0xc374 is a heap corruption error on Windows.

I agree that without a way to reproduce this it will be impossible to track 
down any error.

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35751] traceback.clear_frames manages to deadlock a background task

[Тарас Войнаровський]

New submission from Тарас Войнаровський :

My use case:
I have a background task, say called "coordination". In that task, I want to 
catch any errors and push those to the user waiting in the main task and only 
continue running the background coroutine after the user manually resolves the 
exception. 

Issue:
When testing the behaviour with ``unittest.Case`` and using ``assertRaises`` to 
catch the exception, the background coroutine manages to just freeze. I have 
narrowed it down to ``traceback.clear_frames`` in ``assertRaises`` that causes 
a GeneratorExit in the background coroutine.

I believe this issue is a duplicate to https://bugs.python.org/issue29211, but 
wanted to provide another actual use case where it can pop up. Also even if the 
generator raises a GeneratorExit, why did the background thread freeze is still 
a mystery to me.

Script to reproduce in my case is attached.

--
components: asyncio
files: test_async_deadlock.py
messages: 333759
nosy: asvetlov, yselivanov, Тарас Войнаровський
priority: normal
severity: normal
status: open
title: traceback.clear_frames manages to deadlock a background task
type: behavior
versions: Python 3.6, Python 3.7
Added file: https://bugs.python.org/file48059/test_async_deadlock.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35748] urlparse library detecting wrong hostname leads to open redirect vulnerability

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I just tested other implementations in Ruby and Go and they too return host as 
"evil.com" for "http://www.google@evil.com; along with the user info 
component.

$ ruby -e 'require "uri"; puts URI("http://www.google@evil.com;).hostname'
evil.com
$ cat /tmp/foo.go
package main

import (
"fmt"
"net/url"
)

func main() {
u, _ := url.Parse(`http://www.google@evil.com`)
fmt.Println(u.Host);
fmt.Println(u.User);
}
$ go run /tmp/foo.go
evil.com
www.google.com

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35748] urlparse library detecting wrong hostname leads to open redirect vulnerability

[Christian Heimes]

Christian Heimes  added the comment:

You cannot compare a low level library like Python's urllib module with a user 
interface like a modern browser. Browsers do a lot of extra work to make sense 
of user input. For example Firefox and Chrome mangle your example URL and 
replace \ with /. Firefox even shows a warning when the URL contains user and 
password:

---
You are about to log in to the site “python.org” with the username “user”, but 
the website does not require authentication. This may be an attempt to trick 
you.

Is “python.org” the site you want to visit?
---

--
nosy: +christian.heimes

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35748] urlparse library detecting wrong hostname leads to open redirect vulnerability

[Neeraj Sonaniya]

Neeraj Sonaniya  added the comment:

Hi,

I know that \ (backslash)  should be encoded to url encoding (%5c) but if the 
same url (without urlencoded form) typed into URL bar of browser we are getting 
hostname to 'https://www.google.com'

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35748] urlparse library detecting wrong hostname leads to open redirect vulnerability

[Martin Panter]

Martin Panter  added the comment:

FWIW I understand the backslash should be percent-encoded in URLs, otherwise 
the URL is not valid.

This reminds me of a few other bugs:

* Issue 30500: Made the behaviour of fragment (#. . .) versus userinfo (. . .@) 
consistent, e.g. in //www.google.com#@xxx.com
* Issue 18140: Also about the ambiguity of fragment (#. . .) and query (?. . .) 
versus userinfo (. . .@)
* Issue 23328: Precedence of path segment (/. . .) versus userinfo (. . .@); 
e.g. //user/name:pass/w...@www.google.com

I think people some times come up with these invalid URLs because they are 
trying to make a URL that includes a password with unusual characters (e.g. for 
the “http_proxy” environment variable). So raising an exception or otherwise 
changing the parsing behaviour could break those cases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35749] Ignore exception if event loop wakeup pipe is full

[Andrew Svetlov]

Change by Andrew Svetlov :


--
keywords: +patch, patch, patch
pull_requests: +11254, 11255, 11256
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com