[issue31113] Stack overflow with large program

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I concur with Stefan. Some parts of the compiler are recursive. The crash is 
expected for enough complex programs, and the size of C stack is platform 
depended. There are few hard-coded limits (MAXINDENT, CO_MAXBLOCKS) that may 
prevent the crash by converting it to exception, but they don't take role in 
this case (MAXINDENT is too large (100), and CO_MAXBLOCKS limits only the level 
of nested "for" and "try" blocks).

sys.setrecursionlimit doesn't have relation to C stack.

Increasing the size of C stack on Windows can solve this issue for this 
particular case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31108] add __contains__ for list_iterator (and others) for better performance

[Terry J. Reedy]

Terry J. Reedy added the comment:

Sorry, I mistakenly assumed, without carefully checking the C code, that the 
speedup was from checking the underlying collection, without advancing the 
iterator.  I presume that " ++it->it_index;" is the statement to the contrary. 
I should have either asked or found this sooner.  I unlinked the noisy comment.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31108] add __contains__ for list_iterator (and others) for better performance

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
Removed message: http://bugs.python.org/msg299770

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31108] add __contains__ for list_iterator (and others) for better performance

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Terry, this proposition doesn't change the behavior. It moves the iterator 
forward during searching. The only effect is inlining __next__ in a loop and 
getting rid of the overhead of few indirections and calls.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31113] Stack overflow with large program

[Terry J. Reedy]

Terry J. Reedy added the comment:

We know that compile has undocumented size limits of various sorts that are 
usually more than sufficient for normal human written code but which can be 
overwhelmed by machine-generated code.  We do not regard this as a bug.  
However, 20 levels of if-nesting should not be a problem unless, say, you are 
recursively calling such a function.

How are you running python, on what machine?  What do you mean by 'crash'?  Are 
you running python from a console/terminal, so that there is someplace for 
tracebacks and exceptions to be displayed?  What does 'It did not crash' mean, 
versus the 'crash' label above?

Have you tried increasing the recursion limit with sys.setrecursionlimit.  The 
default for me is 1000.  I have used 1.  On multi-gigabyte machines, 10 
might even work. 

Instead of directly running your code, have you tried a driver program that 
reads your code (one file at a time) into a string and then compiles it with 
compile()?  You might somehow get a better error message, or in any case, find 
out which of the separate files fail.

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31082] reduce takes iterable, not just sequence

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Yes, I think it can. But be aware that "sequence" is used not only in the 
signature, but in following description.

The term "sequence" in the documentation and the docstring looks to me like a 
synonym of "iterable", not a reference to collections.abc.Sequence. I don't 
know how the wording can be improved. Maybe looking at the documentation of 
builtins filter, map, zip, and itertools functions can help.

--
nosy: +ncoghlan, rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31108] add __contains__ for list_iterator (and others) for better performance

[Terry J. Reedy]

Terry J. Reedy added the comment:

On my particular Win 10 machine with 3.6, the times are 1.52 and 2.14.  But to 
me, the times are irrelevant.  A performance enhancement, by definition, should 
not change computational results, but this does. If an iterator is a standard 
iterator, then 'x in iterator' runs the iterator until x is found or iterator 
is exhausted.  The only current code that properly uses this operation on an 
iterator must be depending on the behavior.

So while the change might fix some buggy code, it would break good code and 
divide 'iterator' into two subgroups: standard iterator and contained iterator. 
 That is turn would require us to documents that while generators remain 
standard iterators, builtin collection iterators were now contained iterators.  
This is confusion we should not inflict on users.

This should be rejected, as we have done with all other proposals to add 
features to iterators that only apply to a subset.  (The apparently exception, 
__length_hint__ was defined to not necessarily be accurate, so that there is 
effectively a default __length_hint__ for all iterators, lambda self: n, where 
n depends on the caller.)

--
nosy: +terry.reedy
type: performance -> enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31100] unable to open python on Windows

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
components: +Windows
nosy: +paul.moore, steve.dower, tim.golden, zach.ware
title: unable to open python -> unable to open python on Windows

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31097] itertools.islice not accepting np.int64

[Terry J. Reedy]

Terry J. Reedy added the comment:

LB, an exception is a graceful shutdown, not a crash.  A crash is a seg fault, 
core dump, or an OS message 'Your app has stopped unexpectedly'.

In #30537, the exception was regarded as expected, not a bug, and changing 
islice to accept np ints was regarded as an enhancement, and therefore limited 
to 3.7

--
nosy: +terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31097] itertools.islice not accepting np.int64

[Terry J. Reedy]

Changes by Terry J. Reedy :


--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Using PyNumber_AsSsize_t in itertools.islice
type: crash -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31082] reduce takes iterable, not just sequence

[Terry J. Reedy]

Terry J. Reedy added the comment:

Serhiy, given that the parameters of reduce are currently positional only (by 
experiment, it has not gotten the C.A. treatment yet), can 'sequence' in the 
signature in the docstring be changes to 'iterable'?

--
nosy: +serhiy.storchaka, terry.reedy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31123] fnmatch does not follow Unix fnmatch functionality

[R. David Murray]

R. David Murray added the comment:

Looking at the fnmatch man page, it looks like there are option flags that some 
shells use that our fnmatch doesn't support.  I'm not sure if supporting them 
is a good idea for us or not, but it is probably worth discussing.  I suspect 
our fnmatch was implemented before gnu added those extensions.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0

[Martin Panter]

Martin Panter added the comment:

It might help if you explained what “atrocities” are happening on your network. 
Is there a proxy or man-in-the-middle (or the remote peer) that shuts down TCP 
connections?

If so, perhaps this is similar to Issue 10808. From my memory, in that case an 
OS “recv” or “read” call returns zero to indicate a connection was shut down. 
Python and/or Open SSL correctly treats this as an error, but then assumes 
“errno” is valid. Perhaps Python should be raising SSLEOFError in this 
situation.

Maybe also check the “suppress_ragged_eofs” setting, but I think that only 
affects later stages, after the handshake succeeds.

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29304] dict: simplify lookup functions

[Raymond Hettinger]

Raymond Hettinger added the comment:

It would have been nice to have timed this on ARM chips or 32-bit builds.  I 
suspect that the previous code was mainly beneficial in register starved 
environments when it would have saved some memory accesses whenever there were 
an immediate first hit on a dictionary lookup (which is common).  Also, it is 
more likely to make a difference on larger dicts where the memory accesses 
aren't all in L1 cache.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31123] fnmatch does not follow Unix fnmatch functionality

[R. David Murray]

Changes by R. David Murray :


--
type: behavior -> enhancement
versions:  -Python 2.7, Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31123] fnmatch does not follow Unix fnmatch functionality

[R. David Murray]

R. David Murray added the comment:

I don't believe there is an equivalent unix command.  Are you referring to the 
fnmatch glibc function?  Can you demonstrate the differences?  I doubt we will 
change the functionality, but that would be the minimum starting point for a 
discussion.

--
nosy: +r.david.murray

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31123] fnmatch does not follow Unix fnmatch functionality

[Varun Agrawal]

New submission from Varun Agrawal:

Currently, the fnmatch module in Python does not operate like the Unix 
equivalent command. This is especially the case with patterns such as `dir/` 
which does not match directory path returned by `os` (which would be `dir`) and 
neither does the pattern match the underlying files and sub-directories. Adding 
these functionalities so that fnmatch matches it's Unix equivalent would make 
it much more powerful to use.

Glob is not useful since glob is a special case that matches only with the 
current directory paths rather than the a general purpose filename pattern 
matcher.

--
components: Library (Lib)
messages: 299763
nosy: Varun Agrawal
priority: normal
severity: normal
status: open
title: fnmatch does not follow Unix fnmatch functionality
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31115] Py3.6 threading/reference counting issues with `numexpr`

[Robert McLeod]

Robert McLeod added the comment:

After building with Python3.7 I was able to get a useful error message that 
`PyMem` functions were being called inside GIL release.  I will replace these 
with C-equivalents and try with Python 3.6.

Fatal Python error: Python memory allocator called without holding the GIL

Thread 0x04044e00 (most recent call first):
  File 
"/home/rmcleod/py37/lib/python3.7/site-packages/numexpr3-3.0==19176== 
==19176== Process terminating with default action of signal 6 (SIGABRT)
==19176==at 0x579C428: raise (raise.c:54)
==19176==by 0x579E029: abort (abort.c:89)
==19176==by 0x422DF7: Py_FatalError (pylifecycle.c:1849)
==19176==by 0x41ED4C: _PyMem_DebugCheckGIL (obmalloc.c:1972)
==19176==by 0x41ED23: _PyMem_DebugMalloc (obmalloc.c:1980)
==19176==by 0x41FCAC: PyMem_Malloc (obmalloc.c:418)
==19176==by 0xCEA1920: NumExprObject_copy_threadsafe(NumExprObject 
const*) (interpreter.cpp:147)
==19176==by 0xCEA77CE: th_worker(void*) (module.cpp:73)
==19176==by 0x4E416B9: start_thread (pthread_create.c:333)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31107] copyreg does not properly mangle __slots__ names

[Shane Harvey]

Changes by Shane Harvey :


--
pull_requests: +3039

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31107] copyreg does not properly mangle __slots__ names

[Shane Harvey]

Changes by Shane Harvey :


--
pull_requests: +3038

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31119] Signal tripped flags need memory barriers

[Nathaniel Smith]

Nathaniel Smith added the comment:

@arigo: Technically we also need that the writes to memory are observed to 
happen-before the write() call on the wakeup fd, which is not something that 
Intel is going to make any guarantees about. But *probably* this is also safe 
because the kernel has to use some kind of cross-CPU synchronization to wake up 
the read()ing thread, and that imposes a memory barrier.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31119] Signal tripped flags need memory barriers

[Armin Rigo]

Armin Rigo added the comment:

For reference, no, it can't happen on x86 or x86-64.  I've found that this 
simplified model actually works for reasoning about ordering at the hardware 
level: think about each core as a cache-less machine that always *reads* from 
the central RAM, but that has a delay for writes---i.e. writes issued by a core 
are queued internally, and actually sent to the central RAM at some unspecified 
later time, but in order.

(Of course that model fails on other machines like ARM.)

--
nosy: +arigo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31122] SSLContext.wrap_socket() throws OSError with errno == 0

[Nikolaus Rath]

New submission from Nikolaus Rath:

With a particularly atrocious network connection, I often get the following 
exception:

  File "/usr/lib/python3/dist-packages/dugong/__init__.py", line 503, in connect
self._sock = self.ssl_context.wrap_socket(self._sock, 
server_hostname=server_hostname)
  File "/usr/lib/python3.5/ssl.py", line 385, in wrap_socket
_context=self)
  File "/usr/lib/python3.5/ssl.py", line 760, in __init__
self.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 996, in do_handshake
self._sslobj.do_handshake()
  File "/usr/lib/python3.5/ssl.py", line 641, in do_handshake
self._sslobj.do_handshake()
OSError: [Errno 0] Error

I don't think an error with errno == 0 should ever be raised by Python.

--
assignee: christian.heimes
components: SSL
messages: 299759
nosy: christian.heimes, nikratio
priority: normal
severity: normal
status: open
title: SSLContext.wrap_socket() throws OSError with errno == 0
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31103] Windows Installer Product Version 3.6.2150.0 Offset By 0.0.150.0

[Damon Atkins]

Damon Atkins added the comment:

To Summaries
A)
Name/DisplayName "Python 3.X (64-bit)" or "Python 3.X.Y" (64-bit) "Python 3.X 
(64-bit)" would match the install directory which is \Python3X (this should be 
\Python3.X)

Display Version should start with 3.X.Y or 3.X.Y.

version DWORD in the registry does not need to be set but if set would need to 
be 3.X.Y
B) Registry being placed in in the wrong location
HKU\\Software\Microsoft\Windows\CurrentVersion\Uninstall\{GUID}
should be going into
HKLM\Software\Microsoft\Windows\CurrentVersion\Uninstall

C) Suggest the default install dir is
"c:\Program Files\Python3.6" instead of "c:\Program Files\Python36"

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31103] Windows Installer Product Version 3.6.2150.0 Offset By 0.0.150.0

[Markus Kramer]

Markus Kramer added the comment:

Steve, yes: from my point of view, the version in "Programs and Features" 
should be  3.6.2 or 3.6.2.0 or 3.6.2.150. 


Eryk, thank you for your explanation - I had a wrong memory about the third 
field in ProductVersion to be the micro version and learned something. 


If that would be helpful, I could change the title of this issue.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31119] Signal tripped flags need memory barriers

[Nathaniel Smith]

Nathaniel Smith added the comment:

On further investigation (= a few hours staring at the ceiling last night), it 
looks like there's another explanation for my particular bug... which is good, 
because on further investigation (= a few hours squinting at google results) it 
looks like this probably can't happen on x86/x86-64 (I think? maybe?).

It's still true though that you can't just throw in a 'volatile' and expect 
cross-thread synchronization to work -- that's not C's semantics, and it's not 
true on popular architectures like ARM.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30841] Fix a variable shadowing warning in Python-ast.c

[Łukasz Langa]

Łukasz Langa added the comment:

Thanks!

--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30841] Fix a variable shadowing warning in Python-ast.c

[Łukasz Langa]

Changes by Łukasz Langa :


--
pull_requests: +3037

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30841] Fix a variable shadowing warning in Python-ast.c

[Łukasz Langa]

Łukasz Langa added the comment:

The actual change is just three lines in asdl_c.py. Python-ast.c is generated.

--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23702] docs.python.org/3/howto/descriptor.html still refers to "unbound methods"

[Raymond Hettinger]

Raymond Hettinger added the comment:

We have a sprint in early September and I'll fix it then.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31113] Stack overflow with large program

[Antoine Pitrou]

Antoine Pitrou added the comment:

I'm not an expert in the compiler myself.

--
nosy: +benjamin.peterson, brett.cannon, ncoghlan, serhiy.storchaka, yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26618] _overlapped extension module of asyncio uses deprecated WSAStringToAddressA() function

[Segev Finer]

Segev Finer added the comment:

It was fixed here: 
https://github.com/python/cpython/commit/cc16be85c0b7119854c00fb5c666825deef641cf#diff-1308fc2ab05154373172f10e931c78f0L986.
 So it's fixed for 3.6 and 3.7, but not in 3.5.

--
nosy: +Segev Finer

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31113] Stack overflow with large program

[Manuel Krebber]

Manuel Krebber added the comment:

I have already tried to reduce the nesting, but it still crashes. I have to 
admit that ~20 levels of nesting are still quite a lot. But I am surprised that 
so few levels of nesting already are a problem for the parser... I have 
attached the generated code with reduced nesting.

--
Added file: http://bugs.python.org/file47058/generated.zip

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31113] Stack overflow with large program

[Stefan Behnel]

Stefan Behnel added the comment:

Regarding the user side of the problem, you might(!) be able to work around the 
crash by merging nested if-conditions into and-expressions if they have no 
elif/else. That's also why the split into multiple files doesn't help, it's the 
depth of the nesting and the overall code complexity that strike here, not the 
total length of the program.

Side note: just for fun, I compiled your file with Cython. It took a few 
minutes and then generated a 1.1 GB C file :D - hadn't seen it do that before. 
That's 31MB xz compressed. I was sure it would crash my C compiler if I tried 
to compile that, but since processing time and renewable energy are cheap these 
days, I gave it a try. Now "gcc -O3" is still working on it after 7 hours, 
currently using some 8.5 GB of RAM. It's probably worth first recompiling gcc 
itself with proper C compiler flags next time...

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31120] [2.7] Python 64 bit _ssl compile fails due missing buildinf_amd64.h

[Zachary Ware]

Changes by Zachary Ware :


--
assignee: christian.heimes -> zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31120] [2.7] Python 64 bit _ssl compile fails due missing buildinf_amd64.h

[Hiren Vadalia]

Changes by Hiren Vadalia :


--
nosy: +zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31121] Unable to exit pdb when script becomes invalid

[Jason R. Coombs]

New submission from Jason R. Coombs:

Similar to #16180 and possible a duplicate of #14743, if the script being run 
under pdb becomes invalid such as through a chdir operation, pdb will get 
trapped in a loop, catching its own exception.

$ mkdir foo
$ cat > foo/script.py
import os
os.chdir('foo')
$ python -m pdb foo/script.py
> /Users/jaraco/Dropbox/code/privacymate/harvester/foo/script.py(1)()
-> import os
(Pdb) c
The program finished and will be restarted
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 
1667, in main
pdb._runscript(mainpyfile)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 
1545, in _runscript
with open(filename, "rb") as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'foo/script.py'
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py(1545)_runscript()
-> with open(filename, "rb") as fp:
(Pdb) q
Post mortem debugger finished. The foo/script.py will be restarted
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 
1667, in main
pdb._runscript(mainpyfile)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 
1545, in _runscript
with open(filename, "rb") as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'foo/script.py'
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py(1545)_runscript()
-> with open(filename, "rb") as fp:
(Pdb) q
Post mortem debugger finished. The foo/script.py will be restarted
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 
1667, in main
pdb._runscript(mainpyfile)
  File 
"/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py", line 
1545, in _runscript
with open(filename, "rb") as fp:
FileNotFoundError: [Errno 2] No such file or directory: 'foo/script.py'
Uncaught exception. Entering post mortem debugging
Running 'cont' or 'step' will restart the program
> /Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/pdb.py(1545)_runscript()
-> with open(filename, "rb") as fp:
(Pdb)

I suggest PDB should always pass exceptions when the stack trace terminates in 
the pdb module.

--
components: Library (Lib)
messages: 299748
nosy: jason.coombs
priority: normal
severity: normal
status: open
title: Unable to exit pdb when script becomes invalid
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31120] [2.7] Python 64 bit _ssl compile fails due missing buildinf_amd64.h

[Hiren Vadalia]

Changes by Hiren Vadalia :


--
keywords: +patch
Added file: http://bugs.python.org/file47057/openssl-1.0.2j.diff

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31106] os.posix_fallocate() generate exception with errno 0

[Марк Коренберг]

Марк Коренберг added the comment:

posix_fadvise() is also affected :(

Fixed in this patch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31106] os.posix_fallocate() generate exception with errno 0

[Марк Коренберг]

Марк Коренберг added the comment:

All checks passed.

Please merge.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31120] [2.7] Python 64 bit _ssl compile fails due missing buildinf_amd64.h

[Hiren Vadalia]

Hiren Vadalia added the comment:

Just another update I had chat with Zach for this on core-mentors...@python.org 
and he suggested to supply a patch which I will submit soon.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31120] [2.7] Python 64 bit _ssl compile fails due missing buildinf_amd64.h

[Hiren Vadalia]

Hiren Vadalia added the comment:

I did some more digging into this and I found something strange:
So while compiling Python, it downloads externals from 
http://svn.python.org/projects/external/
Above site has many version of openssl so I looked into all openssl directory 
and found that from openssl version 0.9.8g we have ‘buildinf_amd64.h’ file 
under http://svn.python.org/projects/external/openssl-0.9.8g/crypto/ till 
openssl version 1.0.2f and after that we don’t have ‘*_amd64.h’ file under 
openssl-*/crypro directory and that is the reason 64 bit build is failing when 
user tries to compile Python 2.7.13 using PC\VS9.0 build scripts + Visual 
Studio 2008.

Since all openssl source on http://svn.python.org/projects/external/ is 
pre-configured, don’t you guys think we should have *_amd64.h file under 
openssl-*/crypto directory?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31120] [2.7] Python 64 bit _ssl compile fails due missing buildinf_amd64.h

[Hiren Vadalia]

New submission from Hiren Vadalia:

I am trying to compile Python 2.7.13 
(https://github.com/python/cpython/tree/9c1426de7521299f70eb5483e7e25d1c2a73dbbd)
 in 64 bit but I am getting following error:
Build started: Project: _ssl, Configuration: Release|x64
Performing Pre-Build Event...
Found a working perl at 'c:\perl\bin\perl.exe'
Traceback (most recent call last):
  File "build_ssl.py", line 248, in 
main()
  File "build_ssl.py", line 231, in main
shutil.copy(r"crypto\buildinf_%s.h" % arch, r"crypto\buildinf.h")
  File "C:\projects\windows-work\cpython-2.7.13\lib\shutil.py", line 119, in 
copy
copyfile(src, dst)
  File "C:\projects\windows-work\cpython-2.7.13\lib\shutil.py", line 82, in 
copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'crypto\\buildinf_amd64.h'
Project : error PRJ0019: A tool returned an error code from "Performing 
Pre-Build Event..."
Build log was saved at 
"file://C:\projects\windows-work\cpython-2.7.13\PC\VS9.0\x64-temp-Release\_ssl\BuildLog.htm"
_ssl - 1 error(s), 0 warning(s)
Build started: Project: _hashlib, Configuration: Release|x64
Performing Pre-Build Event...
Found a working perl at 'c:\perl\bin\perl.exe'
Traceback (most recent call last):
  File "build_ssl.py", line 248, in 
main()
  File "build_ssl.py", line 231, in main
shutil.copy(r"crypto\buildinf_%s.h" % arch, r"crypto\buildinf.h")
  File "C:\projects\windows-work\cpython-2.7.13\lib\shutil.py", line 119, in 
copy
copyfile(src, dst)
  File "C:\projects\windows-work\cpython-2.7.13\lib\shutil.py", line 82, in 
copyfile
with open(src, 'rb') as fsrc:
IOError: [Errno 2] No such file or directory: 'crypto\\buildinf_amd64.h'
Project : error PRJ0019: A tool returned an error code from "Performing 
Pre-Build Event..."
Build log was saved at 
"file://C:\projects\windows-work\cpython-2.7.13\PC\VS9.0\x64-temp-Release\_hashlib\BuildLog.htm"
_hashlib - 1 error(s), 0 warning(s)
Build complete: 23 Projects succeeded, 2 Projects failed, 1 Projects skipped


Then I did some search here + buildbots and I found there is already bug for 
it: https://bugs.python.org/issue30368 which seems to be fixed for upcoming 
Python 2.7.14 release.
Also I found that for commit 9c1426de7521299f70eb5483e7e25d1c2a73dbbd AMD64 
buildbot has failed build 
(http://buildbot.python.org/all/builders/AMD64%20Windows7%20SP1%20VS9.0%202.7/builds/23/)

--
assignee: christian.heimes
components: Build, SSL
messages: 299743
nosy: Hiren Vadalia, christian.heimes
priority: normal
severity: normal
status: open
title: [2.7] Python 64 bit _ssl compile fails due missing buildinf_amd64.h
type: compile error
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31106] os.posix_fallocate() generate exception with errno 0

[Марк Коренберг]

Марк Коренберг added the comment:

https://github.com/python/cpython/pull/3000

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31106] os.posix_fallocate() generate exception with errno 0

[Марк Коренберг]

Changes by Марк Коренберг :


--
pull_requests: +3036

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31106] os.posix_fallocate() generate exception with errno 0

[Марк Коренберг]

Марк Коренберг added the comment:

Also, EINTR will not be caught too (!)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31106] os.posix_fallocate() generate exception with errno 0

[Марк Коренберг]

Марк Коренберг added the comment:

man posix_fallocate:

RETURN VALUE

posix_fallocate() returns zero on success, or an error number on failure.  Note 
that errno is not set.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23702] docs.python.org/3/howto/descriptor.html still refers to "unbound methods"

[Johannes Lade]

Johannes Lade added the comment:

And sorry for my lousy manners. Of course I appreciate all the hard work you 
do! It's just frustrating when you get confused by doc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23702] docs.python.org/3/howto/descriptor.html still refers to "unbound methods"

[Johannes Lade]

Johannes Lade added the comment:

This is still a problem. Can please somebody fix this? There are already enough 
confusing discussion full of wrong information about this topic, so it would be 
nice if the official documentation would get it right. Also there's multiple 
Issues for this. Can they be combined into one?
Just one example I found: on 
https://docs.python.org/3.5/howto/descriptor.html#functions-and-methods

Documentation:
>>> class D(object):
... def f(self, x):
... return x
...
>>> d = D()
>>> D.__dict__['f']  # Stored internally as a function

>>> D.f  # Get from a class becomes an unbound method

>>> d.f  # Get from an instance becomes a bound method
>

ipython3.5.3
In [1]: class D(object):
   ...: ... def f(self, x):
   ...: ... return x
   ...: ...

In [2]: d = D()

In [3]: D.__dict__['f']  # Stored internally as a function
Out[3]: 

In [4]: D.f  # Get from a class becomes an unbound method
Out[4]: 

In [5]: d.f  # Get from an instance becomes a bound method
Out[5]: >

--
nosy: +Johannes Lade

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31107] copyreg does not properly mangle __slots__ names

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset c4c9866064f03646c686d7e08b00aeb203c35c19 by Serhiy Storchaka 
(Shane Harvey) in branch 'master':
bpo-31107: Fix copyreg mangled slot names calculation. (#2989)
https://github.com/python/cpython/commit/c4c9866064f03646c686d7e08b00aeb203c35c19


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31119] Signal tripped flags need memory barriers

[Nathaniel Smith]

New submission from Nathaniel Smith:

Sometimes, CPython's signal handler (signalmodule.c:signal_handler) runs in a 
different thread from the main thread. On Unix this is rare but it does happen 
(esp. for SIGCHLD), and on Windows it happens 100% of the time. It turns out 
that there is a subtle race condition that means such signals can be lost.

To notify the main thread that a signal has been received, and which signal it 
is, CPython's signal handler sets two global variables:

  Handlers[sig_num].tripped = 1;
  is_tripped = 1

And then PyErr_CheckSignals does:

  if (is_tripped) {
  is_tripped = 0;
  for (i = 1; i < NSIG; i++) {
  if (Handlers[i].is_tripped) {
  ... run signal handler ...
  }
  }
  }

As some comments in the source note, this logic depends strongly on the 
assumption that when the signal handler sets the two flags, they are 
immediately visible to PyErr_CheckSignals, and that the two assignments happen 
in the given order. For example, suppose that is_tripped were set before 
Handlers[i].is_tripped. Then we could have the following sequence:

1. Thread A sets is_tripped = 1
2. Thread B runs PyErr_CheckSignals
3. Thread B notices is_tripped == 1, and sets it to 0
4. Thread B scans through all the Handlers, sees that none are set, and carries 
on
5. Thread A sets Handlers[i].is_tripped = 1

In this case the signal is lost until another signal arrives, which may not 
happen until an arbitrary amount of time has passed.

Also, the fix for bpo-30038 (making sure that we set the flags before writing 
to the wakeup fd) assumes that setting the flags before writing to the wakeup 
fd means that, well, the flags will be written before the fd.

However, when you have code running in separate threads, none of this is 
actually guaranteed. In general, the compiler is free to re-order writes, and 
more perniciously, the hardware memory hierarchy is also free to arbitrarily 
delay when writes to RAM made by one CPU become visible to another CPU, which 
can also create arbitrary reorderings.

In an attempt to avoid these issues, signalmodule.c declares the signal flags 
as 'volatile sig_atomic_t'. However, the best one can hope for from this is 
that it'll stop the *compiler* from reordering writes. It does nothing to stop 
the *hardware* from reordering writes. Using volatile like this might be 
sufficient if the signal handler runs in the main thread, but not if it runs in 
another thread.

The correct way to handle this is to use the primitives in pyatomic.h to 
get/set the tripped flags.



I noticed this because of the same test case that was failing due to 
bpo-30038... this thing has been frustrating me for months because it's 
intermittent and never seems to happen on my local VM. But specifically what 
I'm seeing at this point is:

Thread A (the thread where the signal handler runs):
A1. takes a lock.
A2. with the lock held, calls the C function raise(SIGINT), which directly 
calls the CPython C-level signal handler. (Verified by consulting the Windows C 
runtime source.)
A3. writes to the wakeup fd & sets both tripped flags, in whatever order
A4. releases the lock

Simultaneously, in thread B (the main thread):
B1. observes that the wakeup fd has been written to
B2. takes the lock
B3. releases the lock
B4. calls repr(), which unconditionally calls PyObject_Repr, which 
unconditionally calls PyErr_CheckSignals

I expect the call in B4 to run the Python-level signal handler, but this does 
not happen. Instead it runs some time later, causing an unexpected 
KeyboardInterrupt.

My reasoning is: the lock guarantees that no matter what the internal details 
of the C-level signal handler actually are, they have to all be complete before 
my code checks for signals. In excruciating detail:

- the write to the wakeup fd must happen-after the lock is acquired (A1)
- therefore, step B1 happens-after step A1
- B2 happens-after B1, so B2 happens-after A1.
- B2 takes the same lock as A1, so if B2 happens-after A1, it must also 
happen-after A4, when the lock is released.
- step B4 happens-after B2, therefore it happens-after A4 and A3. Therefore the 
tripped flags should be visible and the Python-level signal handler should run.

Yet this does not happen, so *something* really weird is going on. The 
hypothesis that it's the lack of memory barriers both explains how this could 
happen -- specifically I think thread B may be running PyErr_CheckSignals as 
part of the wakeup-fd reading logic, and it's clearing is_tripped before 
Handlers[i].tripped becomes visible, like in my code above. And it also 
explains why it happens in like 1/50 runs in Appveyor, but never on my local 
Windows 10 VM that only has 1 virtual CPU.

But regardless of whether this is the cause of my particular weird bug, it 
clearly *is* a bug and should be fixed.

--
messages: 299736
nosy: haypo, njs
priority: normal
severity: normal
status: open
title: Signal tripped flags need 

[issue15988] Inconsistency in overflow error messages of integer argument

[Oren Milman]

Oren Milman added the comment:

How do we proceed?
should I update (if needed) each of the patches I uploaded in March
to eliminate commit conflicts? or can someone review them as they are
now?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30717] Add unicode grapheme cluster break algorithm

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The well known library for Unicode support in C++ and Java is ICU 
(International Components for Unicode). There is a Python wrapper [1].

This is a large complex library that covers many aspects of Unicode support. 
It's interface looks rather Javaic than Pythonic. Some parts of it already are 
covered by other parts of the stdlib (the str class, the codecs and locale 
modules).

[1] https://pypi.python.org/pypi/PyICU/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31118] Make super() work with staticmethod by using __class__ for both arguments to super()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31118] Make super() work with staticmethod by using __class__ for both arguments to super()

[Ashwini Chaudhary]

Changes by Ashwini Chaudhary :


--
components: +Interpreter Core

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31118] Make super() work with staticmethod by using __class__ for both arguments to super()

[Ashwini Chaudhary]

New submission from Ashwini Chaudhary:

>From this question I noticed that super() doesn't work with static methods 
>with no arguments: https://stackoverflow.com/q/45498675/846892

My question is what's the issue with using __class__ for both arguments to 
super() in case of static method? Or was it just an oversight.

--
messages: 299733
nosy: ashwch
priority: normal
severity: normal
status: open
title: Make super() work with staticmethod by using __class__ for both 
arguments to super()
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31113] Stack overflow with large program

[Stefan Behnel]

Stefan Behnel added the comment:

I've looked at the file and it contains a huge amount of deeply nested 
if-statements. Given that parsers and compilers are typically recursive, I can 
well imagine that this is a problem, and my guess is that it's most likely just 
the different C level stack sizes, stack configurations and C compiler 
dependent stack allocation per function call that makes a difference for the 
platforms you tested. It would probably also crash on Linux, just for an even 
larger program.

I'm actually not in the position to decide if something should be done about 
this, so I'm asking in Antoine for a comment.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30717] Add unicode grapheme cluster break algorithm

[Stefan Behnel]

Stefan Behnel added the comment:

Wouldn't this be a typical case where we'd expect a module to evolve and gain 
usage on PyPI first, before adding it to the stdlib?

Searching for "grapheme" in PyPI gives some results for me. Even if they do not 
cover what this ticket asks for, they might give inspiration for a suitable API 
design. And I'm probably missing other related packages by lack of a better 
search term.

https://pypi.python.org/pypi?%3Aaction=search=grapheme

--
nosy: +scoder

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com