[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Guido van Rossum]

Guido van Rossum  added the comment:

I'm glad you've rediscovered pgen2!

I'm in favor of unifying the tokenizers and of updating and moving pgen2 
(though I don't have time to do the work).

I'm not sure if it's technically possible to give tokenize.py the ability to 
tokenize Python 2.7 and up without some version-selection flag -- have you 
researched this part yet?

Also I think you may have to make a distinction between the parser generator 
and its data structures, and between the generated parser for Python vs. the 
parser for other LL(1) grammars one might feed into it.

And I don't think you're proposing to replace Parser/pgen.c with Lib/pgen/, 
right? Nor to replace the CST actually used by CPython's parser with the data 
structures used by pgen2's driver. So the relationship between the CST you 
propose to document and CPython internals wouldn't be quite the same as that 
between the AST used by CPython and the ast module (since those *do* actually 
use the same code).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33315] Allow queue.Queue to be used in type annotations

[Guido van Rossum]

Guido van Rossum  added the comment:

[Raymond]
> I don't think it makes sense to let typing considerations
> start to spill into the rest of the standard library.
> Either add support to typing or not.

We can't possibly add every stdlib class that's generic to typing.py -- it just 
doesn't scale. It also doesn't make sense to import everything from typing.

Arguably there are already some things in typing that don't belong there -- I'm 
thinking in particular of the 'io' and 're' classes. Those were a compromise to 
get the (in my opinion) most popular and fundamental classes quickly typed. But 
we definitely shouldn't continue down this path. Type annotations is here to 
stay (there's an increasing number of PEPs about it) and in the longer term we 
should support it directly in the appropriate stdlib modules. We're not ready 
to do that but we should start thinking about it.

Until we've gotten a firmer decision I don't think we should do one-offs, so I 
don't think Queue should be made generic right now, but we definitely shouldn't 
add it to typing. Users who are using typing should continue to consider 
themselves lucky to be early adopters and will occasionally pay a small price, 
such as putting the type in quotes or (for users on 3.7+) using "from 
__future__ import annotations".

PS. Raymond, please avoid pejoratives like "spill" in the future when talking 
about types.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25478] Consider adding a normalize() method to collections.Counter()

[Raymond Hettinger]

Change by Raymond Hettinger :


--
keywords: +patch
pull_requests: +6275
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33315] Allow queue.Queue to be used in type annotations

[Guido van Rossum]

Guido van Rossum  added the comment:

Disagree strongly. More later.

On Sun, Apr 22, 2018, 18:41 Raymond Hettinger 
wrote:

>
> Raymond Hettinger  added the comment:
>
> > Another solution may be to add a simple `__class_getitem__` to `Queue`
> > that will just return `cls`
>
> I don't think it makes sense to let typing considerations start to spill
> into the rest of the standard library.  Either add support to typing or not.
>
> --
> nosy: +rhettinger
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33315] Allow queue.Queue to be used in type annotations

[Raymond Hettinger]

Raymond Hettinger  added the comment:

> Another solution may be to add a simple `__class_getitem__` to `Queue` 
> that will just return `cls`

I don't think it makes sense to let typing considerations start to spill into 
the rest of the standard library.  Either add support to typing or not.

--
nosy: +rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Łukasz Langa]

Change by Łukasz Langa :


--
pull_requests:  -6270

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Łukasz Langa]

Change by Łukasz Langa :


--
keywords:  -patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Łukasz Langa]

Łukasz Langa  added the comment:

See BPO-8 for an implementation of Step 1.

--
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16152] Trailing whitespace makes tokenize.generate_tokens pathological

[Ned Batchelder]

Ned Batchelder  added the comment:

PR 6273 is mentioned, but I think 6573 is the correct number.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33321] Add a Linux clang ubsan undefined behavior sanitizer buildbot

[kmoultr...@gmail.com]

Change by kmoultr...@gmail.com :


--
versions: +Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33338] [lib2to3] Synchronize token.py and tokenize.py with the standard library

[Łukasz Langa]

Change by Łukasz Langa :


--
pull_requests: +6274

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue2495] tokenize doesn't handle __future__.unicode_literals correctly

[Łukasz Langa]

Change by Łukasz Langa :


--
pull_requests: +6271

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Łukasz Langa]

Change by Łukasz Langa :


--
keywords: +patch
pull_requests: +6270
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue14990] detect_encoding should fail with SyntaxError on invalid encoding

[Łukasz Langa]

Change by Łukasz Langa :


--
pull_requests: +6272

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33338] [lib2to3] Synchronize token.py and tokenize.py with the standard library

[Łukasz Langa]

Change by Łukasz Langa :


--
keywords: +patch
pull_requests: +6269
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16152] Trailing whitespace makes tokenize.generate_tokens pathological

[Łukasz Langa]

Change by Łukasz Langa :


--
pull_requests: +6273

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33338] [lib2to3] Synchronize token.py and tokenize.py with the standard library

[Łukasz Langa]

New submission from Łukasz Langa :

lib2to3's token.py and tokenize.py were initially copies of the respective
files from the standard library.  They were copied to allow Python 3 to read
Python 2's grammar.

Since 2006, lib2to3 grew to be widely used as a Concrete Syntax Tree, also for
parsing Python 3 code.  Additions to support Python 3 grammar were added but
sadly, the main token.py and tokenize.py diverged.

This change brings them back together, minimizing the differences to the bare
minimum that is in fact required by lib2to3.  Before this change, almost every
line in lib2to3/pgen2/tokenize.py was different from tokenize.py.  After this
change, the diff between the two files is only 175 lines long and is entirely
filled with relevant Python 2 compatibility bits.

Merging the implementations, there's numerous fixes to the lib2to3 tokenizer:

+ docstrings made as similar as possible
+ ported `TokenInfo`
+ ported `tokenize.tokenize()` and `tokenize.open()`
+ removed Python 2-only implementation cruft
+ fixes Unicode identifier handling
+ fixes string prefix handling
+ fixes Ellipsis handling
+ Untokenizer backported bugfixes:
- 5e6db313686c200da425a54d2e0c95fa40107b1d
- 9dc3a36c849c15c227a8af218cfb215abe7b3c48
- 5b8d2c3af76e704926cf5915ad0e6af59a232e61
- e411b6629fb5f7bc01bec89df75737875ce6d8f5
- BPO-2495
+ tokenizer doesn't crash on missing newline at the end of the
stream (added \Z (end of string) to PseudoExtras) - BPO-16152
+ `find_cookie` includes file name in error messages, if available
+ `find_cookie` raises SyntaxError on invalid encodings: BPO-14990

Improvements to lib2to3/pgen2/token.py:

+ taken from the current Lib/token.py
+ tokens renumbered to match Lib/token.py
+ `__all__` properly defined
+ ASYNC, AWAIT and BACKQUOTE exist under different numbers (100 + old number)
+ ELLIPSIS added
+ ENCODING added

--
components: 2to3 (2.x to 3.x conversion tool), Library (Lib)
messages: 315639
nosy: lukasz.langa
priority: normal
severity: normal
status: open
title: [lib2to3] Synchronize token.py and tokenize.py with the standard library
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33338] [lib2to3] Synchronize token.py and tokenize.py with the standard library

[Łukasz Langa]

Łukasz Langa  added the comment:

### Diff between files

The unified diff between tokenize implementations is here:
https://gist.github.com/ambv/679018041d85dd1a7497e6d89c45fb86

It clocks at 275 lines but that's because it gives context. The actual diff is
175 lines long.

To make it that small, I needed to move some insignificant bits in
Lib/tokenize.py.  This is what the other PR on this issue is about.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Łukasz Langa]

New submission from Łukasz Langa :

Python includes a set of batteries that enable parsing of Python code.  This
includes its own AST (provided in the standard library under the `ast` module),
as well as a pure Python tokenizer (provided in the standard library under
`tokenize` and `token`).  It also provides an undocumented CST under lib2to3,
which contains its own outdated and patched copies of `tokenize` and `token`.

This situation causes the following issues for users of Python:
- the built-in AST does not preserve comments or whitespace;
- the built-in AST increasingly modifies the tree before presenting it to user
  code (constant folding moved to the AST in Python 3.7);
- the built-in tokenize.py can only be used to parse Python 3.7+ code;
- the version in lib2to3 is partially customized and partially outdated,
  leaving bits of new grammar not supported; new bits of grammar very often get
  overlooked in lib2to3.
- lib2to3 is not documented.

So if users want to write tools that manipulate Python code, the standard
library doesn't provide them with great options.

I suggest the following plan:

1. Bring Lib/lib2to3/pgen2/tokenize.py to the same state as Lib/tokenize.py
   (leaving the bits that allow for parsing of Python 3.6 and older files).

2. Merge the two tokenizers in Python 3.8 so that Lib/tokenize.py now
   officially supports tokenizing Python 2.7 - 3.7 code.

3. Update Lib/lib2to3/pgen2 and move it under Lib/pgen.  Document it as the
   built-in CST provided by Python for use in applications which require code
   modification.  Make it still officially support parsing of Python 2.7 - 3.7
   code.

All three changes are made in a backwards-compatible fashion, existing code
should NOT break.  That being said, the parser under Lib/pgen might grow some
new behavior compared to the compatibility mode for lib2to3, I specifically
seek to improve handling of comments and error recovery.

--
components: Library (Lib)
messages: 315638
nosy: benjamin.peterson, gregory.p.smith, gvanrossum, lukasz.langa, 
serhiy.storchaka
priority: normal
severity: normal
status: open
title: Provide a supported Concrete Syntax Tree implementation in the standard 
library
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue991266] Cookie.py does not correctly quote Morsels

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 9fc998d761591f2741d8e94f5b3009c56ae83882 by Berker Peksag (Miss 
Islington (bot)) in branch '3.7':
bpo-991266: Fix quoting of Comment attribute of SimpleCookie (GH-6555)
https://github.com/python/cpython/commit/9fc998d761591f2741d8e94f5b3009c56ae83882


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue991266] Cookie.py does not correctly quote Morsels

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 8a6f4b4bba950fb8eead1b176c58202d773f2f70 by Berker Peksag (Miss 
Islington (bot)) in branch '3.6':
bpo-991266: Fix quoting of Comment attribute of SimpleCookie (GH-6555)
https://github.com/python/cpython/commit/8a6f4b4bba950fb8eead1b176c58202d773f2f70


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33277] Deprecate __loader__, __package__, __file__, and __cached__ on modules

[Brett Cannon]

Brett Cannon  added the comment:

@Nick: We've had this discussion before about specs being a "receipt" of import 
versus not and the last time we agreed on not. :) That's why we raise a warning 
if __package__ doesn't match __spec__.parent: 
https://github.com/python/cpython/blob/d5a2377c3d70e4143bcbee4a765b3434e21f683a/Lib/importlib/_bootstrap.py#L1056-L1059

It's documented through 
https://docs.python.org/3/reference/import.html#__package__ by saying 
__package__ has to match __spec__.parent. We didn't make it stronger since we 
can't get rid of any of these attributes until Python 2.7 compatibility is no 
longer a concern. And I believe __package__/__spec__.parent is the only 
attribute whose data is in any way directly relied on in importlib which is 
being proposed for removal (e.g. __file__, __cached__, and __loader__ are just 
recorded).

@Raymond: Thanks for the teaching info. I actually opened this after running 
dir() on a module and having there be just as many module-related attributes as 
defined attributes.

@Antoine: we can keep __file__ if necessary (I was on the fence about keeping 
it to begin with). But as Barry pointed out, bugs have occurred because we now 
have two places for most of these details and people don't always know which 
one to update.

@Barry: the tricky bit is what to do if someone sets their own __getattr__() on 
a module? E.g. I just published modutil which provides a __getattr__ for lazy 
importing, so users of this would need a way to not have either function get 
stomped on by the other.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue991266] Cookie.py does not correctly quote Morsels

[miss-islington]

Change by miss-islington :


--
pull_requests: +6268

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue991266] Cookie.py does not correctly quote Morsels

[miss-islington]

Change by miss-islington :


--
pull_requests: +6267

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue991266] Cookie.py does not correctly quote Morsels

[Berker Peksag]

Berker Peksag  added the comment:


New changeset d5a2377c3d70e4143bcbee4a765b3434e21f683a by Berker Peksag in 
branch 'master':
bpo-991266: Fix quoting of Comment attribute of SimpleCookie (GH-6555)
https://github.com/python/cpython/commit/d5a2377c3d70e4143bcbee4a765b3434e21f683a


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33325] Optimize sequences of constants in the compiler

[Mark Shannon]

Mark Shannon  added the comment:

There seems to be an implicit assumption here that fewer bytecodes is better. 
But that isn't always the case.

Do you have evidence that the sequence 
0 LOAD_CONST   5 ((('a', 'b', 'c'), 3, 2, 1))
2 UNPACK_SEQUENCE  4
is actually faster than
0 LOAD_CONST   0 (1)
2 LOAD_CONST   1 (2)
4 LOAD_CONST   2 (3)
6 LOAD_CONST   3 (('a', 'b', 'c'))
?

The second sequence has more bytecodes, but the first has to create a new 
object.

I think you ought to be careful using the word "optimize" unless the output is 
incontrovertibly superior.

--
nosy: +Mark.Shannon

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue18153] python imaplib - error 'unexpected repsonse'

[Matej Cepl]

Change by Matej Cepl :


--
nosy: +mcepl

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33336] [imaplib] MOVE is a legal command

[Matej Cepl]

Change by Matej Cepl :


--
keywords: +patch
pull_requests: +6266
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33336] [imaplib] MOVE is a legal command

[Matej Cepl]

New submission from Matej Cepl :

When running

box = IMAP4_SSL(host)
box.login(user, pass)
msgs = somehowget_uids_of_messsages_to_move()
box.uid('MOVE', msgs, 'target')

I get an error "Unknown IMAP4 UID command: MOVE". The problem is that imaplib 
contains a list Commands of the permitted IMAP commands, and MOVE is not 
included, even though it is perfectly legal (and quite widely supported) 
command according to RFC-6851.

--
components: Library (Lib)
messages: 315632
nosy: mcepl
priority: normal
severity: normal
status: open
title: [imaplib] MOVE is a legal command
versions: Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33333] ConfigParser.items returns items present in `DEFAULTSECT` when section argument given.

[Chris Bradbury]

Change by Chris Bradbury :


--
assignee:  -> docs@python
components: +Documentation -Library (Lib)
nosy: +docs@python
resolution: not a bug -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31457] LoggerAdapter objects cannot be nested

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 26c289dd76ed07dbf37494bfa483c525f6828bd9 by Łukasz Langa (Miss 
Islington (bot)) in branch '3.6':
bpo-31457: Don't omit inner ``process()`` calls with nested LogAdapters 
(GH-4044) (#6568)
https://github.com/python/cpython/commit/26c289dd76ed07dbf37494bfa483c525f6828bd9


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33333] ConfigParser.items returns items present in `DEFAULTSECT` when section argument given.

[Chris Bradbury]

Change by Chris Bradbury :


--
resolution:  -> not a bug

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31457] LoggerAdapter objects cannot be nested

[miss-islington]

Change by miss-islington :


--
pull_requests: +6265

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33333] ConfigParser.items returns items present in `DEFAULTSECT` when section argument given.

[Chris Bradbury]

Chris Bradbury  added the comment:

I'll make the documentation changes instead, not overly surprised to hear this 
is intentional.

--
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33333] ConfigParser.items returns items present in `DEFAULTSECT` when section argument given.

[Łukasz Langa]

Łukasz Langa  added the comment:

See line 379 in configparser.py. BasicInterpolation is also assumed to work for 
values from the default section.  This is why the default section exists in the 
first place.

More importantly, as you're pointing out, even though this is an omission of 
the documentation, existing code in the wild probably assumes that `items()` 
include pairs from the default section.  This is how ConfigParser worked in 
Python 2.  If you don't want this behavior, use cfgparser[section].items() 
instead which does what you want.

If you think this is helpful, go ahead and extend the documentation of 
ConfigParser.items() to point out this weirdness.  But we won't be going 
forward with PR 6567.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33251] ConfigParser.items returns items present in vars

[Łukasz Langa]

Łukasz Langa  added the comment:

> What *actually* was changed in 3.2?

In terms of this issue, nothing.  That piece of documentation was put without 
proper testing, based purely on a comment that somebody put on issue12036.  My 
mistake.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33251] ConfigParser.items returns items present in vars

[Chris Bradbury]

Change by Chris Bradbury :


--
pull_requests: +6264

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33333] ConfigParser.items returns items present in `DEFAULTSECT` when section argument given.

[Chris Bradbury]

Change by Chris Bradbury :


--
pull_requests: +6263

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33015] Fix function cast warning in thread_pthread.h

[Ned Deily]

Change by Ned Deily :


--
nosy: +benjamin.peterson

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33335] turtle.onkey doesn't pass key information when key is None

[J E]

New submission from J E :

I use the turtle module to teach Python to university students. The 
turtle.onkey function allows a program to register a callback function to 
handle a keyboard when the turtle window has a focus. For example, this call 
will result in myfun() getting called when the "s" key is pressed:

def myfun():
  print("You pressed s")

turtle.onkey(myfun, "s")
turtle.listen()

But what if I want to handle all keys? I could create a bunch of functions, and 
register a separate callback for each possible; that's not a great solution. A 
better solution is to let turtle call a single function when any key is 
pressed. Fortunately, this is possible by passing None in for the key:

def myfun():
  print("You pressed some key")

turtle.onkey(myfun, None)
turtle.listen()

However: there is no way for myfun to determine which key was pressed. It is 
called with no parameters, and has no way to recover information about which 
key was pressed, so this approach is useless. The only way to handle all 
keyboard events it to call onkey, with a separate handler, for every key you 
potentially want to handle.

>From the source [1] this looks like it was intended that when key is None, 
>turtle would pass a Tkinter event to the handler, so that it could recover 
>information about the event, but this doesn't happen.

[1] https://hg.python.org/cpython/file/3.6/Lib/turtle.py#l700

I would argue that correct behavior is not to expose Tkinter event to users of 
the higher-level turtle library, but rather to pass the name of the key as a 
string parameter, so you could write this:

def myfun(key):
  print("You pressed "+key)

turtle.onkey(myfun, None)
turtle.listen()

This is an easy fix and I'd be happy to submit a patch.

--
components: Tkinter
messages: 315627
nosy: je1234
priority: normal
severity: normal
status: open
title: turtle.onkey doesn't pass key information when key is None
type: behavior
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

One option would be to create a list of possible defined signals and check if 
the signal is on the list.  For realtime signals, it just a matter to check if 
SIGRTMIN <= signal <= SIGRTMAX.

The glibc defined signals can be checked at tst-signal.c [1] or from main 
signal(7).  It should cover usual ISO C, POSIX, and some linux arch-specific 
signals, but you will still need to check if other OS defined extra signals 
uses elsewhere (another option would to add this check only for Linux/glibc).

[1] 
https://sourceware.org/git/?p=glibc.git;a=blob;f=signal/tst-sigset.c;h=a2b764de5ad66ee960c94ec18df75a07fce4b9a6;hb=HEAD

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28602] `tzinfo.fromutc()` fails when used for a fold-aware tzinfo implementation

[Paul Ganssle]

Change by Paul Ganssle :


--
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33333] ConfigParser.items returns items present in `DEFAULTSECT` when section argument given.

[Chris Bradbury]

Chris Bradbury  added the comment:

The fix for the actual issue is quite trivial (and should fix bpo-33251 at the 
same time). However `ExtendedInterpolation::_interpolate_some` relies on the 
options from the default section always being returned.

Not only does this make the fix more complicated, it also makes me worry that 
aligning behaviour to match the documentation (which I personally feel makes 
more sense) will breake backwards compatibility.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3405] Add support for the new data option supported by event generate (Tk 8.5)

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Fill free to create a pull request. It may need tests and documentation though.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33280] Update link to Tcl/Tk 8.6 man pages in tkinter.rst

[Andrés Delfino]

Andrés Delfino  added the comment:

I have updated the PR:

* Now we mention that python -m tkinter shows the installed Tcl/Tk version so 
users can read the right man pages.

* Only mention the link to documentation page (sadly, there doesn't seem to 
exist an "Archived man pages", and mentioning 8.5/8.4 man pages doesn't seem 
right).

* I have merged the two "See also" sections of the page, separating 
Tkinter-specific documentation from Tcl/Tk general documentation.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33334] Add support of NOP and EXTENDED_ARG in stack_effect()

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +6262
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33334] Add support of NOP and EXTENDED_ARG in stack_effect()

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

Currently dis.stack_effect() doesn't support opcodes NOP and EXTENDED_ARG. NOP 
is never emitted by the standard compiler (it is temporary added in the 
peephole optimizer, but later it is removed). EXTENDED_ARG is a special case, 
it is considered as a part of long instructions.

dis.stack_effect() itself is not used in the dis module. It can be used in the 
third-party code, and the third-party code can produce bytecode with 
non-standard use of NOP and EXTENDED_ARG. Supporting them in dis.stack_effect() 
can avoid the need of special casing them in the third-party code.

I don't know whether this is a bug fix or a new feature.

--
components: Library (Lib)
messages: 315622
nosy: larry, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Add support of NOP and EXTENDED_ARG in stack_effect()
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue3405] Add support for the new data option supported by event generate (Tk 8.5)

[Matthias Kievernagel]

Matthias Kievernagel  added the comment:

I have a patched Python 3.5.3 running mostly following
the comments by O.C. If no one else is active on this
I can try to prepare something for submission.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33277] Deprecate __loader__, __package__, __file__, and __cached__ on modules

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Not that I've thought this through in much detail, but what if we start out by 
proxying those module attributes to the spec via PEP 562 hooks?  We can do that 
in 3.8, issue (silent?) deprecation warnings, and then do a full deprecation 
process for those module attributes?

It's not a terrible thing to keep those attributes as long as they match what's 
in the spec.  It's a problem when they're out of sync (see bpo-32305 and 
bpo-32303).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33331] Clean modules in the reversed order

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33289] tkinter askcolor returning floats for r, g, b values instead of ints

[Cheryl Sabella]

Cheryl Sabella  added the comment:

Adding to what Serhiy said about the askcolor() return value is an issue that 
also affects input:


>>> cc.askcolor('red')
((255, 0, 0), '#ff')
>>> cc.askcolor((255, 0, 0))
((255, 0, 0), '#ff')
>>> cc.askcolor((65535, 0, 0))
Traceback (most recent call last):
  File "", line 1, in 
  File "/home/cheryl/cpython/Lib/tkinter/colorchooser.py", line 75, in askcolor
return Chooser(**options).show()
  File "/home/cheryl/cpython/Lib/tkinter/commondialog.py", line 43, in show
s = w.tk.call(self.command, *w._options(self.options))
_tkinter.TclError: invalid color name "#"


Changing _fixoptions to format initialcolor using "#%04x%04x%04x" % color.

allows for 16-bit hex values, but it's not the same as 8-bit.

>>> cc.askcolor('red')
((255, 0, 0), '#ff')
>>> cc.askcolor((255, 0, 0))
((0, 0, 0), '#00')
>>> cc.askcolor((65535, 0, 0))
((255, 0, 0), '#ff')

(255, 0, 0) in the second call is black and not red.

--
nosy: +csabella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
assignee:  -> pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> Why not export and use the canonical way of 
> sigemptyset/sigfillset/sigaddset/sigdelset/sigismember instead of pushing for 
> more potential non-conformant code? 

I agree this is the proper fix and that's what I plan to do in Python 3.8.  For 
Python 3.7 and earlier, though, we cannot add new features anymore, which is 
why I'm leaning towards a variant of the patch I showed above (which also 
minimizes the risk for regressions by introducing and using a new API).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

Also keep in mind that POSIX [1] specifies that sigaddset *may* fail with 
EINVAL for not support signals, so a conforming implementation may not fail on 
sigaddset and still remove the internal signal in a sigprocmask (uclibc for 
instance).

[1] http://pubs.opengroup.org/onlinepubs/9699919799/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

Why not export and use the canonical way of 
sigemptyset/sigfillset/sigaddset/sigdelset/sigismember instead of pushing for 
more potential non-conformant code? For glibc sigfillset will correctly fill 
all the signal set structure while removing the internal used signals. This is 
par what POSIX specifies [1] where it states that: 

'either sigemptyset() or sigfillset() must be called prior to any other use of 
the signal set'

And more importantly:

'For example, blocking or ignoring an implementation-defined signal may have 
undesirable side-effects, whereas the default action for that signal is 
harmless. In such a case, it would be preferable for such a signal to be 
excluded from the signal set returned by sigfillset().'

Also keep that since is an implementation detail, different libcs can use 
different internal signals. UCLIBC, for instance, uses the same 2 signals as 
GLIBC, however MUSL allocates signal 32, 33, and 34 for internal usage (and 
excludes in sigfillset and warns with EINVAL on sigaddset). 

[1] http://pubs.opengroup.org/onlinepubs/9699919799/

--
nosy: +azanella

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

This indeed makes the tests pass again:

# python3.7 -m test.regrtest test_multiprocessing_fork 
test_multiprocessing_forkserver test_multiprocessing_spawn
Run tests sequentially
0:00:00 load avg: 1.02 [1/3] test_multiprocessing_fork
/usr/lib64/python3.7/multiprocessing/semaphore_tracker.py:55: UserWarning: 
semaphore_tracker: process died unexpectedly, relaunching.  Some semaphores 
might leak.
  warnings.warn('semaphore_tracker: process died unexpectedly, '
0:01:05 load avg: 1.00 [2/3] test_multiprocessing_forkserver -- 
test_multiprocessing_fork passed in 1 min 6 sec
0:02:16 load avg: 1.69 [3/3] test_multiprocessing_spawn -- 
test_multiprocessing_forkserver passed in 1 min 11 sec
test_multiprocessing_spawn passed in 1 min 23 sec
All 3 tests OK.

Total duration: 3 min 40 sec
Tests result: SUCCESS

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Miro, can you give the following patch a try?

diff --git a/Modules/signalmodule.c b/Modules/signalmodule.c
index 7916160..ca76a20 100644
--- a/Modules/signalmodule.c
+++ b/Modules/signalmodule.c
@@ -819,7 +819,6 @@ iterable_to_sigset(PyObject *iterable, sigset_t *mask)
 int result = -1;
 PyObject *iterator, *item;
 long signum;
-int err;
 
 sigemptyset(mask);
 
@@ -841,11 +840,10 @@ iterable_to_sigset(PyObject *iterable, sigset_t *mask)
 Py_DECREF(item);
 if (signum == -1 && PyErr_Occurred())
 goto error;
-if (0 < signum && signum < NSIG)
-err = sigaddset(mask, (int)signum);
-else
-err = 1;
-if (err) {
+if (0 < signum && signum < NSIG) {
+(void) sigaddset(mask, (int)signum);
+}
+else {
 PyErr_Format(PyExc_ValueError,
  "signal number %ld out of range", signum);
 goto error;

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

> We could do that, but I'd rather something not multiprocessing-specific

... and also not Linux-specific.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

We could do that, but I'd rather something not multiprocessing-specific.  I 
think we have to ignore the return value of sigaddset() until issue2 allows 
us to be smarter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

Should the fix be to exclude 32 and 33 from 
multiprocessing.resource_sharer:_serve?

if hasattr(signal, 'pthread_sigmask'):
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Setting as release blocker as it impacts functionality of the multiprocessing 
module.

--
nosy: +ned.deily
priority: critical -> release blocker

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

Python 2 testsuite runs fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33333] ConfigParser.items returns items present in `DEFAULTSECT` when section argument given.

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
components: +Library (Lib)
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33333] ConfigParser.items returns items present in `DEFAULTSECT` when section argument given.

[Chris Bradbury]

Change by Chris Bradbury :


--
keywords: +patch
pull_requests: +6261
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33333] ConfigParser.items returns items present in `DEFAULTSECT` when section argument given.

[Chris Bradbury]

New submission from Chris Bradbury :

According to `ConfigParser.items` docs:

> When section is not given, return a list of section_name, section_proxy 
> pairs, including DEFAULTSECT.

> Otherwise, return a list of name, value pairs for the options in the given 
> section. Optional arguments have the same meaning as for the get() method.

https://docs.python.org/3/library/configparser.html#configparser.ConfigParser.items

However due to `d = self._defaults.copy()` on line 843 of configparser.py the 
default section is always included, even when a section argument is specified.

--
messages: 315608
nosy: chrBrd
priority: normal
severity: normal
status: open
title: ConfigParser.items returns items present in `DEFAULTSECT` when section 
argument given.
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue10117] Tools/scripts/reindent.py fails on non-UTF-8 encodings

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I concur with Éric. Standard input and output are text streams in Python 3. The 
user can control their encoding by setting locale or PYTHONIOENCODING.

I think this issue can be closed now unless somebody want to backport the fix 
to 2.7.

--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33332] Expose valid signal set (sigfillset())

[Antoine Pitrou]

New submission from Antoine Pitrou :

See issue33329: it would be nice to expose a signal module function to get the 
set of user-actionable signals.  Hopefully sigfillset() does that.

--
messages: 315606
nosy: pitrou
priority: normal
severity: normal
stage: needs patch
status: open
title: Expose valid signal set (sigfillset())
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Running the whole test suite sounds fine.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25205] setattr accepts invalid identifiers

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24024] str.__doc__ needs an update

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> rejected
stage: needs patch -> resolved
status: pending -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

I cannot find a corresponding test in 2.7. Running the entire build (incl. 
tests) to see what happens.

Is there a command or a couple I could try instead? I'm afraid I don't 
understand how does that test work (I seems a bit complicated)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33331] Clean modules in the reversed order

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +6260
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33331] Clean modules in the reversed order

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

Since dict is ordered, modules in sys.modules are ordered for the time of 
importing. Currently they are cleaned in PyImport_Cleanup() in the direct order 
-- from imported first to imported later. I wonder if cleaning them in the 
reversed order can solve some problems with the interpreter shutdown.

For example reverting the order fixes issue33328 and may help in other cases.

If revert the order, should only iterating weaklist be reverted (with setting 
all module globals to None), or iterating sys.modules (with setting sys.module 
values to None) too?

--
components: Interpreter Core
messages: 315603
nosy: brett.cannon, eric.snow, ncoghlan, pitrou, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Clean modules in the reversed order
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Miro, could you check whether 2.7 is affected too?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
priority: normal -> critical
stage:  -> needs patch
title: test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal 
number 32 out of range -> sigaddset() can fail on some signal numbers
type:  -> behavior
versions: +Python 3.8 -Python 3.4, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal number 32 out of range

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Ok, this is due to:
https://sourceware.org/bugzilla/show_bug.cgi?id=22391

Short explanation here:
https://unix.stackexchange.com/a/155846

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal number 32 out of range

[Miro Hrončok]

Miro Hrončok  added the comment:

glibc-2.27.9000-13.fc29 ... all ok
glibc-2.27.9000-14.fc29 ... 32, 33 fail
glibc-2.27.9000-15.fc29 (latest built) ... 32, 33 fail

13 to 14 is this in upstream commits:

d39c0a459ef32a41daac4840859bf304d931adab to 
583a27d525ae189bdfaa6784021b92a9a1dae12e

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal number 32 out of range

[Miro Hrončok]

Miro Hrončok  added the comment:

dnf install 
https://kojipkgs.fedoraproject.org/packages/glibc/2.27.9000/7.fc29/x86_64/glibc-headers-2.27.9000-7.fc29.x86_64.rpm
 
https://kojipkgs.fedoraproject.org/packages/glibc/2.27.9000/7.fc29/x86_64/glibc-devel-2.27.9000-7.fc29.x86_64.rpm
dnf install gcc


NSIG = 65
sigaddset(1) returned 0, errno =0
... all ok

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal number 32 out of range

[Miro Hrončok]

Miro Hrončok  added the comment:

"dnf install gcc" upgrades glibc, yes. OK, I'll dig into it a bit more, see why 
and where this change happened. Thanks for hints.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal number 32 out of range

[Miro Hrončok]

Miro Hrončok  added the comment:

glibc-2.27.9000-14.fc29:

NSIG = 65
sigaddset(32) returned -1, errno =22
sigaddset(33) returned -1, errno =22

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal number 32 out of range

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I was able to compile the program above after the system upgrade using your 
Docker image, and the following stands out:

sigaddset(31) returned 0, errno = 0
sigaddset(32) returned -1, errno = 22
sigaddset(33) returned -1, errno = 22
sigaddset(34) returned 0, errno = 0

However, I do not know how to compile it *before* the system upgrade ("dnf 
install gcc" seems to install gcc).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal number 32 out of range

[Miro Hrončok]

Miro Hrončok  added the comment:

I'll try. In the meantime, I've checked and it's glibc update that makes the 
difference:

glibc-2.27.9000-7.fc29 -> glibc-2.27.9000-14.fc29

I'll see what was changed there and whether is was intentional, or bug in 
Fedora.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal number 32 out of range

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Perhaps you can compile and run this C program, before and after the system 
changes, and post the output:


#include 
#include 
#include 

int main(int argc, char** argv)
{
sigset_t set;
int i, ret;

printf("NSIG = %d\n", NSIG);
for (i = 1; i < NSIG; i++) {
errno = 0;
ret = sigaddset(, i);
printf("sigaddset(%d) returned %d, errno =%d\n", i, ret, errno);
}
return 0;
}


(on Ubuntu 16.04, I get NSIG = 65, and all signal numbers work fine)

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33315] Allow queue.Queue to be used in type annotations

[Ivan Levkivskyi]

Ivan Levkivskyi  added the comment:

I think this issue appeared previously on typing tracker. The current 
recommendation is to escape problematic annotations with quotes:

q: 'Queue[int]'

I don't think it will be added to typing, because following this way typing 
will grow infinitely to include all generics in stdlib.

Another solution may be to add a simple `__class_getitem__` to `Queue` that 
will just return `cls` (so that we don't import `typing`). Although it will 
skip all the normal typing checks (so that `Queue[42]` will not raise at 
runtime), static type checkers like mypy will flag such errors. Guido, what do 
you think about such experiment?

--
nosy: +gvanrossum, levkivskyi

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33330] Better error handling in PyImport_Cleanup()

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +6259
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33330] Better error handling in PyImport_Cleanup()

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

The proposed PR adds missed checks for errors and PyErr_Clear() calls in 
PyImport_Cleanup().

It may be worth to use PyErr_WriteUnraisable() in place of PyErr_Clear() (newly 
added and existing) in most cases. But this can add an unexpected noise, and I 
don't wont to do this at this stage of 3.7. It could be done in 3.8 though, and 
backported to 3.7 later.

I'm not sure that PyImport_Cleanup() works properly if sys.modules is not a 
dict. At this stage a custom mapping will likely be broken when most of modules 
are destroyed.

--
components: Interpreter Core
messages: 315592
nosy: brett.cannon, eric.snow, ncoghlan, pitrou, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Better error handling in PyImport_Cleanup()
type: behavior
versions: Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31463] test_multiprocessing_fork hangs test_subprocess

[Miro Hrončok]

Miro Hrončok  added the comment:

> Could you report the signal number issue separately?  Thank you!

https://bugs.python.org/issue33329

Resetting the Versions back.

--
versions:  -Python 3.4, Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] test_multiprocessing_fork, _forkserver, _spawn hangs: ValueError: signal number 32 out of range

[Miro Hrončok]

New submission from Miro Hrončok :

Originally reported in https://bugs.python.org/issue31463

This started to bother us in Fedora rawhide for various Python versions, so 
chances are something changed on the system level.

# python3.7 -m test.regrtest test_multiprocessing_fork 
Run tests sequentially
0:00:00 load avg: 1.24 [1/1] test_multiprocessing_fork
/usr/lib64/python3.7/multiprocessing/semaphore_tracker.py:55: UserWarning: 
semaphore_tracker: process died unexpectedly, relaunching.  Some semaphores 
might leak.
  warnings.warn('semaphore_tracker: process died unexpectedly, '
Exception in thread Thread-26:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
  File "/usr/lib64/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.7/multiprocessing/resource_sharer.py", line 139, in 
_serve
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
  File "/usr/lib64/python3.7/signal.py", line 60, in pthread_sigmask
sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range


(hangs...)

^CProcess Process-184:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/multiprocessing/process.py", line 297, in 
_bootstrap
self.run()
  File "/usr/lib64/python3.7/multiprocessing/process.py", line 99, in run
self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.7/test/_test_multiprocessing.py", line 3328, in 
child_access
w = conn.recv()
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 251, in recv
return _ForkingPickler.loads(buf.getbuffer())
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 960, in 
rebuild_connection
fd = df.detach()
  File "/usr/lib64/python3.7/multiprocessing/resource_sharer.py", line 57, in 
detach
with _resource_sharer.get_connection(self._id) as conn:
  File "/usr/lib64/python3.7/multiprocessing/resource_sharer.py", line 87, in 
get_connection
c = Client(address, authkey=process.current_process().authkey)
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 498, in Client
answer_challenge(c, authkey)
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 741, in 
answer_challenge
message = connection.recv_bytes(256) # reject large message
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 216, in 
recv_bytes
buf = self._recv_bytes(maxlength)
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 407, in 
_recv_bytes
buf = self._recv(4)
  File "/usr/lib64/python3.7/multiprocessing/connection.py", line 379, in _recv
chunk = read(handle, remaining)
KeyboardInterrupt
Warning -- multiprocessing.process._dangling was modified by 
test_multiprocessing_fork
  Before: <_weakrefset.WeakSet object at 0x7f2abd5f6d30>
  After:  <_weakrefset.WeakSet object at 0x7f2abd5f6128> 
Warning -- threading._dangling was modified by test_multiprocessing_fork
  Before: <_weakrefset.WeakSet object at 0x7f2abd5f66d8>
  After:  <_weakrefset.WeakSet object at 0x7f2abc835048> 

Test suite interrupted by signal SIGINT.
1 test omitted:
test_multiprocessing_fork

Total duration: 3 min 43 sec
Tests result: INTERRUPTED



# python3.7 -m test.regrtest test_multiprocessing_forkserver
Run tests sequentially
0:00:00 load avg: 1.00 [1/1] test_multiprocessing_forkserver
Exception in thread Thread-26:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/threading.py", line 917, in _bootstrap_inner
self.run()
  File "/usr/lib64/python3.7/threading.py", line 865, in run
self._target(*self._args, **self._kwargs)
  File "/usr/lib64/python3.7/multiprocessing/resource_sharer.py", line 139, in 
_serve
signal.pthread_sigmask(signal.SIG_BLOCK, range(1, signal.NSIG))
  File "/usr/lib64/python3.7/signal.py", line 60, in pthread_sigmask
sigs_set = _signal.pthread_sigmask(how, mask)
ValueError: signal number 32 out of range

(hangs...)

^CProcess Process-184:
Traceback (most recent call last):
  ...
KeyboardInterrupt
Warning -- multiprocessing.process._dangling was modified by 
test_multiprocessing_forkserver
  Before: <_weakrefset.WeakSet object at 0x7f0323ecde10>
  After:  <_weakrefset.WeakSet object at 0x7f0323ecd208> 
Warning -- threading._dangling was modified by test_multiprocessing_forkserver
  Before: <_weakrefset.WeakSet object at 0x7f0323ecd7b8>
  After:  <_weakrefset.WeakSet object at 0x7f0323ecddd8> 

Test suite interrupted by signal SIGINT.
1 test omitted:
test_multiprocessing_forkserver

Total duration: 55 sec
Tests result: INTERRUPTED



# python3.7 -m test.regrtest test_multiprocessing_spawn
Run tests sequentially
0:00:00 load avg: 1.49 [1/1] test_multiprocessing_spawn
Exception in thread Thread-26:
Traceback (most recent call last):
  File "/usr/lib64/python3.7/threading.py", 

[issue33328] pdb error when stepping through generator

[Rick Teachey]

Rick Teachey  added the comment:

I am on Anaconda 4.5.1 on Windows 10 (Python 3.6.5). I have confirmed that I DO 
get the error on another machine with the same version installed, so the lack 
of an error seems specific to my current machine. I do not know what could be 
causing this; it is very odd.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com