[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> - the built-in AST increasingly modifies the tree before presenting it to user
>   code (constant folding moved to the AST in Python 3.7);

These modification are applied only before bytecodecode generation. The AST 
presented to user is not modified.

> - the built-in tokenize.py can only be used to parse Python 3.7+ code;

Is this a problem? 2.7 is a dead Lib/lib2to3/pgen2/tokenize.pyend, its support 
will be ended in less than 2 years. Even 3.6 will be moved to a security only 
fixes stage short time after releasing 3.8.

I'm in favor of updating Lib/lib2to3/pgen2/tokenize.py, but I don't understand 
why Lib/tokenize.py should parse 2.7.

I'm in favor of reimplementing pgen in Python if this will simplify the code 
and the building process. Python code is simpler than C code, this code is not 
performance critical, and in any case we need an external Python when modify 
grammar of bytecode.

See also issue30455 where I try to get rid of duplications by generating all 
tokens-related data and code from a single source (token.py or external text 
file).

For what purposes the CST is needed besides 2to3? I know only that it could 
help to determine the correct position in docstrings in doctests and similar 
tools which need to process docstrings and report errors. This is not possible 
with AST due to inlined '\n', escaped newlines, and string literals 
concatenation. Changes in 3.7 made this even worse (see issue32911).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Łukasz Langa]

Łukasz Langa  added the comment:

> I'm in favor of unifying the tokenizers and of updating and moving pgen2 
> (though I don't have time to do the work).

I'm willing to do all the work as long as I have somebody to review it. Case in 
point: BPO-8.



> Also I think you may have to make a distinction between the parser generator 
> and its data structures, and between the generated parser for Python vs. the 
> parser for other LL(1) grammars one might feed into it.

Technically pgen2 has the ability to parse any LL(1) grammar but so far the 
plumbing is tightly tied to the tokenizer.  We'd need to enable plugging that 
in, too.



> And I don't think you're proposing to replace Parser/pgen.c with Lib/pgen/, 
> right?

No, I'm not.



> Nor to replace the CST actually used by CPython's parser with the data 
> structures used by pgen2's driver.

No, I'm not.



> So the relationship between the CST you propose to document and CPython 
> internals wouldn't be quite the same as that between the AST used by CPython 
> and the ast module (since those *do* actually use the same code).

Right.  Once we unify the standard library tokenizers (note: *not* tokenizer.c 
which will stay), there wouldn't be much extra documentation to write for 
Lib/tokenize.py.  For Lib/pgen/ itself, we'd need to provide both an API 
rundown and an intro to the high-level functionality (how to create trees from 
files, string, etc.; how to visit trees and edit them; and so on).


> I'm not sure if it's technically possible to give tokenize.py the ability to 
> tokenize Python 2.7 and up without some version-selection flag -- have you 
> researched this part yet?

There's two schools. This is going to take a while to explain :)

One school is to force the caller to declare what Python version they want to 
parse.  This is algorithmically cleaner because we can then literally take 
Grammar/Grammar from various versions of Python and have the user worry about 
picking the right one.

The other school is what lib2to3 does currently, which is to try to implement 
as much of a superset of Python versions as possible.  This is way easier to 
use because the grammar is very forgiving.  However, this has limitations.  
There are three major incompatibilities that we need to deal with, with raising 
degree of severity:
- async/await;
- print statements;
- exec statements.

Async and await became proper keywords in 3.7 and thus broke usage of those as 
names.  It's relatively easy to work around this one seamlessly by keeping the 
grammar trickery we've had in place for 3.5 and 3.6.  This is what lib2to3 does 
today already👍🏻

The print statement is fundamentally incompatible with the print function.  
lib2to3 has two grammar variants and most users by default choose the one 
without the print statement.  Why?  Because it cannot be reliably sniffed 
anymore.  Python 3-only code will not use the __future__ import.  In fact, 2to3 
also doesn't do auto-detection, relies on the user running `2to3 -p` to 
indicate they mean the grammar with the print function.

The exec statement is even worse because there isn't even a __future__ import.  
It's annoying because it creates a third combination. 👎🏻

So now the driver has to attempt three grammars (in this order):
- the almost compatible combined Python 2 + Python 3 one (that assumes exec is 
a function and print is a function);
- the one that assumes exec is a *statement* but print is still a function 
(because __future__ import);
- the one that exposes the legacy exec and print statements.

This approach has one annoying wart.  Imagine you have a file like this:

  print('msg', file=sys.stderr)
  if

Now the driver will attempt all three grammars and fail, and will report that 
the parse error is on the print line.  This can be overcome by comparing syntax 
errors from each grammar and showing the one on the furthest line (which is the 
most likely to be the real culprit).  But it's still annoying and will 
sometimes not do what the user wanted.


-- OK, OK. So which to choose?

And now, while this sounds like more work and is harder to get right, I still 
think the combined grammar with minimal incompatibilities is the better 
approach.  Why?  Two reasons.

1. Nobody ever knows what Python version *exactly* a given file is.  Most files 
aren't even considering compatibility that fine-grained.  And having to attempt 
to parse not three but potentially 8 grammars (3.7 - 3.2, 2.7, 2.6) would be 
prohibitively slow.

2. My tool maybe wants to actually *modify* the compatibility level by, say, 
rewriting ''.format() with f-strings or putting trailing commas where old 
Pythons didn't accept them.  So it would be awkward if the grammar I used to 
read the file wasn't compatible with my later changes.

Unless I'm swayed otherwise, I'd continue on what lib2to3 did, with the 
exception that we need to add a grammar variant without the `exec` statement, 
and the driver needs to attempt parsing with the thr

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Łukasz Langa]

Łukasz Langa  added the comment:

> These modification are applied only before bytecodecode generation. The AST 
> presented to user is not modified.

This bit me when implementing PEP 563 but I was then on the compile path, 
right.  Still, the latest docstring folding would qualify as an example here, 
too, no?


> Is this a problem? 2.7 is a dead end, its support will be ended in less than 
> 2 years. Even 3.6 will be moved to a security only fixes stage short time 
> after releasing 3.8.

Yes, it is a problem.  We will support Python 2 until 2020 but people will be 
running Python 2 code for a decade *at least*.  We need to provide those people 
a way to move their code forward.  Static analysis tools like formatters, 
linters, type checkers, or 2to3-style translators, are all soon going to run on 
Python 3.  It would be a shame if those programs were barred from helping users 
that are still struggling on Python 2.

A closer example is async/await.  It would be a shame if running on Python 3.7 
meant you can't write a tool that renames (or even just *detects*) invalid uses 
of async/await.  I firmly believe that the version of the runtime should be 
indepedent of the version it's able to analyze.


> I'm in favor of updating Lib/lib2to3/pgen2/tokenize.py, but I don't 
> understand why Lib/tokenize.py should parse 2.7.

Hopefully I sufficiently explained that above.


> I'm in favor of reimplementing pgen in Python if this will simplify the code 
> and the building process. Python code is simpler than C code, this code is 
> not performance critical, and in any case we need an external Python when 
> modify grammar of bytecode.

Well, I didn't think about abandoning pgen.  I admit that's mostly because my 
knee-jerk reaction was that it would be too slow.  But you're right that this 
is not performance critical because every `pip install` runs `compileall`.

I guess we could parse in "strict" mode for Python itself but allow for 
multiple grammars for standard library use (as I explained in the reply to 
Guido).  And this would most likely give us opportunity to iterate on grammar 
improvements in the future.

And yet, I'm cautious here.  Even ignoring performance, that sounds like a more 
ambitious task from what I'm attempting.  Unless I find partners in crime for 
this, I wouldn't attempt that.  And I would need thumbs up from the BDFL and 
performance-wary contributors.


> For what purposes the CST is needed besides 2to3?

Anywhere where you need the full view of the code which includes non-semantic 
pieces.  Those include:
- whitespace;
- comments;
- parentheses;
- commas;
- strings prefixes.

The main use case is linters and refactoring tools.  For example mypy is using 
a modified AST to support type comments.  YAPF and Black are based on lib2to3 
because as formatters they can't lose comments, string prefixes, and 
organizational parentheses either.  JEDI is using Parso, a lib2to3 fork, for 
similar reasons.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33338] [lib2to3] Synchronize token.py and tokenize.py with the standard library

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset c2d384dbd7c6ed9bdfaac45f05b463263c743ee7 by Łukasz Langa in 
branch 'master':
bpo-8: [tokenize] Minor code cleanup (#6573)
https://github.com/python/cpython/commit/c2d384dbd7c6ed9bdfaac45f05b463263c743ee7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33319] `subprocess.run` documentation doesn't tell is using `stdout=PIPE` safe

[Pekka Klärck]

Pekka Klärck  added the comment:

My goal is to read stdout. It's good to hear `subprocess.run()` is 
deadlock-safe and I can use it safely. Making the docs explicit about it so 
that others know it's safe would in my opinion be a good idea as well.

Casual users don't know `run()` it uses `communicate()`, not `wait()`, 
internally, or even that this would mean it cannot deadlock. The current 
situation when the docs say that `call()` shouldn't be used with `stdout=PIPE` 
and that `call(...)` is equivalent to `run(...).returncode` indicates 
`stdout=PIPE` is unsafe with `run()` as well.

A separate questions is that if `call(...)` is equivalent to 
`run(...).returncode`, should it also be implemented that way. Based on this 
discussion it would avoid the problem with `stdout=PIPE` also in that case.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33339] Using default encoding with `subprocess.run()` is not obvious

[Pekka Klärck]

New submission from Pekka Klärck :

It is possible to use `subprocess.run()` with the system's default encoding by 
using `universal_newlines=True`. This is very handy, but it's not at all 
obvious (at least for me) that setting such option has effect on encoding. This 
is especially true when the docs don't explain this explicitly:

"""If encoding or errors are specified, or universal_newlines is true, file 
objects for stdin, stdout and stderr are opened in text mode using the 
specified encoding and errors or the io.TextIOWrapper default."""

This is such a useful feature that it ought to be documented better, preferably 
with an example.

>From code reading point of view, I would also consider `encoding=True` to be 
>more explicit that `universal_newlines=True`. I can submit a separate issue 
>about that if others feel the same.

--
messages: 315652
nosy: pekka.klarck
priority: normal
severity: normal
status: open
title: Using default encoding with `subprocess.run()` is not obvious

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33340] Inaccurate docs on `import` behaviour

[sam_b]

New submission from sam_b :

The docs https://docs.python.org/3/tutorial/modules.html#the-module-search-path 
describe:

> When a module named spam is imported, the interpreter first searches for a 
> built-in module with that name. If not found, it then searches for a file 
> named spam.py in a list of directories given by the variable sys.path. 
> sys.path is initialized from these locations:

> -   The directory containing the input script (or the current directory when 
> no file is specified).
> -   PYTHONPATH (a list of directory names, with the same syntax as the shell 
> variable PATH).
> -   The installation-dependent default.


However, it seems like "the directory containing the input script" is checked 
*before* the standard library:

➜  tmp more logging.py
def foo():
print('bar')
➜  tmp python
Python 2.7.15rc1 (default, Apr 15 2018, 21:51:34)
[GCC 7.3.0] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import logging
>>> logging.foo()
bar
>>> logging.WARNING
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: 'module' object has no attribute 'WARNING'
>>> 


Am I misunderstanding the docs?

--
assignee: docs@python
components: Documentation
messages: 315653
nosy: docs@python, sam_b
priority: normal
severity: normal
status: open
title: Inaccurate docs on `import` behaviour
type: behavior
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I don't think we want to be in the business of maintaining a list of 
cross-platform of supported signal values.  Python may be compiled on bizarre 
non-glibc systems (or even systems with a proprietary libc).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33340] Inaccurate docs on `import` behaviour

[Eric V. Smith]

Eric V. Smith  added the comment:

"built-in modules" has the specific meaning of modules that are compiled in to 
the python executable. It doesn't mean modules in the standard library.

See https://docs.python.org/3.6/library/sys.html#sys.builtin_module_names

Python 3.6.4 (default, Jan  7 2018, 15:53:53)
[GCC 6.4.0] on cygwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.builtin_module_names
('_ast', '_codecs', '_collections', '_functools', '_imp', '_io', '_locale', 
'_operator', '_signal', '_sre', '_stat', '_string', '_symtable', '_thread', 
'_tracemalloc', '_warnings', '_weakref', 'atexit', 'builtins', 'errno', 
'faulthandler', 'gc', 'itertools', 'marshal', 'posix', 'pwd', 'sys', 'time', 
'xxsubtype', 'zipimport')

--
nosy: +eric.smith

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33341] python3 fails to build if directory or sysroot contains "*icc*" string

[Gianfranco]

New submission from Gianfranco :

Hello, as said, in yocto we pass in $CC environment, also the --sysroot keyword.

If --sysroot contains the "*icc*" or "*gcc*" in path, the build fails because 
of wrong autoconf checks.

checking in a case switch for *icc* seems bad to me, I propose to change it 
with something like
"*icc" so we might exclude when icc is not the compiler.

I know this isn't a strong patch, but maybe we can apply it while writing some 
better detect code.

The build fails as descripted in the yocto bug
https://bugzilla.yoctoproject.org/show_bug.cgi?id=12703

--
components: Build
files: patch
messages: 315656
nosy: locutusofborg
priority: normal
severity: normal
status: open
title: python3 fails to build if directory or sysroot contains "*icc*" string
type: compile error
versions: Python 3.8
Added file: https://bugs.python.org/file47547/patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33340] Inaccurate docs on `import` behaviour

[sam_b]

sam_b  added the comment:

Thanks, I thought it might be something like that. Would it be worth clarifying 
the distinction in the docs? It was certainly surprising to me that `import 
time` and `import logging` behave differently.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33340] Inaccurate docs on `import` behaviour

[Eric V. Smith]

Eric V. Smith  added the comment:

If you have a suggested improvement, please create a pull request. Thanks!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

Yes I am aware, but I can't see a really portable way to provide the same 
functionality as 'sigfillset'.  Ideally a libc implementation would return 
EINVAL on 'sigaddset' for invalid signals, but even for glibc this is not true 
(the very issue I fixed).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Right, but it seems that even if sigaddset() allowed you to "set" signals 32 
and 33, that would be ignored by pthread_sigmask().

This is what I get here (Ubuntu 16.04, glibc 2.23):

>>> signal.pthread_sigmask(signal.SIG_BLOCK, range(1, 65))
set()
>>> signal.pthread_sigmask(signal.SIG_BLOCK, range(1, 65))
{, , , 
, , , 
, , , 
, , , 
, , 16, , 
, , , 
, , , 
, , , 
, , , 
, , 35, 36, 37, 38, 39, 40, 41, 42, 
43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 
63, }

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +6276
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30750] Update `make patchcheck` to understand Misc/NEWS.d

[Antoine Pitrou]

Change by Antoine Pitrou :


--
pull_requests: +6277

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

Yes, this is the issue I referred in previous comment [1]. Unfortunately it is 
only fixed on master (which will become 2.28).

[1] https://sourceware.org/bugzilla/show_bug.cgi?id=22391

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Not sure what you mean.  In the example above, I try to block signals 32 and 33 
using pthread_sigmask(), but the pthread_sigmask() return value shows they 
weren't blocked, which is ok.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33315] Allow queue.Queue to be used in type annotations

[Semyon Proshev]

Semyon Proshev  added the comment:

I had been thinking about `__class_getitem__` when the issue were created.

I suspected it's not difficult to implement and it follows PEP 560.
So I'm +1 for `__class_item__`, quotes and postponed evaluation are also ok.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31500] IDLE: Tiny font on HiDPI display

[Terry J. Reedy]

Terry J. Reedy  added the comment:

I guess I was not clear enough that I will test on Windows after someone else 
does a backport that works on Linux.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25478] Consider adding a normalize() method to collections.Counter()

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Was __rtruediv__ discussed before? What is the use case for it?

Besides __rtruediv__ all LGTM.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31500] IDLE: Tiny font on HiDPI display

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

You promised to make backports to 2.7 when did the mass files renaming. The 
simple cherry-picking doesn't work, and I even don't know what files in 2.7 
should be changed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Adhemerval Zanella]

Adhemerval Zanella  added the comment:

What I mean is for master glibc, which contains BZ#22391, 'sigaddset' will fail 
do add signal 32 and 33.  But since you are now ignoring 'sigaddset' return 
code, it does not matter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33342] urllib IPv6 parsing fails with special characters in passwords

[benaryorg]

New submission from benaryorg :

The documentation specifies to follow RFC 2396 
(https://tools.ietf.org/html/rfc2396.html) but fails to parse a 
user:password@host url in urllib.parse.urlsplit 
(https://docs.python.org/3/library/urllib.parse.html#urllib.parse.urlsplit) 
when the password contains an '[' character.
This is because the urlsplit code does not strip the authority part (everything 
from index 0 up to and including the last '@') before checking whether the 
hostname contains '[' for detecting whether it's an IPv6 address 
(https://github.com/python/cpython/blob/8a6f4b4bba950fb8eead1b176c58202d773f2f70/Lib/urllib/parse.py#L416-L418).

--
components: Library (Lib)
messages: 315668
nosy: benaryorg
priority: normal
severity: normal
status: open
title: urllib IPv6 parsing fails with special characters in passwords
versions: Python 2.7, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33342] urllib IPv6 parsing fails with special characters in passwords

[benaryorg]

Change by benaryorg :


--
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33315] Allow queue.Queue to be used in type annotations

[Guido van Rossum]

Guido van Rossum  added the comment:

Could you please product a draft PR showing how that would work? It might
not be accepted right away but it would be useful to have.

On Mon, Apr 23, 2018, 05:29 Semyon Proshev  wrote:

>
> Semyon Proshev  added the comment:
>
> I had been thinking about `__class_getitem__` when the issue were created.
>
> I suspected it's not difficult to implement and it follows PEP 560.
> So I'm +1 for `__class_item__`, quotes and postponed evaluation are also
> ok.
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +neologix

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33316] PyThread_release_lock always fails

[Steve Dower]

Steve Dower  added the comment:

Looks okay, but I wonder whether the implementation of PyMUTEX_UNLOCK should 
return something other than zero? What is it meant to return?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33343] Subcommand abbreviations

[Victor Porton]

New submission from Victor Porton :

https://docs.python.org/3/library/argparse.html does not allow abbreviations 
for subparsers ("subcommands").

It seems inconsistent that long options can be abbreviated but subcommands 
cannot.

Either implement subcommand abbreviations or explain in the docs why there is 
none.

--
components: Library (Lib)
messages: 315671
nosy: porton
priority: normal
severity: normal
status: open
title: Subcommand abbreviations
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33289] tkinter askcolor returning floats for r, g, b values instead of ints

[Cheryl Sabella]

Change by Cheryl Sabella :


--
keywords: +patch
pull_requests: +6278
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 25038ecfb665bef641abf8cb61afff7505b0e008 by Antoine Pitrou in 
branch 'master':
bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575)
https://github.com/python/cpython/commit/25038ecfb665bef641abf8cb61afff7505b0e008


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[miss-islington]

Change by miss-islington :


--
pull_requests: +6279

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[miss-islington]

Change by miss-islington :


--
pull_requests: +6280

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33251] ConfigParser.items returns items present in vars

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset 1d4a733cceee237f0850b7887c2945dee707da27 by Łukasz Langa (Chris 
Bradbury) in branch 'master':
bpo-33251: Prevent ConfigParser.items returning items present in vars. (#6446)
https://github.com/python/cpython/commit/1d4a733cceee237f0850b7887c2945dee707da27


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset 75a3e3d5bc0be1ce41289b661e7c53039cf3d5ba by Antoine Pitrou (Miss 
Islington (bot)) in branch '3.7':
bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575) (GH-6579)
https://github.com/python/cpython/commit/75a3e3d5bc0be1ce41289b661e7c53039cf3d5ba


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33332] Expose valid signal set (sigfillset())

[Antoine Pitrou]

Change by Antoine Pitrou :


--
keywords: +patch
pull_requests: +6281
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33332] Expose valid signal set (sigfillset())

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +njs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
nosy: +njs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Change by Antoine Pitrou :


--
pull_requests: +6282

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33251] ConfigParser.items returns items present in vars

[Chris Bradbury]

Change by Chris Bradbury :


--
pull_requests: +6283

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:


New changeset b0ca398cabd2d2ea2d66fa50b08e297a60388c75 by Antoine Pitrou in 
branch '3.6':
[3.6] bpo-33329: Fix multiprocessing regression on newer glibcs (GH-6575) 
(GH-6582)
https://github.com/python/cpython/commit/b0ca398cabd2d2ea2d66fa50b08e297a60388c75


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Miro, this should be fixed now. Please reopen if it isn't.

The sigfillset() functionality will be exposed to Python users in bpo-2.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33329] sigaddset() can fail on some signal numbers

[Miro Hrončok]

Miro Hrončok  added the comment:

Antoine, thank you very much for swift communication and fix.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Gregory P. Smith]

Gregory P. Smith  added the comment:

+1 in general to this work. Łukasz is effectively preaching to the choir by 
looping me in here. :)

It is a big challenge to practically support Python in that we have no good 
ability to parse and understand all language syntax versions via a single API 
that does not depend on the version of the language your own tools process is 
running under.

lib2to3.pgen2 is the closest thing we've got and it used by a notable crop of 
python refactoring tools today because there really wasn't another available 
choice.  All they know is that they've got a ".py" file, they can't know which 
specific language versions it may be intended for.  Nor should they ever need 
to run _on_ that language version.  That situation is a nightmare (ex: pylint 
uses ast and must run on the version of the language it is to analyze as)

I'd love to see a ponycorn module that everything could use to run on top of 
Python 3.recent yet be able to meaningfully process 2.7 and 3.4-3.7 code.  This 
is an area where the language versions we support parsing and analyzing should 
_not_ be limited to the current CPython org still supported releases.

Does this need to go in the CPython project and integrate with its internals 
such as pgen.c or pgen2?  I don't know.  From my perspective this could be a 
PyPI project.  Even if it seems odd that we have stdlib ast and lib2to3.pgen2 
modules and pgen internal to CPython; at some point those could be seen as 
implementation details and made private in favor of tool application code using 
a canonical ponycorn thing on PyPI.  The important part is that it is 
maintained and kept up to date with future language grammar changes while 
maintaining "backwards grammar compatibility".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33251] ConfigParser.items returns items present in vars

[Łukasz Langa]

Łukasz Langa  added the comment:


New changeset e500839796c7fa43998e72edb77165872b9ce034 by Łukasz Langa (Chris 
Bradbury) in branch 'master':
bpo-33251: Update documentation to reflect change. (GH-6446) (#6583)
https://github.com/python/cpython/commit/e500839796c7fa43998e72edb77165872b9ce034


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Łukasz Langa]

Łukasz Langa  added the comment:

> The important part is that it is maintained and kept up to date with future 
> language grammar changes while maintaining "backwards grammar compatibility".

Yes, which is why I have trouble believing this can be effectively outsourced.  
Existing third-party libraries always stalled at some point in this regard.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Guido van Rossum]

Guido van Rossum  added the comment:

But lib2to3 is proof that the stdlib is just as much subject to stalling.
Maybe lib2to3 and pgen2 would have a livelier future if they weren't
limited to updates in sync with Python releases.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Łukasz Langa]

Łukasz Langa  added the comment:

> But lib2to3 is proof that the stdlib is just as much subject to stalling.

The issue here is internal visibility. "lib2to3" is a library that supports 
"2to3" which is rather neglected internally since we started promoting `six` as 
a better migration strategy to Python 3.

Most core devs don't even *know* new syntax is supposed to be added to lib2to3. 
 Case in point: somehow Lib/tokenize.py was updated just in time for f-strings 
to be released but not Lib/lib2to3/pgen2/tokenize.py.

By unifying the tokenizers and moving the CST out of lib2to3's guts (and 
documenting it as a supported feature!), I'm pretty sure we can eliminate the 
danger of forgetting to update it in the future.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31500] IDLE: Tiny font on HiDPI display

[Cheryl Sabella]

Change by Cheryl Sabella :


--
pull_requests: +6284
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Nathaniel Smith]

Nathaniel Smith  added the comment:

It does seem like it'd be unfortunate to end up in a situation like "sorry, 
there's a bug in handling this python 2 code, so black won't be able to 
reformat it until the next major python release". And I assume this issue is 
motivated by running into limitations of the current version; waiting for 3.8 
before you can fix those seems unfortunate too?

Another option to think about: make the library something that's maintained by 
python-dev, but released separately on PyPI.

--
nosy: +njs

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22393] multiprocessing.Pool shouldn't hang forever if a worker process dies unexpectedly

[Oscar Esteban]

Oscar Esteban  added the comment:

We use multiprocessing to parallelize many tasks that run either python code or 
call subprocess.run that are memory hungry.

At times the OOM Killer kicks in. When one of the workers is killed, the queue 
never returns an error for the task being run by the worker.

Are there any plans to merge the patch proposed in this issue?

--
nosy: +Oscar Esteban

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33317] `repr()` of string in NFC and NFD forms does not differ

[Benjamin Peterson]

Benjamin Peterson  added the comment:

As stated, the bug report is invalid: the repr _does_ differ, it's just not 
presented that way by however you're viewing the two reprs. Distinct codepoint 
sequences that look identical under certain circumstances can happen many 
different ways with Unicode. repr's humble mission is to produce a Python 
literal equivalent to its argument not to produce unambiguous representations 
of codepoint sequences after font rendering.

Possibly, this could be converted to a unittest RFE, but I'm not sure if 
there's a good way to detect whether two unicode strings are going to display 
confusingly similarly.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Łukasz Langa]

Change by Łukasz Langa :


--
keywords: +patch
pull_requests: +6285
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33337] Provide a supported Concrete Syntax Tree implementation in the standard library

[Benjamin Peterson]

Benjamin Peterson  added the comment:

The stdlib is a bad place for anything that needs to evolve at a non-glacial 
place. For example, even when 2to3 had not yet fallen out of favor, there were 
effectively 3 versions of it: one 2.7 and two in maintained 3.x branches. That 
was a large pain. 2to3 also could only be updated as quickly as Python is 
released.

--
stage: patch review -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22393] multiprocessing.Pool shouldn't hang forever if a worker process dies unexpectedly

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Oscar, the patch posted here needs updating for the latest git master.

If you want to avoid this issue, you can also use concurrent.futures where the 
issue is fixed.

--
stage:  -> needs patch
versions: +Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com