[issue30811] A venv created and activated from within a virtualenv uses the outer virtualenv's site-packages rather than its own.

[Antony Lee]

Antony Lee  added the comment:

I guess it's reasonable, I'll see whether we can use the workaround you 
proposed.  (Could a fix on virtualenv's side help?)
Thanks for the explanations.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32110] Make codecs.StreamReader.read() more compatible with read() of other files

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +patch
pull_requests: +4437
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32110] Make codecs.StreamReader.read() more compatible with read() of other files

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

Usually the read() method of a file-like object takes one optional argument 
which limits the amount of data (the number of bytes or characters) returned if 
specified.

codecs.StreamReader.read() also has such parameter. But this is the second 
parameter. The first parameter limits the number of bytes read for decoding. 
read(1) can return 70 characters, that will confuse most callers which expect 
either a single character or an empty string (at the end of stream).

Some times ago codecs.open() was recommended as a replacement for the builtin 
open() in programs that should work in 2.x and 3.x (this was before adding 
io.open()), and it is still used in many programs. But this peculiarity makes 
it bad replacement of builtin open().

I wanted to fix this issue long time ago, but forgot, and the question on Stack 
Overflow has reminded me about this. 
https://stackoverflow.com/questions/46437761/codecs-openutf-8-fails-to-read-plain-ascii-file

--
assignee: serhiy.storchaka
components: IO, Library (Lib)
messages: 306701
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Make codecs.StreamReader.read() more compatible with read() of other 
files
type: behavior
versions: Python 2.7, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30811] A venv created and activated from within a virtualenv uses the outer virtualenv's site-packages rather than its own.

[Vinay Sajip]

Vinay Sajip  added the comment:

Is it OK if I close this? It's not really a bug, nor a case that's been 
designed for (a stdlib module supporting specific external packages is very 
unusual - pip support via ensurepip is perhaps the one exception I can think 
of). Given there's a way to create a venv from a virtualenv's Python, as posted 
above, I'm not sure what more I can do.

If, on the other hand, you look into the details and find something I've 
missed, I'll certainly look again.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30811] A venv created and activated from within a virtualenv uses the outer virtualenv's site-packages rather than its own.

[Vinay Sajip]

Vinay Sajip  added the comment:

> I don't understand why things would be different when nesting?

Specifically because venv keeps "a pointer" to the Python environment it was 
created from. Usually that's a system Python. If a venv ("inner") is created 
from a virtualenv's interpreter, the pointer points back to that environment 
("outer", in this case).

Also, virtualenv can re-invoke itself with a different interpreter easily - 
that's business as usual. However, the venv module is part of a specific Python 
stdlib and doesn't reinvoke itself with a different interpreter.

To find the exact mechanisms which lead to these behaviours, you would need to 
examine the code of virtualenv and the venv module and perhaps do some stepping 
through in a debugger!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32109] Separated square brackets will generate a tuple instead of a list.

[Berker Peksag]

New submission from Berker Peksag :

Thank you for your report and for the PR, but I think you misunderstood 
documentation:

Using square brackets, separating items with commas: [a], [a, b, c]

The comma is used to show two different lists: a list with one item ``[a]`` 
*and* a list with three items ``[a, b, c]``, not a tuple with two lists ``[a], 
[a, b, c]`` (In Python, both ``(1,)`` and ``1,`` create a tuple)

--
nosy: +berker.peksag
resolution:  -> not a bug
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30811] A venv created and activated from within a virtualenv uses the outer virtualenv's site-packages rather than its own.

[Antony Lee]

Antony Lee  added the comment:

> venv, on the other hand, doesn't, but keeps a reference to its original 
> Python environment. That is, I think, the reason for the difference in 
> behaviour.

But for example, using the system Python to create a venv (no nesting), 
packages installed system-wide (e.g. using a linux package manager) are not 
visible in the venv.  Or, as you note, nesting a venv into another works fine.  
I don't understand why things would be different when nesting?

In any case thanks for the workaround.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32109] Separated square brackets will generate a tuple instead of a list.

[YCmove]

Change by YCmove :


--
keywords: +patch
pull_requests: +4435
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32109] Separated square brackets will generate a tuple instead of a list.

[YCmove]

Change by YCmove :


--
assignee: docs@python
components: Documentation
nosy: YCmove, docs@python
priority: normal
severity: normal
status: open
title: Separated square brackets will generate a tuple instead of a list.
type: enhancement
versions: Python 2.7, Python 3.4, Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31299] Add "ignore_modules" option to TracebackException.format()

[Dmitry Kazakov]

Change by Dmitry Kazakov :


--
pull_requests:  -3274

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Ignored ResourceWarning warnings leak memory in warnings registries

[Decorater]

Decorater  added the comment:

If it was me I would store the warning registry in an error log or something in 
the current directory that ran python itself. (maybe something like ``[main 
script name].log``? This way it generates the warnings like usual and does not 
eat up memory. And then the log could to turned off when they dont want any 
warnings (when ignored).

--
nosy: +Decorater

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32099] Use range in itertools roundrobin recipe

[Raymond Hettinger]

Change by Raymond Hettinger :


--
pull_requests: +4434

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Ignored ResourceWarning warnings leak memory in warnings registries

[STINNER Victor]

STINNER Victor  added the comment:

Attached bench_ignore_warn.py: microbenchmark (using the perf module) to 
measure the performance of emitting a warning when the warning is ignored.

I added two basic optimizations to my PR 4489. With these optimizations, the 
slowdown is +17% on such microbenchmark:

$ ./python -m perf compare_to ref.json patch.json 
Mean +- std dev: [ref] 903 ns +- 70 ns -> [patch] 1.06 us +- 0.06 us: 1.17x 
slower (+17%)

The slowdown was much larger without optimizations, +42%:

$ ./python -m perf compare_to ref.json ignore.json 
Mean +- std dev: [ref] 881 ns +- 59 ns -> [ignore] 1.25 us +- 0.08 us: 1.42x 
slower (+42%)


About the memory vs CPU tradeoff, we are talking around ~1000 ns. IMHO 1000 ns 
is "cheap" (fast) and emitting warnings is a rare event Python. I prefer to 
make warnings slower than "leaking" memory (current behaviour: warnings 
registry growing with no limit).

--
Added file: https://bugs.python.org/file47282/bench_ignore_warn.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31324] support._match_test() used by test.bisect is very inefficient

[STINNER Victor]

STINNER Victor  added the comment:

I merged my PR 4421 which is based on Serhiy's PR 4420. Thank you Serhiy for 
your reviews!

--

Serhiy Storchaka: "We could split patterns on two parts and create both 
matchers (...) I don't know whether it is worth to do."

My use bisect is to find a regression using the test.bisect tool. According to 
my tests, the fastest set().__contains__ matcher is always used, except of 
test_json. If you bisect a bug in test_json, it will work, but just be a little 
bit slower.

Honestly, I don't think that we can go further in term of optimization. In my 
tests, it seems like creating the matcher function and filter all tests take 
less than 1 second, so it's enough for my use case ;-)

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31324] support._match_test() used by test.bisect is very inefficient

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 803ddd8ce22f0de3ab42fb98a225a704c000ef06 by Victor Stinner in 
branch 'master':
bpo-31324: Optimize support._match_test() (#4421)
https://github.com/python/cpython/commit/803ddd8ce22f0de3ab42fb98a225a704c000ef06


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32100] IDLE: PathBrowser isn't working

[Cheryl Sabella]

Cheryl Sabella  added the comment:

I added a test for PathBrowser, but I didn't change the existing tests, except 
to move them to their own test class.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30811] A venv created and activated from within a virtualenv uses the outer virtualenv's site-packages rather than its own.

[Vinay Sajip]

Vinay Sajip  added the comment:

The problem for venv is that it's tied to the running interpreter, which is (in 
the case you mention) the one from the outer virtualenv. Unlike virtualenv, 
venv does not provide a mechanism to restart itself with a different Python 
interpreter.

I don't know how Conda works in detail, but virtualenv makes copies of parts of 
the stdlib; venv, on the other hand, doesn't, but keeps a reference to its 
original Python environment. That is, I think, the reason for the difference in 
behaviour.

Instead of just using 'python' to invoke the command which runs the "-mvenv", 
you could get the underlying Python and create the venv using that. For 
example, using this script "upvenv.py":

#!/usr/bin/env python
def make_venv(venvpath):
import os
import subprocess
import sysconfig

python = os.path.join(sysconfig.get_config_var('BINDIR'), 'python3')
cmd = [python, '-mvenv', venvpath]
subprocess.run(cmd)

if __name__ == '__main__':
import sys
assert len(sys.argv) == 2
try:
rc = make_venv(sys.argv[1])
except Exception as e:
print('Failed: %s' % e)
rc = 1
sys.exit(rc)

I get:

vinay@ubuntu:/tmp$ cd /tmp
vinay@ubuntu:/tmp$ ve15 --version
15.1.0
vinay@ubuntu:/tmp$ ve15 -p python3 outer
Running virtualenv with interpreter /usr/bin/python3
Using base prefix '/usr'
New python executable in /tmp/outer/bin/python3
Also creating executable in /tmp/outer/bin/python
Installing setuptools, pip, wheel...done.
vinay@ubuntu:/tmp$ source outer/bin/activate
(outer)  vinay@ubuntu:/tmp$ python upvenv.py inner
(outer)  vinay@ubuntu:/tmp$ source inner/bin/activate
(inner)  vinay@ubuntu:/tmp$ python -minspect -d pip
Target: pip
Origin: /tmp/inner/lib/python3.5/site-packages/pip/__init__.py
Cached: 
/tmp/inner/lib/python3.5/site-packages/pip/__pycache__/__init__.cpython-35.pyc
Loader: <_frozen_importlib_external.SourceFileLoader object at 0x7f6917e9e860>
Submodule search path: ['/tmp/inner/lib/python3.5/site-packages/pip']

which seems OK.

This approach can't be transplanted into venv because it doesn't have the 
ability to restart with a different interpreter.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32106] Move sysmodule.c to Modules?

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The builtins and sys modules are special as they are directly initialized at 
interpreter startup. They are parts of the interpreter core and this is why 
they are in the Python directory.

--
nosy: +serhiy.storchaka
resolution:  -> rejected
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32096] Py_DecodeLocale() fails if used before the runtime is initialized.

[Eric Snow]

Eric Snow  added the comment:

> IMHO for the long term, the best would be to have a structure for
> pre-Py_Initialize configuration, maybe _PyCoreConfig, and pass it
> to functions that can be called before Py_Initialize().

+1

As an alternative to that, we could also deprecate using any of those functions 
before initializing the runtime.   Instead of calling them, you would set the 
relevant info on the runtime "config" struct that you pass to the replacement 
for PyInitialize().  At that point we would not need some of those functions 
any longer and we could remove them (eventually, once backward-compatibility is 
resolved).  Given that the community of CPython embedders is relatively small, 
we're still in a position to iron this out

Regardless, I see where you're coming from.  I'm okay with reverting the 
Object/obmalloc.c parts, but, like I said, I'd rather avoid it if possible.

Solution #2 (that I listed above), AKA PR #4481, is focused and effective.  
Unfortunately, it feels like a bit like a hack to me, though it is a 
well-contained hack.  So I'm not convinced it's the best solution.  However, I 
like it as much as I like reverting the allocators.

Solution #3, AKA PR #4495, is nice and clean, but potentially adds a little 
overhead to all PyMem_RawMalloc() and PyMem_RawFree() calls.  All the other 
PyMem_* functions are unaffected, so perhaps the overall impact is not 
significant enough to worry.


>> 2. statically initialize the "raw" allocator with defaults, enough
>>   to make PyMem_RawMalloc() and PyMem_RawFree() work pre-init (this
>>   is what my PR does)
>
> As I explained, the code to initialize PyMem_Raw allocator is complex
> and I would really prefer to only initialize it "partially" to prevent
> bad surprises.

The surprises would only be pre-initialization, right?  After the runtime is 
initialized, the allocators are in the proper fully-initialized state.  So it 
mostly boils down to what parts of the C-API embedders can use before 
initialization and how those functions interact with the raw memory allocator.  
Those constraints narrow down the scope of potential problems to a manageable 
size (I won't say small, but it feels that way).

Ultimately, I favor solution #3 if we can see that it does not impact 
performance.  If we can't come to an agreement in a timely fashion then I'll go 
along with #1 (revert), so that we don't leave the embedding story broken.  If 
we go that route, do you think we could resolve the initialization issues 
within the 3.7 timeframe?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32095] AIX: ModuleNotFoundError: No module named '_ctypes' - make install fails

[Zachary Ware]

Zachary Ware  added the comment:

You removed me from the nosy list, so I didn't see your reply until now when I 
happened to refresh the tab that happened to still be open :)

The warning I was talking about in that message was a warning that libffi was 
found but is outdated, which wouldn't be any help to you here.  There are 
already messages in the build log about libffi being missing and/or _ctypes not 
being built.

We do still bundle libffi for OSX and Windows, but we're trying to remove them. 
 I definitely don't want to add another bundled version.

You already have some install dependencies, or are missing some optional 
modules.  In particular, the lzma module depends on the xz package, readline 
depends on libreadline, _ssl depends on OpenSSL, _bz2 depends on bzip2, etc.  
There is a message at the end of `make` about which modules were not built and 
why.

Another workaround you could use (at the expense of not having pip installed) 
is to configure with `./configure --without-ensurepip`.

Finally, since we've nailed down that lack of libffi was in fact the source of 
the problem, this is a duplicate of #31652.

--
nosy: +zach.ware
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> make install fails: no module _ctypes
versions:  -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32071] Add py.test-like "-k" test selection to unittest

[Jonas H.]

Jonas H.  added the comment:

Interesting, Victor. I've had a look at the code you mentioned, but I'm afraid 
it doesn't really make sense to re-use any of the code.

Here's a new patch, implemented in the loader as suggested by Antoine, and with 
tests.

I'm happy to write documentation etc. once we're through with code review.

https://github.com/python/cpython/pull/4496

--
keywords: +patch
pull_requests: +4433
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] test_uuid uses the incorrect bitmask

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

On Nov 21, 2017, at 15:47, Serhiy Storchaka  wrote:

> If your fix is correct (and it looks correct to me), we should fix not only 
> the test, but, first of all, the _*_getnode functions. Perhaps they should 
> ignore locally administered MAC addresses and continue searching better 
> variants or fall back to other method.

I think it does make sense to ignore locally administered MACs from _getnode() 
and friends.  E.g. there’s no use in returning the interface for the Touch Bar 
bridge :)

I don’t have time right now to elaborate on that, but I’ll work on improving 
the PR with that change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] test_uuid uses the incorrect bitmask

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

If your fix is correct (and it looks correct to me), we should fix not only the 
test, but, first of all, the _*_getnode functions. Perhaps they should ignore 
locally administered MAC addresses and continue searching better variants or 
fall back to other method.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] test_uuid uses the incorrect bitmask

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Yikes.  It's entirely possible that these tests are tainted by environmental 
leakage.  I was looking into why Travis fails on my PR:

https://travis-ci.org/python/cpython/jobs/305433725

and stepping through _ifconfig_getnode() on my Mac.  The "ibridge" interface is 
getting returned, which has a MAC address of:

>>> from uuid import _ifconfig_getnode
>>> mac = _ifconfig_getnode()
>>> hex(mac)
'0xacde48001122'

That's for the en5 interface, which according to this article is the bridge to 
the Touch Bar, and *the same on every Mac*.

https://discussions.apple.com/thread/7763102?start=0=0

Why I think that's problematic for this particular test is that whatever gets 
returned is going to be highly dependent on the hardware the test is run on, 
and it's entirely possible that the MAC address returned is indeed locally 
administered and not tied to a physical external (and thus required to be 
universally administered) MAC address.

Mocking _ifconfig_*() probably isn't a good idea because these tests are 
worthless that way.  But it's also true that the _ifconfig_*() methods can 
match unexpected interfaces which cause the test to fail incorrectly.  It's a 
mess, and I'm not sure what to do about it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30855] [2.7] test_tk: test_use() of test_tkinter.test_widgets randomly fails with "integer value too large to represent" on with AMD64 Windows8 3.5

[Zachary Ware]

Zachary Ware  added the comment:

If you'd like to do that update, there are some instructions at 
https://github.com/python/cpython-source-deps.  You can get AppVeyor to use 
dependencies from your fork of `cpython-source-deps` by adjusting 
2.7/PCbuild/get_externals.bat:32 to point to your username temporarily.

If you don't want to mess with it, I'll get to it sooner or later :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32108] configparser bug: section is emptied if you assign a section to itself

[Łukasz Langa]

Łukasz Langa  added the comment:

Confirmed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32096] Py_DecodeLocale() fails if used before the runtime is initialized.

[Eric Snow]

Change by Eric Snow :


--
pull_requests: +4432
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32099] Use range in itertools roundrobin recipe

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

But why use less efficient code? Is my code worse?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32099] Use range in itertools roundrobin recipe

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Really, I don't give a whit about the micro-benchmarks -- that completely 
misses the point.  The recipes are primarily intended to have educational value 
on ways to use itertools, deques, and whatnot.

For itertools, I'm satisfied with new variable name and the additional comment. 
 For collections, there is an open question about whether adding an extra 
example would make users better off.   Beyond that, I have little interest is 
exploring all the little variants or wasting further time on this (that is what 
ASPN, StackOverflow, and blog posts are for).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30811] A venv created and activated from within a virtualenv uses the outer virtualenv's site-packages rather than its own.

[Antony Lee]

Antony Lee  added the comment:

Travis uses virtualenvs as toplevel containers for running CI.  Some projects 
need (or would like to) set up venvs as part of their test suites.  On example 
is https://github.com/airspeed-velocity/asv, which profiles a project's speed 
across revisions; in order to do so it sets up an environment for each revision 
to run the test suite.  Currently, it can either set up a virtualenv or a conda 
env, but it would make sense for it to be able to set up a venv too 
(https://github.com/airspeed-velocity/asv/pull/526).  But the unability to nest 
a venv into a virtualenv makes this impossible to test on Travis.

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] test_uuid uses the incorrect bitmask

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Pure chance I believe.  It all depends on the MAC address of the machines the 
test runs on (which is perhaps an unacceptable environmental variability in the 
test suite, and should be fixed/mocked?).

FWIW, I saw the failures on my 2017 MacBook Pro, where the MAC address did have 
that bit set on two of its interfaces.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30811] A venv created and activated from within a virtualenv uses the outer virtualenv's site-packages rather than its own.

[Vinay Sajip]

Vinay Sajip  added the comment:

Can you explain why virtualenv and venv need to be mixed in this way, other 
than as an academic exercise? If you use -m venv for both inner and outer 
venvs, then it seems to work as expected.

I'm planning to close this as "not a bug" unless you can provide a reason why 
this use case needs to be supported.

--
status: open -> pending

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32100] IDLE: PathBrowser isn't working

[Terry J. Reedy]

Terry J. Reedy  added the comment:

The issue is this:
1. PathBrowser subclasses ModuleBrowser.
2. ModuleBrowser.init() has the common code for initializing both.
3. #31460 changed the signature of ModuleBrowser.__init__ *and* .init.
4. I must not have tested pathbrowser after the .init change.  Bad.
5. test_pathbrowser needs a new gui test class that at least calls 
PathBrowser.__init__, which will call ModuleBrowser.init

3.6.3 is OK, 3.7.0a2 has this bug.  3.7.0a3 is due 11/27.  Patch should be 
merged before this with or without new unittests.
---

Module items can be expanded either by clicking [+] or double-clicking.  The 
latter also opens the file so that lines can be highlighted.  I have thought 
about disabling both, and opening a module browser instead, but I can see the 
usefulness of expanding several files one level deep to get an overview of 
parts or all of a package.  On the other hand, we might have double-click open 
both the module and a module browser.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32108] configparser bug: section is emptied if you assign a section to itself

[R. David Murray]

Change by R. David Murray :


--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32108] configparser bug: section is emptied if you assign a section to itself

[Simon Lambourn]

New submission from Simon Lambourn :

If you assign a ConfigParser section back to the parent ConfigParser object 
(say after updating the section), the section is emptied.

(I realise now that you don't need to assign the section back to the parent as 
it's a proxy for the section in the parent already - but still it does not 
behave as you would expect):
code: 
from configparser import ConfigParser
config = ConfigParser()
config['test'] = {'key': 'value'}
section = config['test']
section['key'] = 'different'
print("before: config['test'] is %s" % dict(config['test']))
config['test'] = section
print("after: config['test'] is %s" % dict(config['test']))
# the section is now printed as {}

--
components: Library (Lib)
messages: 306675
nosy: simonltwick
priority: normal
severity: normal
status: open
title: configparser bug: section is emptied if you assign a section to itself
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] test_uuid uses the incorrect bitmask

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

I'm just wondering why tests were passing successfully before.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32099] Use range in itertools roundrobin recipe

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Today I have published a similar recipe on Python-Ideas. It uses popleft/append 
instead of __getitem__/rotate.

def roundrobin(*iterables):
"roundrobin('ABC', 'D', 'EF') --> A D E B F C"
nexts = deque(iter(it).__next__ for it in iterables)
popleft = nexts.popleft
append = nexts.append
while nexts:
next = popleft()
try:
yield next()
except StopIteration:
pass
else:
append(next)

It is faster (10-25%) in all microbenchmarks that I did (Steven's benchmarks 
for small number of iterables and my examples for large number of iterables).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] test_uuid uses the incorrect bitmask

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
keywords: +patch
pull_requests: +4431
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32107] test_uuid uses the incorrect bitmask

[Barry A. Warsaw]

New submission from Barry A. Warsaw :

The check_node() function in test_uuid.py uses the wrong bitmask, causing the 
tests to fail on valid MAC addresses.  Also, the description in the comment is 
incorrect.  From my reading of the wikipedia page, the test is trying to ensure 
that the MAC address is "universally administered", which is designated by a 
zero value in "the second least significant digit of the first octet".

Thus the bitmask value *and* the comment are both wrong AFAICT, given that the 
original value is:

% ./python.exe -c "from math import log2; print(log2(0x0100))"
40.0

I have a fix that passes on my machine, so I'll PR that and see if it passes on 
CI.

--
assignee: barry
components: Tests
messages: 306672
nosy: barry
priority: normal
severity: normal
status: open
title: test_uuid uses the incorrect bitmask
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32106] Move sysmodule.c to Modules?

[Decorater]

Change by Decorater :


--
components: +Interpreter Core, Library (Lib)

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28562] test_asyncio fails on Android upon calling getaddrinfo()

[Xavier de Gaye]

Xavier de Gaye  added the comment:

test_create_connection_service_name does not fail on android-24-x86_64.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32099] Use range in itertools roundrobin recipe

[Raymond Hettinger]

Raymond Hettinger  added the comment:

While we're on the topic, I had some thought of also adding a similar recipe to 
https://docs.python.org/3/library/collections.html#deque-recipes .  The 
alternative recipe is slower is common cases but doesn't degrade when there are 
a huge number of iterables:  

def roundrobin(*iterables):
"roundrobin('ABC', 'D', 'EF') --> A D E B F C"
nexts = deque(iter(it).__next__ for it in iterables)
while nexts:
try:
while True:
yield nexts[0]()
nexts.rotate(-1)
except StopIteration:
nexts.popleft()

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32106] Move sysmodule.c to Modules?

[Decorater]

New submission from Decorater :

After looking in the folder Python sysmodule.c is in there instead of in 
Modules. I am wondering if it has any reason to be in that folder instead of in 
Modules/* with the other builtin modules in python.

If not I think it is best to move it so that way it is easier to find for those 
who think sysmodule is in the Modules folder because it is an builtin module.

--
messages: 306671
nosy: Decorater
priority: normal
severity: normal
status: open
title: Move sysmodule.c to Modules?
versions: Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32103] Inconsistent text at TypeError in concatenation

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

This is a duplicate of issue29116.

--
nosy: +serhiy.storchaka
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Make str and bytes error messages on concatenation conform with 
other sequences

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32100] IDLE: PathBrowser isn't working

[Terry J. Reedy]

Terry J. Reedy  added the comment:

Error is printed in console window if IDLE started from one.
>>> import idlelib.idle
# Select pathbrowser in File menu
Exception in Tkinter callback
Traceback (most recent call last):
  File "F:\dev\3x\lib\tkinter\__init__.py", line 1699, in __call__
return self.func(*args)
  File "F:\dev\3x\lib\idlelib\editor.py", line 671, in open_path_browser
pathbrowser.PathBrowser(self.flist)
  File "F:\dev\3x\lib\idlelib\pathbrowser.py", line 18, in __init__
self.init(flist)
TypeError: init() takes 1 positional argument but 2 were given

--
type: crash -> behavior
versions: +Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31324] support._match_test() used by test.bisect is very inefficient

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

We could split patterns on two parts and create both matchers. Then the final 
matching function could look like:

return _match_test_patterns is None or test_id in id_set or 
regex_match(test_id) or any(map(regex_match, test_id.split(".")))


I don't know whether it is worth to do.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32105] Please add asyncio.BaseEventLoop.connect_accepted_socket Version Added to documentation

[Yury Selivanov]

Change by Yury Selivanov :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32105] Please add asyncio.BaseEventLoop.connect_accepted_socket Version Added to documentation

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset d7ed48c2b8c11bb99da3661e8fe0bf5ae7f6b8d7 by Yury Selivanov (Miss 
Islington (bot)) in branch '3.6':
bpo-32105: add asyncio.BaseEventLoop.connect_accepted_socket versionadded to 
documentation. (GH-4491) (#4493)
https://github.com/python/cpython/commit/d7ed48c2b8c11bb99da3661e8fe0bf5ae7f6b8d7


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Ignored ResourceWarning warnings leak memory in warnings registries

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

> While I don't see how we can avoid "leaking memory" (growing the registry) 
> for actions like "once", I think that it's ok to don't touch the registry for 
> the "ignore" action. So at least, an application ignoring ResourceWarning 
> will not leak anymore.

This is a caching for speed. If the same line of code emits the same warning, 
it is enough to tests the registry and don't apply all warning filters.

I think we should find better compromise between performance if all messages 
equal and memory usage if all messages are different. Since caching for the 
"ignore" action affects only performance, not semantic, we could check whether 
there are too much warnings for the "ignore" action are cached in the same 
line. The problem is how to make this effectively. And what is "too much".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32105] Please add asyncio.BaseEventLoop.connect_accepted_socket Version Added to documentation

[Yury Selivanov]

Yury Selivanov  added the comment:


New changeset 431665bf1971e66c51f59abf0693f700ff7919e8 by Yury Selivanov 
(AraHaan) in branch 'master':
bpo-32105: add asyncio.BaseEventLoop.connect_accepted_socket versionadded to 
documentation. (#4491)
https://github.com/python/cpython/commit/431665bf1971e66c51f59abf0693f700ff7919e8


--
nosy: +yselivanov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32105] Please add asyncio.BaseEventLoop.connect_accepted_socket Version Added to documentation

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4430

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29040] building Android with android-ndk-r14

[Xavier de Gaye]

Change by Xavier de Gaye :


--
keywords: +patch
pull_requests: +4429
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32096] Py_DecodeLocale() fails if used before the runtime is initialized.

[STINNER Victor]

STINNER Victor  added the comment:

IMHO for the long term, the best would be to have a structure for 
pre-Py_Initialize configuration, maybe _PyCoreConfig, and pass it to functions 
that can be called before Py_Initialize().

For example, I'm working on a variant of Py_GetPath() which accepts a 
_PyCoreConfig to be able to pass the value of the PYTHONPATH environment 
variable.

That's a more complex solution, so I proposed to first revert, to have more 
time to design the "correct API".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Ignored ResourceWarning warnings leak memory in warnings registries

[Jonathan Bastien-Filiatrault]

Jonathan Bastien-Filiatrault  added the comment:

> But it will still "leak" when you display ResourceWarning warnings with an 
> action different than "always". In this case, IMHO the root issue is more the 
> code which doesn't close the resource, than Python itself.

Not closing a file is a bug, but under normal circumstances it causes no leak 
by itself. The fact that the warnings module leaks in this case seems a 
problem. Had I logged warnings correctly, I would have found the bug by looking 
at the application log rather than by investigating the cause of the OOM killer 
invocation.

IMHO, the warnings module should have upper bounds on memory consumption to 
avoid DOSing itself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32096] Py_DecodeLocale() fails if used before the runtime is initialized.

[STINNER Victor]

STINNER Victor  added the comment:

> 2. statically initialize the "raw" allocator with defaults, enough
>   to make PyMem_RawMalloc() and PyMem_RawFree() work pre-init (this
>   is what my PR does)

As I explained, the code to initialize PyMem_Raw allocator is complex and I 
would really prefer to only initialize it "partially" to prevent bad surprises.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32096] Py_DecodeLocale() fails if used before the runtime is initialized.

[STINNER Victor]

STINNER Victor  added the comment:

Nick: "Should we deprecate our implied support for calling Py_DecodeLocale() 
before calling Py_Initialize()?"

Please don't do that. Py_DecodeLocale() is the best available function to 
decode paths and environment variables to initialize Python. The implementation 
of Py_DecodeLocale() is very complex, but we need this complexity to decode 
"correctly" OS data.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32096] Py_DecodeLocale() fails if used before the runtime is initialized.

[STINNER Victor]

STINNER Victor  added the comment:

I marked bpo-32096 as a duplicate of this one. I don't want to discuss the same 
issue in 3 places (2 bpo and python-dev).

--
superseder: C API: Clarify which C functions are safe to be called before 
Py_Initialize() -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32086] C API: Clarify which C functions are safe to be called before Py_Initialize()

[STINNER Victor]

STINNER Victor  added the comment:

I mark this issue as a duplicate of bpo-32096.

--
resolution:  -> duplicate
stage:  -> resolved
status: open -> closed
superseder:  -> Py_DecodeLocale() fails if used before the runtime is 
initialized.

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32096] Py_DecodeLocale() fails if used before the runtime is initialized.

[STINNER Victor]

STINNER Victor  added the comment:

"3. use hard-coded defaults in PyMem_RawMalloc() and PyMem_RawFree() if the 
runtime has not been initialized yet"

I dislike this option since it can have a negative impact on performances. The 
PEP 445 already added a new level of indirection and so made memory allocations 
a little bit slower.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32096] Py_DecodeLocale() fails if used before the runtime is initialized.

[Eric Snow]

Eric Snow  added the comment:

I thought issue #32086 was about documentation (which is worth having a 
separate issue for), not about a fix to the regression.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32105] Please add asyncio.BaseEventLoop.connect_accepted_socket Version Added to documentation

[Decorater]

Change by Decorater :


--
keywords: +patch
pull_requests: +4428
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32096] Py_DecodeLocale() fails if used before the runtime is initialized.

[Eric Snow]

Eric Snow  added the comment:

I see at least 3 ways to sort this out:

 1. partially revert the _PyRuntime change, sort of temporarily
   ("revert the change on memory allocators, and retry later to fix
   it, once other initializations issues are fixed", as suggested by
   Victor in the email thread)
 2. statically initialize the "raw" allocator with defaults, enough
   to make PyMem_RawMalloc() and PyMem_RawFree() work pre-init (this
   is what my PR does)
 3. use hard-coded defaults in PyMem_RawMalloc() and PyMem_RawFree()
   if the runtime has not been initialized yet

I considered implementing #3 instead, but wasn't sure about the performance 
impact.  It would add a pointer comparison to NULL and a branch on each 
PyMem_RawMalloc() and PyMem_RawFree() call.  I'm not convinved it would make 
much of a difference.  Furthermore, I consider #3 to be the simplest solution, 
both to implement and to maintain, so I'll probably try it out.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32105] Please add asyncio.BaseEventLoop.connect_accepted_socket Version Added to documentation

[Decorater]

Decorater  added the comment:

Will see whhat I can do.

--
nosy: +Decorater

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Ignored ResourceWarning warnings leak memory in warnings registries

[STINNER Victor]

STINNER Victor  added the comment:

> The best way to fix this is excluding file name from warning message. As a 
> result, the message for every file will be the same and warnings registry 
> will not grow.

Your warning comes from the io module which uses the message: "unclosed file 
%r" % file.

I dislike the idea of removing the filename from this warning message, since 
the filename is very useful to identify where the bug comes from.

Different things are discussed here:

* First of all, if an application is correctly written, ResourceWarning is not 
emitted, and so Python doesn't leak memory

* Using the "always" action, ResourceWarning doesn't leak neither.

* Using a different action like "default", ResourceWarning does leak memory in 
hidden warning registries.

While I don't see how we can avoid "leaking memory" (growing the registry) for 
actions like "once", I think that it's ok to don't touch the registry for the 
"ignore" action. So at least, an application ignoring ResourceWarning will not 
leak anymore.

But it will still "leak" when you display ResourceWarning warnings with an 
action different than "always". In this case, IMHO the root issue is more the 
code which doesn't close the resource, than Python itself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31672] string.Template should use re.ASCII flag

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31672] string.Template should use re.ASCII flag

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:


New changeset e256b408889eba867e1d90e5e1a0904843256255 by Barry Warsaw in 
branch 'master':
bpo-31672 - Add one last minor clarification for idpattern (#4483)
https://github.com/python/cpython/commit/e256b408889eba867e1d90e5e1a0904843256255


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32105] Please add asyncio.BaseEventLoop.connect_accepted_socket Version Added to documentation

[Thomas Eldon Allred]

New submission from Thomas Eldon Allred :

asyncio.BaseEventLoop.connect_accepted_socket was added in v3.5.3 Please add a 
note to the documentation.

--
assignee: docs@python
components: Documentation
messages: 306650
nosy: Thomas Eldon Allred, docs@python
priority: normal
severity: normal
status: open
title: Please add asyncio.BaseEventLoop.connect_accepted_socket Version Added 
to documentation
type: enhancement
versions: Python 3.5, Python 3.6, Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Ignored ResourceWarning warnings leak memory in warnings registries

[STINNER Victor]

Change by STINNER Victor :


--
title: Memory leaks when opening tons of files -> Ignored ResourceWarning 
warnings leak memory in warnings registries

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Memory leaks when opening tons of files

[Jonathan Bastien-Filiatrault]

Jonathan Bastien-Filiatrault  added the comment:

@vstinner Yes, from what I saw, the leak was from the registry / deduplication 
logic.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Memory leaks when opening tons of files

[STINNER Victor]

STINNER Victor  added the comment:

I created PR 4489 to fix the memory leak for the "ignore "action". IMHO it's 
interesting to modify the "ignore" action because Python uses ignore many 
warnings by default:

haypo@selma$ python3 -c 'import pprint, warnings; 
pprint.pprint(warnings.filters)'
[('ignore', None, , None, 0),
 ('ignore', None, , None, 0),
 ('ignore', None, , None, 0),
 ('ignore', None, , None, 0),
 ('ignore', None, , None, 0)]


I created memory2.py (based on attached memory.py) to measure the leak.

Currently, the "always" action doesn't leak, but "ignore" and "default" actions 
leak:

haypo@selma$ ./python -W always memory2.py 
Memory peak grow: +3.2 kB
Warning filters:
[('always',
  re.compile('', re.IGNORECASE),
  ,
  re.compile(''),
  0),
 ('ignore', None, , None, 0),
 ('always', None, , None, 0)]

haypo@selma$ ./python -W ignore memory2.py 
Memory peak grow: +26692.3 kB
Warning filters:
[('ignore',
  re.compile('', re.IGNORECASE),
  ,
  re.compile(''),
  0),
 ('ignore', None, , None, 0),
 ('always', None, , None, 0)]

haypo@selma$ ./python -W default memory2.py 
Memory peak grow: +26693.2 kB
Warning filters:
[('default',
  re.compile('', re.IGNORECASE),
  ,
  re.compile(''),
  0),
 ('ignore', None, , None, 0),
 ('always', None, , None, 0)]


With my PR 4489, the "ignore" action doesn't leak anymore:

haypo@selma$ ./python -W ignore memory2.py 
Memory peak grow: +2.4 kB
Warning filters:
[('ignore',
  re.compile('', re.IGNORECASE),
  ,
  re.compile(''),
  0),
 ('ignore', None, , None, 0),
 ('always', None, , None, 0)]

--
Added file: https://bugs.python.org/file47281/memory2.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Memory leaks when opening tons of files

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +4427
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32104] add method throw() to asyncio.Task

[Yury Selivanov]

Yury Selivanov  added the comment:

> in order to write advanced Tasks there should be a way
> to throw custom exceptions into active Task. 
> so it can react accordingly.  

What is an "advanced" task?  Why CancelledError is not enough?  What's the 
actual use case?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32089] In developer mode (-X dev), ResourceWarning is only emited once per line numbers

[STINNER Victor]

STINNER Victor  added the comment:

Antoine:
> If they have different names, they will be logged separately.

Oh wow, nice behaviour, I like it :-)

Antoine:
>> For ResourceWarning, your rationale only concerns pydebug build, no?
> Why?  I'm talking about "-X dev", not pydebug builds.

Oops sorry, typo (brain error), I mean: "also concerns".

Serhiy:
> See also issue27535.

Oh, not leaking memory in warnings registies is a nice side effect of the 
"always" action :-)

--

It's much more complicated than what I expected to get the "correct behaviour" 
for warnings :-/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Memory leaks when opening tons of files

[STINNER Victor]

STINNER Victor  added the comment:

The problem is not the function displaying the warning, but warnings registries 
(__warningregistry__) no?

The behaviour depends on the action of the filter matching the ResourceWarning 
warning:

* always: don't touch the registry => no leak
* ignore: add the key to registry => leak!
* another action: add the key to registry => leak!

The key is the tuple (text, category, lineno).

I understand why always doesn't touch the registry. But why does "ignore" 
action touch the registry? Not only the user will not see the message, but the 
memory will slowly grow in the background.

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32104] add method throw() to asyncio.Task

[Oleg K]

New submission from Oleg K :

currently there is no other way to interrupt task but to call
cancel() which will: 

"This arranges for a CancelledError to be thrown into the wrapped coroutine on 
the next cycle through the event loop."

in order to write advanced Tasks there should be a way
to throw custom exceptions into active Task. 
so it can react accordingly.  

i am looking for the same thing as "generator.throw()" is providing,
allowing to interrupt generators different way.

--
components: asyncio
messages: 306644
nosy: Oleg K2, yselivanov
priority: normal
severity: normal
status: open
title: add method throw() to asyncio.Task
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32102] Add "capture_output=True" option to subprocess.run

[Barry A. Warsaw]

Barry A. Warsaw  added the comment:

Yeah, I find it pretty common to set both stdout and stderr to PIPE, but I'm 
with you in hesitating to add YAO to the subprocess API.  If you do move 
forward with this though, I think capture_output would have to be mutually 
exclusive to stdout and stderr.  You should not be allowed to provide either of 
the latter two if the former is given.

--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32103] Inconsistent text at TypeError in concatenation

[Carl]

New submission from Carl :

>>> a = b"jan"
>>> b = "jan"
>>> a+b
Traceback (most recent call last):
  File "", line 1, in 
TypeError: can't concat str to bytes
>>> b+a
Traceback (most recent call last):
  File "", line 1, in 
TypeError: must be str, not bytes
>>>

IMHO The latter TypeError text should be "TypeError: can't concat bytes to str"

--
components: Interpreter Core
messages: 306642
nosy: wolfc01
priority: normal
severity: normal
status: open
title: Inconsistent text at TypeError in concatenation
type: enhancement
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32089] In developer mode (-X dev), ResourceWarning is only emited once per line numbers

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

See also issue27535.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue31324] support._match_test() used by test.bisect is very inefficient

[STINNER Victor]

STINNER Victor  added the comment:

I created a list of all test cases using:

./python -m test --list-cases > all_cases

The list contains 29,569 test cases. Sadly, the set().__contains__ matcher of 
my PR 4421 cannot be taken because test_json produces two test cases called 
"json". IMHO it's a bug in test_json, all test identifiers should be "fully 
qualified". test_builtin has a similar issue: it produces identifiers like 
"builtins.bin" instead of "test.test_builtin.builtins.bin".

If I removed the two "json" from all_cases, "./python -u -m test.bisect -i 
all_cases -v" takes less than one second before running the first test method, 
and so it's fast enough.

With PR 4420, it takes around 17 seconds before running the first test. The 
compilation of the giant regex and matching test identifiers are slow. My PR 
has the same bad performance when the regex path is taken.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32089] In developer mode (-X dev), ResourceWarning is only emited once per line numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I forgot: the snippet above was made with "python -Wdefault::ResourceWarning".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32089] In developer mode (-X dev), ResourceWarning is only emited once per line numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Le 21/11/2017 à 14:46, STINNER Victor a écrit :
> 
> I don't know the rationale of the "always" action rather than "default".

Neither do I.  But I don't think it matters a lot.  pydebug builds are
almost only used by Python core developers.  "developer mode" is target
at every Python developer.

> The problem of ResourceWarning is that the warning logs where the last 
> reference to the resource was cleared, or where the a garbage collection was 
> triggered. Multiple resources can be created from different places, but all 
> "die" at the same place.

If they have different names, they will be logged separately.

>>> def f(fname):
... open(fname)
...
>>> f('setup.py')
__main__:2: ResourceWarning: unclosed file <_io.TextIOWrapper
name='setup.py' mode='r' encoding='UTF-8'>
>>> f('setup.py')
>>> f('LICENSE.txt')
__main__:2: ResourceWarning: unclosed file <_io.TextIOWrapper
name='LICENSE.txt' mode='r' encoding='UTF-8'>
>>> f('LICENSE.txt')
>>>

> For ResourceWarning, your rationale only concerns pydebug build, no?

Why?  I'm talking about "-X dev", not pydebug builds.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32089] In developer mode (-X dev), ResourceWarning is only emited once per line numbers

[STINNER Victor]

STINNER Victor  added the comment:

> If a program calls func() 1000 times in a row, they will show the same 
> warning 1000 times.  What does it bring to have the exact same warning (and 
> line number) displayed 1000 times instead of once? Nothing.  There is a 
> reason the filter "default" isn't the same as "always", and it's that 
> "always" can be completely overwhelming.

I don't know the rationale of the "always" action rather than "default". The 
choice was made 7 years ago, when Georg Brandl implemented the new 
ResourceWarning category: commit 08be72d0aa0112118b79d271479598c218adfd23.

The problem of ResourceWarning is that the warning logs where the last 
reference to the resource was cleared, or where the a garbage collection was 
triggered. Multiple resources can be created from different places, but all 
"die" at the same place.

In Python 3.6, I enhanced ResourceWarning to log also the traceback where the 
leaked resource has been created, if the tracemalloc is tracing memory 
allocations. When tracemalloc is used, it's useful to log ResourceWarning 
multiple times even if the warning itself is logged multipletimes. Well, that's 
an unusual usage of the ResourceWarning, we can also suggest to use -W 
always::ResourceWarning in that case.


> I don't agree with this.  You're comparing "-X dev" with a pydebug build of 
> Python, not with a normal Python.

In general, I would like to offer the same experience in "development mode" (-X 
dev) and with a pydebug build.

For ResourceWarning, your rationale only concerns pydebug build, no?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27535] Memory leaks when opening tons of files

[Jonathan Bastien-Filiatrault]

Jonathan Bastien-Filiatrault  added the comment:

We just got hit by this. We had one specific case where files with unique names 
were not being closed and Python was leaking a lot of memory over a few days.

--
nosy: +Jonathan Bastien-Filiatrault

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32089] In developer mode (-X dev), ResourceWarning is only emited once per line numbers

[Antoine Pitrou]

Antoine Pitrou  added the comment:

I don't agree with this.  You're comparing "-X dev" with a pydebug build of 
Python, not with a normal Python.

Your example shows the problem.  If a program calls func() 1000 times in a row, 
they will show the same warning 1000 times.  What does it bring to have the 
exact same warning (and line number) displayed 1000 times instead of once?  
Nothing.  There is a reason the filter "default" isn't the same as "always", 
and it's that "always" can be completely overwhelming.

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29040] building Android with android-ndk-r14

[Xavier de Gaye]

Change by Xavier de Gaye :


--
versions:  -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28762] lockf() is available now on Android API level 24, but the F_LOCK macro is not defined

[Xavier de Gaye]

Xavier de Gaye  added the comment:

This change is not needed anymore now that Unified Headers are supported by 
android-ndk-r14 (see issue 29040)

--
resolution: fixed -> not a bug
stage: resolved -> patch review
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28762] lockf() is available now on Android API level 24, but the F_LOCK macro is not defined

[Xavier de Gaye]

Change by Xavier de Gaye :


--
pull_requests: +4426

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32099] Use range in itertools roundrobin recipe

[Roundup Robot]

Change by Roundup Robot :


--
pull_requests: +4425

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com