[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

Can you please attach a single and standalone file without dependencies like 
attrs so that it would help in bisecting the issue?

--
nosy: +xtreak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36546] Add quantiles() to the statistics module

[Raymond Hettinger]

Raymond Hettinger  added the comment:

One other thought:  The PR implements both methods 6 and 7 which means that 
that we can reproduce the results of every major stats tool except for 
Mathematica.  The summary of properties chart of  in Hyndman & Fan lists our 
default as satisfying 5 of 6 desirable properties.  This is the same as for 
method 8 which almost no one uses by default.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36640] python ibm_db setup.py post install script does not seem to work from Anaconda

[Saba Kauser]

Saba Kauser  added the comment:

I have debugged this further.
The problem seems be happening due to usage of get_python_lib().

e.g:
if('darwin' in sys.platform):
class PostInstall(install):
""" Post installation - run install_name_tool on Darwin """
def run(self):
install.run(self)
clipath = os.getenv('IBM_DB_HOME', '@loader_path/clidriver')
print("in PostInstall with {}".format(clipath))
for so in glob.glob(get_python_lib()+r'/ibm_db*.so'):
os.system("install_name_tool -change libdb2.dylib 
{}/lib/libdb2.dylib {}".format(clipath, so))

cmd_class = dict(install = PostInstall)

The get_python_lib() returns following path and hence my script is never able 
to fetch the matching lib name.
example output of get_python_lib() usage:

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Users/skauser/anaconda3/lib/python3.7/site-packages/CHANGES

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Users/skauser/anaconda3/lib/python3.7/site-packages/LICENSE

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Users/skauser/anaconda3/lib/python3.7/site-packages/README.md

/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/site-packages/Users/skauser/anaconda3/lib/python3.7/site-packages/config.py.sample


I have both anaconda and standalone python installations and get_python_lib() 
seem to get confused with different paths.
It sometimes returns the duplicate path as well:
e.g: 

/Users/skauser/anaconda3/lib/python3.7/site-packages/Users/skauser/anaconda3/lib/python3.7/site-packages/CHANGES

/Users/skauser/anaconda3/lib/python3.7/site-packages/Users/skauser/anaconda3/lib/python3.7/site-packages/LICENSE

/Users/skauser/anaconda3/lib/python3.7/site-packages/Users/skauser/anaconda3/lib/python3.7/site-packages/README.md

/Users/skauser/anaconda3/lib/python3.7/site-packages/Users/skauser/anaconda3/lib/python3.7/site-packages/config.py.sample

Above examples are taken from one usage of get_python_lib() for copying certain 
data files while installation to site-packages. 
My strong belief is that, its the same problem with the post-install script as 
well.

Why is get_python_lib() behavior not consistent? 
What are the other alternatives to get the current site-package path for the 
running installation?

When I disable cache of pip and install, get_python_lib() path as retrieved 
while install behaves correctly.
e.g: pip --no-cache-dir install ibm_db

Kindly help! This is blocking production of many users.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36546] Add quantiles() to the statistics module

[Raymond Hettinger]

Raymond Hettinger  added the comment:

Steven, as requested I added a documentation note reserving the right to add 
other interpolation methods.  We can make the note stronger if you like.  
Otherwise, I think we're good to go now. 

Mark, thanks for the link.  I've read all the posts and agree that we might 
consider changing the default interpolation method prior to the release.  For 
now, matching what Excel and SciPy does seems like a reasonable starting point.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36693] Reversing large ranges results in a minor type inconsistency

[Raymond Hettinger]

Raymond Hettinger  added the comment:

This all seems proper to me.  In each case, an iterator is returned -- that is 
the documented behavior.  The specific type of iterator is allowed to vary in 
ways that are convenient for the implementation.

Specifically, reversed() will call __reversed__() which must return an iterator 
but its type is allowed to vary.  That is really convenient for handling 
extreme ranges separately from the fast, common case of smaller ranges.

--
nosy: +rhettinger
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36695] Change (regression?) in v3.8.0a3 doctest output after capturing the stderr output from a raised warning

[Brian Skinn]

New submission from Brian Skinn :

In [this project](https://github.com/bskinn/stdio-mgr) of mine, I have a tox 
matrix set up with Pythons from 3.3. to 3.8. I have pytest set up to run 
doctest on my 
[`README.rst`](https://github.com/bskinn/stdio-mgr/blob/6444cce8e5866e2d519c1c0630551d8867f30c9a/README.rst).
  For Pythons 3.4 to 3.7 (3.4.10, 3.5.7, 3.6.8, 3.7.2), the following doctest 
example passes:

```
>>> import warnings
>>> with stdio_mgr() as (in_, out_, err_):
... warnings.warn("'foo' has no 'bar'")
... err_cap = err_.getvalue()
>>> err_cap
"...UserWarning: 'foo' has no 'bar'\n..."
```

Under Python 3.8.0a3, though, it fails (actual local paths elided):

```
$ tox -re py38-attrs_latest
.package recreate: .../.tox/.package
.package installdeps: wheel, setuptools, attrs>=17.1
py38-attrs_latest recreate: .../.tox/py38-attrs_latest
py38-attrs_latest installdeps: attrs, pytest
py38-attrs_latest inst: .../.tox/.tmp/package/1/stdio-mgr-1.0.2.dev1.tar.gz
py38-attrs_latest installed: 
atomicwrites==1.3.0,attrs==19.1.0,more-itertools==7.0.0,pluggy==0.9.0,py==1.8.0,pytest==4.4.1,six==1.12.0,stdio-mgr==1.0.2.dev1
py38-attrs_latest run-test-pre: PYTHONHASHSEED='2720295779'
py38-attrs_latest run-test: commands[0] | pytest
===
 test session starts 

platform linux -- Python 3.8.0a3, pytest-4.4.1, py-1.8.0, pluggy-0.9.0
cachedir: .tox/py38-attrs_latest/.pytest_cache
rootdir: ..., inifile: tox.ini
collected 6 items   

   

README.rst F

 [ 16%]
tests/test_stdiomgr_base.py .   

 [100%]

=
 FAILURES 
=
___
 [doctest] README.rst 
___
077 
078 **Mock** ``stderr``\ **:**
079 
080 .. code ::
081 
082 >>> import warnings
083 >>> with stdio_mgr() as (in_, out_, err_):
084 ... warnings.warn("'foo' has no 'bar'")
085 ... err_cap = err_.getvalue()
086 >>> err_cap
Expected:
"...UserWarning: 'foo' has no 'bar'\n..."
Got:
':2: UserWarning: \'foo\' has no \'bar\'\n  
warnings.warn("\'foo\' has no \'bar\'")\n'

.../README.rst:86: DocTestFailure

 1 failed, 5 passed in 0.06 seconds 

ERROR: InvocationError for command .../.tox/py38-attrs_latest/bin/pytest 
(exited with code 1)
_
 summary 
__
ERROR:   py38-attrs_latest: commands failed

```

If I change the doctest in README to the following, where the expected output 
is surrounded by single-quotes instead of double-quotes, and the internal 
single quotes are escaped, it passes fine in 3.8.0a3:

```
>>> import warnings
>>> with stdio_mgr() as (in_, out_, err_):
... warnings.warn("'foo' has no 'bar'")
... err_cap = err_.getvalue()
>>> err_cap
'...UserWarning: \'foo\' has no \'bar\'\n...'
```

But, naturally, it fails in 3.7 and below.

It *looks* like this is probably a glitch somewhere in 3.8.0a3, where this 
string containing single quotes is rendered (at the REPL?) using enclosing 
single quotes and escaped internal single quotes, rather than enclosing 
double-quotes and non-escaped internal single-quotes?

--
components: Library (Lib)
messages: 340637
nosy: bskinn
priority: normal
severity: normal
status: open
title: Change (regression?) in v3.8.0a3 doctest output after capturing the 
stderr output from a raised warning
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-

[issue23078] unittest.mock patch autospec doesn't work on staticmethods

[Berker Peksag]

Berker Peksag  added the comment:

Thank you!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23078] unittest.mock patch autospec doesn't work on staticmethods

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 15a57a3cadb992bb1752302333ff593e7eab284c by Berker Peksag in 
branch '3.7':
bpo-23078: Add support for {class,static}method to mock.create_autospec() 
(GH-11613)
https://github.com/python/cpython/commit/15a57a3cadb992bb1752302333ff593e7eab284c


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35586] Open pyexpat compilation, Make shows error（missing separator）

[Aaron Hurst]

Aaron Hurst  added the comment:

This is the same issue as https://bugs.python.org/issue35184

I can reproduce this issue by uncommenting the pyexpat line in Setup.dist and 
compiling.

The issue is with -DXML_POOR_ENTROPY=1.  The equals character causes the line 
to be incorrectly interpreted as a macro definition by makesetup.  This results 
in an invalid Makefile output.

I've submitted a PR, but a quick work-around is to remove the "=1".  It is not 
necessary.

--
keywords: +patch
nosy: +ahurst
pull_requests: +12830
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23078] unittest.mock patch autospec doesn't work on staticmethods

[Berker Peksag]

Change by Berker Peksag :


--
pull_requests: +12829

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23078] unittest.mock patch autospec doesn't work on staticmethods

[Berker Peksag]

Change by Berker Peksag :


--
pull_requests:  -11349

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23078] unittest.mock patch autospec doesn't work on staticmethods

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 9b21856b0fcda949de239edc7aa6cf3f2f4f77a3 by Berker Peksag 
(Xtreak) in branch 'master':
bpo-23078:  Add support for {class,static}method to mock.create_autospec() 
(GH-11613)
https://github.com/python/cpython/commit/9b21856b0fcda949de239edc7aa6cf3f2f4f77a3


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36670] test suite broken due to cpu usage feature on win 10/ german

[Steve Dower]

Steve Dower  added the comment:

If the code page is set to UTF-8 then mbcs will still be the right encoding :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36518] Avoid conflicts when pass arbitrary keyword arguments to Python function

[Steve Dower]

Steve Dower  added the comment:

No, now there is syntax that anyone using built in functions will have already 
learned I see no reason to create another way to solve the problem.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35184] Makefile is not correctly generated when compiling pyextat with DXML_POOR_ENTROPY=1

[Aaron Hurst]

Aaron Hurst  added the comment:

I can reproduce this issue by uncommenting the pyexpat line in Setup.dist and 
compiling.

The issue is with -DXML_POOR_ENTROPY=1.  The equals character causes the line 
to be incorrectly interpreted as a macro definition by makesetup.  This results 
in an invalid Makefile output.

I've submitted a PR, but a quick work-around is to remove the "=1".  It is not 
necessary.

--
nosy: +ahurst

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35184] Makefile is not correctly generated when compiling pyextat with DXML_POOR_ENTROPY=1

[Aaron Hurst]

Change by Aaron Hurst :


--
keywords: +patch
pull_requests: +12827
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24011] Add error checks to PyInit_signal()

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 9541bd321a94f13dc41163a5d7a1a847816fac84 by Berker Peksag 
(Joannah Nanjekye) in branch 'master':
bpo-24011: Use PyModule_Add{Object,IntMacro} in PyInit__signal() (GH-12765)
https://github.com/python/cpython/commit/9541bd321a94f13dc41163a5d7a1a847816fac84


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue24011] Add error checks to PyInit_signal()

[Berker Peksag]

Berker Peksag  added the comment:

Thank you, Joannah. There's no need to backport PR 12765 to maintenance 
branches, so I'm closing this issue as 'fixed'.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36678] duplicate method definitions in Lib/test/test_dataclasses.py

[Windson Yang]

Change by Windson Yang :


--
keywords: +patch
pull_requests: +12826
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36523] Add docstring to io.IOBase.writelines

[Marcin Niemira]

Marcin Niemira  added the comment:

Hey,
Is there something I can improve in this PR?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36645] re.sub() library entry does not adequately document surprising change in behavior between versions

[Berker Peksag]

Berker Peksag  added the comment:

Thank you!

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
type:  -> behavior
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36645] re.sub() library entry does not adequately document surprising change in behavior between versions

[miss-islington]

miss-islington  added the comment:


New changeset 71b88827f6ad368eafa17983bd979175d24da888 by Miss Islington (bot) 
in branch '3.7':
bpo-36645: Fix ambiguous formatting in re.sub() documentation (GH-12879)
https://github.com/python/cpython/commit/71b88827f6ad368eafa17983bd979175d24da888


--
nosy: +miss-islington

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36645] re.sub() library entry does not adequately document surprising change in behavior between versions

[miss-islington]

Change by miss-islington :


--
pull_requests: +12824

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36645] re.sub() library entry does not adequately document surprising change in behavior between versions

[Berker Peksag]

Berker Peksag  added the comment:


New changeset 5ebfa840a1c9967da299356733da41b532688988 by Berker Peksag 
(mollison) in branch 'master':
bpo-36645: Fix ambiguous formatting in re.sub() documentation (GH-12879)
https://github.com/python/cpython/commit/5ebfa840a1c9967da299356733da41b532688988


--
nosy: +berker.peksag

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36685] C implementation of xml.etree.ElementTree does not make a copy of attrib argument when creating new Element

[Gordon P. Hemsley]

Change by Gordon P. Hemsley :


--
keywords: +patch
pull_requests: +12823
stage: needs patch -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35149] pip3 show causing Error for ConfigParaser

[Eryk Sun]

Eryk Sun  added the comment:

In Windows, Python defaults to the system ANSI codepage (e.g. 1252 in the West) 
for non-console standard I/O. For the case of a `for /f` loop in CMD, stdout is 
a pipe, so Python defaults to writing ANSI encoded text to its end of the pipe. 
I recommend overriding the encoding to UTF-8 using the PYTHONIOENCODING 
environment variable. 

CMD uses the console's output codepage to decode bytes read from its end of the 
pipe, so the batch script should temporarily change the console codepage to 
UTF-8 via `chcp.com 65001`. Note that this won't work if CMD is running without 
a console (i.e. a DETACHED_PROCESS), in which case it defaults to ANSI. (I 
don't recommend running without a console. If no window is required, use 
CREATE_NO_WINDOW or a hidden window instead.) First save the current console 
codepage, parsed from the output of running `chcp.com` without arguments. Then 
you can restore the original console codepage after the loop.

After decoding the text, CMD's `echo` command writes to the console using the 
wide-character WriteConsoleW function, so there's no problem at this stage -- 
up to the limits of the console's text support. FYI, in lieu of Python getting 
the blame for this too, the Windows console can only render Basic Multilingual 
Plane (i.e. UCS-2) text, and it doesn't support automatic font fallback or 
complex scripts. If the console can't display a character, it displays the 
font's default glyph (e.g. an empty rectangle), or two default glyphs for a 
surrogate pair. However, we can still copy text from the console in this case.

--
resolution:  -> not a bug
stage:  -> resolved
status: open -> closed
type:  -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36691] SystemExit & sys.exit : Allow both exit status and message

[Barry A. Warsaw]

Change by Barry A. Warsaw :


--
nosy: +barry

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36693] Reversing large ranges results in a minor type inconsistency

[Paul Ganssle]

Paul Ganssle  added the comment:

I believe the relevant code is here:

https://github.com/python/cpython/blob/bb86bf4c4eaa30b1f5192dab9f389ce0bb61114d/Objects/rangeobject.c#L1038

It looks like it's a performance enhancement and that for ranges where the 
beginning and end can fit in a C long, a faster iterator that uses C types 
under the hood is returned, and for ranges where the boundaries *can't* be 
represented by a C long, it defaults to the slower `longrange_iterator`, that 
uses Python integers.

It *may* be possible to disguise this from the end user, but I'm not sure if 
doing so is warranted. I have always treated the specific type returned by 
`iter()` to be an implementation detail, other than the fact that it is an 
iterator.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35149] pip3 show causing Error for ConfigParaser

[Karthikeyan Singaravelan]

Karthikeyan Singaravelan  added the comment:

I am able to reproduce this by explicitly setting PYTHONIOENCODING as ascii . 
But I am not sure about the defaults in Windows (related to your environment, 
os only?) and perhaps something has changed since 3.6 since it fails when 
executed under batch script or to close this as a pip related third party issue.

(py37-venv) ➜  cpython git:(master) ✗ PYTHONIOENCODING=ascii pip3 show 
ConfigParser
Name: configparser
Version: 3.7.4
Summary: Updated configparser from Python 3.7 for Python 2.6+.
Home-page: https://github.com/jaraco/configparser/
--- Logging error ---
Traceback (most recent call last):
  File 
"/Library/Frameworks/Python.framework/Versions/3.7/lib/python3.7/logging/__init__.py",
 line 1036, in emit
stream.write(msg)
UnicodeEncodeError: 'ascii' codec can't encode character '\u0141' in position 
8: ordinal not in range(128)
Call stack:
  File "/Users/karthikeyansingaravelan/stuff/python/py37-venv/bin/pip3", line 
11, in 
sys.exit(main())
  File 
"/Users/karthikeyansingaravelan/stuff/python/py37-venv/lib/python3.7/site-packages/pip/_internal/__init__.py",
 line 78, in main
return command.main(cmd_args)
  File 
"/Users/karthikeyansingaravelan/stuff/python/py37-venv/lib/python3.7/site-packages/pip/_internal/cli/base_command.py",
 line 176, in main
status = self.run(options, args)
  File 
"/Users/karthikeyansingaravelan/stuff/python/py37-venv/lib/python3.7/site-packages/pip/_internal/commands/show.py",
 line 47, in run
results, list_files=options.files, verbose=options.verbose):
  File 
"/Users/karthikeyansingaravelan/stuff/python/py37-venv/lib/python3.7/site-packages/pip/_internal/commands/show.py",
 line 145, in print_results
logger.info("Author: %s", dist.get('author', ''))
Message: 'Author: %s'
Arguments: ('\u0141ukasz Langa',)
Author-email: luk...@langa.pl
License: UNKNOWN

--
nosy: +eryksun

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36693] Reversing large ranges results in a minor type inconsistency

[Paul Ganssle]

Paul Ganssle  added the comment:

I have edited the title to be a bit more and give more context. Donald, if you 
feel I have misrepresented your issue, please feel free to tweak it further.

--
nosy: +p-ganssle
title: Minor inconsistancy with types. -> Reversing large ranges results in a 
minor type inconsistency

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36694] Excessive memory use or memory fragmentation when unpickling many small objects

[Paul Ellenbogen]

Change by Paul Ellenbogen :


Removed file: https://bugs.python.org/file48278/dump.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36694] Excessive memory use or memory fragmentation when unpickling many small objects

[Paul Ellenbogen]

Change by Paul Ellenbogen :


Removed file: https://bugs.python.org/file48281/dump.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36694] Excessive memory use or memory fragmentation when unpickling many small objects

[Paul Ellenbogen]

Change by Paul Ellenbogen :


Added file: https://bugs.python.org/file48282/dump.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36694] Excessive memory use or memory fragmentation when unpickling many small objects

[Paul Ellenbogen]

Paul Ellenbogen  added the comment:

Good point. I have created a new version of dump that uses random() instead. 
float reuse explains the getsizeof difference, but there is still a significant 
memory usage difference. This makes sense to me because the original code I saw 
this issue in is more analogous to random()

--
Added file: https://bugs.python.org/file48281/dump.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36694] Excessive memory use or memory fragmentation when unpickling many small objects

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The difference is because in the first case all floats are the same float 
object 0.0, but in the second case they are different objects. For more 
reaĺistic comparison use different floats (for example random()).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36693] Minor inconsistancy with types.

[SilentGhost]

Change by SilentGhost :


--
nosy: +christian.heimes
versions: +Python 3.7, Python 3.8 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36694] Excessive memory use or memory fragmentation when unpickling many small objects

[Paul Ellenbogen]

Change by Paul Ellenbogen :


Added file: https://bugs.python.org/file48280/common.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36694] Excessive memory use or memory fragmentation when unpickling many small objects

[Paul Ellenbogen]

New submission from Paul Ellenbogen :

Python encounters significant memory fragmentation when unpickling many small 
objects.

I have attached two scripts that I believe demonstrate the issue. When you run 
"dumpy.py" it will generate a large list of namedtuples, then write that list 
to a file using pickle. Before it does so, it pauses for user input. Before 
exiting the script you can view the memory usage in htop or whatever your 
preferred method is.

The "load.py" script loads the file written by dump.py. After loading the data 
is complete, it waits for user input. The memory usage at the point where the 
script is waiting for user input is (more than) twice as much in the "load" 
case as the "dump" case.

The small objects in the list I am storing have 3 values, and I have tested 
three alternative representations: tuple, namedtuple, and a custom class. The 
namedtuple and custom class both have the memory use/fragmentation issue. The 
built in tuple type does not have this issue. Using optimize in pickletools 
doesn't seem to make a difference.

Matthew Cowles from the python help list had some good suggestions, and found 
that the object size themselves, as observed by sys.getsizeof was different 
before and after pickling. Perhaps this is something other than memory 
fragmentation, or something in addition to memory fragmentation.

Although high water mark is similar for both scripts, the pickling script 
settles down on a reasonably smaller memory footprint. I would still consider 
the long run memory waste of unpickling a bug. For example in my use case I 
will run one instance of the equivalent of pickling script, then run many many 
instances of the script that unpickles.


These scripts were run with Python 3.6.7 (GCC 8.2.0) on Ubuntu 18.10.

--
components: Library (Lib)
files: dump.py
messages: 340615
nosy: Ellenbogen, alexandre.vassalotti
priority: normal
severity: normal
status: open
title: Excessive memory use or memory fragmentation when unpickling many small 
objects
type: resource usage
versions: Python 3.6
Added file: https://bugs.python.org/file48278/dump.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36694] Excessive memory use or memory fragmentation when unpickling many small objects

[Paul Ellenbogen]

Change by Paul Ellenbogen :


Added file: https://bugs.python.org/file48279/load.py

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32424] Synchronize copy methods between Python and C implementations of xml.etree.ElementTree.Element

[Gordon P. Hemsley]

Change by Gordon P. Hemsley :


--
pull_requests: +12822

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36693] Minor inconsistancy with types.

[Donald Hobson]

New submission from Donald Hobson :

Almost all of python makes the abstraction that ints are a single type of 
thing, the fact that some ints are too big to store in 8 bytes of memory is 
abstracted away. This abstraction fails when you try to reverse large ranges. 

>>> reversed(range(1<<63))

>>> reversed(range(1<<63-1))

>>> type(reversed(range(1<<63-1)))

>>> type(reversed(range(1<<63)))

>>> type(reversed(range(1<<63-2)))==type(reversed(range(1<<63-1)))
True
>>> type(reversed(range(1<<63-1)))==type(reversed(range(1<<63)))
False

--
components: Interpreter Core
messages: 340614
nosy: Donald Hobson
priority: normal
severity: normal
status: open
title: Minor inconsistancy with types.
type: behavior
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36684] codecov.io code coverage has not updated since 2019-04-13

[Gordon P. Hemsley]

Gordon P. Hemsley  added the comment:

I'm not sure when the gcc build started failing, but it looks like the commits 
at the boundary failed due to max build time. Something is apparently too slow 
to run.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36684] codecov.io code coverage has not updated since 2019-04-13

[SilentGhost]

Change by SilentGhost :


--
components: +Build
nosy: +zach.ware
versions: +Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36684] codecov.io code coverage has not updated since 2019-04-13

[Gordon P. Hemsley]

Gordon P. Hemsley  added the comment:

It looks like this may be the result of the code coverage being provided by the 
optional gcc build, which has been failing for quite a while.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36684] codecov.io code coverage has not updated since 2019-04-13

[Gordon P. Hemsley]

Gordon P. Hemsley  added the comment:

In fact, it seems the last commit on master was 
962b028b0c20abcf39594f08b1e5f8c36c4e5f6f 3 months ago, which doesn't even have 
valid report.

The previous commit, 9932a22897ef9905161dac7476e6976370e13515, was the last to 
have a functioning report.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36691] SystemExit & sys.exit : Allow both exit status and message

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

The default handler writes the message of non-integer code to stderr, but in 
case of '--help', the message should be written to stdout. So this feature is 
not applicable for it. Are there other examples where this feature can be used?

In such cases it is better to write the message explicitly and call sys.exit() 
with the required exit code. For tests you can use contextlib.redirect_stderr() 
(or manually replace sys.stderr) to capture the stderr output.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36690] A typing error in demo rpython.py

[SilentGhost]

Change by SilentGhost :


--
keywords: +easy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36690] A typing error in demo rpython.py

[SilentGhost]

SilentGhost  added the comment:

Would you be interested in submitting a PR on github? Guidelines can be found 
on https://devguide.python.org/pullrequest/ if you're not sure how to proceed.

--
nosy: +SilentGhost
stage:  -> needs patch
type: crash -> behavior
versions: +Python 3.7, Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36692] Unexpected stderr output from test_sys_settrace

[Nick Coghlan]

New submission from Nick Coghlan :

The test output from test_sys_settrace makes it look like a couple of the async 
tracing tests aren't cleaning up after themselves properly:

```
[ncoghlan@localhost cpython]$ ./python -m test test_sys_settrace
Run tests sequentially
0:00:00 load avg: 1.27 [1/1] test_sys_settrace
unhandled exception during asyncio.run() shutdown
task: ()> exception=RuntimeError("can't send non-None value to a 
just-started coroutine")>
RuntimeError: can't send non-None value to a just-started coroutine
unhandled exception during asyncio.run() shutdown
task: ()> exception=RuntimeError("can't send non-None value to a 
just-started coroutine")>
RuntimeError: can't send non-None value to a just-started coroutine

== Tests result: SUCCESS ==

1 test OK.

Total duration: 102 ms
Tests result: SUCCESS

```

If that output is actually expected as part of the test, it would be helpful if 
the test printed a message beforehand saying to expect it. Otherwise, it would 
be desirable for the test to clean up after itself and keep the messages from 
being displayed in the first place.

--
components: Tests
messages: 340608
nosy: asvetlov, ncoghlan, yselivanov
priority: low
severity: normal
stage: needs patch
status: open
title: Unexpected stderr output from test_sys_settrace
type: behavior
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue35998] test_asyncio: test_start_tls_server_1() TimeoutError on Fedora 29

[Nick Coghlan]

Change by Nick Coghlan :


--
nosy: +ncoghlan

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36691] SystemExit & sys.exit : Allow both exit status and message

[Thomas Kluyver]

New submission from Thomas Kluyver :

The SystemExit exception, and consequently the sys.exit() function,  take a 
single parameter which is either an integer exit status for the process, or a 
message to print to stderr before exiting - in which case the exit status is 
implicitly 1.

In certain situations, it would be useful to pass both an exit status and a 
message. E.g. when argparse handles '--help', it wants to display a message and 
exit successfully (status 0). You may also use specific exit codes to indicate 
different kinds of failure.

Printing the message separately before raising SystemExit is not an entirely 
satisfactory subsitute, because the message attached to the exception is only 
printed if it is unhandled. E.g. for testing code that may raise SystemExit, 
it's useful to have the message as part of the exception.

I imagine that the trickiest bit of changing this would be ensuring as much 
backwards compatibility as possible. In particular, SystemExit exceptions have 
a 'code' attribute which can be either the exit status or the message.

--
messages: 340607
nosy: takluyver
priority: normal
severity: normal
status: open
title: SystemExit & sys.exit : Allow both exit status and message
type: enhancement

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36690] A typing error in demo rpython.py

[jiawei zhou]

New submission from jiawei zhou :

Hi. There is an error in file `Tools/demo/rpython.py` at line 22.
The original statement is  `port = int(port[i+1:])`, but it will crash if the 
port is specified as parameters. 
The correct code should be `port = int(host[i+1:])`. Then the program can read 
specified port from parameter sys.argv[1].

--
components: Demos and Tools
messages: 340606
nosy: jiawei zhou
priority: normal
severity: normal
status: open
title: A typing error in demo rpython.py
type: crash

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36518] Avoid conflicts when pass arbitrary keyword arguments to Python function

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Steve, do you mind to create a PEP? I can help with examples and comments, but 
it is too hard to me to write it myself.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36689] docs: os.path.commonpath raises ValueError for different drives

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
keywords: +easy
stage:  -> needs patch
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue36689] docs: os.path.commonpath raises ValueError for different drives

[Christoph Reiter]

New submission from Christoph Reiter :

Since I just got bit by this despite reading the docs:

https://docs.python.org/3.8/library/os.path.html#os.path.commonpath

It lists various error cases where ValueError is raised but is missing the case 
where absolute paths on Windows are on different drives and I forgot to handle 
that:

  File "C:/building/msys64/mingw64/lib/python3.7\ntpath.py", line 631, in 
commonpath
raise ValueError("Paths don't have the same drive")
ValueError: Paths don't have the same drive

--
assignee: docs@python
components: Documentation
messages: 340604
nosy: docs@python, lazka
priority: normal
severity: normal
status: open
title: docs: os.path.commonpath raises ValueError for different drives
type: enhancement
versions: Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32780] ctypes: memoryview gives incorrect PEP3118 format strings for both packed and unpacked structs

[Stefan Krah]

Stefan Krah  added the comment:

Since Terry added me: Yes, this is clearly a bug, but it is a ctypes issue and 
not a memoryview issue.

ctypes issues unfortunately tend to take some time until someone reviews.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26746] struct.pack(): trailing padding bytes on x64

[Stefan Krah]

Change by Stefan Krah :


--
nosy: +Eric.Wieser

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue32424] Synchronize copy methods between Python and C implementations of xml.etree.ElementTree.Element

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Concur with Stefan. Adding the method deprecated from the born looks silly. We 
should either deprecate the copy() method in the Python implementation, or add 
an undeprecated method in the C implementation. The latter should be considered 
as a new feature.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue33777] dummy_threading: .is_alive method returns True after execution has completed

[Nate Atkinson]

Nate Atkinson  added the comment:

To be clear-- is_alive() doesn't *always* return True. It returns True until 
.join() is called.


Python 3.6.7 (default, Oct 22 2018, 11:32:17)
[GCC 8.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from dummy_threading import Thread
>>> def f(): print('foo')
...
>>> t = Thread(target=f)
>>> t.start()
foo
>>> t.is_alive()
True
>>> t.join()
>>> t.is_alive()
False



I would expect is_alive to return True while the target function is executing 
and return False after the execution has completed. Instead, .is_alive is 
continuing to return True after execution of the target function has completed.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com