[issue29881] Add a new private API clear private variables, which are initialized once, at Python shutdown

[Nick Coghlan]

Nick Coghlan added the comment:

For module level variables, I'd expect this API to need to be used in tandem 
with switching the module over to PEP 489 multi-phase initialization.

By calling the new API in their _Py_mod_exec slot implementation, extension 
modules should be able to handle multiple initialize/finalize cycles correctly: 
https://www.python.org/dev/peps/pep-0489/#subinterpreters-and-interpreter-reloading

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API clear private variables, which are initialized once, at Python shutdown

[Nick Coghlan]

Nick Coghlan added the comment:

The main potential benefit I see to keeping the special variable declaration is 
that it may help avoid a certain category of error: calling _Py_ONCE_VAR_INIT 
on a stack local pointer reference (which would leave the global array with a 
reference to nonsense). While we don't care if the once_vars are static or not, 
we do care that they're not allocated on the call stack, as otherwise they 
won't be around for Py_Finalize() to clean up.

On the other hand, _Py_SET_ONCE is nice and easy to explain "it's similar to 
_Py_SETREF, but: 1) doesn't do anything if the reference is already set; and 2) 
registers the reference to be cleaned up in Py_Finalize"

Also interesting is the fact that you can still use _Py_SETREF to change a 
reference that was initialized with _Py_SET_ONCE without breaking anything. 
From that point of view, a better name might be _Py_SET_FINALIZED, emphasising 
the fact that it registers the pointer for finalization over the fact that it's 
a no-op when run on an already set pointer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29893] create_subprocess_exec doc doesn't match software

[Torrin Jones]

New submission from Torrin Jones:

The documentation for asyncio.create_subprocess_exec says this is the 
definition . . .

asyncio.create_subprocess_exec(*args, stdin=None, stdout=None, stderr=None, 
loop=None, limit=None, **kwds)

The actual definition is this . . .

def create_subprocess_exec(program, *args, stdin=None, stdout=None,
   stderr=None, loop=None, 
   limit=streams._DEFAULT_LIMIT, **kwds)

Notice the first argument (program) at the start of the actual definition.

--
components: asyncio
messages: 290077
nosy: Torrin Jones, yselivanov
priority: normal
severity: normal
status: open
title: create_subprocess_exec doc doesn't match software
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29243] --enable-optimizations makes common build commands always need to compile from scratch

[Xiang Zhang]

Xiang Zhang added the comment:

Another complaint from #29889.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29889] test_asyncio fails always

[Xiang Zhang]

Xiang Zhang added the comment:

> Then, when doing a 'make install', it builds the entire thing again and runs 
> another profile generation run which also fails, then a new profiled build. 
> Possibly because the first profile run failed?

This is a known issue. Some build commands always clear the environment and 
rebuild. See #29243.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29862] Fix grammar typo in importlib.reload() exception

[Mandeep]

Mandeep added the comment:

Hi Brett, I'd like to take this on as my first contribution to CPython if 
that's okay with you.

--
nosy: +mandeepb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29882] Add an efficient popcount method for integers

[Case Van Horsen]

Case Van Horsen added the comment:

I like the name bit_count and I'll gladly add it to gmpy2 with the appropriate 
changes to exceptions, etc.

--
nosy: +casevh

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28041] Inconsistent behavior: Get st_nlink from os.stat() and os.scandir()

[Josh Rosenberg]

Josh Rosenberg added the comment:

This is documented behavior. Per the docs for os.DirEntry's stat method (the 
objects yielded by os.scandir):

>On Windows, the st_ino, st_dev and st_nlink attributes of the stat_result are 
>always set to zero. Call os.stat() to get these attributes.

It might be nice if those values could be cached on read (through a lazily 
initialized value on property access or the like), but this is not a bug, it's 
working as documented.

--
nosy: +josh.r

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29892] change statement for open() is splited into two part in middle of sentence.

[Jim Fasarakis-Hilliard]

Jim Fasarakis-Hilliard added the comment:

Indeed, nice catch! Submit a PR for it if you want to (if not, someone else 
will pick it up soon :-)

--
nosy: +Jim Fasarakis-Hilliard

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29892] change statement for open() is splited into two part in middle of sentence.

[OSAMU NAKAMURA]

New submission from OSAMU NAKAMURA:

In https://docs.python.org/3.6/library/functions.html#open , 
Following sentence is wrongly separated by extra asterisk.

```
FileExistsError is now raised if the file opened in exclusive
creation mode ('x') already exists.
```

This mistake is introduced by 
https://github.com/python/cpython/commit/3929499914d47365ae744df312e16da8955c90ac#diff-30d76a3dc0c885f86917b7d307ccf279

--
assignee: docs@python
components: Documentation
messages: 290070
nosy: OSAMU.NAKAMURA, docs@python
priority: normal
severity: normal
status: open
title: change statement for open() is splited into two part in middle of 
sentence.
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29890] Constructor of ipaddress.IPv*Interface does not follow documentation

[Eric V. Smith]

Eric V. Smith added the comment:

This should be easy enough to fix, at least in IPv4Interface.__init__. It needs 
to copy some of IPv4Network.__init__, dealing with address[1] and calling 
_make_netmask(). Currently, it just calls int(address[1]).

I haven't looked at IPv6Interface.

Tests are also needed, of course.

--
keywords: +easy
nosy: +eric.smith
stage:  -> needs patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29642] Why does unittest.TestLoader.discover still rely on existence of __init__.py files?

[Ned Batchelder]

Changes by Ned Batchelder :


--
nosy: +nedbat

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue28041] Inconsistent behavior: Get st_nlink from os.stat() and os.scandir()

[Mark Becwar]

Changes by Mark Becwar :


--
pull_requests: +700

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29889] test_asyncio fails always

[Thomas Knox]

Thomas Knox added the comment:

The build succeeds, but the profile generation fails (only the test_asyncio 
portion fails), then does a new profiled build after the profiling run 
completes which succeeds.

Then, when doing a 'make install', it builds the entire thing again and runs 
another profile generation run which also fails, then a new profiled build. 
Possibly because the first profile run failed?

On an RPi, this is exceedingly painful as a build/profile/build cycle takes 
about 10 hours. So doing it twice is double-unfun.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29889] test_asyncio fails always

[Zachary Ware]

Zachary Ware added the comment:

Does the build fail, or just the test in the profile generation?

--
nosy: +zach.ware

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29889] test_asyncio fails always

[Thomas Knox]

Thomas Knox added the comment:

I changed the configuration flags to be:

pi@pi3:~/Source/Python-3.6.1 $ ./configure --enable-optimizations 
--enable-loadable-sqlite-extensions --disable-ipv6 --with-system-expat 
--with-system-ffi --with-threads

The profiling filed again with:

0:04:47 [ 25/405] test_asyncio
Executing .start() done, defined at 
/home/pi/Source/Python-3.6.1/Lib/test/test_asyncio/test_pep492.py:150> 
result=None created at 
/home/pi/Source/Python-3.6.1/Lib/asyncio/base_events.py:446> took 1.187 seconds
0:08:45 [ 26/405] test_asyncore -- test_asyncio failed in 3 min 59 sec

But at least it failed 2.5 minutes faster. :)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29804] test_ctypes test_pass_by_value fails on arm64 (aarch64) architecture

[Ned Deily]

Ned Deily added the comment:

Technically speaking, we do not officially support arm64 in our release process 
as we have no arm64 buildbots nor an identified core developer for the 
platform; see PEP 11 for the policy details.  So there is no place to test a 
fix if there was one identified.  That said, it would be good to fix the 
problem if someone is willing to provide a fix and test it or determine that 
the issue here is a libffi problem.  It would be even better if we could make 
arm64 an officially supported platform as outlined.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29891] urllib.request.Request accepts but doesn't check bytes headers

[Maciej Szulik]

Changes by Maciej Szulik :


--
nosy: +maciej.szulik

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29642] Why does unittest.TestLoader.discover still rely on existence of __init__.py files?

[Andrei Fokau]

Andrei Fokau added the comment:

I was wrong. The ticket can be closed now.

--
nosy: +Andrei Fokau2

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29891] urllib.request.Request accepts but doesn't check bytes headers

[Ezio Melotti]

New submission from Ezio Melotti:

urllib.request.Request allows the user to create a request object like:
  req = Request(url, headers={b'Content-Type': b'application/json'})

When calling urlopen(req, data), urllib will check if a 'Content-Type' header 
is present and fail to recognize b'Content-Type' because it's bytes.
urrlib will therefore add the default Content-Type 
'application/x-www-form-urlencoded', and the request will then be sent with 
both Content-Types.  This will result in difficult-to-debug errors because the 
server will sometimes pick one and sometimes the other, depending on the order.

urllib should either reject bytes headers, or check for both bytes and strings. 
 The docs also don't seem to specify that the headers should be strings.

--
components: Library (Lib)
messages: 290063
nosy: ezio.melotti, orsenthil
priority: normal
severity: normal
stage: test needed
status: open
title: urllib.request.Request accepts but doesn't check bytes headers
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29890] Constructor of ipaddress.IPv*Interface does not follow documentation

[Ilya Kulakov]

New submission from Ilya Kulakov:

As per documentation, it should understand the same arguments as IPv*Network.

Unfortunately it does not recognize netmask in string form. Hence the following 
code will fail:

ipaddress.ip_interface(('192.168.1.10', '255.255.255.0'))

while the following will work:

ipaddress.ip_network(('192.168.1.10', '255.255.255.0'), strict=False)

--
messages: 290062
nosy: Ilya.Kulakov
priority: normal
severity: normal
status: open
title: Constructor of ipaddress.IPv*Interface does not follow documentation
type: behavior
versions: Python 3.5, Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22962] ipaddress: Add optional prefixlen argument to ip_interface and ip_network

[Ilya Kulakov]

Ilya Kulakov added the comment:

You can initialize ip_interface via a tuple of 2 elements: IP address and a 
prefix (prefixlen or string representation of a netmask).

I believe the issue can be closed now.

--
nosy: +Ilya.Kulakov

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29889] test_asyncio fails always

[Ned Deily]

Ned Deily added the comment:

There have been other reports (e.g. Issue29712) of failures when using 
--enable-optimizations with --enable-shared.  Can you see if removing 
--enable-shared makes a difference for your configurations?

--
components: +asyncio -Tests
nosy: +haypo, ned.deily, yselivanov
type: compile error -> 

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue22005] datetime.__setstate__ fails decoding python2 pickle

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +699

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29889] test_asyncio fails always

[Thomas Knox]

New submission from Thomas Knox:

Downloaded Python 3.6.1 source code onto CentOS 7.3 (64 bit), Fedora 25 (64 
bit), Ubuntu 16.10 (64 bit) and Raspberry Pi 8.0 (32 bit).

Configured with
./configure --enable-shared --enable-optimizations 
--enable-loadable-sqlite-extensions --disable-ipv6 --with-system-expat 
--with-system-ffi --with-threads

On every platform, when running the profile generation code, test_asyncio fails 
with this error message:

0:06:45 [ 25/405] test_asyncio
Executing .start() done, defined at 
/home/pi/Source/Python-3.6.1/Lib/test/test_asyncio/test_pep492.py:150> 
result=None created at 
/home/pi/Source/Python-3.6.1/Lib/asyncio/base_events.py:446> took 2.106 seconds
0:13:11 [ 26/405] test_asyncore -- test_asyncio failed in 6 min 27 sec

--
components: Tests
messages: 290059
nosy: Thomas Knox
priority: normal
severity: normal
status: open
title: test_asyncio fails always
type: compile error
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API clear private variables, which are initialized once, at Python shutdown

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Some code (_PyUnicode_FromId, bool_repr, create_filter) could be simpler if 
make _PY_ONCEVAR_INIT returning the value (NULL in case of error).

The signature of _PY_ONCEVAR_INIT() is the same as of _Py_SETREF(). If var == 
NULL both `_PY_ONCEVAR_INIT(var, expr)` and `_Py_SETREF(var, expr)` are 
equivalent to just `var = expr`. Maybe rename _PY_ONCEVAR_INIT to _Py_SET_ONCE?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16510] Using appropriate checks in tests

[Mark Lawrence]

Changes by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20548] Use specific asserts in warnings and exceptions tests

[Mark Lawrence]

Changes by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20554] Use specific asserts in optparse test

[Mark Lawrence]

Changes by Mark Lawrence :


--
nosy:  -BreamoreBoy

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29888] The link referring to "Python download page" is broken

[Kinebuchi Tomohiko]

New submission from Kinebuchi Tomohiko:

The download page [1]_ contains a link intended to refer to the release page of 
the corresponding Python version [2]_.

.. [1] `Download Python 2.7.13 Documentation 
`_

.. [2] e.g. `Python 2.7.8 Release 
`_

Although, this link is broken for three reasons.

1. Wrong template syntax

   `Present code 
`_::

  {% trans download_page="https://www.python.org/download/releases/{{ 
release[:5] }}/" %}HTML Help
  (.chm) files are made available in the "Windows" section
  on the Python download page.{% endtrans 
%}

   Fixed code::

  {% trans download_page="https://www.python.org/download/releases/; + 
release[:5] + "/" %}HTML Help
  (.chm) files are made available in the "Windows" section
  on the Python download page.{% endtrans 
%}

2. Unexpected version number

   The URL contains a Python version string (i.e. ``release[:5]``), but for 
Python 2.7.13, ``release[:5]`` evaluates to ``'2.7.1'`` which obviously wrong 
as a version string.

3. Non-existent release pages for some versions

   www.python.org has pages which URLs are 
https://www.python.org/download/releases// with  = 2.7.1--8, 
although has no pages with  = 2.7.9 and so on.

   Is https://www.python.org/downloads/release/python-2713/ an appropriate page 
to refer?

--
assignee: docs@python
components: Documentation
messages: 290057
nosy: cocoatomo, docs@python
priority: normal
severity: normal
status: open
title: The link referring to "Python download page" is broken
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API clear private variables, which are initialized once, at Python shutdown

[STINNER Victor]

STINNER Victor added the comment:

I modified more modules to use the new API. I'm not sure that using the API in 
modules is safe.

It's safe to use the API in functions which begins with trying to initialize 
the variable. In such case, if the variable is cleared, calling the function 
later initialize again the variable and it's fine.

When for module variables only initialized when the module is initialized, 
there is a risk that a module function tries to access a variable which has 
been cleared previously during Python shutdown.

See for example changes in the _datetime module.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API clear private variables, which are initialized once, at Python shutdown

[STINNER Victor]

Changes by STINNER Victor :


--
title: Add a new private API for "static C variables" (_PyStaticVar) to clear 
them at exit -> Add a new private API clear private variables, which are 
initialized once, at Python shutdown

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23699] Add a macro to ease writing rich comparisons

[Charalampos Stratakis]

Charalampos Stratakis added the comment:

Sent a PR against the master branch. What do you think about it?

Would it make sense as well for python 3.6 now?

--
nosy: +cstratak

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23699] Add a macro to ease writing rich comparisons

[Charalampos Stratakis]

Changes by Charalampos Stratakis :


--
pull_requests: +698

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16510] Using appropriate checks in tests

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +697

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20547] Use specific asserts in bigmem tests

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +696

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29883] asyncio: Windows Proactor Event Loop UDP Support

[Brett Cannon]

Changes by Brett Cannon :


--
nosy: +giampaolo.rodola, haypo

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20552] Use specific asserts in bytes tests

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

I think this should only go to 3.7 now.

--
nosy: +Mariatta
versions: +Python 3.7 -Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20552] Use specific asserts in bytes tests

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +695

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20550] Use specific asserts in collections tests

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +694

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20545] Use specific asserts in unicode tests

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +692

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20548] Use specific asserts in warnings and exceptions tests

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +693

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20554] Use specific asserts in optparse test

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +691

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29887] test_normalization doesn't work

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

It needs to fetch http://www.pythontest.net/unicode/9.0.0/NormalizationTest.txt 
(8.0.0 in 3.5) but get the 404 error.

--
components: Tests
messages: 290053
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: test_normalization doesn't work
type: behavior
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[STINNER Victor]

STINNER Victor added the comment:

About _PY_ONCEVAR_INIT() vs _Py_IDENTIFIER.

Using _Py_IDENTIFIER works well if you have an API accepting directly a 
_Py_IDENTIFIER*. If you call functions requesting a PyObject*, you need to call 
_PyUnicode_FromId() and test for failure. If you start by calling 
_PyUnicode_FromId() when the object is not initialized yet, above you have to 
use var.object, whereas previously Serhiy and Nick weren't confortable with 
this specific case.

I prefer to use _PY_ONCEVAR_INIT() to keep a regular PyObject* variable and 
makes the code simpler.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29886] links between binascii.{un,}hexlify / bytes.{,to}hex

[Roundup Robot]

Changes by Roundup Robot :


--
pull_requests: +689

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29886] links between binascii.{un,}hexlify / bytes.{,to}hex

[chrysn]

New submission from chrysn:

The function binascii.{un,}hexlify and bytes.{,to}hex do almost the same things 
(plus/minus details of whether they accept whitespace, and whether the 
hex-encoded data is accepted/returned as strings or bytes).

I think that it would help users to point that out in the documentation, eg. by 
adding a "Similar functionality is provided by the ... function." lines at the 
ends of the functions' documentations.

--
assignee: docs@python
components: Documentation
messages: 290050
nosy: chrysn, docs@python
priority: normal
severity: normal
status: open
title: links between binascii.{un,}hexlify / bytes.{,to}hex
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21895] signal.pause() and signal handlers don't react to SIGCHLD in non-main thread

[Guido van Rossum]

Changes by Guido van Rossum :


--
nosy:  -gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[STINNER Victor]

STINNER Victor added the comment:

3rd round of the API:

static PyObject *assertion_error = NULL;
...
if (_PY_ONCEVAR_INIT(assertion_error,
 PyUnicode_InternFromString("AssertionError"))) {
return 0;
}
...

(Not the best example, _Py_IDENTIFIER() would be more appropriate here ;-))

--

I just added a second commit to remove the next field from _Py_Identifier and 
reuse _PyOnceVar_Set() in _PyUnicode_FromId().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think we can get rid of _Py_ONCEVAR(). Just keep current declarations. 
_PY_ONCEVAR_INIT() can work even with non-static global variables. It could 
work even with non-PyObject pointers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29879] typing.Text not available in python 3.5.1

[Charles Bouchard-Légaré]

Changes by Charles Bouchard-Légaré :


--
pull_requests: +688

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29640] _PyThreadState_Init and fork race leads to inconsistent key list

[Charalampos Stratakis]

Changes by Charalampos Stratakis :


--
pull_requests: +687

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[STINNER Victor]

STINNER Victor added the comment:

About the pull request: to be clear, I know that some modified local variables 
should use _Py_IDENTIFIER() rather than _Py_ONCEVAR(), but I chose to use 
_Py_ONCEVAR() just to give examples of the API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[STINNER Victor]

STINNER Victor added the comment:

> * do we ever declare non-statics this way? (OTOH, if we do, this could be an 
> opportunity to hide them behind read-only accessor functions)
> * do we ever declare more specific types than PyObject * this way? (OTOH, if 
> that's the uncommon case, requiring a cast to the more specific type probably 
> wouldn't hurt)

Well, with my 2nd API, I'm not sure that the macro to declare a variable is 
very simple:

+/* Declare a static PyObject* variable which is only initialized once.
+   _PyOnceVar_Fini() will clear the variable at Python finalization. */
+#define _Py_ONCEVAR(var) \
+static PyObject* var = NULL

Technically, the variable doesn't have to be static. But do we want to use this 
API for global variables initialized once? It would increase the memory usage, 
whereas currently we have specialized code like _PyUnicode_Fini() which clears 
its unicode_empty singleton. I don't want to touch this code. At least, not yet.

If you want to support other types than PyObject*, _PY_ONCEVAR_INIT() macro can 
cast the first argument to PyObject*.

I would prefer to start with something simpler, and discuss case by case for 
other variables.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[Nick Coghlan]

Nick Coghlan added the comment:

Passing var_decl was based on not knowing the answer to two questions:

* do we ever declare non-statics this way? (OTOH, if we do, this could be an 
opportunity to hide them behind read-only accessor functions)
* do we ever declare more specific types than PyObject * this way? (OTOH, if 
that's the uncommon case, requiring a cast to the more specific type probably 
wouldn't hurt)

As far as stack usage goes, all _Py_OnceVar instances should be in the data 
segment when using the linked list approach, so they shouldn't impact stack 
usage, and doing so retains the benefit of avoiding dynamic memory allocation 
just to track the static variable declarations. Since the macro is 
dereferencing the address of a static variable in the initialiser of another 
static variable, that shouldn't require any runtime stack space either. OTOH, I 
haven't actually ever tried compiling a macro like that, so it's entirely 
possible compilers won't like it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26657] Directory traversal with http.server and SimpleHTTPServer on windows

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +686

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29854] Segfault when readline history is more then 2 * history size

[Martin Panter]

Martin Panter added the comment:

Gnu Readline comes includes its own documentation (e.g. 
/usr/share/info/history.info.gz on my computer). It is also at 
.

Perhaps the history_base value is relevant; see some of the comments starting 
at .

It would be interesting to see if Apple Editline is affected. According to the 
comment in the get_history_item function, history_get might crash before 
returning the null pointer. Is there some other workaround that avoids calling 
history_get?

--
nosy: +martin.panter

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The new API rather combines pthread_once() and pthread_cleanup_push().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20631] python readline module crashing on NULL access

[Martin Panter]

Changes by Martin Panter :


--
superseder:  -> Make libedit support more generic; port readline / libedit to 
FreeBSD

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20631] python readline module crashing on NULL access

[Martin Panter]

Changes by Martin Panter :


--
resolution:  -> duplicate
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue20631] python readline module crashing on NULL access

[Martin Panter]

Martin Panter added the comment:

Closing in favour of Issue 13501, since the report was apparently about using a 
non-Apple Editline rather than Gnu Readline. However see also Issue 29854, 
where the same symptom is seen with Gnu Readline, and it will probably get the 
same fix.

--
status: pending -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue6532] thread.get_ident() should return unsigned value

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +685

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

> "use gc_next/gc_refs for managing a linked list"

Forgot about this idea. Not all objects have the GC header.

But if they have it they could be removed from the GC list and added in the 
separate list since in any case the garbage collector shouldn't destroy objects 
referenced by static variables.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[STINNER Victor]

STINNER Victor added the comment:

I wrote a new different API: https://github.com/python/cpython/pull/780

New C API for variables only initialized once to be able to clear
them at exit:

New macro _Py_ONCEVAR(var) to declare a variable
New macro _PY_ONCEVAR_INIT(var, expr) to initialize a variable
once
New function _PyOnceVar_Set() to explicitly set a variable once
to initialize it
New _PyOnceVar_Fini() function clearing all variables (initialized
once) at exit

Variables keep their PyObject* type, but the API hides an internal C array 
tracking all Python objects to Py_DECREF them at exit in _PyOnceVar_Fini().

I used Nick's naming scheme since it seems like pthread has an existing API 
which is similar, but different (my API doesn't require an initialization 
callback).

I really prefer the second API using PyObject*.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[STINNER Victor]

Changes by STINNER Victor :


--
pull_requests: +684

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

My apologies Victor. It seems that the browser on my netbook shown only the 
part of changes and I seen only the change in array_array___reduce_ex__().

I like Nick's idea for hiding the indirection, but passing var_decl doesn't 
look having much sense to me. It should always be "static PyObject *".

I don't have a preference for the name, but the purpose of a new API is not 
just to ensure that a piece of initialization code is executed at most once, 
but that the object will be destroyed and the reference will be cleared after 
finalizing the interpreter. Seems this is different from the purpose of 
pthread_once().

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[STINNER Victor]

STINNER Victor added the comment:

Serhiy: "use gc_next/gc_refs for managing a linked list"

I'm not sure that I understand your idea. These fields are already used 
internally by the garbage collector. If I modify one of these fields, it would 
corrupt the GC, no?

Serhiy: "You can use a dynamic array of PyObject** instead of a linked list for 
collecting references to "static variables""

Ah yes, I like the idea of using a single array to track all variables. An 
array reduces memory fragmentation and is simple to maintain, since the API 
only needs two operations: list.append(obj) and list.clear(). The API already 
requires to check for errors, so another memory allocation failure wouldn't be 
suprising.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[STINNER Victor]

STINNER Victor added the comment:

#define _Py_ONCE_VAR(var_decl, var) \
  var_decl var = NULL;
  static _Py_OnceVar var ## _once_meta = {.next = NULL, .obj_ref = 
(PyObject **) }

Yeah, I had a similar idea, but I fear that it will increase the usage of the C 
stack memory.

See the issue #28858: my old _PyObject_CallArg1() macro allocated an implicit 
array on the stack and increased the stack usage, whereas the purpose of the 
macro was to *reduce* the stack usage (just the opposite!). So I removed the 
macro.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[STINNER Victor]

STINNER Victor added the comment:

Serhiy: "The patch contains an example of using _Py_STATICVAR(). But it doesn't 
use _PY_STATICVAR_INIT()."

I'm not sure that I understand your comment. All modified code use 
_Py_STATICVAR() to declare the declare. All modified code use 
_PY_STATICVAR_INIT() to initialize the variable, except of 
array_array___reduce_ex__() which uses explicitly _PyStaticVar_Set() beause its 
initialization code is more complex.

_PyStaticVar_Set() is the low-level function, I would prefer to avoid it since 
it fails with an assertion error if it's called twice. _PY_STATICVAR_INIT() 
helper should be perfer since it makes the code shorter, but it's similar to 
simple expressions like PyUnicode_FromString("\n").

I tried to include examples of usage of both APIs to give you an idea of the 
API.

I chose to *not* patch the whole Python code base, because I would like to 
first get a review of a the API. It seems like alternatives have been proposed 
;-)


> I like the idea in general, but I want to see more examples.

Which kind of other examples do you need?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue25658] PyThread assumes pthread_key_t is an integer, which is against POSIX

[STINNER Victor]

STINNER Victor added the comment:

Hi people working on the new TLS API: I would like your opinion on a related 
API, issue #29881: Add a new private API for "static C variables" 
(_PyStaticVar) to clear them at exit !

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[Nick Coghlan]

Nick Coghlan added the comment:

I like this idea in principle, and suspect it may be helpful in the 
implementation of the new thread-specific-storage API proposed in PEP 539 (also 
see http://bugs.python.org/issue25658 ).

How would you feel about calling it _Py_ONCE_VAR? The use case here is quite 
similar to the concepts behind the pthread_once API (just with the lifecycle 
tied to Py_Initialize/Py_Finalize rather than the C level process), and it's 
the "initialise-on-first-use" behaviour that's significant here, moreso than 
the fact that the typical storage target is a static variable.

As far as the linked list goes, the nice aspect of Victor's approach is that it 
doesn't need to do any additional runtime memory allocations - all the storage 
involved is still statically allocated in the modules that initialise the 
values, there are just some extra  pointer assignments to link everything 
together (with the GIL protecting against race conditions).

However, the user visible ".obj" indirection could still be avoided at the cost 
of an additional pointer per entry:

typedef struct _Py_OnceVar {
struct _Py_OnceVar *next;
PyObject **obj_ref;
} _Py_OnceVar;

#define _Py_ONCE_VAR(var_decl, var) \
  var_decl var = NULL;
  static _Py_OnceVar var ## _once_meta = {.next = NULL, .obj_ref = 
(PyObject **) }

Intended declaration:

_Py_ONCE_VAR(static PyObject *, array_reconstructor);

_Py_ONCE_VAR_INIT would similarly be adjusted to assign the supplied value to 
"var", while also setting "var_once_meta.next" to hook the value into the 
linked list.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue16572] Bad multi-inheritance support in some libs like threading or multiprocessing

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The _Verbose class and verbose arguments for threading classes and functions 
were removed in issue13550. Thus this is 2.7-only issue now.

--
nosy: +haypo
versions:  -Python 3.3, Python 3.4

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27572] Support bytes-like objects when base is given to int()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Created PR 779 for the deprecation.

--
stage:  -> patch review
versions: +Python 3.7 -Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27572] Support bytes-like objects when base is given to int()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +683

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21895] signal.pause() and signal handlers don't react to SIGCHLD in non-main thread

[Antoine Pitrou]

Antoine Pitrou added the comment:

That said, the pthread_kill() solution deserves testing as well.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue21895] signal.pause() and signal handlers don't react to SIGCHLD in non-main thread

[Antoine Pitrou]

Antoine Pitrou added the comment:

> Using set_wakeup_fd would fix them... except that it generates spurious 
> warning messages when the wakeup fd buffer is full, and there's no way to 
> stop it

Are you using a pipe or a socket to set_wakeup_fd?  Pipes have rather small 
buffers.  In any case, since, as you mention, Tornado and Twisted use it, and 
they never complained, perhaps your example is too contrived or artificial.

--
nosy: +gvanrossum

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29885] Allow GMT timezones to be used in datetime.

[Martin Panter]

Martin Panter added the comment:

Does the “astimezone” method work for you? 


>>> from datetime import *
>>> aedt = timezone(+timedelta(hours=11))
>>> local = datetime.now(aedt)
>>> format(local)
'2017-03-23 19:14:41.410334+11:00'
>>> gmt = local.astimezone(timezone.utc)
>>> format(gmt)
'2017-03-23 08:14:41.410334+00:00'

--
nosy: +martin.panter
resolution:  -> works for me

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29885] Allow GMT timezones to be used in datetime.

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Use pytz (https://pypi.python.org/pypi/pytz).

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API for "static C variables" (_PyStaticVar) to clear them at exit

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

The patch contains an example of using _Py_STATICVAR(). But it doesn't use 
_PY_STATICVAR_INIT(). _PY_STATICVAR_INIT() can be used if extract the code of 
getting _array_reconstructor into separate function.

I like the idea in general, but I want to see more examples. Ideally -- the 
patch should use a new API for *all* static PyObject* variables. This will help 
to design good API.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29885] Allow GMT timezones to be used in datetime.

[Decorater]

New submission from Decorater:

I noticed that there is no ways to convert local times to GMT if I realize that 
some other object (sometimes from a server) is using GMT and they happen to be 
ahead of my current time. As such it would be great if one can convert their 
current time that can be in an datetime object to an GMT time to see how much 
time has passed since something happened on their zone if so desired (otherwise 
can cause undesired or undefined consequences).

--
components: Library (Lib)
messages: 290027
nosy: Decorater
priority: normal
severity: normal
status: open
title: Allow GMT timezones to be used in datetime.
versions: Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com