[issue30045] Bad parameter name in re.escape()

[Mandeep Bhutani]

Mandeep Bhutani added the comment:

I agree that pattern seems right. However I think that the parameter in the 
docs should be changed from string to pattern for consistency purposes.

--
nosy: +mandeepb

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29692] contextlib.contextmanager may incorrectly unchain RuntimeError

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
pull_requests: +1244

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26985] Information about CodeType in inspect documentation is outdated

[Xiang Zhang]

Changes by Xiang Zhang :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26985] Information about CodeType in inspect documentation is outdated

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset 72b1d419ac5f7cd9ef82ffd2ffe21aa9b34e21d2 by Xiang Zhang in branch 
'3.5':
bpo-26985: Add missing info of code object in inspect documentation (GH-1090) 
(GH-1100)
https://github.com/python/cpython/commit/72b1d419ac5f7cd9ef82ffd2ffe21aa9b34e21d2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29861] multiprocessing Pool keeps objects (tasks, args, results) alive too long

[Xiang Zhang]

Xiang Zhang added the comment:

Hi, Antoine, after this change, I sometimes see tests fail for 3.5 branch, for 
example 
http://buildbot.python.org/all/builders/x86%20Ubuntu%20Shared%203.5/builds/194/steps/test/logs/stdio.

--
nosy: +xiang.zhang

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26985] Information about CodeType in inspect documentation is outdated

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset 14944c62300f741488c2f28cb91ad1e3fef7343b by Xiang Zhang in branch 
'3.6':
bpo-26985: Add missing info of code object in inspect documentation (GH-1090) 
(GH-1099)
https://github.com/python/cpython/commit/14944c62300f741488c2f28cb91ad1e3fef7343b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30047] Typos in Doc/library/select.rst

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

Backported to 3.6.
Closing this now.
Thanks all :)

--
resolution:  -> fixed
stage: backport needed -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30047] Typos in Doc/library/select.rst

[Mariatta Wijaya]

Mariatta Wijaya added the comment:


New changeset f0c416799b33742af36d7c6608b9a422615fb3d2 by Mariatta in branch 
'3.6':
[3.6] bpo-30047: Fix a typo in Doc/library/select.rst (GH-1086) (GH-1098)
https://github.com/python/cpython/commit/f0c416799b33742af36d7c6608b9a422615fb3d2


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26985] Information about CodeType in inspect documentation is outdated

[Xiang Zhang]

Changes by Xiang Zhang :


--
pull_requests: +1243

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26985] Information about CodeType in inspect documentation is outdated

[Xiang Zhang]

Changes by Xiang Zhang :


--
pull_requests: +1242

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30047] Typos in Doc/library/select.rst

[Mariatta Wijaya]

Changes by Mariatta Wijaya :


--
pull_requests: +1241

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30063] DeprecationWarning in json/encoder.py

[Paul Durack]

Paul Durack added the comment:

I have attempted to quieten these using:

In [2]: import warnings
   ...: warnings.filterwarnings('ignore', category=DeprecationWarning)
In [3]: import vcs
/home/user/anaconda2/envs/cdatcmornclnco/lib/python2.7/json/encoder.py:207: 
DeprecationWarning: Interpreting naive datetime as local 2017-04-12 
19:45:31.251897. Please add timezone info to timestamps.
  chunks = self.iterencode(o, _one_shot=True)

However, this doesn't seem to be working

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30063] DeprecationWarning in json/encoder.py

[Paul Durack]

New submission from Paul Durack:

I have started receiving the following warnings which are starting to prevent 
an ipython session from functioning:

/home/user/anaconda2/envs/cdatcmornclnco/lib/python2.7/json/encoder.py:207: 
DeprecationWarning: Interpreting naive datetime as local 2017-04-12 
17:15:36.235571. Please add timezone info to timestamps.
  chunks = self.iterencode(o, _one_shot=True)
/home/user/anaconda2/envs/cdatcmornclnco/lib/python2.7/json/encoder.py:207: 
DeprecationWarning: Interpreting naive datetime as local 2017-04-12 
17:15:36.267401. Please add timezone info to timestamps.
  chunks = self.iterencode(o, _one_shot=True)

The only way I can continue is to terminate the ipython shell and open a new 
instance.

Can someone tell me what I need to do to solve the issue? Is there a json 
import somewhere that requires some new arguments?

--
components: Library (Lib)
messages: 291582
nosy: Paul Durack
priority: normal
severity: normal
status: open
title: DeprecationWarning in json/encoder.py
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26985] Information about CodeType in inspect documentation is outdated

[Xiang Zhang]

Xiang Zhang added the comment:


New changeset a6902e662c18dc837d40664eaafe50a44aae6366 by Xiang Zhang in branch 
'master':
bpo-26985: Add missing info of code object in inspect documentation (GH-1090)
https://github.com/python/cpython/commit/a6902e662c18dc837d40664eaafe50a44aae6366


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30048] If a task is canceled at the right moment, the cancellation is ignored

[INADA Naoki]

Changes by INADA Naoki :


--
pull_requests: +1240

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30048] If a task is canceled at the right moment, the cancellation is ignored

[INADA Naoki]

INADA Naoki added the comment:

> In Evgeny's example the 'task' is 'coro1' (not 'coro2'). It has plenty of 
> yield points after being cancelled.

Since coro1 waiting coro2 when cancelling, Task(coro1).cancel() redirect to 
Task(coro2).cancel().

But I was wrong about "CancelledError is thrown to coro2."
Task(coro2) is cancelled in last step.  CancelledError is not thrown to coro2 
actually.
So there is chance to detect it.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30048] If a task is canceled at the right moment, the cancellation is ignored

[Yury Selivanov]

Yury Selivanov added the comment:

In Evgeny's example the 'task' is 'coro1' (not 'coro2'). It has plenty of yield 
points after being cancelled.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30048] If a task is canceled at the right moment, the cancellation is ignored

[INADA Naoki]

INADA Naoki added the comment:

> The problem is that the task doesn't catch CancelledError, yet it disappears.

The problem is CancelledError is not raised, even it's thrown.
Task can't catch exception not raised.  See below example which demonstrates 
how task works.

---
from asyncio import CancelledError

cancelled = False

def coro():
global cancelled
print(1)
yield (1,)
print("cancel")
cancelled = True
#print(2)
#yield (2,)  # uncomment this line makes cancel success.

c = coro()

while True:
try:
if cancelled:
r = c.throw(CancelledError)
else:
r = c.send(None)
except StopIteration:
print("end")
break
except CancelledError:
print("cancelled")
break

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30048] If a task is canceled at the right moment, the cancellation is ignored

[Evgeny Kapun]

Evgeny Kapun added the comment:

The problem is that the task doesn't catch CancelledError, yet it disappears.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30062] datetime in Python 3.6+ no longer respects 'TZ' environment variable

[Adam Williamson]

Adam Williamson added the comment:

Hmm, after a bit more poking I found this:

https://docs.python.org/3/library/time.html#time.tzset

"Note

Although in many cases, changing the TZ environment variable may affect the 
output of functions like localtime() without calling tzset(), this behavior 
should not be relied on."

It seems like that's kinda what we're dealing with here. If I extend my tests 
to change TZ, call the test function, then call `time.tzset()` and call the 
test function again, the *second* call to the test function gives the different 
result, i.e. the `time.tzset()` call does what it claims and changes Python's 
conception of the 'current' timezone.

So while that note has been there all along, it seems like the behaviour 
actually changed between 3.5 and 3.6, and a change to 'TZ' is now less likely 
to be respected without a `tzset()` call. But given the doc note, perhaps that 
can't be considered a bug.

anaconda doesn't call `time.tzset()` anywhere at present. It's also 
multi-threaded, so making sure all the threads call `time.tzset()` after any 
thread has changed what the 'current' timezone is will be lots of fun to 
implement, I guess :/

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30052] URL Quoting page links to function Bytes instead of defintion

[Cheryl Sabella]

Cheryl Sabella added the comment:

I've looked into this a little more and it seems that, in all instances, the 
:class:`bytes` and :class:`bytearray` mark-up link to the Built-In Functions 
page instead of the Built-In Types page.

It also looks like the Built-In Functions page has specific code for many types 
(dict, frozenset, memoryview, list, tuple, str, etc) so that it wouldn't link 
to this (the functions) page.  However, bytes and bytearray aren't part of that 
list.  I don't know if they were excluded on purpose or if their links should 
behave like the other types.

Please let me know if this should change.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30062] datetime in Python 3.6+ no longer respects 'TZ' environment variable

[Berker Peksag]

Changes by Berker Peksag :


--
nosy: +belopolsky

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30062] datetime in Python 3.6+ no longer respects 'TZ' environment variable

[Adam Williamson]

New submission from Adam Williamson:

I can't figure out yet why this is, but it's very easy to demonstrate:

[adamw@adam anaconda (time-log %)]$ python35 
Python 3.5.2 (default, Feb 11 2017, 18:09:24) 
[GCC 7.0.1 20170209 (Red Hat 7.0.1-0.7)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import datetime
>>> os.environ['TZ'] = 'America/Winnipeg'
>>> datetime.datetime.fromtimestamp(0)
datetime.datetime(1969, 12, 31, 18, 0)
>>> os.environ['TZ'] = 'Europe/London'
>>> datetime.datetime.fromtimestamp(0)
datetime.datetime(1970, 1, 1, 1, 0)
>>> 

[adamw@adam anaconda (time-log %)]$ python3
Python 3.6.0 (default, Mar 21 2017, 17:30:34) 
[GCC 7.0.1 20170225 (Red Hat 7.0.1-0.10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> import datetime
>>> os.environ['TZ'] = 'America/Winnipeg'
>>> datetime.datetime.fromtimestamp(0)
datetime.datetime(1969, 12, 31, 16, 0)
>>> os.environ['TZ'] = 'Europe/London'
>>> datetime.datetime.fromtimestamp(0)
datetime.datetime(1969, 12, 31, 16, 0)
>>> 

That is, when deciding what timezone to use for operations that involve one, if 
the 'TZ' environment variable was set, Python 3.5 would use the timezone it was 
set to. Python 3.6 does not, it ignores it.

As you can see, if I twiddle the 'TZ' setting and call 
`datetime.datetime.fromtimestamp(0)` repeatedly under Python 3.5, I get 
different results - each one is the wall clock time at the epoch (timestamp 0) 
in the timezone specified as 'TZ'. If I do the same on Python 3.6, the 'TZ' 
setting is ignored and I always get the same result (the wall clock time of 
'the epoch' in Vancouver, which is my real timezone, and which I guess is being 
picked up from /etc/localtime or whatever).

This wound up causing a problem in the Fedora / Red Hat installer, anaconda:

https://bugzilla.redhat.com/show_bug.cgi?id=1433560

The 'current time zone' can be changed in anaconda. Shortly after it starts up, 
it automatically tries to guess the correct time zone via geolocation, and the 
user can also explicitly choose a timezone in the installer interface (or set 
one in a kickstart). Whenever the timezone is set in this way, an underlying 
library (libtimezonemap - https://launchpad.net/timezonemap) sets 'TZ' to the 
chosen timezone. It turns out other code in anaconda relies on Python 
respecting that setting, which Python 3.6 does not do. As a consequence, 
anaconda with Python 3.6 winds up setting the system time incorrectly. Also, 
the timestamps on all its log files are different now, and there may well be 
other consequences I didn't figure out yet.

The same applies to, e.g., `datetime.datetime.now()`: you can perform the same 
experiment with Python 3.5 and 3.6. If you change the 'TZ' env var while 
calling `datetime.datetime.now()` after each change, on Python 3.5, the naive 
datetime object it returns is the current time *in that timezone*. On Python 
3.6, regardless of what 'TZ' is set to, it always gives you the same time.

Is this an intended and/or desired change that we should adjust to somehow? Is 
there another way a running Python process can change what "the current" 
timezone is, for the purposes of datetime calculations like this?

--
components: Library (Lib)
messages: 291574
nosy: adamwill
priority: normal
severity: normal
status: open
title: datetime in Python 3.6+ no longer respects 'TZ' environment variable
versions: Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30061] Check if PyObject_Size() raised an error

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1239

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30061] Check if PyObject_Size() raised an error

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30061] Check if PyObject_Size() raised an error

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

PyObject_Size(), PySequence_Size() and PyMapping_Size() can raise an exception. 
But not always this is checked after using them. This can lead to a crash. For 
example:

>>> import io
>>> class R(io.IOBase):
... def readline(self): return None
... 
>>> next(R())
Fatal Python error: a function returned a result with an error set
TypeError: object of type 'NoneType' has no len()   

   

The above exception was the direct cause of the following exception:

SystemError:  returned a result with an error set

Current thread 0xb749c700 (most recent call first):
  File "", line 1 in 

--
components: Extension Modules, IO, Interpreter Core
messages: 291573
nosy: serhiy.storchaka
priority: normal
severity: normal
status: open
title: Check if PyObject_Size() raised an error
type: crash
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue23894] lib2to3 doesn't recognize rb'...' and f'...' in Python 3.6

[Łukasz Langa]

Łukasz Langa added the comment:

No Serhiy, I need a new patch with tests, etc.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30059] No documentation for C type Py_Ellipsis

[Michael Seifert]

Michael Seifert added the comment:

> I'm wondering if it is worth to add a separate section for Ellipsis at the 
> same level as "Slice Objects", but in the same file.

I'm not sure either. 

There seems to be no precedent in the documentation, for example 
"Py_NotImplemented" is also just another (but mentioned first) point in the 
Object protocol [https://docs.python.org/3/c-api/object.html#object-protocol] 
and "Py_None" even has a complete page 
[https://docs.python.org/3/c-api/none.html].

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30059] No documentation for C type Py_Ellipsis

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I'm wondering if it is worth to add a separate section for Ellipsis at the same 
level as "Slice Objects", but in the same file. See for example "Instance 
Method Objects" and "Method Objects" in method.rst.

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30047] Typos in Doc/library/select.rst

[Mariatta Wijaya]

Mariatta Wijaya added the comment:

This needs backport, which I will do later today. Thanks.

--
assignee: docs@python -> Mariatta
nosy: +Mariatta
stage:  -> backport needed
versions: +Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30060] Crash on Py_Finalize if Py_NoSiteFlag is used

[Stephen Kelly]

New submission from Stephen Kelly:

When attempting to use PyImport_ImportModule("os") (or to import many other 
libraries), there is a crash on Py_Finalize if Py_NoSiteFlag is set. The issue 
appears to be the use of frozenset() as a result of importing the module.

I reproduced this on Windows after building 2.7.13 with VS 2015 by applying the 
following patch:


Python changes.

--- Include\\fileobject.h
+++ Include\\fileobject.h
@@ -70,7 +70,7 @@
 */
 int _PyFile_SanitizeMode(char *mode);
 
-#if defined _MSC_VER && _MSC_VER >= 1400
+#if defined _MSC_VER && _MSC_VER >= 1400 && _MSC_VER < 1900
 /* A routine to check if a file descriptor is valid on Windows.  Returns 0
  * and sets errno to EBADF if it isn't.  This is to avoid Assertions
  * from various functions in the Windows CRT beginning with
--- Modules\\posixmodule.c
+++ Modules\\posixmodule.c
@@ -529,7 +529,7 @@
 #endif
 
 
-#if defined _MSC_VER && _MSC_VER >= 1400
+#if defined _MSC_VER && _MSC_VER >= 1400 && _MSC_VER < 1900
 /* Microsoft CRT in VS2005 and higher will verify that a filehandle is
  * valid and raise an assertion if it isn't.
  * Normally, an invalid fd is likely to be a C program error and therefore
--- Modules\\timemodule.c
+++ Modules\\timemodule.c
@@ -68,6 +70,9 @@
 #if defined(MS_WINDOWS) && !defined(__BORLANDC__)
 /* Win32 has better clock replacement; we have our own version below. */
 #undef HAVE_CLOCK
+#define timezone _timezone
+#define tzname _tzname
+#define daylight _daylight
 #endif /* MS_WINDOWS && !defined(__BORLANDC__) */
 
 #if defined(PYOS_OS2)



Backtrace:


KernelBase.dll!7ff963466142()   Unknown
>   python27_d.dll!Py_FatalError(const char * msg) Line 1700C
python27_d.dll!PyThreadState_Get() Line 332 C
python27_d.dll!set_dealloc(_setobject * so) Line 553C
python27_d.dll!_Py_Dealloc(_object * op) Line 2263  C
python27_d.dll!PySet_Fini() Line 1084   C
python27_d.dll!Py_Finalize() Line 526   C
mn.exe!main(int argc, char * * argv) Line 40C
[External Code] 



Reproducing code:


#include 

int main(int argc, char** argv)
{
// http://www.awasu.com/weblog/embedding-python/threads

//  Comment this to avoid crash
Py_NoSiteFlag = 1;

Py_Initialize();
PyEval_InitThreads(); // nb: creates and locks the GIL
// NOTE: We save the current thread state, and restore 
it when we unload,
// so that we can clean up properly.
PyThreadState* pMainThreadState = PyEval_SaveThread(); // nb: this also 
releases the GIL

PyEval_AcquireLock(); // nb: get the GIL

PyThreadState* pThreadState = Py_NewInterpreter();
assert(pThreadState != NULL);
PyEval_ReleaseThread(pThreadState); // nb: this also releases the GIL

PyEval_AcquireThread(pThreadState);

// Can reproduce by importing the os module, but the issue actually appears 
// because of the use of frozenset, so simplify to that.
#if 0
PyObject* osModule = PyImport_ImportModule("os");
Py_DECREF(osModule);
#endif

// As in abc.py ABCMeta class
PyRun_SimpleString("abstractmethods = frozenset(set())");

PyEval_ReleaseThread(pThreadState);

// release the interpreter 
PyEval_AcquireThread(pThreadState); // nb: this also locks the GIL
Py_EndInterpreter(pThreadState);
PyEval_ReleaseLock(); // nb: release the GIL
  
// clean up
PyEval_RestoreThread(pMainThreadState); // nb: this also locks the GIL
Py_Finalize();
}

--
components: Interpreter Core
messages: 291568
nosy: steveire
priority: normal
severity: normal
status: open
title: Crash on Py_Finalize if Py_NoSiteFlag is used
type: crash
versions: Python 2.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30059] No documentation for C type Py_Ellipsis

[Michael Seifert]

New submission from Michael Seifert:

The "Py_Ellipsis" object is part of the public C-API but it isn't documented 
anywhere.

It is defined in "sliceobject.o/.h" so I created a PR and added it to the 
"slice" documentation.

--
assignee: docs@python
components: Documentation
messages: 291567
nosy: MSeifert, docs@python
priority: normal
pull_requests: 1238
severity: normal
status: open
title: No documentation for C type Py_Ellipsis
type: enhancement
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29994] site.USER_SITE is None for Windows embeddable Python 3.6

[Steve Dower]

Steve Dower added the comment:

Actually, I suspect you're not importing site on startup at all (which is great 
for performance), but that means that site.main() is not being run. I'll post 
more details on the pip bug.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30058] Buffer overflow in kqueue.control()

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

Proposed patch presumably fixes the issue. But since this is *BSD specific I 
can't even check that it is compiled.

--
stage:  -> patch review

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30058] Buffer overflow in kqueue.control()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1237

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29994] site.USER_SITE is None for Windows embeddable Python 3.6

[Steve Dower]

Steve Dower added the comment:

Yes, fixing pip in this case is the right approach.

Since you're going to be including a copy of pip with your application, you can 
also just patch that yourself to get going while the pip team decides how to go 
from here. Or you could add a sitecustomize.py file (which will only be 
imported if you have "import site" in your ._pth file) that sets site.USER_SITE 
to something so that pip does not crash.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30058] Buffer overflow in kqueue.control()

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

The first parameter of kqueue.control() is documented as an iterable. But 
actually it should have a length. kqueue.control() uses PyObject_Size() for 
allocating an array and PyObject_GetIter()+PyIter_Next() for iterating kevent 
objects and filling the array. If the length and the iterator are not 
consistent this can lead to writing past the end of the array.

--
components: Extension Modules, FreeBSD
messages: 291563
nosy: koobs, serhiy.storchaka
priority: normal
severity: normal
status: open
title: Buffer overflow in kqueue.control()
type: crash
versions: Python 2.7, Python 3.5, Python 3.6, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30057] signal.signal should check tripped signals

[Jeroen Demeyer]

Changes by Jeroen Demeyer :


Added file: http://bugs.python.org/file46802/fix30057-py3.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30057] signal.signal should check tripped signals

[Jeroen Demeyer]

Changes by Jeroen Demeyer :


--
keywords: +patch
Added file: http://bugs.python.org/file46801/fix30057-py2.patch

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30056] RuntimeWarning: invalid value encountered in maximum/minimum

[R. David Murray]

R. David Murray added the comment:

Numpy is not part of the python standard library.  You should report this issue 
to the numpy bug tracker, which appears to be here: 
https://github.com/numpy/numpy/issues.  Or perhaps first ask the numpy 
community if this is really a bug: the new version might have fixed a bug in 
minimum/maximum that is now detecting a problem with your code that wasn't 
detected previously.

--
nosy: +r.david.murray
resolution:  -> third party
stage:  -> resolved
status: open -> closed
type: compile error -> behavior

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30057] signal.signal should check tripped signals

[Jeroen Demeyer]

New submission from Jeroen Demeyer:

There is a race condition in calling signal.signal() if the signal arrives 
while (or right before) signal.signal() is being executed: the function 
signal.signal(sig, action) marks the signal "sig" as not tripped. Because of 
this, signals can get lost. Instead, it would be better to call 
PyErr_CheckSignals() to check for pending signals.

--
components: Interpreter Core
messages: 291561
nosy: jdemeyer
priority: normal
severity: normal
status: open
title: signal.signal should check tripped signals
versions: Python 2.7, Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30056] RuntimeWarning: invalid value encountered in maximum/minimum

[Amy]

New submission from Amy:

I just updated to numpy 1.12.1 and am getting this Runtime Warning when using 
numpy.minimum or numpy.maximum:

RuntimeWarning: invalid value encountered in maximum

Prior to updating, I was using numpy 1.10.x and had no issues running 
numpy.minimum or numpy.maximum

--
messages: 291560
nosy: aching
priority: normal
severity: normal
status: open
title: RuntimeWarning: invalid value encountered in maximum/minimum
type: compile error
versions: Python 3.5

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30055] Missed testcleanup in decimal.rst

[Marco Buttu]

Changes by Marco Buttu :


--
pull_requests: +1236

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30055] Missed testcleanup in decimal.rst

[Marco Buttu]

New submission from Marco Buttu:

The testsetup in Doc/library/decimal.rst is not enough for isolating the tests 
in respect to the other rst files.  Currently we have the following testsetup, 
without a testcleanup:

 .. testsetup:: *
 
import decimal
import math
from decimal import *
# make sure each group gets a fresh context
setcontext(Context())

Without a testcleanup, the changes on the context will affect the other files 
that use the context (like Doc/library/statistics.rst).  We should better 
isolate the tests adding also a testcleanup:

.. testcleanup:: *

   # make sure other tests (outside this file) get a fresh context
   setcontext(Context())

I am opening a PR.

--
assignee: docs@python
components: Documentation
messages: 291559
nosy: docs@python, marco.buttu, skrah
priority: normal
severity: normal
status: open
title: Missed testcleanup in decimal.rst
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30054] Expose tracemalloc C API to track/untrack memory blocks

[Julian Taylor]

Julian Taylor added the comment:

With this changeset it would:
https://github.com/numpy/numpy/pull/8885

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30054] Expose tracemalloc C API to track/untrack memory blocks

[STINNER Victor]

STINNER Victor added the comment:

What's the status in numpy? Is the current numpy release able to use CPython 
3.6 tracemalloc private functions?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29791] print documentation: flush is also a keyword argument

[Berker Peksag]

Changes by Berker Peksag :


--
nosy: +berker.peksag
stage:  -> patch review
type: enhancement -> behavior
versions: +Python 3.5, Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29791] print documentation: flush is also a keyword argument

[Berker Peksag]

Changes by Berker Peksag :


--
pull_requests: +1235

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30054] Expose tracemalloc C API to track/untrack memory blocks

[Julian Taylor]

Julian Taylor added the comment:

I am not sure if _PyTraceMalloc_GetTraceback really needs to be a public 
function.
Exposing the tracing information should probably just go over python interfaces.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30054] Expose tracemalloc C API to track/untrack memory blocks

[STINNER Victor]

New submission from STINNER Victor:

The issue #26530 added a private C API to manually track/untrack memory blocks 
in tracemalloc. I was just validated by Julian Taylor who confirms that the API 
works as expected:
http://bugs.python.org/issue26530#msg291551

So I propose to make the 3 newly added functions public and document them:

* _PyTraceMalloc_Track()
* _PyTraceMalloc_Untrack()
* _PyTraceMalloc_GetTraceback()

--
messages: 291554
nosy: haypo, jtaylor
priority: normal
severity: normal
status: open
title: Expose tracemalloc C API to track/untrack memory blocks
type: enhancement
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26530] tracemalloc: add C API to manually track/untrack memory allocations

[STINNER Victor]

STINNER Victor added the comment:

"I don't see any reason why not to.": Ok, I created the issue #30054.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26530] tracemalloc: add C API to manually track/untrack memory allocations

[Julian Taylor]

Julian Taylor added the comment:

I don't see any reason why not to.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26530] tracemalloc: add C API to manually track/untrack memory allocations

[STINNER Victor]

STINNER Victor added the comment:

Julian Taylor: "The api looks good to me. Works fine in numpy."

Cool! Should it be made public in that case? (Remove _ prefix and document it.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26530] tracemalloc: add C API to manually track/untrack memory allocations

[Julian Taylor]

Julian Taylor added the comment:

The api looks good to me. Works fine in numpy.

--
nosy: +jtaylor

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30017] zlib.error: Error -2 while flushing: inconsistent stream state

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1234

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30017] zlib.error: Error -2 while flushing: inconsistent stream state

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 4c0d9ea995da595e90e08813b89510de59907802 by Serhiy Storchaka in 
branch 'master':
bpo-30017: Allowed calling the close() method of the zip entry writer object 
(#1041)
https://github.com/python/cpython/commit/4c0d9ea995da595e90e08813b89510de59907802


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30027] test_xml_etree and test_xml_etree_c fail due to AssertionError: unhandled warning DeprecationWarning

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
resolution:  -> fixed
stage:  -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30027] test_xml_etree and test_xml_etree_c fail due to AssertionError: unhandled warning DeprecationWarning

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 65c5b096ac2c6608d296f1603cd4792086108c95 by Serhiy Storchaka in 
branch '2.7':
bpo-30027: Fix Py3k warnings in test_xml_etree. (#1065)
https://github.com/python/cpython/commit/65c5b096ac2c6608d296f1603cd4792086108c95


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue13285] signal module ignores external signal changes

[Jeroen Demeyer]

Jeroen Demeyer added the comment:

I have a preliminary implementation (in the cysignals package) at 
https://github.com/sagemath/cysignals/pull/53

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30053] Problems building with --enable-profiling on macOS

[Brecht Machiels]

Changes by Brecht Machiels :


Added file: http://bugs.python.org/file46800/make_gcc.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30053] Problems building with --enable-profiling on macOS

[Brecht Machiels]

Changes by Brecht Machiels :


Added file: http://bugs.python.org/file46799/make.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30053] Problems building with --enable-profiling on macOS

[Brecht Machiels]

New submission from Brecht Machiels:

The python.exe produced during the build process is somehow broken:

$ ./python.exe -S
Killed: 9

Strangely, it works when run from gdb:

$ gdb -args ./python.exe -S
GNU gdb (GDB) 7.12.1
Copyright (C) 2017 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 

This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-apple-darwin16.3.0".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
.
Find the GDB manual and other documentation resources online at:
.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from ./python.exe...done.
(gdb) start
Temporary breakpoint 1 at 0x1109c: file ../Programs/python.c, line 28.
Starting program: /Users/brechtm/Documents/Code/cpython/profile/python.exe 
-S
[New Thread 0x1403 of process 62753]
warning: unhandled dyld version (15)

Thread 2 hit Temporary breakpoint 1, main (argc=2, argv=0x7fff5bfff460) at 
../Programs/python.c:28
28  (void)_PyMem_SetupAllocators("malloc");
(gdb) c
Continuing.
Could not find platform dependent libraries 
Consider setting $PYTHONHOME to [:]
Python 3.7.0a0 (heads/master:3e0f1fc4e0, Apr 12 2017, 14:39:47)
[GCC 4.2.1 Compatible Apple LLVM 8.1.0 (clang-802.0.41)] on darwin
Module readline not available.
>>> ^D
[Inferior 1 (process 62753) exited normally]
(gdb) q

I'm running macOS Sierra 10.12.4 (16E195) and XCode 8.3.1.

$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr 
--with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.1.0 (clang-802.0.41)
Target: x86_64-apple-darwin16.5.0
Thread model: posix
InstalledDir: 
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin

Forcing the use of the Homebrew-provided GCC by invoking "make CC=gcc-6 
CXX=g++-6" has the same result.

$ gcc-6 --version
gcc-6 (Homebrew GCC 6.3.0_1) 6.3.0
Copyright (C) 2016 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

The build is fine when --enable-profiling is not specified.

--
components: macOS
files: configure.txt
messages: 291547
nosy: brechtm, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: Problems building with --enable-profiling on macOS
type: compile error
versions: Python 3.7
Added file: http://bugs.python.org/file46798/configure.txt

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30052] URL Quoting page links to function Bytes instead of defintion

[Cheryl Sabella]

New submission from Cheryl Sabella:

On the URL Quoting page, the following line:

`string may be either a str or a bytes.`

Has the `str` link to:
https://docs.python.org/3/library/stdtypes.html#str

But the `bytes` link to:
https://docs.python.org/3/library/functions.html#bytes

Should the `bytes` link to?
https://docs.python.org/3/library/stdtypes.html#bytes

--
assignee: docs@python
components: Documentation
messages: 291546
nosy: csabella, docs@python
priority: normal
severity: normal
status: open
title: URL Quoting page links to function Bytes instead of defintion
versions: Python 3.6

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30049] Don't cache tp_iternext

[Raymond Hettinger]

Changes by Raymond Hettinger :


--
assignee:  -> rhettinger

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API clear private variables, which are initialized once, at Python shutdown

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

What if use the term "singleton"? It includes "initialize once" and "finalize 
at shutdown" meanings. _Py_INIT_SINGLETON() or _Py_SET_SINGLETON()?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30051] Document that the random module doesn't support fork

[Serhiy Storchaka]

Serhiy Storchaka added the comment:

I think it is worth documenting.

Even if multiprocessing reseeds the module-global random generator, it doesn't 
reseed other instances of Random.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30047] Typos in Doc/library/select.rst

[Serhiy Storchaka]

Serhiy Storchaka added the comment:


New changeset 3e0f1fc4e0ffcfcc706015fa3d67c262948ef171 by Serhiy Storchaka 
(NAKAMURA Osamu) in branch 'master':
bpo-30047: Fix a typo in Doc/library/select.rst (#1086)
https://github.com/python/cpython/commit/3e0f1fc4e0ffcfcc706015fa3d67c262948ef171


--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue26985] Information about CodeType in inspect documentation is outdated

[Xiang Zhang]

Changes by Xiang Zhang :


--
pull_requests: +1233

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29881] Add a new private API clear private variables, which are initialized once, at Python shutdown

[STINNER Victor]

STINNER Victor added the comment:

I dislike _Py_SET_FINALIZED and _Py_SET_ONCE name.

I prefer to mention an "initialization" because it's a common pattern in 
programming and it is known that it only requires to be done once.

_PY_ONCEVAR_INIT() macro is unusual: it only evaluate the second parameter 
parameter, a C expression, if the variable is not initialized yet.

Py_SETREF() is different because it always evaluates its second argument, and 
in most cases, the second argument is an existing variable and not an 
expression.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30051] Document that the random module doesn't support fork

[Antoine Pitrou]

Antoine Pitrou added the comment:

I mean mentioning fork(), which people usually don't call directly, is not 
necessary, so, indeed, a note isn't necessary.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30051] Document that the random module doesn't support fork

[STINNER Victor]

STINNER Victor added the comment:

> I'm not sure that's necessary.

Do you mean that adding a note is not necessary? Or proposing a
solution in the note?

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29995] re.escape() escapes too much

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee:  -> serhiy.storchaka
dependencies: +Add examples for re.escape()

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29694] race condition in pathlib mkdir with flags parents=True

[Armin Rigo]

Armin Rigo added the comment:

https://github.com/python/cpython/pull/1089

(I fixed the problem with my CLA check.  Now 
https://cpython-devguide.readthedocs.io/pullrequest.html#licensing says "you 
can ask for the CLA check to be run again" but doesn't tell how to do that, so 
as far as I can tell, I have to ask e.g. here.)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30021] Add examples for re.escape()

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
assignee: docs@python -> serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29694] race condition in pathlib mkdir with flags parents=True

[Roundup Robot]

Changes by Roundup Robot :


--
pull_requests: +1232

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29178] Adding bytes.frombuffer(byteslike) constructor

[INADA Naoki]

INADA Naoki added the comment:

FYI, Tornado 4.5b switched buffer implementation from deque-of-bytes
to bytearray like asyncio.  I sent PR for fixing "unreleased memoryview".
https://github.com/tornadoweb/tornado/pull/2008

This is common pitfall when implementing buffer.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30051] Document that the random module doesn't support fork

[Antoine Pitrou]

Antoine Pitrou added the comment:

I'm not sure that's necessary.  fork() is a low-level primitive, people 
can/should use multiprocessing.Process instead, which does re-seed the PRNG.

--
nosy: +pitrou

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30050] Please provide a way to disable the warning printed if the signal module's wakeup fd overflows

[Nathaniel Smith]

Nathaniel Smith added the comment:

I haven't noticed the error in the wild because I don't use set_wakeup_fd on 
Linux/MacOS, because of this issue :-). But on MacOS literally all it would 
take is to receive two signals in quick succession, or to receive one signal at 
a moment when someone has recently scheduled a callback from a thread (which 
also writes to the wakeup fd). In principle it can also happen on Windows, but 
on Windows apparently the smallest buffer you get for a socketpair is like 500 
kilobytes, so it doesn't come up except in rather artificial situations.

And yes, a possible workaround is to use an artificially large buffer on 
Linux/MacOS as well. I'm not saying this is a huge showstopper. But it's silly 
for downstream libraries to carry around workarounds for issues in CPython when 
we could just fix CPython.

> If the pipe becomes full, you loose signals: it's an hard error. I decided to 
> log an error into fd 2 because in a signal handler, the number of allowed 
> functions is very limited.

This makes sense for asyncio, but not for most other async libraries in Python, 
because they *don't* lose signals when the pipe becomes full. That's why I'm 
suggesting that the signal module should have an option to let users decide 
whether they want the warning or not.

(It might also make sense to switch asyncio to stop using the fd as a 
communications channel. Theoretically this would be the ideal solution, but I 
understand why it's not a priority.)

> Related question: does asyncio support signals on Windows? I added support 
> for set_wakeup_fd() to Windows for asyncio, but then I forgot this TODO item 
> :-)

No, asyncio doesn't use set_wakeup_fd on Windows, so code like 
'loop.run_until_complete(asyncio.sleep(9))' can't be interrupted. I'd 
appreciate if you could wait a few days to fix this though, because the blog 
post that I'm almost finished writing about signal handling currently uses this 
as an example of how trio is more awesome than asyncio ;-).

(This is a joke. Well, I mean, mostly. My draft does literally have that sleep 
example in it, but I won't be grumpy if you fix it first. Or at least, not 
*very* grumpy :-).)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30051] Document that the random module doesn't support fork

[STINNER Victor]

Changes by STINNER Victor :


--
assignee:  -> docs@python
components: +Documentation
nosy: +docs@python, rhettinger, serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30030] Simplify _RandomNameSequence

[STINNER Victor]

STINNER Victor added the comment:

After reviewing the change, I created the issue #30051: "Document that the 
random module doesn't support fork".

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30051] Document that the random module doesn't support fork

[STINNER Victor]

New submission from STINNER Victor:

When reviewing the issue #30030, it reminded me that the random module "doesn't 
support fork": after fork, the parent and the child produce the same "random" 
number sequence.

I suggest to add a quick note about that: not a warning, just a note, as a 
reminder.

There is an exception: SystemRandom produces a different sequence after fork, 
since it uses os.urandom() (which runs in the kernel).

I am tempted to propose a solution in the note like using SystemRandom, but I'm 
not sure that it's a good idea. Some users may misunderstood the note and 
always use SystemRandom where random.Random is just fine for their needs.

Proposed note:

The random module doesn't support fork: the parent and the child process will 
produce the same number sequence.

Proposed solution:

If your code uses os.fork(), a workaround is to check if os.getpid() changes 
and in that case, create a new Random instance or reseed the RNG in the child 
process.

--

The tempfile module reminds the pid and instanciates a new RNG on fork.

Another option is to not add a note, but implement the workaround directly into 
the random module. But I don't think that it's worth it. Forking is a rare 
usecase, and calling os.getpid() may slowdown the random module. (I don't 
recall if os.getpid() requires a syscall or not, I'm quite sure that it's 
optimized at least on Linux to avoid a real syscall.)

--
messages: 291534
nosy: haypo
priority: normal
severity: normal
status: open
title: Document that the random module doesn't support fork
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30050] Please provide a way to disable the warning printed if the signal module's wakeup fd overflows

[STINNER Victor]

STINNER Victor added the comment:

> One way that writing to the wakeup fd can fail is if the pipe or socket's 
> buffer is already full, in which case we get EWOULDBLOCK or WSAEWOULDBLOCK.

Is it a theorical question or did you notice the error in the wild? I mean: 
without sending a signal in a loop.

Right now, we write the signal number into the pipe each time a signal is 
received. Do you need to get N bytes if the signal is received N times? Maybe 
we can use a flag and only write the byte once by signal number?


> Printing to the console isn't a great solution, but it's better than letting 
> the error pass silently.

If the pipe becomes full, you loose signals: it's an hard error. I decided to 
log an error into fd 2 because in a signal handler, the number of allowed 
functions is very limited.

I suggest to have a flag to remind if the pipe overflowed, and in that case: 
schedule a Python callback. The callback can be a new parameter to 
set_wakeup_fd()?

A pipe seems limited to 64 KB, limit hardcoded in Linux kernel code. Is it the 
case if we use a socket pair, as we already do on Windows? Or is it possible to 
increase the socket buffer size?


> This is particularly annoying for trio, because I try to minimize the size of 
> the wakeup fd's send buffer to avoid wasting non-swappable kernel memory on 
> what's essentially an overgrown bool.

asyncio uses wakeup fd to get signal numbers: it's not only a bool. See the 
issue #21645 for the rationale of using the FD to get signal numbers in asyncio.

It was a bool on Python 2 which only writes NUL bytes into the FD.


Related question: does asyncio support signals on Windows? I added support for 
set_wakeup_fd() to Windows for asyncio, but then I forgot this TODO item :-)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30050] Please provide a way to disable the warning printed if the signal module's wakeup fd overflows

[Nathaniel Smith]

New submission from Nathaniel Smith:

When a wakeup fd is registered via signal.set_wakeup_fd, then the C level 
signal handler writes a byte to the wakeup fd on each signal received. If this 
write fails, then it prints an error message to the console.

Some projects use the wakeup fd as a way to see which signals have occurred 
(asyncio, curio). Others use it only as a toggle for "a wake up is needed", and 
transmit the actual data out-of-line (twisted, tornado, trio – I guess it has 
something to do with the letter "t").

One way that writing to the wakeup fd can fail is if the pipe or socket's 
buffer is already full, in which case we get EWOULDBLOCK or WSAEWOULDBLOCK. For 
asyncio/curio, this is a problem: it indicates a lost signal! Printing to the 
console isn't a great solution, but it's better than letting the error pass 
silently.

For twisted/tornado/trio, this is a normal and expected thing – the semantics 
we want are that after a signal is received then the fd will be readable, and 
if its buffer is full then it's certainly readable! So for them, 
EWOULDBLOCK/WSAEWOULDBLOCK are *success* conditions. Yet currently, the signal 
module insists on printing a scary message to the console whenever we succeed 
in this way.

It would be nice if there were a way to disable this; perhaps something like: 
signal.set_wakeup_fd(fd, warn_on_full_buffer=False)

This is particularly annoying for trio, because I try to minimize the size of 
the wakeup fd's send buffer to avoid wasting non-swappable kernel memory on 
what's essentially an overgrown bool. This ends up meaning that on Linux the 
buffer is 6 bytes, and on MacOS it's 1 byte. So currently I don't use the 
wakeup fd on Linux/MacOS, which is *mostly*  OK but it would be better if we 
could use it. Trio bug with a few more details: 
https://github.com/python-trio/trio/issues/109

--
messages: 291532
nosy: haypo, njs
priority: normal
severity: normal
status: open
title: Please provide a way to disable the warning printed if the signal 
module's wakeup fd overflows
versions: Python 3.7

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30049] Don't cache tp_iternext

[Serhiy Storchaka]

Changes by Serhiy Storchaka :


--
pull_requests: +1231

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30048] If a task is canceled at the right moment, the cancellation is ignored

[INADA Naoki]

INADA Naoki added the comment:

This behavior is documented as:

https://docs.python.org/3.6/library/asyncio-task.html#asyncio.Task.cancel

> Unlike Future.cancel(), this does not guarantee that the task will be 
> cancelled: the exception might be caught and acted upon, delaying 
> cancellation of the task or preventing cancellation completely. The task may 
> also return a value or raise a different exception.
>
> Immediately after this method is called, cancelled() will not return True 
> (unless the task was already cancelled). A task will be marked as cancelled 
> when the wrapped coroutine terminates with a CancelledError exception (even 
> if cancel() was not called).

I agree that this behavior is somewhat surprising.
But I don't know how can I fix the behavior.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30049] Don't cache tp_iternext

[Serhiy Storchaka]

New submission from Serhiy Storchaka:

Some operations cache the value of the tp_iternext slot and call it in a loop. 
But calling tp_iternext can run arbitrary code, release GIL and change the 
tp_iternext slot in the same or other thread. This can leads to visible 
behavior difference, such as different iterated values, different raised 
exceptions or infinite loop. In the past this even could cause a crash (see 
issue3720), but seems now this is impossible.

Examples (list constructor caches tp_iternext, tuple constructor doesn't):

>>> def make_iter():
... class Iterator:
... def __iter__(self):
... return self
... def __next__(self):
... del Iterator.__next__
... return 1
... return Iterator()
... 
>>> tuple(make_iter())
Traceback (most recent call last):
  File "", line 1, in 
TypeError: 'Iterator' object is not iterable
>>> list(make_iter())
Traceback (most recent call last):
  File "", line 1, in 
AttributeError: __next__
>>> 
>>> def make_iter2():
... it2 = iter((2,))
... def subiter():
... Iterator.__next__ = Iterator.next2
... yield 1
... class Iterator(filter):
... def next2(self):
... return next(it2)
... return Iterator(lambda x: True, subiter())
... 
>>> tuple(make_iter2())
(1, 2)
>>> list(make_iter2())
[1]

The tp_iternext is cached for performance, and removing the caching can cause 
performance regression. But actually the difference is very small. I found a 
measurable difference (up to 5%) only in following artificial examples in which 
tp_iternext is called in very tight and long loops:

$ ./python -m perf timeit --compare-to ./python-orig -s 'a = [0]*100+[1]' 
-- 'next(filter(None, a))'python-orig: . 47.2 ms +- 0.2 ms
python: . 49.7 ms +- 0.3 ms

Mean +- std dev: [python-orig] 47.2 ms +- 0.2 ms -> [python] 49.7 ms +- 0.3 ms: 
1.05x slower (+5%)

$ ./python -m perf timeit --compare-to ./python-orig -s 'from itertools import 
repeat, islice' -- 'next(islice(repeat(1), 100, None))'
python-orig: . 15.4 ms +- 0.1 ms
python: . 16.0 ms +- 0.2 ms

Mean +- std dev: [python-orig] 15.4 ms +- 0.1 ms -> [python] 16.0 ms +- 0.2 ms: 
1.04x slower (+4%)

$ ./python -m perf timeit --compare-to ./python-orig -s 'from itertools import 
repeat; from collections import deque' -- 'deque(repeat(1, 100), 0)'
python-orig: . 14.2 ms +- 0.1 ms
python: . 14.8 ms +- 0.2 ms

Mean +- std dev: [python-orig] 14.2 ms +- 0.1 ms -> [python] 14.8 ms +- 0.2 ms: 
1.05x slower (+5%)

In all other other cases, when involved creation of a collection (list, 
bytearray, deque (with maxlen != 0) constructors, ''.join), or calling other 
code (builtins all, max, map, itertools functions), or for shorter loops the 
difference is hardly distinguished from the random noise.

$ ./python -m perf timeit --compare-to ./python-orig -s 'a = [0]*1000' -- 
'list(iter(a))'
python-orig: . 31.8 us +- 0.3 us
python: . 31.8 us +- 0.4 us

Mean +- std dev: [python-orig] 31.8 us +- 0.3 us -> [python] 31.8 us +- 0.4 us: 
1.00x faster (-0%)
Not significant!

$ ./python -m perf timeit --compare-to ./python-orig -s 'a = [1]*1000' -- 
'all(a)'
python-orig: . 47.4 us +- 0.2 us
python: . 48.0 us +- 0.3 us

Mean +- std dev: [python-orig] 47.4 us +- 0.2 us -> [python] 48.0 us +- 0.3 us: 
1.01x slower (+1%)

$ ./python -m perf timeit --compare-to ./python-orig -s 'a = [1]*1000' -- 
'max(a)'
python-orig: . 108 us +- 1 us
python: . 108 us +- 1 us

Mean +- std dev: [python-orig] 108 us +- 1 us -> [python] 108 us +- 1 us: 1.00x 
faster (-0%)
Not significant!

$ ./python -m perf timeit --compare-to ./python-orig -s 'a = [0]*100+[1]' 
-- 'next(filter(lambda x: x, a))'
python-orig: . 527 ms +- 8 ms
python: . 528 ms +- 2 ms

Mean +- std dev: [python-orig] 527 ms +- 8 ms -> [python] 528 ms +- 2 ms: 1.00x 
slower (+0%)
Not significant!

$ ./python -m perf timeit --compare-to ./python-orig -s 'from itertools import 
repeat, islice' -- 'next(islice(repeat(1), 100, None))'
python-orig: . 4.72 us +- 0.05 us
python: . 4.72 us +- 0.04 us

Mean +- std dev: [python-orig] 4.72 us +- 0.05 us -> [python] 4.72 us +- 0.04 
us: 1.00x faster (-0%)
Not significant!

$ ./python -m perf timeit --compare-to ./python-orig -s 'from itertools import 
repeat; from collections import deque' -- 'deque(repeat(1, 100), 0)'
python-orig: . 4.16 us +- 0.11 us
python: . 4.11 us +- 0.05 us

Mean +- std dev: [python-orig] 4.16 us +- 0.11 us -> [python] 4.11 us +- 0.05 
us: 1.01x faster (-1%)

--
components: Extension Modules, Interpreter Core
messages: 291530
nosy: rhettinger, serhiy.storchaka
priority: normal

[issue13285] signal module ignores external signal changes

[Jeroen Demeyer]

Jeroen Demeyer added the comment:

Ping? I'll try to implement this in cysignals (which is more difficult than 
doing it in CPython because I cannot access all internals of the Python signal 
module).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue30048] If a task is canceled at the right moment, the cancellation is ignored

[INADA Naoki]

INADA Naoki added the comment:

When task.cancel() called, CancelledError is thrown to coro2.
But coro2 doesn't call `yield from` after task.cancel().
So the exception is never raised.

If you add `yield from` after `task.cancel()`, the script runs expected.
---
$ cat at.py
import asyncio as a

@a.coroutine
def coro1():
yield from a.ensure_future(coro2())
print("Still here")
yield from a.sleep(.1)
print("Still here 2")

@a.coroutine
def coro2():
yield from a.sleep(.1)
res = task.cancel()
print("Canceled task:", res)
yield from a.sleep(.1)  # !!! added this line

loop = a.get_event_loop()
task = a.ensure_future(coro1())
loop.run_until_complete(task)

$ ./python.exe at.py
Canceled task: True
Traceback (most recent call last):
  File "at.py", line 19, in 
loop.run_until_complete(task)
  File "/Users/inada-n/work/python/cpython/Lib/asyncio/base_events.py", line 
465, in run_until_complete
return future.result()
concurrent.futures._base.CancelledError

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29694] race condition in pathlib mkdir with flags parents=True

[Armin Rigo]

Armin Rigo added the comment:

Update: a review didn't show any other similar problems (pathlib.py is a thin 
layer after all).  Applied the fix and test (x2.diff) inside PyPy.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue27400] Datetime NoneType after calling Py_Finalize and Py_Initialize

[Frank Blankenburg]

Changes by Frank Blankenburg :


--
nosy: +FrankBlabu

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue29994] site.USER_SITE is None for Windows embeddable Python 3.6

[Brecht Machiels]

Brecht Machiels added the comment:

I see. In that case I think pip should be able to handle the case when 
site.USER_SITE is None. I have created a ticket here: 
https://github.com/pypa/pip/issues/4437.

I am using the embeddable Python what's it intended for, to make a 
self-contained distribution of my application (using pynsist). I'm using pip to 
install optional resources (fonts).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com