[issue42011] Documentation for logging.Filter.filter reports the wrong return type
New submission from Riccardo Coccioli : The documentation for the logging.Filter().filter() method states: #- Is the specified record to be logged? Returns 0 for no, nonzero for yes. If deemed appropriate, the record may be modified in-place. #- While its implementation returns a boolean in Lib/logging/__init__.py. Moreover the most recent version of mypy (0.790) reports it as an error if a custom class inherit from logging.Filter and implement a filter() method that returns an int as specified in the documentation: #- error: Return type "int" of "filter" incompatible with return type "bool" in supertype "Filter" [override] #- P.S. As a side note, the API for filter() is quite counter-intuitive as it requires the filter() method to return False to filter out the record and True to include it. -- assignee: docs@python components: Documentation messages: 378480 nosy: Riccardo Coccioli, docs@python priority: normal severity: normal status: open title: Documentation for logging.Filter.filter reports the wrong return type type: behavior versions: Python 3.10, Python 3.5, Python 3.6, Python 3.7, Python 3.8, Python 3.9 ___ Python tracker <https://bugs.python.org/issue42011> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue36284] importlib.import_module() not thread safe if Exception is raised (3.4, 3.5)
New submission from Riccardo Coccioli :
It seems that importlib.import_module() is not thread-safe if the loaded module
raises an Exception on Python 3.4 and 3.5. I didn't find any thread-unsafe
related information in Python's documentation.
The frequency of the failure appears to be random.
This is the setup to reproduce the issue:
#- FILES STRUCTURE
├── fail.py
└── test.py
#-
#- CONTENT OF fail.py
ACCESSIBLE = 'accessible'
import nonexistent # raise RuntimeError('failed') is basically the same
NOT_ACCESSIBLE = 'not accessible'
#-
#- CONTENT OF test.py
import importlib
import concurrent.futures
def f():
try:
mod = importlib.import_module('fail')
# importlib.reload(mod) # WORKAROUND
try:
val = mod.NOT_ACCESSIBLE
except AttributeError as e:
val = str(e)
return (mod.__name__, type(mod), mod.ACCESSIBLE, val)
except ImportError as e:
return str(e)
with concurrent.futures.ThreadPoolExecutor(max_workers=3) as executor:
futures = [executor.submit(f) for i in range(5)]
for future in concurrent.futures.as_completed(futures):
print(future.result())
#-
Expected result:
#-
No module named 'nonexistent'
No module named 'nonexistent'
No module named 'nonexistent'
No module named 'nonexistent'
No module named 'nonexistent'
#-
Actual result:
#-
No module named 'nonexistent'
No module named 'nonexistent'
No module named 'nonexistent'
('fail', , 'accessible', "'module' object has no attribute
'NOT_ACCESSIBLE'")
('fail', , 'accessible', "'module' object has no attribute
'NOT_ACCESSIBLE'")
#-
In the unexpected output lines, the module has been "partially" imported. The
'mod' object contains a module object, and trying to access an attribute
defined before the import that raises Exception works fine, but trying to
access an attribute defined after the failing import, fails.
It seems like the Exception was not properly raised at module load time, but at
the same time the module is only partially loaded up to the failing import.
The actual number of half-imported modules varies between runs and picking
different values for max_workers and range() and can also be zero (normal
behaviour). Also the frequency of the issue varies.
Using multiprocessing.pool.ThreadPool() and apply_async() instead of
concurrent.futures.ThreadPoolExecutor has the same effect.
I was able to reproduce the issue with the following Python versions and
platforms:
- 3.4.2 and 3.5.3 on Linux Debian
- 3.4.9 and 3.5.6 on macOS High Sierra 10.13.6
While the issue doesn't show up at the best of my knowledge on:
- 3.6.7 and 3.7.2 on macOS High Sierra 10.13.6
Thanks to a colleague suggestion I also found a hacky workaround. Uncommenting
the line in test.py marked as 'WORKAROUND' a reload of the module is forced.
With that modification the actual result is:
#-
No module named 'nonexistent'
No module named 'nonexistent'
No module named 'nonexistent'
module fail not in sys.modules
module fail not in sys.modules
#-
While this doesn't solve the issue per se, it actually raises the same
ImportError that the module was supposed to raise in the first place, just with
a different message, allowing the code to continue it's normal execution.
--
components: Library (Lib)
messages: 337887
nosy: Riccardo Coccioli
priority: normal
severity: normal
status: open
title: importlib.import_module() not thread safe if Exception is raised (3.4,
3.5)
type: behavior
versions: Python 3.4, Python 3.5
___
Python tracker
<https://bugs.python.org/issue36284>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31334] select.poll.poll fails on BSDs with arbitrary negative timeouts
Change by Riccardo Coccioli : -- pull_requests: +4008 ___ Python tracker <https://bugs.python.org/issue31334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31786] In select.poll.poll() ms can be 0 if timeout < 0
Change by Riccardo Coccioli : -- nosy: +Riccardo Coccioli ___ Python tracker <https://bugs.python.org/issue31786> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31334] select.poll.poll fails on BSDs with arbitrary negative timeouts
Riccardo Coccioli added the comment: Although it's documented as -1 in Linux man page [1], from my quick tests I was not able to get any error with negative values different from -1 and it seems to wait indefinitely as expected. Looking also at its implementation in [2], it doesn't seem to differentiate between negative values. It could be argued that is implementation dependent at the moment and the behaviour might change in the future. But, on a related note, the Python documentation doesn't say much either as what are acceptable values for the timeout parameter, see [3]. So at the moment there isn't any discrepancy between the Python documentation and the current behaviour IMHO, but I'm happy to open a separate task and send a PR if you think this should be improved/fixed too. [1] http://man7.org/linux/man-pages/man2/epoll_wait.2.html [2] http://elixir.free-electrons.com/linux/latest/source/fs/eventpoll.c#L1754 [3] https://docs.python.org/3.7/library/select.html -- ___ Python tracker <https://bugs.python.org/issue31334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31334] select.poll.poll fails on BSDs with arbitrary negative timeouts
Riccardo Coccioli added the comment: This can actually be reproduced with Python 2.7 too (thanks @thiell to let me know). At first I thought that it was not because it doesn't repro with the stock macOS-shipped Python 2.7.10 on macOS Sierra 10.12.6, where the select.poll() is not available at all, see below. Updated list of version where I was able to reproduce the error: - macOS Sierra 10.12.6 with those Python versions: 2.7.10, 2.7.13, 3.3.6, 3.4.6, 3.5.3, 3.6.2, 3.7.0a0 (heads/master:2ef37607b7) - FreeBSD 11.1 with those Python versions: 2.7.13, 3.7.0a0 (heads/master:2ef37607b7) For reference, the repro code executed with the stock macOS-shipped Python: #-- $ /usr/bin/python Python 2.7.10 (default, Feb 7 2017, 00:08:15) [GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.34)] on darwin Type "help", "copyright", "credits" or "license" for more information. >>> import select >>> p = select.poll() Traceback (most recent call last): File "", line 1, in AttributeError: 'module' object has no attribute 'poll' #-- If the PR 3277 that I've sent against the master branch with the fix will be accepted, I'm ready to send additional PRs to backport the fix in all affected versions. -- versions: +Python 2.7 ___ Python tracker <http://bugs.python.org/issue31334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31334] select.poll.poll fails on BSDs with arbitrary negative timeouts
Changes by Riccardo Coccioli : -- pull_requests: +3320 ___ Python tracker <http://bugs.python.org/issue31334> ___ ___ Python-bugs-list mailing list Unsubscribe: https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
[issue31334] select.poll.poll fails on BSDs with arbitrary negative timeouts
New submission from Riccardo Coccioli:
According to the Python documentation for the 'poll.poll([timeout])' method in
the 'select' module, any negative value for the 'timeout' parameter is valid
and should have the same behaviour [1]:
"If timeout is omitted, negative, or None, the call will block until there
is an event for this poll object."
Unfortunately, unlike the Linux, on many other OSes, including, but not limited
to, macOS and {Free,Open,Net}BSD, the 'poll()' system call requires that the
'timeout' parameter is a non-negative integer or exactly -1 (sometimes defined
as INFTIM). Any other negative value throws an error, see [2], [3], [4] and [5].
This is a snippet of code to reproduce the error:
#-
import select
p = select.poll()
p.poll(-100)
#-
Expected behaviour: block until there is an event for the poll object, in this
case block indefinitely
Current behaviour on macOS and FreeBSD: OSError: [Errno 22] Invalid argument
I was able to reproduce the error on:
- macOS Sierra 10.12.6 with those Python versions: 3.3.6, 3.4.6, 3.5.3, 3.6.2,
3.7.0a0 (heads/master:2ef37607b7)
- FreeBSD 11.1 with Python 3.7.0a0 (heads/master:2ef37607b7)
On Linux this doesn't happen because the 'poll()' system call accept any
negative value to wait indefinitely, see [6].
To adhere with the Python documentation described behaviour, I'm sending a pull
request to propose to force the 'timeout' value passed to the 'poll()' system
call to be exactly -1 (or INFTIM where defined) when a negative value is given.
This will not change the current behaviour on Linux and will have the behaviour
described in the documentation on other OSes where is currently failing with an
error.
[1] https://docs.python.org/3/library/select.html#poll-objects
[2] https://www.freebsd.org/cgi/man.cgi?poll
[3] https://man.openbsd.org/poll.2
[4] http://netbsd.gw.com/cgi-bin/man-cgi/man?poll
[5] From macOS 'man poll': "If timeout is greater than zero, it specifies a
maximum interval (in milliseconds) to wait for any file descriptor to become
ready. If timeout is zero, then poll() will return without blocking. If the
value of timeout is -1, the poll blocks indefinitely."
[6] http://man7.org/linux/man-pages/man2/poll.2.html
--
components: Extension Modules, FreeBSD, macOS
messages: 301202
nosy: Riccardo Coccioli, koobs, ned.deily, ronaldoussoren
priority: normal
severity: normal
status: open
title: select.poll.poll fails on BSDs with arbitrary negative timeouts
type: behavior
versions: Python 3.3, Python 3.4, Python 3.5, Python 3.6, Python 3.7
___
Python tracker
<http://bugs.python.org/issue31334>
___
___
Python-bugs-list mailing list
Unsubscribe:
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com
