[issue45042] Many multiprocessing tests are silently skipped since 3.9

[STINNER Victor]

STINNER Victor  added the comment:

See also bpo-45128 (fixed): "test_multiprocessing_fork fails if run 
sequentially after test_genericalias and test_logging".

--
nosy: +vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Nikita Sobolev]

Change by Nikita Sobolev :


--
pull_requests: +26631
pull_request: https://github.com/python/cpython/pull/28182

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset ab58269ab3b784bef33a613cd0a68914065a9134 by Serhiy Storchaka in 
branch '3.9':
[3.9] bpo-45042: Now test classes decorated with `requires_hashdigest` are not 
skipped (GH-28060) (GH-28169)
https://github.com/python/cpython/commit/ab58269ab3b784bef33a613cd0a68914065a9134


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[miss-islington]

miss-islington  added the comment:


New changeset e5976dd2e6e966183da59df99978ebcb4b3a32df by Miss Islington (bot) 
in branch '3.10':
bpo-45042: Now test classes decorated with `requires_hashdigest` are not 
skipped (GH-28060)
https://github.com/python/cpython/commit/e5976dd2e6e966183da59df99978ebcb4b3a32df


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
pull_requests: +26598
pull_request: https://github.com/python/cpython/pull/28169

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:


New changeset dd7b816ac87e468e2fa65ce83c2a03fe1da8503e by Nikita Sobolev in 
branch 'main':
bpo-45042: Now test classes decorated with `requires_hashdigest` are not 
skipped (GH-28060)
https://github.com/python/cpython/commit/dd7b816ac87e468e2fa65ce83c2a03fe1da8503e


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[miss-islington]

Change by miss-islington :


--
nosy: +miss-islington
nosy_count: 5.0 -> 6.0
pull_requests: +26597
pull_request: https://github.com/python/cpython/pull/28168

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[STINNER Victor]

Change by STINNER Victor :


--
nosy:  -vstinner

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Nikita Sobolev]

Nikita Sobolev  added the comment:

Two new issues created:
1. https://bugs.python.org/issue45053
2. https://bugs.python.org/issue45052

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Łukasz Langa]

Łukasz Langa  added the comment:

FYI, there seem to be two Windows-specific regressions since the tests were 
unintentionally disabled, namely test_shared_memory_basics and 
test_checksum_fodder.

Following Serhiy's advice, I elect to have those tests skipped for now on 
Windows and fix them through separate issues. We will be releasing 3.9.7 
according to schedule today.

--
nosy: +lukasz.langa

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Nikita Sobolev]

Nikita Sobolev  added the comment:

Serhiy, yes, you are right. I guess, it is safe to assume that 
load-time/test-time might make a difference, especially with `_hashlib`.

I've commited your suggestion, thanks a lot for your help!

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Here is the implementation of the second option:

def requires_hashdigest(digestname, openssl=None, usedforsecurity=True):
def decorator(func):
if isinstance(func, type):
setUpClass = func.__dict__.get('setUpClass')
if setUpClass is None:
def setUpClass(cls):
super(func, cls).setUpClass()
setUpClass.__qualname__ = func.__qualname__ + '.setUpClass'
setUpClass.__module__ = func.__module__
else:
setUpClass = setUpClass.__func__
setUpClass = classmethod(decorator(setUpClass))
func.setUpClass = setUpClass
return func
@functools.wraps(func)
def wrapper(*args, **kwargs):
try:
if openssl and _hashlib is not None:
_hashlib.new(digestname, usedforsecurity=usedforsecurity)
else:
hashlib.new(digestname, usedforsecurity=usedforsecurity)
except ValueError:
raise unittest.SkipTest(
f"hash digest '{digestname}' is not available."
)
return func(*args, **kwargs)
return wrapper
return decorator

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Thank you Nikita.

The difference between the original and the proposed in PR 28060 code is that 
in the original code the conditional was tested at the testing time, while in 
the proposed code it is tested at the loading time. I do not know what effect  
it causes on tests. If it does not matter, the code can be simpler:

   return unittest.skipIf(should_be_skipped,
  f"hash digest '{digestname}' is not available.")

But if it matters, there are two options:

1. In requires_hashdigest() raise exception if func is a class. Remove the 
decorator from classes and add it to test and setup methods or just to the 
setUpClass() classmethod.

2. In requires_hashdigest() check if func is a class, and if it is true, patch 
its test and setup methods (or just add a decorated setUpClass() classmethod).

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Nikita Sobolev]

Nikita Sobolev  added the comment:

Looks like it did, two newly-unignored tests are failing:
https://github.com/python/cpython/pull/28060#issuecomment-908295411

пн, 30 авг. 2021 г. в 15:16, Antoine Pitrou :

>
> Antoine Pitrou  added the comment:
>
> Wow. I hope this didn't hide any regression :-(
>
> --
>
> ___
> Python tracker 
> 
> ___
>

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Antoine Pitrou]

Antoine Pitrou  added the comment:

Wow. I hope this didn't hide any regression :-(

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Nikita Sobolev]

Change by Nikita Sobolev :


--
keywords: +patch
pull_requests: +26504
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/28060

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Nikita Sobolev]

Nikita Sobolev  added the comment:

Looks like that this happens because tests classes are decorated with 
`@hashlib_helper.requires_hashdigest('md5')`: 
https://github.com/python/cpython/blob/2a8127cafe1d196f858a3ecabf5f1df3eebf9a12/Lib/test/_test_multiprocessing.py#L4482-L4485

Tests above run fine without this decorator.
It was added in https://bugs.python.org/issue17258

The problem is that `` was treating a decorated entity as a function, not a 
class. 
https://github.com/python/cpython/blob/e6497fe698f6e87344501a68ffdea106eafcb257/Lib/test/support/hashlib_helper.py#L25
 But, it was decorating classes as well.
So, it needs to changed to respect classes.

I will make a PR shortly.

--
nosy: +sobolevn

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Serhiy Storchaka]

Change by Serhiy Storchaka :


--
priority: normal -> high

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue45042] Many multiprocessing tests are silently skipped since 3.9

[Serhiy Storchaka]

New submission from Serhiy Storchaka :

Here is a list of multiprocessing tests which are run in 3.8 but are not found 
in 3.9+:

OtherTest.test_answer_challenge_auth_failure
OtherTest.test_deliver_challenge_auth_failure
TestInitializers.test_manager_initializer
TestInitializers.test_pool_initializer
TestSyncManagerTypes.test_array
TestSyncManagerTypes.test_barrier
TestSyncManagerTypes.test_bounded_semaphore
TestSyncManagerTypes.test_condition
TestSyncManagerTypes.test_dict
TestSyncManagerTypes.test_event
TestSyncManagerTypes.test_joinable_queue
TestSyncManagerTypes.test_list
TestSyncManagerTypes.test_lock
TestSyncManagerTypes.test_namespace
TestSyncManagerTypes.test_pool
TestSyncManagerTypes.test_queue
TestSyncManagerTypes.test_rlock
TestSyncManagerTypes.test_semaphore
TestSyncManagerTypes.test_value
WithManagerTestBarrier.test_abort
WithManagerTestBarrier.test_abort_and_reset
WithManagerTestBarrier.test_action
WithManagerTestBarrier.test_barrier
WithManagerTestBarrier.test_barrier_10
WithManagerTestBarrier.test_default_timeout
WithManagerTestBarrier.test_reset
WithManagerTestBarrier.test_single_thread
WithManagerTestBarrier.test_thousand
WithManagerTestBarrier.test_timeout
WithManagerTestBarrier.test_wait_return
WithManagerTestCondition.test_notify
WithManagerTestCondition.test_notify_all
WithManagerTestCondition.test_notify_n
WithManagerTestCondition.test_timeout
WithManagerTestCondition.test_wait_result
WithManagerTestCondition.test_waitfor
WithManagerTestCondition.test_waitfor_timeout
WithManagerTestContainers.test_dict
WithManagerTestContainers.test_dict_iter
WithManagerTestContainers.test_dict_proxy_nested
WithManagerTestContainers.test_list
WithManagerTestContainers.test_list_iter
WithManagerTestContainers.test_list_proxy_in_list
WithManagerTestContainers.test_namespace
WithManagerTestEvent.test_event
WithManagerTestLock.test_lock
WithManagerTestLock.test_lock_context
WithManagerTestLock.test_rlock
WithManagerTestManagerRestart.test_rapid_restart
WithManagerTestMyManager.test_mymanager
WithManagerTestMyManager.test_mymanager_context
WithManagerTestMyManager.test_mymanager_context_prestarted
WithManagerTestPool.test_apply
WithManagerTestPool.test_async
WithManagerTestPool.test_async_timeout
WithManagerTestPool.test_context
WithManagerTestPool.test_empty_iterable
WithManagerTestPool.test_enter
WithManagerTestPool.test_imap
WithManagerTestPool.test_imap_handle_iterable_exception
WithManagerTestPool.test_imap_unordered
WithManagerTestPool.test_imap_unordered_handle_iterable_exception
WithManagerTestPool.test_make_pool
WithManagerTestPool.test_map
WithManagerTestPool.test_map_async
WithManagerTestPool.test_map_async_callbacks
WithManagerTestPool.test_map_chunksize
WithManagerTestPool.test_map_handle_iterable_exception
WithManagerTestPool.test_map_no_failfast
WithManagerTestPool.test_map_unplicklable
WithManagerTestPool.test_release_task_refs
WithManagerTestPool.test_resource_warning
WithManagerTestPool.test_starmap
WithManagerTestPool.test_starmap_async
WithManagerTestPool.test_terminate
WithManagerTestPool.test_traceback
WithManagerTestPool.test_wrapped_exception
WithManagerTestQueue.test_closed_queue_put_get_exceptions
WithManagerTestQueue.test_fork
WithManagerTestQueue.test_get
WithManagerTestQueue.test_no_import_lock_contention
WithManagerTestQueue.test_put
WithManagerTestQueue.test_qsize
WithManagerTestQueue.test_queue_feeder_donot_stop_onexc
WithManagerTestQueue.test_queue_feeder_on_queue_feeder_error
WithManagerTestQueue.test_task_done
WithManagerTestQueue.test_timeout
WithManagerTestRemoteManager.test_remote
WithManagerTestSemaphore.test_bounded_semaphore
WithManagerTestSemaphore.test_semaphore
WithManagerTestSemaphore.test_timeout
WithProcessesTestManagerRestart.test_rapid_restart
WithProcessesTestPicklingConnections.test_access
WithProcessesTestPicklingConnections.test_pickling
WithProcessesTestSharedMemory.test_shared_memory_ShareableList_basics
WithProcessesTestSharedMemory.test_shared_memory_ShareableList_pickling
WithProcessesTestSharedMemory.test_shared_memory_SharedMemoryManager_basics
WithProcessesTestSharedMemory.test_shared_memory_SharedMemoryManager_reuses_resource_tracker
WithProcessesTestSharedMemory.test_shared_memory_SharedMemoryServer_ignores_sigint
WithProcessesTestSharedMemory.test_shared_memory_across_processes
WithProcessesTestSharedMemory.test_shared_memory_basics
WithProcessesTestSharedMemory.test_shared_memory_cleaned_after_process_termination
WithThreadsTestManagerRestart.test_rapid_restart

--
components: Tests
keywords: 3.9regression
messages: 400521
nosy: davin, pitrou, serhiy.storchaka, vstinner
priority: normal
severity: normal
status: open
title: Many multiprocessing tests are silently skipped since 3.9
type: behavior
versions: Python 3.10, Python 3.11, Python 3.9

___
Python tracker 

___
___