[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:

> https://github.com/python/buildmaster-config/pull/314

I merged this PR as well. The new buildbot configuration will be deployed soon.

--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 359bc392ba2b8f2acca223426c8210bb74f724c6 by Victor Stinner in 
branch '3.9':
[3.10] bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632) (GH-31634) 
(GH-31644)
https://github.com/python/cpython/commit/359bc392ba2b8f2acca223426c8210bb74f724c6


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29764
pull_request: https://github.com/python/cpython/pull/31644

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 93264452d952d9ba604bacf2947c2df5dd477931 by Victor Stinner in 
branch '3.10':
[3.10] bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632) (GH-31634)
https://github.com/python/cpython/commit/93264452d952d9ba604bacf2947c2df5dd477931


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

Change by STINNER Victor :


--
pull_requests: +29756
pull_request: https://github.com/python/cpython/pull/31634

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 9204bb72a2da5885facc747e63d2bd2d654606fe by Victor Stinner in 
branch 'main':
bpo-46633: Skip tests on ASAN and/or MSAN builds (GH-31632)
https://github.com/python/cpython/commit/9204bb72a2da5885facc747e63d2bd2d654606fe


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:

"""
No longer skipped:

* test_capi
* test_ctypes
* test_faulthandler
* test_interpreters
"""

I built Python manually with:

  ./configure --with-pydebug CC=clang LD=clang --with-address-sanitizer 

These tests no longer with ASAN_OPTIONS used on our CI:

ASAN_OPTIONS='detect_leaks=0:allocator_may_return_null=1:handle_segv=0' 
./python -m test (...)

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:

I created bpo-46887: ./Programs/_freeze_module fails with MSAN: Uninitialized 
value was created by an allocation of 'stat.i'.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:

Status with the 2 pending PRs:

* GH-31632
* https://github.com/python/buildmaster-config/pull/314

Tests only skipped on ASAN:

* _test_multiprocessing

Skip on ASAN and MSAN:

* test___all__
* test_concurrent_futures
* test_crypt
* test_decimal.test_maxcontext_exact_arith()
* test_idle
* test_peg_generator
* test_tix
* test_tk
* test_tools
* test_ttk_guionly

No longer skipped:

* test_capi
* test_ctypes
* test_faulthandler
* test_interpreters

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

Change by STINNER Victor :


--
keywords: +patch
pull_requests: +29754
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/31632

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:

I created a PR to no longer skip tests in buildbots:
https://github.com/python/buildmaster-config/pull/314

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

I have recently added some decorators in test.
support to deactivate tests if running under the sanitizers.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

STINNER Victor  added the comment:

Notes:

* test.support has check_sanitizer() function. Tests using it:

  * test_crypt
  * test_idle
  * test_tix
  * test_tk
  * test_ttk_guionly


* test_decimal also checks for memory sanitizer and skip some tests if it's 
present:

MEMORY_SANITIZER = (
'-fsanitize=memory' in _cflags or
'--with-memory-sanitizer' in _config_args
)   

* test_faulthandler suppress crash report, similir to 
support.SuppressCrashReport, directly in the C code used by tests: 
faulthandler_suppress_crash_report() function.

* Objects/obmalloc.c checks "#if __has_feature(address_sanitizer)": is ASAN 
enabled?

* GitHub Action config: .github/workflows/build.yml. Skipped tests:

  * test___all__
  * test_concurrent_futures
  * test_multiprocessing_fork
  * test_multiprocessing_forkserver
  * test_multiprocessing_spawn
  * test_peg_generator
  * test_tools

Comment:
---
# Skip test_tools test_peg_generator test_concurrent_futures because
# there are too slow: between 5 and 20 minutes on this CI.
#
# Skip multiprocessing and concurrent.futures tests which are affected by
# bpo-45200 bug: libasan dead lock in pthread_create().
#
# test___all__ is skipped because importing some modules directly can trigger
# known problems with ASAN (like tk or crypt).
---

* Buildbot configuration: UnixAsanBuild class of master/custom/factories.py. 
Options:

  ASAN_OPTIONS='detect_leaks=0:allocator_may_return_null=1:handle_segv=0'

With the comment:
---
# See https://bugs.python.org/issue42985 for more context on why
# SIGSEGV is ignored on purpose.
---

  Skipped tests:
  
  * test_ctypes
  * test_capi
  * test_crypt
  * test_decimal
  * test_faulthandler
  * test_interpreters

With the comment:
---
# These tests are currently raising false positives or are interfering with the 
ASAN mechanism,
# so we need to skip them unfortunately.
---

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue46633] AddressSanitizer: Skip tests directly in Python, not with external config

[STINNER Victor]

New submission from STINNER Victor :

It seems like bpo-45200: "Address Sanitizer: libasan dead lock in 
pthread_create() (test_multiprocessing_fork.test_get() hangs)" is not fixed yet.

In the GitHub Action job, test_multiprocessing_fork is skipped because it's too 
slow, so the job doesn't hang.

Yesterday, I modified the ASAN buildbot to double its timeout from 20 to 40 
minutes:
https://github.com/python/buildmaster-config/commit/5a37411e75c9475d48eabaac18102a3c9fc2d467

But it's useful, when it hangs, it hangs forever. Exmaple on the AMD64 Arch 
Linux Asan Debug 3.9 buildbot (with the new config):

---
(test.test_multiprocessing_fork.WithProcessesTestPicklingConnections) ... ok
Timeout (0:35:00)!
---
https://buildbot.python.org/all/#/builders/588/builds/332

Tests are tuned for ASAN, but the configuration is copied and inconsistent 
between the GitHub Actions job and the buildbot configuration.

I propose to move this configuration directly into Python.

test_decimal.py checks for "-fsanitize=address" in CFLAGS and skip some tests 
if it's present.

--
components: Tests
messages: 412499
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: AddressSanitizer: Skip tests directly in Python, not with external config
versions: Python 3.11

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com