[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[STINNER Victor]

STINNER Victor  added the comment:


New changeset b894b669c98cc365b84cbb8d20f531f1d0686f59 by Victor Stinner in 
branch '3.7':
Update libregrtest from master (GH-19517)
https://github.com/python/cpython/commit/b894b669c98cc365b84cbb8d20f531f1d0686f59


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[STINNER Victor]

STINNER Victor  added the comment:


New changeset 67b8a1f0f0f78ec38b8626fa9f5b2f5a55c17e15 by Victor Stinner in 
branch '3.8':
[3.8] Update libregrtest from master (GH-19516)
https://github.com/python/cpython/commit/67b8a1f0f0f78ec38b8626fa9f5b2f5a55c17e15


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

Ok, I reopen until the backports

--
resolution: fixed -> 
status: closed -> open

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[STINNER Victor]

STINNER Victor  added the comment:

I'm trying to keep regrtest in sync between 3.7, 3.8 and master branches. Maybe 
backport this change to 3.7 and 3.8 branches once buildbots validated the 
change? I'm keeping them in sync mostly to make bugfixes easier, but also to 
make my life easier when I debug issues in 3.7 or 3.8 branch.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
resolution:  -> fixed
stage: patch review -> resolved
status: open -> closed
versions:  -Python 3.8

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:


New changeset e0cd8aa70a3ce19c3d3712568940aa0cbd9aa97b by Pablo Galindo in 
branch 'master':
bpo-37957: Allow regrtest to receive a file with test (and subtests) to ignore 
(GH-16989)
https://github.com/python/cpython/commit/e0cd8aa70a3ce19c3d3712568940aa0cbd9aa97b


--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[Pablo Galindo Salgado]

Change by Pablo Galindo Salgado :


--
keywords: +patch
pull_requests: +16515
stage:  -> patch review
pull_request: https://github.com/python/cpython/pull/16989

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[STINNER Victor]

STINNER Victor  added the comment:

FYI in the past, Fedora and RHEL packages skipped multiple tests using 
downstream patches. Hopefully, we managed to reenable almost all tests.

Currently in Fedora, the python3 package use regrtest -x to skip some tests:

  # Run the upstream test suite
  # test_gdb skipped on s390x:
  #   https://bugzilla.redhat.com/show_bug.cgi?id=1678277
  # test_gdb skipped everywhere:
  #   https://bugzilla.redhat.com/show_bug.cgi?id=1734327
  # test_distutils
  #   distutils.tests.test_bdist_rpm tests fail when bootstraping the Python
  #   package: rpmbuild requires /usr/bin/pythonX.Y to be installed
  LD_LIBRARY_PATH=$ConfDir $ConfDir/python -m test.regrtest \
-wW --slowest -j0 \
%if %{with bootstrap}
-x test_distutils \
%endif
-x test_gdb \
%ifarch %{mips64}
-x test_ctypes \
%endif

https://src.fedoraproject.org/rpms/python3/blob/master/f/python3.spec

We could only skip some test methods, but it's simpler for us to skip a whole 
test file.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[Pablo Galindo Salgado]

Pablo Galindo Salgado  added the comment:

> Tests that fail on some platforms can be marked to skip in code.

One of the use cases of this feature is for people to package Python at 
distributions or companies and want to run the test suite without modifying the 
code or applying patches.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[STINNER Victor]

STINNER Victor  added the comment:

> Tests that fail on some platforms can be marked to skip in code.

We provide best-effort support for AIX, but I consider that Solaris is no 
longer supported, and so I would prefer to avoid code specific to Soliars in 
the Python source code. Except if Pablo wants to provide again best-effort 
support for Solaris?
https://pythondev.readthedocs.io/platforms.html

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[Serhiy Storchaka]

Serhiy Storchaka  added the comment:

Tests that fail on some platforms can be marked to skip in code.

For example:

@unittest.skipIf(sys.platform.startswith('aix'),
 'bpo-29972: broken test on AIX')
def test_strcoll_with_diacritic(self):
self.assertLess(locale.strcoll('à', 'b'), 0)

--
nosy: +serhiy.storchaka

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[STINNER Victor]

STINNER Victor  added the comment:

I'm fine with adding an --excludefile=filename option to regrtest. Do you want 
to try to implement it? A test can be added in test_regrtest.

--

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com

[issue37957] Allow regrtest to receive a file with test (and subtests) to ignore

[Pablo Galindo Salgado]

New submission from Pablo Galindo Salgado :

When building Python in some uncommon platforms (I am looking at you Solaris 
and AIX) there are some known tests that will fail. Right now, regrtest has the 
ability to ignore entire tests using the -x option and to receive a filter file 
using the --matchfile filter.

The problem with the --matchfile option is that it receives a file with 
patterns to accept and when you want to ignore a couple of tests and subtests, 
is too cumbersome to lists ALL tests that are not the ones that you want to 
accept.

The problem with -x is that is not easy to ignore just a subtests that fail and 
the whole tests needs to be ignored.

So I suggest to add a new command line option similar to --matchfile but the 
other way around. Another possibility is allowing to reverse the meaning of the 
matchfile argument, but I find that a bit more confusing.

--
components: Tests
messages: 350584
nosy: pablogsal, vstinner
priority: normal
severity: normal
status: open
title: Allow regrtest to receive a file with test (and subtests) to ignore
type: enhancement
versions: Python 3.8, Python 3.9

___
Python tracker 

___
___
Python-bugs-list mailing list
Unsubscribe: 
https://mail.python.org/mailman/options/python-bugs-list/archive%40mail-archive.com