On Wed, May 13, 2026 at 10:02 PM Branko Čibej <[email protected]> wrote:
> On 13. 5. 26 16:41, Nathan Hartman wrote: > > On Wed, May 13, 2026 at 8:17 AM Ivan Zhakov <[email protected]> wrote: > >> On Mon, 27 Apr 2026 at 21:41, Evgeny Kotkov via dev < >> [email protected]> wrote: >> >>> The 1.15.0-rc2 release artifacts are now available for testing/signing. >>> Please get the tarballs from >>> https://dist.apache.org/repos/dist/dev/subversion >>> and add your signatures there. >>> >>> >> I encountered several new issues while trying to run the test suite on >> Windows: >> >> 1) I typically use the Python embeddable package distributed as a .zip >> archive [1][2]. This worked well for Subversion 1.14, but fails for 1.15, >> because the embeddable package does not include the venv module: >> [[[ >> File >> "C:\Ivan\SVN\svn-1.15.x\subversion\tests\cmdline\svntest\__init__.py", line >> 59, in <module> >> from . import main >> File "C:\Ivan\SVN\svn-1.15.x\subversion\tests\cmdline\svntest\main.py", >> line 43, in <module> >> import venv >> ModuleNotFoundError: No module named 'venv' >> ]]] >> >> The workaround is to perform a full installation of Python via an >> exe-based installer or package manager. This could be inconvenient for >> package maintainers and automated tools, as it requires modifying the host >> environment rather than simply extracting and using a portable binary. >> >> 2) Running the tests in an offline environment now introduces a 75-second >> delay (5 retries x 15s) before the tests start. This is caused by attempts >> to fetch Python packages: >> [[[ >> Testing Release configuration on local repository. >> WARNING: Retrying (Retry(total=4, connect=None, read=None, >> redirect=None, status=None)) after connection broken by >> 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object >> at 0x00000298BBEEB010>, 'Connection to pypi.org timed out. (connect >> timeout=15)')': /simple/lxml/ >> ... >> ]]] >> >> This may reduce reproducibility and potentially cause issues for package >> maintainers building in offline or hardened environments. >> >> For example, here is a quote from Debian Policy [3]: >> [[[ >> Except for packages in the non-free archive with the Autobuild control >> field unset or set to no, required targets must not attempt network access >> to other hosts. Only access via the loopback interface to services on the >> build host that have been started by the build is allowed. >> ]]] >> >> 3) I haven't checked the details yet, but if we don't pin the >> versions/signatures of the fetched packages, this may introduce a supply >> chain attack threat for anyone running the Python tests. Because the test >> runner now includes a step that automatically downloads and executes code >> from a remote URL. >> >> Thoughts? >> >> [1] https://www.python.org/downloads/windows/ >> [2] >> https://www.python.org/ftp/python/3.14.5/python-3.14.5-embed-amd64.zip >> [3] https://www.debian.org/doc/debian-policy/ch-source.html >> >> -- >> Ivan Zhakov >> > > > I have the same concerns and didn't have an opportunity to articulate > them. Thank you for doing so! > > This was added in r1925717 and followed up in r1925899. > > Prior to r1925717 we didn't run pip from the test suite at all. Now we > are using pip to install lxml [1] and rnc2rng [2]. > > Normally I would ask if we could just copy the relevant files but > unfortunately these appear to be pretty hefty packages in their own > right. > > It looks like nothing will be fetched if the system happens to have > lxml and rnc2rng already. > > Otherwise... At the very least, we should pin the versions/signatures > as Ivan suggests, both for consistency across runs and to mitigate > potential supply chain attacks. > > May I suggest going even further: if the system doesn't have these > packages, the tests that require them could just be skipped. Yes, it's > convenient to have the test suite fetch dependencies automatically, > but I think it isn't the test suite's job to install stuff. It should > only run tests. Just my .02... > > > > No need to skip the tests, svntest.verify.validate_xml_schema() already > has a graceful fallback if lxml isn't found, and so does > svntest.main.is_bad_xml_fatal(). > > As for "the system not having these dependencies", no system will unless > you force developers to create a Python venv just for tests. Instead, we > create that venv for tests in the test suite. > > As for package sizes, lxml especially depends on a platform-specific > native library. Including that in our repo is a non-starter. As it is now, > the test suite creates that virtual environment once in the test binary > directory. > > We can do (one of or all of) the following: > > > 1. Pin the dependency versions. This is a one-liner change of > SVN_TESTS_REQUIRE in svntest.main > 2. Do not fail if the 'venv' module isn't available. That is a > one-liner change in svntest.main where the module is imported. Just my 2 > cents, a Python3 installation without that module is not a Python3 > installation. > 3. Have an option to turn off creating the virtualenv and installing > dependencies even if we have the venv module. There's already an option > --python-venv, we could introduce a special value that means "don't create > the venv". It would be up to whoever is running the test suite in an > isolated environment to turn on that option. > > > Then there is schema conversion, that's built into Makefile.in and > probably not replicated in the CMake build. > > Personally I find it unacceptable that we had all these schemas but never > actually tested the --xml output of our commands against them. It has been > a while since I did this but I think there were a couple cases where we > generated invalid XML. This addition to the tests caught those cases. > We already ask users to provide their own version of APR, Serf, Expat, Zlib, lz4, utf8proc, sqlite, httpd - and those are only required dependencies not even including more exotic ones. Of course because it's a more unix-ish solution where the environment defines how an application should be built and configured. What's so exceptional about the Python test-suite? Maybe I want some specific version of lxml? Perhaps there is a bug that I have locally fixed? -- Timofei Zhakov
