On Thu, 14 May 2026, 13:51 Timofei Zhakov, <[email protected]> wrote:
> On Wed, May 13, 2026 at 10:02 PM Branko Čibej <[email protected]> wrote: > >> On 13. 5. 26 16:41, Nathan Hartman wrote: >> >> On Wed, May 13, 2026 at 8:17 AM Ivan Zhakov <[email protected]> wrote: >> >>> On Mon, 27 Apr 2026 at 21:41, Evgeny Kotkov via dev < >>> [email protected]> wrote: >>> >>>> The 1.15.0-rc2 release artifacts are now available for testing/signing. >>>> Please get the tarballs from >>>> https://dist.apache.org/repos/dist/dev/subversion >>>> and add your signatures there. >>>> >>>> >>> I encountered several new issues while trying to run the test suite on >>> Windows: >>> >>> 1) I typically use the Python embeddable package distributed as a .zip >>> archive [1][2]. This worked well for Subversion 1.14, but fails for 1.15, >>> because the embeddable package does not include the venv module: >>> [[[ >>> File >>> "C:\Ivan\SVN\svn-1.15.x\subversion\tests\cmdline\svntest\__init__.py", line >>> 59, in <module> >>> from . import main >>> File >>> "C:\Ivan\SVN\svn-1.15.x\subversion\tests\cmdline\svntest\main.py", line 43, >>> in <module> >>> import venv >>> ModuleNotFoundError: No module named 'venv' >>> ]]] >>> >>> The workaround is to perform a full installation of Python via an >>> exe-based installer or package manager. This could be inconvenient for >>> package maintainers and automated tools, as it requires modifying the host >>> environment rather than simply extracting and using a portable binary. >>> >>> 2) Running the tests in an offline environment now introduces a >>> 75-second delay (5 retries x 15s) before the tests start. This is caused by >>> attempts to fetch Python packages: >>> [[[ >>> Testing Release configuration on local repository. >>> WARNING: Retrying (Retry(total=4, connect=None, read=None, >>> redirect=None, status=None)) after connection broken by >>> 'ConnectTimeoutError(<pip._vendor.urllib3.connection.HTTPSConnection object >>> at 0x00000298BBEEB010>, 'Connection to pypi.org timed out. (connect >>> timeout=15)')': /simple/lxml/ >>> ... >>> ]]] >>> >>> This may reduce reproducibility and potentially cause issues for package >>> maintainers building in offline or hardened environments. >>> >>> For example, here is a quote from Debian Policy [3]: >>> [[[ >>> Except for packages in the non-free archive with the Autobuild control >>> field unset or set to no, required targets must not attempt network access >>> to other hosts. Only access via the loopback interface to services on the >>> build host that have been started by the build is allowed. >>> ]]] >>> >>> 3) I haven't checked the details yet, but if we don't pin the >>> versions/signatures of the fetched packages, this may introduce a supply >>> chain attack threat for anyone running the Python tests. Because the test >>> runner now includes a step that automatically downloads and executes code >>> from a remote URL. >>> >>> Thoughts? >>> >>> [1] https://www.python.org/downloads/windows/ >>> [2] >>> https://www.python.org/ftp/python/3.14.5/python-3.14.5-embed-amd64.zip >>> [3] https://www.debian.org/doc/debian-policy/ch-source.html >>> >>> -- >>> Ivan Zhakov >>> >> >> >> I have the same concerns and didn't have an opportunity to articulate >> them. Thank you for doing so! >> >> This was added in r1925717 and followed up in r1925899. >> >> Prior to r1925717 we didn't run pip from the test suite at all. Now we >> are using pip to install lxml [1] and rnc2rng [2]. >> >> Normally I would ask if we could just copy the relevant files but >> unfortunately these appear to be pretty hefty packages in their own >> right. >> >> It looks like nothing will be fetched if the system happens to have >> lxml and rnc2rng already. >> >> Otherwise... At the very least, we should pin the versions/signatures >> as Ivan suggests, both for consistency across runs and to mitigate >> potential supply chain attacks. >> >> May I suggest going even further: if the system doesn't have these >> packages, the tests that require them could just be skipped. Yes, it's >> convenient to have the test suite fetch dependencies automatically, >> but I think it isn't the test suite's job to install stuff. It should >> only run tests. Just my .02... >> >> >> >> No need to skip the tests, svntest.verify.validate_xml_schema() already >> has a graceful fallback if lxml isn't found, and so does >> svntest.main.is_bad_xml_fatal(). >> >> As for "the system not having these dependencies", no system will unless >> you force developers to create a Python venv just for tests. Instead, we >> create that venv for tests in the test suite. >> >> As for package sizes, lxml especially depends on a platform-specific >> native library. Including that in our repo is a non-starter. As it is now, >> the test suite creates that virtual environment once in the test binary >> directory. >> >> We can do (one of or all of) the following: >> >> >> 1. Pin the dependency versions. This is a one-liner change of >> SVN_TESTS_REQUIRE in svntest.main >> 2. Do not fail if the 'venv' module isn't available. That is a >> one-liner change in svntest.main where the module is imported. Just my 2 >> cents, a Python3 installation without that module is not a Python3 >> installation. >> 3. Have an option to turn off creating the virtualenv and installing >> dependencies even if we have the venv module. There's already an option >> --python-venv, we could introduce a special value that means "don't create >> the venv". It would be up to whoever is running the test suite in an >> isolated environment to turn on that option. >> >> >> Then there is schema conversion, that's built into Makefile.in and >> probably not replicated in the CMake build. >> >> Personally I find it unacceptable that we had all these schemas but never >> actually tested the --xml output of our commands against them. It has been >> a while since I did this but I think there were a couple cases where we >> generated invalid XML. This addition to the tests caught those cases. >> > > We already ask users to provide their own version of APR, Serf, Expat, > Zlib, lz4, utf8proc, sqlite, httpd - and those are only required > dependencies not even including more exotic ones. > > Of course because it's a more unix-ish solution where the environment > defines how an application should be built and configured. What's so > exceptional about the Python test-suite? Maybe I want some specific version > of lxml? Perhaps there is a bug that I have locally fixed? > > -- > Timofei Zhakov > The point is that I didn't want these libs to be mandatory dependencies, but do want the majority of developer and CI tests to include the XML schema check. No amount of documentation is going to replace actual code that installs those modules in a temporary location. You'll forget about them and run your tests and they'll pass and you'll be happy and we still, after 2 decades, have edge cases of 'svn log' producing invalid XML. Anyway, I've started to work on a change that takes a slightly different approach, to support testing in environments that don't have 'venv' or internet access. This may take the form of the test suite explicitly failing tests if XML validation isn't performed, unless specifically told not to. However I'm now AFK in hospital and it will take a week or two before I can get back to this. -- Brane
