On Thu, Aug 10, 2023 at 6:53 PM <kjohn...@eclypse.org> wrote: > > Let me start by saying that I recognize that this situation is probably > specific to Debian. Even so, it seems possible that someone here might be > able to give me helpful ideas about where or how to start looking to track > down the symptom I am seeing. > > A set of svnadmin dump commands that run as part of a backup procedure seem > to be _much_ slower in Bookworm than in Bullseye. Prior to the upgrade to > Bullseye, these commands took slightly less than one hour. After the > upgrade, similar commands (dumping a few more revisions) require more than > six hours. The script that performs these commands has not been changed. > > Bookworm has subversion 1.14.2. Bullseye had 1.14.1. > > If you had these symptoms, where would you start to look? > > Thanks, > > Ken
That's a pretty significant slowdown. Unfortunately there's no easy way to say what might cause it, since many things can happen (and change) between OS releases. Subversion is only one of many, probably thousands, of components that changed during the OS upgrade. Whatever is causing the slowdown could be in Subversion or in another component. I assume the two releases are running on the same hardware? Same storage hardware? Same file system format? Can you think of anything else significant that may have changed in the system besides the OS upgrade itself? Has anything very large, or a very large number of revisions, been committed to the repository after the OS upgrade took place? Could anything significant be happening in the background? Especially something related to disks? The strategy should be divide and conquer, trying to isolate the problem to a specific component or at least rule out as many things as possible to narrow down the search space. Since you've noticed a slowdown related to dumping the Subversion repositories, and if there's nothing else that you can easily test first, then you might try to rule out (or rule in) whether the slowdown is caused by changes in Subversion between 1.14.1 and 1.14.2. Since it was "fast" with 1.14.1 and "slow" with 1.14.2, the way I might approach it would be to build both 1.14.1 and 1.14.2 on the new OS and try the dump with each. You don't need to build with all the options and dependencies; you only need the minimal dependencies (APR, APR-util, sqlite-amalgamation, and utf8proc) to get a functional svnadmin. You probably shouldn't do builds and tests on a production system that is serving repositories; I would use another machine for the tests. If your builds of 1.14.1 and 1.14.2 take approximately the same time to perform the dump, then the problem is not in Subversion itself, you've ruled that out, and you'd have to look elsewhere in the system: If both are "slow" then something else changed in the OS or hardware or the repository being dumped. If both are "fast" while the package manager's 1.14.2 is "slow" then I'd wonder what's different about the packaged binaries. If, however, there is a significant speed difference -- i.e., if 1.14.1 is "fast" and 1.14.2 is "slow" -- then I'd start looking into what changed in Subversion; if we get to this stage, then I would go into strategies of how to minimize the search effort within Subversion. But first, we have to know whether Subversion is the component to dive further into. By the way, this is the list of changes between 1.14.1 and 1.14.2, with the trunk revision from which each change was merged to the 1.14.x branch. Before I wrote the above, I looked over this list to see if anything glaringly obvious caught my attention; I didn't see anything like that, which is why I'm suggesting to rule out effects from other components first. * r1889487 swig-py: Fix doubly destroying memory pool with cyclic garbage collector. * r1896877 Fix misleading -r option documentation for some svnadmin subcommands. * r1887704 Follow up to r1865987, r1866588: Unbreak a msgid. * r1891908 Fix a bug where «make davautocheck» immediately after configure failed hard because of an unbuilt dependency. * r1885199 swig-py: Skip some tests on Python 2 if default encoding is 'utf-8'. * r1886708 swig-py: Fix dependency of make copy-swig-py target. * r1883838, r1883989, r1886460, r1886582 Add test coverage for CVE-2020-17525 (mod_authz_svn NULL deref) * r1887641, r1890013 Fix encoding of error message on failure of system() call. * r1889629 Document how the port number is passed to custom tunnels. * r1897449 tests: Include additional information in an error message. * r1894734 Fix issue #4880, "Use-after-free of object-pools when running in httpd" * r1892121 Fix an error message when running make davautocheck. * r1883355 Use the APR-1.4+ API for flushing file contents to disk. * r1899227 Don't show unreadable copyfrom paths in 'svn log -v' * r1898633 Fix sporadic testCrash_RequestChannel_nativeRead_AfterException failure * r1892470, r1892471, r1892541 Fix an assertion failure triggered by the conflict resolver. * r1881534 Fix issue #4864 "build/ac-macros/macosx.m4: workaround AC_RUN_IFELSE" * r1896609 Bumps copyright date to 2022 * r1890231 README: Document new home of our IRC channels at libera.chat * r1894491 INSTALL: Document not to build ZLib with ASM optimizations * r1884474 Fixes authz tests on Windows Hope this helps, Nathan
