Den tors 19 nov. 2026 kl 12:36 skrev Ananya Srivastava < [email protected]>:
> Hi Pavel, >> >> Thank you for your suggestions. I've now completed the initial checks on >> our SVN server and reviewed the Apache error logs. >> >> We are still experiencing performance degradation across multiple >> projects. SVN updates, commits and checkouts are taking considerably >> longer, and the TortoiseSVN Repository Browser sometimes takes 9-10 minutes >> to load. These slowdowns are intermittent, with operations sometimes >> completing normally and becoming extremely slow at other times. >> >> *Server-side software versions:* >> >> - >> >> Subversion: 1.14.0 (compiled August 17, 2020) >> - >> >> Apache HTTPD: 2.4.43 >> - >> >> APR: 1.5.2 >> - >> >> APR-Util: 1.5.4 >> - >> >> Operating System: Ubuntu 20.04.5 LTS >> - >> >> Kernel: Linux 5.15 >> >> Both dav_svn_module and authz_svn_module are loaded. >> >> I've also verified that our repositories use FSFS, repository format 8, >> with a sharded layout of 1000. >> >> *Additional findings from the Apache error logs:* >> >> We are seeing repeated errors involving the SQLite representation cache >> database (rep-cache.db) across multiple repositories.. >> >> The recurring errors include: >> >> - >> >> sqlite[S5]: database is locked >> - >> >> Couldn't open rep-cache database >> - >> >> Couldn't perform atomic initialization >> >> We also found several instances where SVN commits succeeded, but >> subsequent filesystem processing reported errors involving rep-cache.db. >> >> For example: >> >> commit of r24669 succeeded, but an error occurred after the commit: >> sqlite[S5]: database is locked >> >> Additionally, the logs contain HTTP 500 errors, connection resets and >> errors while streaming REPORT responses. >> >> *Some infrastructure changes we are currently considering:* >> >> 1. >> >> *Artist-side SVN client upgrade:* I have started upgrading the SVN >> client to version 1.15 on some artist workstations. The central SVN server >> is still running version 1.14.0. >> 2. >> >> *Repository distribution:* Currently, all repositories for our Unreal >> projects are hosted on a single SVN server. We are planning to split and >> distribute these repositories across multiple servers rather than keeping >> all projects on one server so that the traffic is controlled. >> 3. >> >> *Storage migration:* We are also discussing the possibility of moving >> the repositories from CIFS/SMB network storage to local SSD storage. >> >> I agree with what Pavel has already written - the most likely culprit is storing the repository on a network share. Local storage doesn't necessarily have to be directly attached to the server - if you have a proper SAN where you mount an iSCSI-target on the server that should work as well. TortoiseSVN has a FAQ entry about storing the repository (and working copy) on network shares: https://tortoisesvn.net/faq.html#repoonshare SQLite also has an entry about this: https://sqlite.org/useovernet.html TLDR; Don't do it. > Considering our current setup and the findings above, I would appreciate >> your guidance on the following: >> >> 1. >> >> Could the repeated SQLite rep-cache.db locking errors be related to >> hosting FSFS repositories on CIFS/SMB storage? >> >> Yes, certainly. If the SMB server and/or client isn't doing locking properly you will run into problems. > >> 1. >> >> Could these errors contribute to the performance degradation we're >> experiencing, or might they be separate symptoms of an underlying storage >> issue? >> 2. >> >> What read-only diagnostics would you recommend to distinguish between >> CIFS/SMB latency, locking behaviour, SQLite contention and other possible >> bottlenecks? >> 3. >> >> Would migrating the repositories to local SSD storage be your >> recommended long-term solution? Are there any specific precautions we >> should take during such a migration? >> >> Migration should be easy: Just shut down Apache so no user can access the repository. Then move it to local storage and reconfigure Apache to the new locations. > >> 1. >> >> Given that we are currently running Subversion 1.14.0 with older APR >> dependencies, would you recommend evaluating a Subversion and Apache stack >> upgrade separately from the storage migration? >> >> I don't think there should be any significant difference running a later version but there is always a chance some bugs have been fixed. > >> 1. >> >> Would splitting the repositories across multiple SVN servers help >> improve performance and scalability in our situation? Are there any >> particular considerations when distributing FSFS repositories across >> multiple servers? >> >> That probably depends on the load of the server but I don't see a particular need to do that unless, after migration to local storage, you continue to see high load and high IO load. > >> 1. >> >> Is upgrading the artist-side SVN clients to 1.15 while the central >> server remains on 1.14.0 a reasonable approach? >> >> That would probably not help much. If the clients start to use "pristine on demand" (new feature in 1.15) it might even make matter worse since that trades local storage for network traffic/server load. Our primary objective is to identify the root cause of the slow SVN >> operations and restore consistent performance across projects. >> >> We would prefer to understand the underlying issues and establish the >> right infrastructure approach before making production changes. >> >> I can share the relevant Apache error-log excerpts if required. >> >> Thank you again for your time and guidance. >> >> Best regards, >> Ananya Srivastava >> > Hope this helps Daniel > > > ____________________________________________________________________________ > Disclaimer: - This email and any files transmitted with it are strictly > confidential and intended solely for the use of the individual or entity to > whom they are addressed. If you have received this email in error, please > notify the system manager. This message contains confidential information > and is intended only for the individual named. If you are not the named > addressee, you should not disseminate, distribute or copy this email. > Please notify the sender immediately by email if you have received this > email by mistake and delete this email from your system. If you are not the > intended recipient, you are notified that disclosing, copying, distributing > or taking any action in reliance on the contents of this information is > strictly prohibited. >
