>
> Hi Pavel,
>
> Thank you for your suggestions. I've now completed the initial checks on
> our SVN server and reviewed the Apache error logs.
>
> We are still experiencing performance degradation across multiple
> projects. SVN updates, commits and checkouts are taking considerably
> longer, and the TortoiseSVN Repository Browser sometimes takes 9-10 minutes
> to load. These slowdowns are intermittent, with operations sometimes
> completing normally and becoming extremely slow at other times.
>
> *Server-side software versions:*
>
>    -
>
>    Subversion: 1.14.0 (compiled August 17, 2020)
>    -
>
>    Apache HTTPD: 2.4.43
>    -
>
>    APR: 1.5.2
>    -
>
>    APR-Util: 1.5.4
>    -
>
>    Operating System: Ubuntu 20.04.5 LTS
>    -
>
>    Kernel: Linux 5.15
>
> Both dav_svn_module and authz_svn_module are loaded.
>
> I've also verified that our repositories use FSFS, repository format 8,
> with a sharded layout of 1000.
>
> *Additional findings from the Apache error logs:*
>
> We are seeing repeated errors involving the SQLite representation cache
> database (rep-cache.db) across multiple repositories..
>
> The recurring errors include:
>
>    -
>
>    sqlite[S5]: database is locked
>    -
>
>    Couldn't open rep-cache database
>    -
>
>    Couldn't perform atomic initialization
>
> We also found several instances where SVN commits succeeded, but
> subsequent filesystem processing reported errors involving rep-cache.db.
>
> For example:
>
> commit of r24669 succeeded, but an error occurred after the commit:
> sqlite[S5]: database is locked
>
> Additionally, the logs contain HTTP 500 errors, connection resets and
> errors while streaming REPORT responses.
>
> *Some infrastructure changes we are currently considering:*
>
>    1.
>
>    *Artist-side SVN client upgrade:* I have started upgrading the SVN
>    client to version 1.15 on some artist workstations. The central SVN server
>    is still running version 1.14.0.
>    2.
>
>    *Repository distribution:* Currently, all repositories for our Unreal
>    projects are hosted on a single SVN server. We are planning to split and
>    distribute these repositories across multiple servers rather than keeping
>    all projects on one server so that the traffic is controlled.
>    3.
>
>    *Storage migration:* We are also discussing the possibility of moving
>    the repositories from CIFS/SMB network storage to local SSD storage.
>
> Considering our current setup and the findings above, I would appreciate
> your guidance on the following:
>
>    1.
>
>    Could the repeated SQLite rep-cache.db locking errors be related to
>    hosting FSFS repositories on CIFS/SMB storage?
>    2.
>
>    Could these errors contribute to the performance degradation we're
>    experiencing, or might they be separate symptoms of an underlying storage
>    issue?
>    3.
>
>    What read-only diagnostics would you recommend to distinguish between
>    CIFS/SMB latency, locking behaviour, SQLite contention and other possible
>    bottlenecks?
>    4.
>
>    Would migrating the repositories to local SSD storage be your
>    recommended long-term solution? Are there any specific precautions we
>    should take during such a migration?
>    5.
>
>    Given that we are currently running Subversion 1.14.0 with older APR
>    dependencies, would you recommend evaluating a Subversion and Apache stack
>    upgrade separately from the storage migration?
>    6.
>
>    Would splitting the repositories across multiple SVN servers help
>    improve performance and scalability in our situation? Are there any
>    particular considerations when distributing FSFS repositories across
>    multiple servers?
>    7.
>
>    Is upgrading the artist-side SVN clients to 1.15 while the central
>    server remains on 1.14.0 a reasonable approach?
>
> Our primary objective is to identify the root cause of the slow SVN
> operations and restore consistent performance across projects.
>
> We would prefer to understand the underlying issues and establish the
> right infrastructure approach before making production changes.
>
> I can share the relevant Apache error-log excerpts if required.
>
> Thank you again for your time and guidance.
>
> Best regards,
> Ananya Srivastava
>

-- 


____________________________________________________________________________

Disclaimer: - This email and any files transmitted with it are strictly 
confidential and intended solely for the use of the individual or entity to 
whom they are addressed. If you have received this email in error, please 
notify the system manager. This message contains confidential information 
and is intended only for the individual named. If you are not the named 
addressee, you should not disseminate, distribute or copy this email. 
Please notify the sender immediately by email if you have received this 
email by mistake and delete this email from your system. If you are not the 
intended recipient, you are notified that disclosing, copying, distributing 
or taking any action in reliance on the contents of this information is 
strictly prohibited.


Reply via email to