> On Dec 20, 2025, at 3:09 PM, sebb <[email protected]> wrote: > > On Sat, 20 Dec 2025 at 18:44, Craig Russell <[email protected] > <mailto:[email protected]>> wrote: >> >> >> >>> On Dec 19, 2025, at 16:53, Greg Stein <[email protected]> wrote: >>> >>> On Fri, Dec 19, 2025 at 9:32 AM Craig Russell <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>>>> On Dec 18, 2025, at 18:02, Greg Stein <[email protected]> wrote: >>>>> >>>>> Because the svn-based solution is hitting operational snags. It MUST be >>>>> replaced. >>>>> >>>>> svn itself has no problem with a repository measured in dozens of >>>>> terabytes. But that totally blows up operational concerns. >>>> >>>> Last I heard (and I do pay attention) svn is where the projects put their >>>> stuff. After that, infra makes dozens of copies and distributes them around >>>> the world and delivers them via dyn/closer and/or dlcdn. Svn does not have >>>> to scale. >>> >>> >>> Read what I wrote. >>> >>> svn scales just fine. That is NOT the problem. Trying to backup dozens of >>> terabytes, or to copy them to a new machine ... that is the *operational* >>> concern of such a massive svn repository. Not svn itself. >> >>> >>> Compare trying to manage (1) 40T file (an svn repository is many files, but >>> logically must be considered as a single file; much like we look at a >>> single file being composed as many sectors on a hard drive), compared to >>> managing 100,000 files, each at 400M. ... The latter is much easier. You >>> can thing up. Spread them out. Copy little bits at a time. So many more >>> options. Serving the artifacts can become easier because of serving off >>> disk, rather than via mod_dav_svn. (we serve off disk, actually, but that >>> requires svnwcsub which introduces its own set of synchronization >>> problems). Throw in an rsync service to just keep things toasty. >> >> There are two aspects of scalability that might be relevant here. >> >> 1. Volume of requests to the repository. This is apparently solved by having >> dyn/closer and dlcdn serving artifacts distributed around the world. >> >> 2. Size of the repository. If the entire dist repository is in one file, >> that's a recipe for infra overload. >> >> An alternative is that without changing the external view of the repo, each >> of the subdirectories could be its own svn repository. That would cut down >> the size of the dist repo to 1% of its current size.
That would make the cron jobs that do the svn up much more complex and subject to lots of maintenance issues. >> >> Of course, changing the structure of dist would impact anyone who has the >> repo checked out to a local copy, but... > > I assumed downloads.a.o was a checkout of the dist/release directory > tree - is that not the case? > Or is there problem using SVN to maintain such a checkout? downloads.a.o of which there are currently three boxes and archives.a.o which is singular are rsyncs of rsync.a.o which has the checkout of svn dist release. We are not using svn at (1) Infra’s request and (2) we need a different, more fine-grained policy on who can write to the release distribution. To be investigated is what it means for a Release to be a Trusted Release. This is likely to mean that no single individual approves adding or deleting a release. In addition other policies like upload size limitations are currently done using svn tricks. Which PMCs allow RMs to be non-PMC committers is done via svn directory permissions, but svn is the only place that PMC choice is recorded. SVN dist is a policy enforcement nightmare. Greg is correct in pointing out this is policy and not “security” while I’m inclined to consider trust a security issue. I’m not going to debate this more right now. Best, Dave > >> Craig >> >>> >>> Dave says there is something related to security, but I'm not buying that >>> whatsoever. svn has fine-grained permissions, which is implemented for >>> repos/dist/, so I don't know what he's talking about. Maybe he should ask >>> an svn expert. >>> >>> Regards, >>> -g >> >> Craig L Russell >> [email protected] >> > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > <mailto:[email protected]> > For additional commands, e-mail: [email protected] > <mailto:[email protected]>
