On Fri, Jun 26, 2015 at 11:04:35AM -0600, Stephen John Smoogen wrote: > On 26 June 2015 at 10:11, Pierre-Yves Chibon <[email protected]> wrote: > > So the UMDL script currently supports three ways of crawling the tree: > > * file > > * rsync > > * directory > > > > We, in Fedora, are only using the last one. I believe the `rsync` mode was > > added > > to support Ubuntu and the file mode is basically a simplified version of the > > directory mode, but that we do not use at at the moment. > > > > I would like to propose that we drop support for rsync. I feel that it may > > be > > simpler and easier to create an UMDL and a crawler for each distro that > > would > > like to use MirrorManager than maintaining a one-script-fits-all UMDL that > > is > > in fact tested for only one of the scenario. > > That being said, if we ever have interest from Ubuntu, CentOS or any other > > communities, we should definitively look into making the UMDL and crawler as > > re-usable as possible for them, but keeping the distro-specific bits > > separated. > > > > I thought rsync was added for speed versus Ubuntu and that we actually > use it a lot on the crawlers... because that is the reason we needed > to move the memory usage up.. it uses a lot of memory but takes a lot > shorter period to 'crawl' or am I conflating two different things?
That's the rsync of the crawler scanning the mirrors, that's much faster
compared to http/ftp. umdl analyzing the master mirror can either use
the local filesystem or rsync or a file as input.
Adrian
pgpXrYl5WhphJ.pgp
Description: PGP signature
_______________________________________________ infrastructure mailing list [email protected] https://admin.fedoraproject.org/mailman/listinfo/infrastructure
