On 11 February 2015 at 06:10, Pierre-Yves Chibon <[email protected]> wrote:
> Hi everyone, > > We currently have MirrorManager2 running on staging. It's apparently not > 100% > set-up since we get emails once in a while that one of the crons failed > (iirc among other we need to finish configuring fedmsg). > > Other that this, MirrorManager2 is currently in a decent shape I think. > However, we really need to make sure nothing broke in the re-write and we > want to make sure we won't break it in the future. > To try to ensure that last part, I have try to write some tests for the UI > but > also for the backend part (all the different scripts). > The pull-request is opened for review: > https://github.com/fedora-infra/mirrormanager2/pull/14 > > I have also been trying to capitalize on the knowledge we acquired during > the > FAD by starting to write down how mirrormanager works in the documentation: > https://github.com/fedora-infra/mirrormanager2/compare/tests...doc > (pull-request to be opened once the tests branch is merged) > I would appreciate if those that were at the FAD could go through this > branch/changes and adjust as needed. > I have been thinking about asking Matt to do the review so that we can > adjust > and improve the documentation. > > > Before we move MirrorManager2 to prod, here is what I think would be nice > to > do/have: > > - Pickle validation > - Figure a way to validate a pickle after its creation and before moving > it to > the mirrorlist boxes > - Find out if we can improve our tests some more > (to improve our confidence that we're ready) > - Engage the mirror mailing list and try to get them to react on the coming > changes > > The pickle validation might also be an interesting idea to check if there > is a > difference between the pickle generated by prod and the one generated in > stg. > > > Finally, at DevConf we have been speaking quite a bit with Dennis around > updates and MirrorManager and here is some of the ideas we spoke about: > > - Be able to run the UMDL script on only a part of the tree > (ie: be able to say, we updated f21-updates and we only update this part) > - Crawl the mirrors for only a part of the tree > (This goes together with updating only part of the tree via UMDL) > For a dumb optimization (which may be there laready) I would only crawl the trees which we know change "hourly" (updates/X/, development/X/) and only scan releases etc daily or weekly because it should not change that much. > - Consider if we should/could drop the content of the host_category_dir > table > before running the crawler > - Mirror versioning: > - run UMDL, detect changes, increase master mirror's version by 1 > - run the crawler, check for the changes, align that mirror's version > with the > master mirror one > - be able to see the difference between two versions > - be able to crawl a mirror only for the difference between the version > it is > at and the version the master mirror is at > note: we might still want to run a full crawl once in a while (daily? > bi-daily?) > > Those last couple of steps would be very useful in cutting down load from mirrors running rsync against enchillada but only needing a couple of packages in updates since the last time they did it. > This list of ideas is more a long term todo list, not something we would > want to > have working for pushing MirrorManager2 to prod. > > > Thoughts? Agreements? Disagreements? > > Thanks, > Pierre > > _______________________________________________ > infrastructure mailing list > [email protected] > https://admin.fedoraproject.org/mailman/listinfo/infrastructure > -- Stephen J Smoogen.
_______________________________________________ infrastructure mailing list [email protected] https://admin.fedoraproject.org/mailman/listinfo/infrastructure
