Re: About MirrorManager2
On 02/14/2015 01:27 AM, Jason L Tibbitts III wrote: NC == Nick Coghlan ncogh...@redhat.com writes: NC In that general vein, an interesting project Robert Collins NC introduced me to last year is his lmirror work: NC https://pypi.python.org/pypi/lmirror That is quite interesting. It's obvious that rsync is kind of breaking down, given that with my current mirroring setup (just running a private internal mirror) I can rarely get a complete copy across the wire due to timeouts while receiving the file set. Not a lot of commits happening upstream, though. Is this code actually being used anywhere? When we were talking about the project at PyCon NZ, Robert said he planned to use it for something in relation to OpenStack's infrastructure, but I don't recall the details. If you wanted to know more, it may be worth getting in touch with him directly. Cheers, Nick. -- Nick Coghlan Red Hat Hosted Shared Services Software Engineering Development, Brisbane HSS Provisioning Architect ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: About MirrorManager2
On 02/12/2015 02:43 AM, Stephen John Smoogen wrote: - Mirror versioning: - run UMDL, detect changes, increase master mirror's version by 1 - run the crawler, check for the changes, align that mirror's version with the master mirror one - be able to see the difference between two versions - be able to crawl a mirror only for the difference between the version it is at and the version the master mirror is at note: we might still want to run a full crawl once in a while (daily? bi-daily?) Those last couple of steps would be very useful in cutting down load from mirrors running rsync against enchillada but only needing a couple of packages in updates since the last time they did it. In that general vein, an interesting project Robert Collins introduced me to last year is his lmirror work: https://pypi.python.org/pypi/lmirror It's designed to take advantage of various HTTP/2 features and the knowledge that it's specifically being used to create read-only mirrors to make the network operations more efficient than is feasible with the more general purpose rsync. Design details: https://bazaar.launchpad.net/~lmirror/lmirror/master/view/head:/doc/DESIGN.txt Regards, Nick. -- Nick Coghlan Red Hat Hosted Shared Services Software Engineering Development, Brisbane HSS Provisioning Architect ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Code Search for Fedora
On 11/19/2014 05:58 PM, Michael Stapelberg wrote: On Tue, Nov 18, 2014 at 2:12 PM, Kevin Fenzi ke...@scrye.com wrote: On Tue, 18 Nov 2014 13:24:02 -0800 Michael Stapelberg michael+fed...@stapelberg.ch wrote: I think before we go looking into hardware requirements, we should discuss the software? Whats it written in? Is there a bunch of people who work on it? or just you? It’s written in Go, and mostly I’m working on it, with a few random contributions from other people from time to time. We are very heavily a python shop here, I don't know if any of our applications folks have really done much with go, but I'm sure they will chime in if they have. ;) I see. Go is pretty easy to pick up, and the vast majority of people in my filter bubble who gave it a try found it pleasant to work with. The various Python folks I know that have done work in Go have all considered it a pleasant language for writing network services. Another point worth noting is that because Go is precompiled (ala C/C++), working with deployed systems written in it should be more like working with the system components written in C/C++ than with the Python web services. There's also the fact that given Docker Kubernetes are both written in Go, it seems likely that it will eventually make in appearance somewhere in Fedora's infrastructure, even independent of this proposal. Cheers, Nick. -- Nick Coghlan Red Hat Hosted Shared Services Software Engineering Development, Brisbane HSS Provisioning Architect ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Congrats to Tim Flink
On 08/10/2014 02:35 AM, Tim Flink wrote: On Sat, 9 Aug 2014 06:49:46 -0600 Kevin Fenzi ke...@scrye.com wrote: I'm happy to announce that I have approved Tim into the sysadmin-main group. This is the core group of trusted folks that high level access to most things. Tim is part of the Fedora QA group and has been working very hard on taskotron and resultsdb and related deployment. Being in the main group will allow him to not block on other folks adding data or doing things he needs. Also, he may be able to help out with other tasks as his time permits. Congrats and use your powers for good! :) Only for good!? So much for my plan to create a taskotron-managed cluster of dogecoin miners :'( Thanks for the welcome. Hopefully I'll be able to help out some but at least I won't be pestering you all quite so often once I learn how to do a few things. Congrats Tim! We've been a little busy with a few other things, but we'll getting back to beaker.fedoraproject.org soon, with an upgrade to Beaker 0.17 :) Cheers, Nick. -- Nick Coghlan Red Hat Hosted Shared Services Software Engineering Development, Brisbane HSS Provisioning Architect ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
Re: Followup on Atomic composes
On 07/22/2014 10:07 PM, Colin Walters wrote: From: 15:36:25 dgilmore walters: i have it working for the TC/RC process. I need to change up a few things. not yet looked at doing it for rawhide/branched yet Are there any blockers to setting up the rawhide/branched parts? Anything I can help with? In particular, will these composes write trees to the mirrors? If we're only doing a tree compose for TC, things are highly likely not to work because it'll be the first time we've run it in prod. What I'm thinking about for a plan is: Dennis (and any help he needs) work on the mainline rel-eng thread for F21. In the meantime though, Atomic still needs a mechanism to *try* code that's in development. So I may look at having the internal compose server be a pull from COPR type thing and thus be a remix (again), but improve on the current situation by mirroring to dl.fp.org or so. Note that this is also why I'm keen to see a draft compose *somewhere* that we can actually try importing into beaker.fedoraproject.org - so long as it is at least somewhat close to the structure of a regular tree (in terms of Beaker being able to find the kernel and initrd files and be able to generate appropriate netboot files), we should be able to make it available for installation. Amit also came up with a custom kickstart to let us run the test harness in a container rather than directly on the host, so even though the default kickstarts won't cope with Atomic yet, we would be able to start experimenting to see what works and what breaks. Cheers, Nick. -- Nick Coghlan Red Hat Hosted Shared Services Software Engineering Development, Brisbane HSS Provisioning Architect ___ infrastructure mailing list infrastructure@lists.fedoraproject.org https://admin.fedoraproject.org/mailman/listinfo/infrastructure
