Re: [foreman-dev] Koji repo sync metadata problem
I started an mrepo run after adding CentOS in a screen session. The run appears to be running a lot longer than previous and it's not clear to me what exactly it's doing at present or whats safe to do. Lukas could you check on it in your morning and see if you spot anything odd about why it seems to have stalled or slowed down? On Thu, Nov 16, 2017 at 6:01 AM, Ewoud Kohl van Wijngaarden < ew...@kohlvanwijngaarden.nl> wrote: > Those sound like good arguments, but if we don't test on older releases > then we don't know for sure they work either. IMHO it's acceptable to > explicitly only support EL 7.4 and up. We don't have the resources to > support older versions especially since CentOS doesn't support that either. > If users want verified working versions on EL < 7.4 then they should either > help us properly support it or buy support from a vendor that does want to > promise it works. > > > On Thu, Nov 16, 2017 at 08:56:39AM +0100, Lukas Zapletal wrote: > >> I would be against syncing regularly but I don't do much packaging so >> it's up to you. The problem I see is "random" breakage. You come to >> work on Monday after sync and if you don't realize the package was >> deleted from EPEL, you can burn hour or something to figure it out >> sometimes. This was pretty clear case but as you know there are >> situations (SCL) when it is not that clear. >> >> If we ever deploy new koji, I'd vote to avoid getting OS updates >> completely, just base channels and explicit updates when we need it. >> We need reproducible builds, with autosync we are not getting it. >> >> One note, if we sync CentOS or RHEL base to particular version, >> chances are that SELinux won't load on older releases. So basically we >> must be ready to do minimum requirements change here. >> >> Thanks for solving the problem. >> >> LZ >> >> On Wed, Nov 15, 2017 at 10:05 PM, Ewoud Kohl van Wijngaarden >> wrote: >> >>> I think we should sync & update. CentOS has no concept of supported minor >>> releases and we should be testing with a supported release. >>> >>> >>> On Wed, Nov 15, 2017 at 03:57:40PM -0500, Eric D Helms wrote: >>> This now appears to be working again. One out standing question that affects nodejs- packaging builds. As part of the RHEL 7.4 release, http-parser was removed from EPEL. With this latest round of Koji external repositories update we now have a newer EPEL with this package removed. We did not update our CentOS external repository that would include this package. Not updating the base OS has been our policy for a while now it would seem. We have two options: 1) Sync and update CentOS 2) Add http-parser to the overrides tag for foreman-nightly On Wed, Nov 15, 2017 at 2:11 PM, Eric D Helms wrote: I had forgotten to run the repo-regen on Koji. That is finishing up now. > I > will run another test after and report back. > > On Wed, Nov 15, 2017 at 2:08 PM, Patrick Creech > wrote: > > On Wed, 2017-11-15 at 14:00 -0500, Patrick Creech wrote: >> > On Wed, 2017-11-15 at 19:56 +0100, Lukas Zapletal wrote: >> > > I do not have access here, can someone take a look? Are repodata >> > > present for EPEL7? >> > >> > If I have the correct path, it does not appear so: >> > >> > [root@koji repodata]# pwd >> > /mnt/koji/external-repos/epel7-x86_64/epel/repodata >> > [root@koji repodata]# ls >> > [root@koji repodata]# >> > >> >> I might not have, this path DOES have repodata: >> >> /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/repodata >> >> > > LZ >> > > >> > > On Wed, Nov 15, 2017 at 6:15 PM, Ewoud Kohl van Wijngaarden >> > > wrote: >> > > > I still get errors when building: >> > > > >> > > > http://koji.katello.org/koji/taskinfo?taskID=52363 >> > > > http://koji.katello.org/kojifiles/work/tasks/2363/52363/ >> root.log >> > > > >> > > > DEBUG util.py:439: Error downloading packages: >> > > > DEBUG util.py:439:1:nodejs-6.10.3-1.el7.x86_64: failed to >> retrieve >> > > > nodejs-6.10.3-1.el7.x86_64.rpm from build >> > > > DEBUG util.py:439: error was [Errno 2] Local file does not >> exist: >> > > > /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/nodejs-6 >> .10.3-1.el7.x86_64.rpm >> > > > DEBUG util.py:439:http-parser-2.7.1-3.el7.x86_64: failed to >> retrieve >> > > > http-parser-2.7.1-3.el7.x86_64.rpm from build >> > > > DEBUG util.py:439: error was [Errno 2] Local file does not >> exist: >> > > > /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/http-par >> ser-2.7.1-3.el7.x86_64.rpm >> > > > DEBUG util.py:439:1:npm-3.10.10-1.6.10.3.1.el7.x86_64: >> failed >> to >> > > > retrieve npm-3.10.10-1.6.10.3.1.el7.x86_64.rpm from build >> > > > DEBUG util.py:439: error was [Errno 2] Local file does not >> exist:
[foreman-dev] Getting Foreman and Smart-Proxy to run in FIPS environment.
What is FIPS? >From Wikipedia [1]: The Federal Information Processing Standard (FIPS) Publication 140-2, (FIPS PUB 140-2), is a U.S. government computer security standard used to approve cryptographic modules. The title is Security Requirements for Cryptographic Modules. What are Implications of FIPS 140-2 Support for Foreman, Katello, and Smart-Proxy? Linux system, or rather an SSL library in FIPS-compatible mode will only have a set of ciphers and hash functions compatible with FIPS. [2] contains the list of approved cryptographic functions, Oracle graciously compiled the list of not approved ones, which is more useful and can be found at [3]. OpenSSL in FIPS mode My understanding is that only OpenSSL versions 1.0.1 and 1.0.2 have FIPS 140-2 validated cryptographic modules. OpenSSL raises ABRT signal when it receives a call to one of the unapproved ciphers/functions. Foreman in FIPS mode I haven’t looked at pulp, candlepin, qpid, goferd, etc, and at point don’t know how and if these can be made to work in FIPS mode. All tests I’ve done so far were against Rails 5.0, Considering the number of dependencies, we will need to limit FIPS support to just one version of Rails. Rails and other (ruby) dependencies. MD5 is used (hard-coded) in a few places in Rails, at this point I’m quite certain that its use is constrained to various built-in caches. I had to disable *all* Rails caches to be able to run Foreman in FIPS mode. Additionally, strong ETAG’s cannot be used, I’m not sure if they are used, or there are plans for them. Spring uses MD5 to generate application ID, but will use one in SPRING_APPLICATION_ID environment variable if it’s available. Gravatar uses MD5 hashes in their urls, doesn’t look like other hashes are supported. I think apipie cache uses MD5, but I will need to verify this. Foreman app/services/password_crypt uses MD5 for grub2 passwords, which will need to be switched to SHA512. MD5 will need to be removed from the list of hash functions SshKey#generate_fingerprint, call to SSHKey.fingerprint uses MD5 A note: with caching disabled, and issues above fixed, I was able to get Foreman suite of tests to pass, and get Foreman to start. Smart-Proxy Smart-Proxy codebase appears to be compatible with FIPS (ran and passed tests ok without any changes), but there are issues with external depdencies. DHCPD uses MD5-based omapi shared secret. DHCPD shared secret with bind is also md5-based. BIND when used with dhcpd uses MD5 hashes stored in TXT as host id. Puppet needs to be run in FIPS mode (FIPS-compatible hash function needs to be configured). I assume this covers all of puppet, including mcollective, puppet run, puppetca. BMC/IPMI authentication can use MD5 or lower based hashes, older clients may not have newer hash functions. Salt appears to use MD5 hashes by default, individual nodes must be configured to use other hash_type Any 3rd party SSL certificates that may need to be verified or decoded by either Foreman or Smart-Proxy must be generated using FIPS-compatible algorithms/hash functions. How we can reach FIPS compatibility The easiest first step would be to replace offending cryptographic and hash functions in Foreman, and in Smart-Proxy case, 3rd party configuration files with FIPS-compatible ones. Additionally, any new code changes that employ MD5 or other non approved functions shouldn’t be accepted. The next step would be to create a CI job that will continuously execute the the full suite of tests on a VM with FIPS mode enabled. GDB configured with Ruby’s project .gdbinit [4] and a tiny batch [5] of commands can be used to report on FIPS-related failures. Considering the amount of dependencies Foreman and Smart-Proxy have, I think would be useful to have all CI environments switched to run in FIPS mode: this should increase the probability of discovering of new FIPS-related issues before our users. Lastly, a FIPS-compatible caching solution for Rails needs to be found, if none exist, an existing one needs to be modified to support FIPS. Any feedback would be appreciated, -d [1] Wikipedia article on FIPS 140-2, https://en.wikipedia.org/wiki/ FIPS_140-2 [2] Approved Security Functions for FIPS 140-2, https://csrc.nist.gov/csrc/media/publications/fips/140/2/final/documents/ fips1402annexa.pdf [3] List of algorithms not approved for FIPS 140-2, https://docs.oracle.com/cd/E36784_01/html/E54953/fips-notok-1.html [4] Ruby project’s gdb helper functions, https://github.com/ruby/ruby/blob/trunk/.gdbinit [5] Catching SIGABRTs with gdb and ruby-specific .gdbinit, https://gist.github.com/witlessbird/904fefb0031c2eda96da61bd19424c86 -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] Re: [UX] Facets and Host UI - roadmap discussion.
Specifically talking to the new host wizard, it was designed to allow users to basically choose a certain type of workflow which would populate the wizard with the relevant data entry forms. This obviously only addresses the creation of things, not necessarily the editing or viewing of them. I don't think this type of interaction is in contrast with anything that's been said as it's really just a delivery vehicle. To Tom's point about customizable tables - The Host Merge design I am currently working on will introduce this type of table. On Wed, Nov 15, 2017 at 8:43 AM, Tom McKay wrote: > Perhaps another (or additional) approach to consider is having different > "views" of the objects. > > Consider hosts currently. Depending on the user's role for that moment, > the information and flows needed will be vastly different. The first > engineer may, for example, be the provisioning one. Perhaps the puppet flow > is paramount, or perhaps it's ansible, or whatever. This engineer doesn't > need or care about the other flavors. Next task for this engineer, or a > different one, is to confirm subscriptions. Another task is to check on > package versions. If I could choose which "view" I wanted for the > particular task of the day, that would be great. > > The implementation could simply be that the main table had different view > choices (which columns to show) which would then lead to different > click-through pages. > > As a plugin writer, I may wish to provide a view entirely my own. > Alternatively, I may wish to be included in details on other views. To me > both inline (deface style) and additional tabs (easier) are needed but > cramming every aspect of a host into a single view is not ideal. > > > On Wed, Nov 15, 2017 at 8:24 AM, Tomer Brisker > wrote: > >> On Wed, Nov 15, 2017 at 2:23 PM, wrote: >> > >> > Marek, you lead me to an interesting conclusion: >> > >> > I think we have to distinguish two things here - there are workflows >> (such >> > as provisioning, config management, fact reporting) and there are >> > information aspects. >> > An information aspect is a set of properties that describe a host in >> > different system. For example puppet facet would store all properties >> that >> > are needed to describe a host in puppet. Ovirt facet would store all >> > properties that describe the host in ovirt system. >> > Workflow, on the other hand, is a set of actions that needed to be >> taken in >> > a certain order to achieve some operational goal. Examples to that >> would be >> > provisioning - a set of actions that involve different systems (dhcp, >> dns, >> > vm infrastructure and OS installer) that result in a fully operational >> > server. Another example would be monitoring - in this case we will want >> > multiple systems (like puppet's facter, vm's power status e.t.c.) to >> report >> > to the user. >> > >> > Now, once we have those concepts, we can try and translate this into >> the UI: >> > In my opinion, data entry should be done from screens centered on >> > information aspects, regardless of the workflows where this information >> > could be used. On the other hand each workflow deserves a "summary" read >> > only screen, where we will combine data from multiple facets to show >> which >> > data would be used for that particular workflow. >> >> I have to disagree on this point. Users shouldn't care about the >> "behind the scenes" of Foreman. They want a host provisioned in their >> environment and don't care if we store the data needed for that in 1 >> table or 10. The most important point is that the user's workflow is >> easy as possible for the majority of cases, with ability to add more >> information if needed in special cases. Entering a lot of info that >> may or may not be needed before they can take any action with that >> info feels to me like more complication, not less. >> >> > >> > From a more practical point of view, our current screens serve both >> > workflows and data entry. It means that we have to establish for each >> screen >> > what it tries to achieve - either it's a data entry page, such as host's >> > new\edit form or it's a workflow preview/result page such as host show >> page. >> > Data entry pages should be rigidly grouped by facets, but workflow pages >> > should be extensible, so each facet will be able to show relevant >> > information on the same page. >> > >> > How does this sound to you? >> > Roxanne, does it fit with your vision of form's next generation? >> > >> > >> > >> > On Wednesday, November 15, 2017 at 8:33:35 AM UTC+2, Marek Hulan wrote: >> >> >> >> I think these screenshots illustrate that pure option 2 can have >> negative >> >> impact on usability. If I'm a puppet user, the puppet environment is >> one >> >> of >> >> the most important thing to set. Having it in last tab completely >> separate >> >> from hostgroup does not feel right. If some field of facet is part of >> >> provisioning workflow, it should be extending provisioning
Re: [foreman-dev] 1.17 branching schedule
On Thu, Nov 16, 2017 at 3:18 PM, Eric D Helms wrote: > Howdy, > > I am not clear on what the shape of everything else is, but I think there > are two larger updates that are in risk if we stick to our hard time based > releases here: > > 1) Dynflow as a stand-alone service used by Foreman which is targeted for > 1.16 GA but needs to also make 1.17 > 2) 1.17 with Rails 5.1 > > We've had a lot of packaging breakages which has prevented work on and > subsequent testing of Rails 5.1 SCL for use in 1.17. Further, nightlies > have been broken more than not which should lower confidence levels > approaching a 1.17 branch date. I say this not to be an alarmist but to > raise concerns such that we target a strong and stable release. > > While I understand the packaging challenges, I do think that code quality and the time passed since last branching IMHO make sense to try and recover and cut a release. Having said that, if your ETA on rails 5.1 is over a month of effort, maybe it make sense to revisit that, but as it stands, I'm in favor in branching more or less on schedule. > > Eric > > On Thu, Nov 16, 2017 at 6:58 AM, Ondrej Prazak wrote: > >> Hi, >> I am sending this just to let you know that, according to the schedule, >> the 1.17 branching should happen in 2 weeks. >> >> O. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "foreman-dev" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to foreman-dev+unsubscr...@googlegroups.com. >> For more options, visit https://groups.google.com/d/optout. >> > > > > -- > Eric D. Helms > Red Hat Engineering > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] 1.17 branching schedule
Howdy, I am not clear on what the shape of everything else is, but I think there are two larger updates that are in risk if we stick to our hard time based releases here: 1) Dynflow as a stand-alone service used by Foreman which is targeted for 1.16 GA but needs to also make 1.17 2) 1.17 with Rails 5.1 We've had a lot of packaging breakages which has prevented work on and subsequent testing of Rails 5.1 SCL for use in 1.17. Further, nightlies have been broken more than not which should lower confidence levels approaching a 1.17 branch date. I say this not to be an alarmist but to raise concerns such that we target a strong and stable release. Eric On Thu, Nov 16, 2017 at 6:58 AM, Ondrej Prazak wrote: > Hi, > I am sending this just to let you know that, according to the schedule, > the 1.17 branching should happen in 2 weeks. > > O. > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- Eric D. Helms Red Hat Engineering -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[foreman-dev] 1.17 branching schedule
Hi, I am sending this just to let you know that, according to the schedule, the 1.17 branching should happen in 2 weeks. O. -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] Regular Foreman-core issue triage?
On 16/11/17 11:51, Greg Sutcliffe wrote: > I've just finished rebasing our Redmine patches onto each of the stable > branches in the upstream repo (see [1]). Next up is testing the DB > migrates correctly, and then I can add the question plugin Marek > mentioned as part of the upgrade. Links help :facepalm: [1] https://github.com/GregSutcliffe/redmine/branches/yours There's more to say on this but it's derailing the triage discussion, I'll start a new thread for that. Greg -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] Regular Foreman-core issue triage?
On 16/11/17 11:23, Ivan Necas wrote: > +0, or at least not relying on it as the only way the triage happens. > > +1 for resolving the needinfo feature in redmine: the triage mtg has bigger > potential with it I've just finished rebasing our Redmine patches onto each of the stable branches in the upstream repo (see [1]). Next up is testing the DB migrates correctly, and then I can add the question plugin Marek mentioned as part of the upgrade. Greg -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] Regular Foreman-core issue triage?
On Thu, 16 Nov 2017 at 12:13, Ewoud Kohl van Wijngaarden < ew...@kohlvanwijngaarden.nl> wrote: > +1 the open issues need to be reviewed. I don't know if this is the best > solution but can't hurt to get more eyes on them. > > What I don't see is something to go over old issues that have been > triaged at some point. I did that a while back for the installer and > could close a few issues that had been fixed already or could be closed > with WONTFIX. > > On Tue, Nov 14, 2017 at 05:12:36PM -0500, Eric D Helms wrote: > >Bump - any thoughts here from the -dev community? Simple +1 or -1 or even > a > >+0 for indifference would be great to just know where folks stand and > >whether this would be worth while. +0, or at least not relying on it as the only way the triage happens. +1 for resolving the needinfo feature in redmine: the triage mtg has bigger potential with it -- Ivan > > > >On Thu, Nov 9, 2017 at 8:28 AM, Eric D Helms > wrote: > > > >> Writing this up inspired me to capture it for the long term [1]. I'd be > >> happy to run the first one or two given my experience with it (and > assuming > >> the timeslot works) just to get into the groove. Note that our process > for > >> triaging does require some overall Redmine process change with the way > we > >> uses some of the empty and Recycle Bin releases. > >> > >> > >> [1] https://github.com/theforeman/theforeman.org/pull/970 > >> > >> On Thu, Nov 9, 2017 at 7:54 AM, Greg Sutcliffe > > >> wrote: > >> > >>> On 08/11/17 16:47, Eric D Helms wrote: > >>> [tons of useful stuff] > >>> > >>> Thanks Eric! I think that format will work for us too, might take a > >>> little practice. We'll need volunteers to be the runner, ofc ;) > >>> > >>> On 09/11/17 07:03, Marek Hulan wrote: > >>> > I'd join regularly, after few years for which I receive all > >>> > notifications from redmine, I can confirm there are bugs without much > >>> > attention. > >>> > > >>> > If we won't have representatives from all areas, we might need some > >>> > tooling to ping people in redmine tickets. Again, after few years, I > can > >>> > tell that mentioning name in comment does not always work. There > used to > >>> > be a question plugin which works similarly as needinfo in BZ. > Perhaps we > >>> > could install it? > >>> > >>> http://www.redmine.org/projects/redmine/wiki/PluginQuestion is what > >>> you're referring to right? Looks nice, I can look into adding it - some > >>> Redmine work is definitely on my short-term todo list anyway. > >>> > >>> > Thanks Greg for bringing this up > >>> > >>> Still looking for suggested time slots. Perhaps I should create a poll? > >>> > >>> Greg > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. > -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] Regular Foreman-core issue triage?
+1 the open issues need to be reviewed. I don't know if this is the best solution but can't hurt to get more eyes on them. What I don't see is something to go over old issues that have been triaged at some point. I did that a while back for the installer and could close a few issues that had been fixed already or could be closed with WONTFIX. On Tue, Nov 14, 2017 at 05:12:36PM -0500, Eric D Helms wrote: Bump - any thoughts here from the -dev community? Simple +1 or -1 or even a +0 for indifference would be great to just know where folks stand and whether this would be worth while. On Thu, Nov 9, 2017 at 8:28 AM, Eric D Helms wrote: Writing this up inspired me to capture it for the long term [1]. I'd be happy to run the first one or two given my experience with it (and assuming the timeslot works) just to get into the groove. Note that our process for triaging does require some overall Redmine process change with the way we uses some of the empty and Recycle Bin releases. [1] https://github.com/theforeman/theforeman.org/pull/970 On Thu, Nov 9, 2017 at 7:54 AM, Greg Sutcliffe wrote: On 08/11/17 16:47, Eric D Helms wrote: [tons of useful stuff] Thanks Eric! I think that format will work for us too, might take a little practice. We'll need volunteers to be the runner, ofc ;) On 09/11/17 07:03, Marek Hulan wrote: > I'd join regularly, after few years for which I receive all > notifications from redmine, I can confirm there are bugs without much > attention. > > If we won't have representatives from all areas, we might need some > tooling to ping people in redmine tickets. Again, after few years, I can > tell that mentioning name in comment does not always work. There used to > be a question plugin which works similarly as needinfo in BZ. Perhaps we > could install it? http://www.redmine.org/projects/redmine/wiki/PluginQuestion is what you're referring to right? Looks nice, I can look into adding it - some Redmine work is definitely on my short-term todo list anyway. > Thanks Greg for bringing this up Still looking for suggested time slots. Perhaps I should create a poll? Greg -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] Koji repo sync metadata problem
Those sound like good arguments, but if we don't test on older releases then we don't know for sure they work either. IMHO it's acceptable to explicitly only support EL 7.4 and up. We don't have the resources to support older versions especially since CentOS doesn't support that either. If users want verified working versions on EL < 7.4 then they should either help us properly support it or buy support from a vendor that does want to promise it works. On Thu, Nov 16, 2017 at 08:56:39AM +0100, Lukas Zapletal wrote: I would be against syncing regularly but I don't do much packaging so it's up to you. The problem I see is "random" breakage. You come to work on Monday after sync and if you don't realize the package was deleted from EPEL, you can burn hour or something to figure it out sometimes. This was pretty clear case but as you know there are situations (SCL) when it is not that clear. If we ever deploy new koji, I'd vote to avoid getting OS updates completely, just base channels and explicit updates when we need it. We need reproducible builds, with autosync we are not getting it. One note, if we sync CentOS or RHEL base to particular version, chances are that SELinux won't load on older releases. So basically we must be ready to do minimum requirements change here. Thanks for solving the problem. LZ On Wed, Nov 15, 2017 at 10:05 PM, Ewoud Kohl van Wijngaarden wrote: I think we should sync & update. CentOS has no concept of supported minor releases and we should be testing with a supported release. On Wed, Nov 15, 2017 at 03:57:40PM -0500, Eric D Helms wrote: This now appears to be working again. One out standing question that affects nodejs- packaging builds. As part of the RHEL 7.4 release, http-parser was removed from EPEL. With this latest round of Koji external repositories update we now have a newer EPEL with this package removed. We did not update our CentOS external repository that would include this package. Not updating the base OS has been our policy for a while now it would seem. We have two options: 1) Sync and update CentOS 2) Add http-parser to the overrides tag for foreman-nightly On Wed, Nov 15, 2017 at 2:11 PM, Eric D Helms wrote: I had forgotten to run the repo-regen on Koji. That is finishing up now. I will run another test after and report back. On Wed, Nov 15, 2017 at 2:08 PM, Patrick Creech wrote: On Wed, 2017-11-15 at 14:00 -0500, Patrick Creech wrote: > On Wed, 2017-11-15 at 19:56 +0100, Lukas Zapletal wrote: > > I do not have access here, can someone take a look? Are repodata > > present for EPEL7? > > If I have the correct path, it does not appear so: > > [root@koji repodata]# pwd > /mnt/koji/external-repos/epel7-x86_64/epel/repodata > [root@koji repodata]# ls > [root@koji repodata]# > I might not have, this path DOES have repodata: /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/repodata > > LZ > > > > On Wed, Nov 15, 2017 at 6:15 PM, Ewoud Kohl van Wijngaarden > > wrote: > > > I still get errors when building: > > > > > > http://koji.katello.org/koji/taskinfo?taskID=52363 > > > http://koji.katello.org/kojifiles/work/tasks/2363/52363/root.log > > > > > > DEBUG util.py:439: Error downloading packages: > > > DEBUG util.py:439:1:nodejs-6.10.3-1.el7.x86_64: failed to retrieve > > > nodejs-6.10.3-1.el7.x86_64.rpm from build > > > DEBUG util.py:439: error was [Errno 2] Local file does not exist: > > > /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/nodejs-6 .10.3-1.el7.x86_64.rpm > > > DEBUG util.py:439:http-parser-2.7.1-3.el7.x86_64: failed to retrieve > > > http-parser-2.7.1-3.el7.x86_64.rpm from build > > > DEBUG util.py:439: error was [Errno 2] Local file does not exist: > > > /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/http-par ser-2.7.1-3.el7.x86_64.rpm > > > DEBUG util.py:439:1:npm-3.10.10-1.6.10.3.1.el7.x86_64: failed to > > > retrieve npm-3.10.10-1.6.10.3.1.el7.x86_64.rpm from build > > > DEBUG util.py:439: error was [Errno 2] Local file does not exist: > > > /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/npm-3.10 .10-1.6.10.3.1.el7.x86_64.rpm > > > > > > > > > On Wed, Nov 15, 2017 at 11:44:24AM -0500, Eric D Helms wrote: > > > > > > > > Looks like everything is back up and working as expected. For packagers, > > > > keep in mind that this updated some of our external repositories to their > > > > latest versions if you see any oddities. This also means that Fedora 27 is > > > > available as an external repository for Pulp and Katello > > > > clients. > > > > > > > > On Wed, Nov 15, 2017 at 9:32 AM, Lukas Zapletal > > > > wrote: > > > > > > > > > Hey, > > > > > > > > > > when I initiated mrepo -n (dry run) this morning to see how mrepo > > > > > works on our koji in order to test if we are able to add > > > > > Fedora 27 for > > > > > Pulp, I learned that this "dry run" is actually a real run and mrepo > > > > > initiated full resync of our content without metadata regeneration. > > > > > > > > > > Thi
Re: [foreman-dev] Koji repo sync metadata problem
Thanks Eric, I added this section about the process for the future:
http://projects.theforeman.org/projects/foreman/wiki/KojiSetup#Adding-external-repository
On Wed, Nov 15, 2017 at 9:57 PM, Eric D Helms wrote:
> This now appears to be working again. One out standing question that affects
> nodejs- packaging builds.
>
> As part of the RHEL 7.4 release, http-parser was removed from EPEL. With
> this latest round of Koji external repositories update we now have a newer
> EPEL with this package removed. We did not update our CentOS external
> repository that would include this package. Not updating the base OS has
> been our policy for a while now it would seem. We have two options:
>
> 1) Sync and update CentOS
> 2) Add http-parser to the overrides tag for foreman-nightly
>
> On Wed, Nov 15, 2017 at 2:11 PM, Eric D Helms wrote:
>>
>> I had forgotten to run the repo-regen on Koji. That is finishing up now. I
>> will run another test after and report back.
>>
>> On Wed, Nov 15, 2017 at 2:08 PM, Patrick Creech
>> wrote:
>>>
>>> On Wed, 2017-11-15 at 14:00 -0500, Patrick Creech wrote:
>>> > On Wed, 2017-11-15 at 19:56 +0100, Lukas Zapletal wrote:
>>> > > I do not have access here, can someone take a look? Are repodata
>>> > > present for EPEL7?
>>> >
>>> > If I have the correct path, it does not appear so:
>>> >
>>> > [root@koji repodata]# pwd
>>> > /mnt/koji/external-repos/epel7-x86_64/epel/repodata
>>> > [root@koji repodata]# ls
>>> > [root@koji repodata]#
>>> >
>>>
>>> I might not have, this path DOES have repodata:
>>>
>>> /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/repodata
>>>
>>> > > LZ
>>> > >
>>> > > On Wed, Nov 15, 2017 at 6:15 PM, Ewoud Kohl van Wijngaarden
>>> > > wrote:
>>> > > > I still get errors when building:
>>> > > >
>>> > > > http://koji.katello.org/koji/taskinfo?taskID=52363
>>> > > > http://koji.katello.org/kojifiles/work/tasks/2363/52363/root.log
>>> > > >
>>> > > > DEBUG util.py:439: Error downloading packages:
>>> > > > DEBUG util.py:439:1:nodejs-6.10.3-1.el7.x86_64: failed to
>>> > > > retrieve
>>> > > > nodejs-6.10.3-1.el7.x86_64.rpm from build
>>> > > > DEBUG util.py:439: error was [Errno 2] Local file does not exist:
>>> > > >
>>> > > > /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/nodejs-6.10.3-1.el7.x86_64.rpm
>>> > > > DEBUG util.py:439:http-parser-2.7.1-3.el7.x86_64: failed to
>>> > > > retrieve
>>> > > > http-parser-2.7.1-3.el7.x86_64.rpm from build
>>> > > > DEBUG util.py:439: error was [Errno 2] Local file does not exist:
>>> > > >
>>> > > > /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/http-parser-2.7.1-3.el7.x86_64.rpm
>>> > > > DEBUG util.py:439:1:npm-3.10.10-1.6.10.3.1.el7.x86_64: failed
>>> > > > to
>>> > > > retrieve npm-3.10.10-1.6.10.3.1.el7.x86_64.rpm from build
>>> > > > DEBUG util.py:439: error was [Errno 2] Local file does not exist:
>>> > > >
>>> > > > /mnt/koji/external-repos/www/epel7-x86_64/RPMS.epel/npm-3.10.10-1.6.10.3.1.el7.x86_64.rpm
>>> > > >
>>> > > >
>>> > > > On Wed, Nov 15, 2017 at 11:44:24AM -0500, Eric D Helms wrote:
>>> > > > >
>>> > > > > Looks like everything is back up and working as expected. For
>>> > > > > packagers,
>>> > > > > keep in mind that this updated some of our external repositories
>>> > > > > to their
>>> > > > > latest versions if you see any oddities. This also means that
>>> > > > > Fedora 27 is
>>> > > > > available as an external repository for Pulp and Katello clients.
>>> > > > >
>>> > > > > On Wed, Nov 15, 2017 at 9:32 AM, Lukas Zapletal
>>> > > > > wrote:
>>> > > > >
>>> > > > > > Hey,
>>> > > > > >
>>> > > > > > when I initiated mrepo -n (dry run) this morning to see how
>>> > > > > > mrepo
>>> > > > > > works on our koji in order to test if we are able to add Fedora
>>> > > > > > 27 for
>>> > > > > > Pulp, I learned that this "dry run" is actually a real run and
>>> > > > > > mrepo
>>> > > > > > initiated full resync of our content without metadata
>>> > > > > > regeneration.
>>> > > > > >
>>> > > > > > This rendered our external repositories unusable as all
>>> > > > > > metadata were
>>> > > > > > deleted. All builds will fail today with missing dependencies.
>>> > > > > >
>>> > > > > > The plan: we need to let mrepo finish "dry run" downloading of
>>> > > > > > Fedora
>>> > > > > > 27 and then we need to reexecute it with proper flags:
>>> > > > > >
>>> > > > > > cat /usr/local/bin/koji-sync-external-repos
>>> > > > > > #!/bin/bash
>>> > > > > > mrepo -qugf
>>> > > > > > koji list-targets --quiet | awk '{print $2}' | sort -u | xargs
>>> > > > > > -I '{}'
>>> > > > > > koji regen-repo --nowait '{}'
>>> > > > > >
>>> > > > > > Note the -u -g -f flags for mrepo, this will cause all metadata
>>> > > > > > to be
>>> > > > > > regenerated. Once this command is done, koji will see missing
>>> > > > > > dependencies to appear.
>>> > > > > >
>>> > > > > > The dry run of mrepo is running in screen and it is about to
>>> > > > > > finish
>>> > > > > > hopefully soon. I will defer this
Re: [foreman-dev] Re: Discourse summary week 2 (ish)
Hello, After giving discourse a quick spin, I have to say I like it a lot. The UI is great for new comers, the ability to have tags that you can follow and teams is great for focusing on things that require your attention or interest you (or grab someone else's attention when needed). Honestly, I think I would prefer only logging in once-twice a day to read new messages and only set up mail notifications for things that require my attention, so that the amount of mail notifications I get will go down and allow me better focus. >From what I saw, it looks like people who prefer sticking to mail can still do that with no problem - they will get a bit less "bells and whistles" but their workflow will be preserved, so I have to say I'm having a bit of a hard time understanding the objections - you can keep using your mail client just as before, while giving new-comers an easier on-boarding and allowing those who want to to leverage the nice UI and extra features that we currently don't have. In fact, I might also consider suggesting a move to discourse for the local FOSS non-profit I'm on the board of as a way to attract new members and increase engagement instead of the mailing list. Thank you Greg for setting this up and driving the discussion! On Thu, Nov 16, 2017 at 5:54 AM, Andrew Schofield wrote: > I'm generally silent in here. It's certainly a +1 from me. I like the > formatting. I like the categories. I like the tags. I like the suggestions. > Will it take some getting used to? Of course it will. > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- Have a nice day, Tomer Brisker Red Hat Engineering -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] Request to add Fedora 27 to Koji
We still have 180GB space left, it will be shrinking as we will continue building but let's just wait with Fedora removal until the next release. Eric fixed the building issues anyway. And I just did: [root@koji ~]# koji add-external-repo fedora27-external-repo file:///mnt/koji/external-repos/www/fedora27-\$arch/RPMS.os/ Created external repo 58 [root@koji ~]# koji add-external-repo fedora27-updates-external-repo file:///mnt/koji/external-repos/www/fedora27-\$arch/RPMS.updates/ Created external repo 59 I believe you can now add required tags and start your builds. On Wed, Nov 15, 2017 at 4:00 PM, Patrick Creech wrote: > On Wed, 2017-11-15 at 15:42 +0100, Lukas Zapletal wrote: >> We have 200GB spare space, one more Fedora could fit. Can we delete >> Fedoras 24 and 25? This is definitely the last time before we'd need >> to expand space again. > > With regards to pulp needs, Fedora 24 can go, as that is no longer supported > by Fedora. I'd prefer > waiting till 2.14.3 is GA next tuesday (as we still have f24 builds in that > pipeline) and we have a > switchover of the release process aligned with that point. > > Fedora 25 should be good to drop early december based on pulp needs. > > With that said, if needed, do what you gotta do. > >> I initiated mrepo dry run which turned into real run and koji is now >> out of service due to missing metadata, we are working on this. Will >> get back to you, will take several hours. > > I saw unfortunately. I always hate it when things go awry from something I > initiated. We are very > appreciative, and if there's anything I can do to help along the way, please > let me know. > > > Thanks, > Patrick > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- Later, Lukas @lzap Zapletal -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
