[foreman-dev] Test failures and merging PRs
Hello devs, since there was a discussion on foreman-dev IRC channel recently about merging PRs in Foreman core even if there's some build failed, I talked to few people and decided to describe here what I think is current way of how it works. I'm also attaching one suggestion at the end that came up after the discussion. Please, add questions, comments or simple +1 so we all know whether we're on the same page. Core PR runs 7 checks - foreman, katello, codeclimate, hound, prprocessor, upgrade, continuous-integration/travis-ci/pr. In ideal case they are all green and after review, the PR is merged. There are several cases where we can merge even if the PR is red. 1) codeclimate is red This can be ignored, we never agreed on using this as a hard metric for the PR. The motivation to introduce it was mainly to save some time to reviewer. We don't have to run it manually to get indications whether there's something introducing a big complexity [1]. From my experience it sometimes leads to worse code, since author splits the logic into more methods to lower e.g. cyclomatic complexity but it should be judged separately in every case. 2) foreman is red This can happen because of intermittent tests failures. If the PR is clearly not causing new ones and commiter is aware of this error, the PR is merged with message like "test unrelated" comments. If we are not sure, we retrigger the run, If Foreman develop branch is broken, we need to merge the PR to fix it so this is another exception. Usually we trigger the jenkins job manually first to see that the PR fixes the issue. 3) katello is red If katello becomes red only for this PR, we analyze what causes it. Usually it means that we change some internal things that have impact on Katello. In such case, we're doing our best to send a fixing PR to Katello or we ping someone with better knowledge in this area. We don't merge the PR until it's resolved, then usually we merge both parts at the same time. If it turns out there are more PRs that are failing with same errors, we merge PRs if we're sure they don't introduce new Katello failures. At this time, we're not blocking merges until Katello is green again. (*) here the suggestion applies 4) upgrade is red this is very similar to katello job, if there's some issue in upgrade, we should not merge the PR. I remember though, there was a time when we knew the job is broken which fall under "known to be broken" category. 5) There's no 5, all the rest must be green. Sometimes hound service does not respond and remains in "running" state, then it's retriggered by the reviewer. prprocessor and travis must always be happy. Now the promised suggestion. When we see katello/upgrade job is broken on multiple PRs, the first reviewer who spots it should notify the rest of developers about "from now on, we ignore tests XY". The ideal channel seems to be this list. This helps Katello devs to find out when it started, what were the commits that first induced it. [1] https://groups.google.com/forum/#!topic/foreman-dev/p7ESagXwNwk Thanks for comments -- Marek -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] [Infra] Openshift V2 Sunsetting: Redmine and Prprocessor
Update from today's testing... many thanks to Evgeni for his help! There are 3 apps listed on the v2 platform: # Pluginmatrix http://pluginmatrix-theforeman.rhcloud.com/ I don't think this is in active use, but if it's still needed, we can add it to the migration list. No action taken so far. # PrProcessor We got this running on v3 without too much effort. I updated the webhook in foreman_templates to test against the new URL and saw a post from GitHub in the logs, things seemed to happen. We think this is ready for larger testing - ideally we'd like to use Foreman core as lots happens there. Process would be to alter the url of the webhook to the new processor and let it run for a day or two to check all is fine. We can then look at updating the webhooks on all the configured repos (there is a script to do this in the codebase). There are some small niggle to figure out as part of that, such as getting a sensible custom url, and making cron work. we're confident we can get that running. Does anyone have concerns over enabling this in a few days time on foreman/foreman for testing? I'll notify again before I do so, just in case. # Redmine Obviously more complex, but initial tests have gone well. We got Psql and Redmine (from our fork, with a few extra patches) running, db migrated (clean, not a copy of ours), and could see the web interface. Cron will also need enabling here (as above) and outgoing email needs testing too, but so far it's promising. We're requesting more resources to our account so that we can try a larger test, and I'll update again once that's done. I *think we may be able to run them side-by-side for a bit, which will allow some user load testing - this is important as we have to move up to Ruby 2.2 or 2.3 as part of this (v2 is on 1.9.3), so making sure nothing subtle is broken will help. So far so good. Thanks again to Evgeni for the help! Greg -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] [Infra] Openshift V2 Sunsetting: Redmine and Prprocessor
Hi, On Mon, Aug 28, 2017 at 09:55:02AM +0200, Daniel Lobato Garcia wrote: > Isn't theforeman.org also running on OpenShift v2? Nope, this is running on web02.rackspace.theforeman.org. Regards -- Michael Moll -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
Re: [foreman-dev] [Infra] Openshift V2 Sunsetting: Redmine and Prprocessor
On 08/25, Eric D Helms wrote: > All, > > I wanted to make everyone aware that its been announced that Openshift V2 > will be closing down on September 30th. We currently run our Redmine and > Prprocessor on this infrastructure. This means that we will either need to > find new homes for these or migrate them to Openshift V3. More details will > follow as we work toward a solution. If anyone has concerns, or different > suggestions feel free to share them here. Isn't theforeman.org also running on OpenShift v2? > > -- > Eric D. Helms > Red Hat Engineering > > -- > You received this message because you are subscribed to the Google Groups > "foreman-dev" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to foreman-dev+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- Daniel Lobato Garcia @dLobatog blog.daniellobato.me daniellobato.me GPG: http://keys.gnupg.net/pks/lookup?op=get&search=0x7A92D6DD38D6DE30 Keybase: https://keybase.io/elobato -- You received this message because you are subscribed to the Google Groups "foreman-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to foreman-dev+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout. signature.asc Description: PGP signature
