Re: [OpenStack-Infra] What's the future for git-review?

[Jeremy Stanley]

On 2018-07-06 23:54:37 +0100 (+0100), Darragh Bailey wrote:
> On Thu, 5 Jul 2018, 02:57 Jeremy Stanley,  wrote:
[...]
> > The change listing feature really seems increasingly out of place to
> > me, and most of the "fixes" I saw related to it were about
> > supporting more and more of Gerrit's query language and terminal
> > formatting. If we deprecated/removed that and recommended
> > interacting directly with Gerrit or alternative utilities for change
> > searches (there are a lot more options for this than there were back
> > when git-review was first written) all of those would become
> > unnecessary and the code would be simplified at the same time.
> 
> That's interesting, I'd consider the ability to query for what is
> available for review a step before downloading a change for
> review, and understanding that it might be bringing multiple
> chances down that aren't merged useful.
> 
> If this was moved out of git-review, I suspect it might still need
> to know a bit about git-review and be able to use some of its
> configuration.
[...]

I guess it comes down to whether we think "finding changes in
Gerrit" is really within scope. For me it hasn't been for a very
long time, as there are other tools far better at this. If I use
git-review to retrieve a change I pretty much already know the
change number (either because I had it pulled up in gertty or the
Gerrit WebUI or saw it mentioned by an IRC bot or in an E-mail
update notification or because someone directly asked me about it).

> > I too find the notes refs handy but have a global configuration
> > in my ~/.gitconfig which seems to do the trick already so I'm
> > curious to find out how git-review might improve on that.
> 
> I didn't think that this could be done for the 'origin' remote and
> would be ignored by fit for other projects where it doesn't exist?
> Or are you using the 'gerrit' remote?

Yes, I do it globally for all remotes named "origin". At least the
git versions I use and the non-Gerrit origins with which I interact
don't seem to get confused when there's no refs/notes/review in a
repo. And for Gerrit-hosted repos, Gerrit replication includes notes
refs even if I'm cloning from the replica rather than directly from
Gerrit. If there are corner cases this breaks, I'd appreciate
knowing about them so I can help with a better implementation, but
so far it's worked out great for me.

> But to the main advantage is that it opens this up to many people
> that might not have been aware it exists. And also opens up to
> asking the user if they'd like the review notes displayed along
> with the logs by default for this repo by setting core.notesRef.
[...]

Yes, this is one of the reasons I consider it to still be in scope.
It's a bootstrapping situation.

> > I'd love to know what about git-review is focused on OpenStack's
> > workflow. We tried to make it as generic as possible. If there
> > are any OpenStack-specific features still lingering in there, we
> > should see about ripping them out as soon as is feasible.
[...]
> There may be others, I recall this coming up before around being
> able to set review scores for labels at the same time as uploading
> the change. Think it was around the 'Workflow' label. Maybe this
> is a case for a different command, but it seems likely to break
> the flow doe anyone using git-review to submit. However the labels
> for each project are customizable so it seems likely this would
> need the correct set to be worked out at setup time if included.
[...]

Some of that situation hails from the dark ages when the OpenStack
community ran a fork of Gerrit with a "work in progress" feature we
were unable to get upstreamed. There was pressure to get git-review
to support it (a -w flag landed in the codebase at some point for
this) but that was a very clear OpenStackism and even long before
the OpenStack community switched back to mainline Gerrit and started
using a "Workflow" label to indicate work in progress status (among
other things) the short-lived -w option was ripped out of git-review
in a mass revert to put the brakes on the runaway train it was
becoming: https://review.openstack.org/13890

As mentioned in the more recent review(s), setting arbitrary labels
at upload might be acceptable if we can come up with a clean
solution for that, but encoding OpenStack community assumptions like
"Workflow=-1 means work-in-progress" must be avoided. Problem is
that each deployment's (even each repository's/ACL's) use of
non-default labels would need some semantic policy description
language in the .gitreview file unless we really do want it to just
be a insert-arbitrary-label-votes-here sort of thing.

An interesting aside, latest versions of Gerrit implement a "work in
progress" feature to replace the terrible "draft" functionality, so
the OpenStack community's use of Workflow=-1 to signal that may
cease to be relevant soon after the next review.openstack.org
upgrade.
-- 
Jeremy Stanley


signature.asc

Re: [OpenStack-Infra] What's the future for git-review?

[Darragh Bailey]

Perhaps sending that email right before taking a few days off meaning I
couldn't reply straight away wasn't the most helpful ;-)


On Thu, 5 Jul 2018, 02:57 Jeremy Stanley,  wrote:

> On 2018-07-04 22:32:53 +0100 (+0100), Darragh Bailey wrote:
> [...]
> > Based on the comments at
> >
> https://git.openstack.org/cgit/openstack-infra/git-review/tree/CONTRIBUTING.rst#n5
> ,
> > git-review is considered feature complete, and as a consequence it
> > seems that reviewers have mostly moved onto other projects so it
> > can take quite some time to get reviews. Perfectly understandable,
> > everyone can only do so much and needs to pick something(s) to
> > prioritise. However this is such a useful tool for working with
> > Gerrit from the command line it seems to be the defacto git
> > subcommand for interfacing with Gerrit that it seems a shame to
> > limit it.
>
> At one time git-review had started to suffer from scope creep and
> was getting more random proposals for new features than actual
> stability improvements. Its test coverage was, for quite a long
> while, also sub-par so that some of the feature additions which did
> get accepted introduced regressions that went unnoticed sometimes
> for weeks or months before we'd discover they needed reverting. The
> original authors intended for git-review to be primarily focused on
> bootstrapping Gerrit connectivity from a cloned Git repository as
> well as simplifying the basic Git commands for retrieving changes
> from and pushing changes to a Gerrit. That update to the
> CONTRIBUTING.rst file was intended to put the brakes on future scope
> creep, especially in cases where an added feature would work just as
> well as its own separate git subcommand.
>

You've made one suggestion below on that applying for the list behaviour.
Made me think that an etherpad listing the current outstanding or abandoned
changes and discussing which would be better off outside of git-review
would be a useful starting place.

> While I think there are a number of current reviews that would be
> > beneficial to git-review, as well as some pieces that don't appear
> > to be there currently, I'm reluctant to invest much time as it
> > seems unlikely enhancements would be accepted due to the current
> > state of feature complete. Instead of putting together various
> > changes to see if they might be reviewed and accepted, hoping a
> > chat about what paths might be available could save a bit of time.
>
> I've tried to go in and approve changes from time to time, but in
> all honesty the negativity I've received in the past when attempting
> to push back on feature additions has caused me to deprioritize
> reviewing more changes for it. I should probably just buck up and go
> in with a (very polite) machete anyway.
>

I think it's all due to needing to prioritise time spent and git-review has
mostly done what's needed.


> There are a couple of things that I would like to work towards:
> >
> > * Change the tests to use a single gerrit with separate projects
> > instead of separate instances (faster testing)
>
> This seems reasonable if it doesn't introduce new races or odd test
> interdependencies from the reduced fixture isolation. I really have
> never been fond of the integration-testing-only model we ended up
> with though. I originally recommended lower-level unit testing with
> mocks for the Git and SSH interactions, but the one volunteer we got
> to implement a testsuite chose to automate Gerrit installation so it
> is what it is at the moment.
>

More mocks around ssh/http should work well, but I've found that it's not
necessarily beneficial doing the same around git, as with some simple
fixtures it can be tested very quickly.

The existing tests are still useful, and changing to a single gerrit
instance per runner would require some thought in the logging side (adding
markers should be sufficient I think), but thetas probably the main issue.


> * Allow the tests to run against multiple versions of Gerrit (ensure
> > compatibility)
>
> This seems reasonable. We should have been bumping the Gerrit
> versions in the tests and/or running more jobs for different
> releases of it but the way version selection was implemented would
> need a bit of an overhaul to accommodate that.
>

I'm thinking also for compatibility across a few releases would be good as
well.

> * Fix and land many of the changes making it easier to download
> > changes, list changes ordered with their dependencies, stashing
> > when downloading, etc
>
> The change listing feature really seems increasingly out of place to
> me, and most of the "fixes" I saw related to it were about
> supporting more and more of Gerrit's query language and terminal
> formatting. If we deprecated/removed that and recommended
> interacting directly with Gerrit or alternative utilities for change
> searches (there are a lot more options for this than there were back
> when git-review was first written) all of those would become
> 

Re: [OpenStack-Infra] Brainstorming winterstack service branding

[Clark Boylan]

On Fri, Jul 6, 2018, at 10:40 AM, Clark Boylan wrote:
> As mentioned in this week's meeting I offered to start a draft of what 
> the branding sets might look like for winterstack services. Specifically 
> which would not be whitelabeled and which we'd expect to explicitly 
> whitelabel for projects. I've got a really early (and rough) list going 
> at https://etherpad.openstack.org/p/winterscale-service-branding along 
> with some simple criteria I used for putting things in one bucket or 
> another.
> 
> Feel free to add your thoughts there or move things around. Once we get 
> a list we are reasonable happy with I think we can share it more broadly 
> as an indicator for what projects should be able to rely on once we get 
> winterstack moving.
> 

As noted on IRC it should be "winterscale" not "winterstack". I got it right on 
the etherpad, but then my typing drivers failed and got it wrong when writing 
this email.

Sorry for the confusion,
Clark

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] Brainstorming winterstack service branding

[Clark Boylan]

As mentioned in this week's meeting I offered to start a draft of what the 
branding sets might look like for winterstack services. Specifically which 
would not be whitelabeled and which we'd expect to explicitly whitelabel for 
projects. I've got a really early (and rough) list going at 
https://etherpad.openstack.org/p/winterscale-service-branding along with some 
simple criteria I used for putting things in one bucket or another.

Feel free to add your thoughts there or move things around. Once we get a list 
we are reasonable happy with I think we can share it more broadly as an 
indicator for what projects should be able to rely on once we get winterstack 
moving.

Clark

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Followup on the future of Infra config management specs

[Clark Boylan]

On Tue, May 8, 2018, at 3:00 PM, Clark Boylan wrote:
> Hello everyone,
> 
> Last week we got all three of the promised potential future config 
> management system specs pushed to Gerrit.
> 
> They can be found here:
> * https://review.openstack.org/449933 Puppet 4 Infra
> * https://review.openstack.org/469983 Ansible Infra
> * https://review.openstack.org/565550 Containerized Infra
> 
> A good chunk of us appear to have reviewed them at this point. During 
> today's Infra meeting I asked for some initial thoughts and the 
> direction people thought they saw us going in.
> 
> The general mood seems to be using a system that decouples applications 
> from their host platforms (containers as packaging essentially) and 
> config management to build the base platform(s) that doesn't require 
> every server have specific versions of specific tools (Ansible) would be 
> a helpful long term goal. That said any transition will take time and 
> the puppet upgrade is long over due.
> 
> With all of this considered the rough plan that I propose is: "life 
> support puppet4 short/medium term, transition to ansible base + 
> container application "packaging" longer term, eventually having zuul do 
> deployments (but this last bit should be its own spec and is out of 
> scope of current effort)".
> 
> I think this gives us a good short term option that should be doable 
> (upgrade puppetry to puppet 4). Then we can transition in the goodness 
> of not tightly coupling our config management tooling and applications 
> themselves to the platforms we run. Monty has volunteered to do the 
> combining of the specs to reflect what this more concrete plan would 
> look like.
> 
> I know not everyone can attend the meetings so wanted to make sure 
> everyone saw this and hence this thread. Please provide feedback if you 
> feel strongly about this plan (think it is terrible or think it is 
> great, info is useful in both cases).

Monty has put together a spec for this at https://review.openstack.org/565550. 
A couple of us have reviewed it and I think it is really close to being ready. 
In this week's meeting I suggested that we target the week of July 17 for 
approving this if we can get reviewers to look it over and help refine it 
futher (as necessary).

Then we get almost two months to work on initial tasks before meeting at the 
PTG to focus on the bits we've found could use face to face time.

All that to say please review this spec. I'm hopeful we can approve it with a 
reasonable amount of consensus the week of July 17.

Thank you,
Clark

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] How do I add a third party CI by ZuulV3?

[James E. Blair]

We could consider hosting a config-project with pipeline definitions for
third-party CI as an optional service folks could use.  It would not,
however, be able to support customized reporting messages or recheck
syntax.

-Jim

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] How do I add a third party CI by ZuulV3?

[Clark Boylan]

On Fri, Jul 6, 2018, at 1:49 AM, Rikimaru Honjo wrote:
> Hello,
> 
> I'd like to add a third party CI of networking-spp project.[1]
> But, I have some question about it.
> I'd appreciate it if you give information.
> 
> My wishes are the following:
> 
> * I'd like to run my test on my environment.
>Because my test requires special environment.
> * I'm planning that check new patch-sets and run my test by ZuulV3.
> 
> So I built ZuulV3 and nodepool on my environment, and pushed .zuul.yaml
> to gerrit.[2]
> 
> But, the following error was returned.
> Should I add settings of my third party CI to project-config in this case?
> If it is "Yes", is there documents about the way?
> 
> I confirmed ,
> but there was no information for ZuulV3.
> 
> > Zuul encountered a syntax error while parsing its configuration in the
> > repo openstack/networking-spp on branch master.  The error was:
> > 
> >   Pipelines may not be defined in untrusted repos, they may only be
> >   defined in config repos.
> 
> 
> [1]
> https://github.com/openstack/networking-spp
> 
> [2]
> https://review.openstack.org/#/c/580561/1

The Zuul config in the projects that OpenStack Infra hosts apply to the 
OpenStack Zuul instance. Certain aspects of this config must be defined in a 
trusted repo to protect this instance from unintended (or even malicious) 
updates in the repos we host. The error you ran into is a case of this.

In particular pipelines define when and how zuul should run jobs so we don't 
want anyone to be able to update that without review in central trusted config.

As for how to do this for third party CI, your Zuul would need to have its own 
trusted config (for the same reasons as above, but protecting your Zuul 
instance not ours). That config will have pipelines defined. If the project is 
comfortable with it you can define the jobs and playbooks and roles for third 
party CI in the upstream project. Then you would select to run those jobs in 
your Zuul's local config and report the results back to Gerrit from there.

Or if the upstream project wants to keep that data out of tree you can 
configure all of it in your Zuul config locally. One drawback to hosting the 
job config upstream would be that changes to the job config can be made without 
gating them and ensuring that they work (because third party CI can only vote 
+/-1). This problem is likely less of an issue if reviewers respect the third 
party CI results.

I think to start I would mostly keep what you've done, but move the pipeline 
definitions and project config that says what jobs to run into your Zuul's 
config.

Hope this helps,
Clark

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] How do I add a third party CI by ZuulV3?

[Rikimaru Honjo]

Hello,

I'd like to add a third party CI of networking-spp project.[1]
But, I have some question about it.
I'd appreciate it if you give information.

My wishes are the following:

* I'd like to run my test on my environment.
  Because my test requires special environment.
* I'm planning that check new patch-sets and run my test by ZuulV3.

So I built ZuulV3 and nodepool on my environment, and pushed .zuul.yaml
to gerrit.[2]

But, the following error was returned.
Should I add settings of my third party CI to project-config in this case?
If it is "Yes", is there documents about the way?

I confirmed ,
but there was no information for ZuulV3.


Zuul encountered a syntax error while parsing its configuration in the
repo openstack/networking-spp on branch master.  The error was:

  Pipelines may not be defined in untrusted repos, they may only be
  defined in config repos.



[1]
https://github.com/openstack/networking-spp

[2]
https://review.openstack.org/#/c/580561/1

Best regards,
--
_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/_/
Rikimaru Honjo
E-mail:honjo.rikim...@po.ntt-tx.co.jp


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra