[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

[Nir Soffer]

On Wed, Dec 1, 2021 at 5:50 PM Nir Soffer  wrote:

> On Wed, Dec 1, 2021 at 11:11 AM Michal Skrivanek <
> michal.skriva...@redhat.com> wrote:
>
>> Hi all,
>> so far we haven't encounter any blocking issue with this effort, I wanted
>> to propose to decide on oVirt development moving  to GitHub, COPR and CBS.
>> Recent issue with decommissioning of our CI datacenter is a good reminder
>> why we are doing that...
>> What do we want to do?
>> 1) move "ovirt-master-snapshot" compose to COPR
>> it is feasible for all projects except ovirt-node and appliance due to
>> COPR limitations, for these two we plan to use a self-hosted runner in
>> github env.
>> it replaces the "build-artifacts" stdci stage
>> 2) move release to CentOS Community Build System to simplify our oVirt
>> releases
>> replaces our custom releng-tools process and aligns us better with CentOS
>> that is our main (and only) platform we support.
>> 3) move development from Gerrit to GitHub
>> this is a very visible change and affects every oVirt developer. We need
>> a way how to test posted patches and the current stdci "check-patch" stage
>> is overly complex and slow to run, we lack people for stdci maintenance in
>> general (bluntly, it's a dead project). Out of the various options that
>> exist we ended up converging to Github. Why? Just because it's the most
>> simple thing to do for us, with least amount of effort, least amount of
>> additional people and hw resources, with a manageable learning curve. It
>> comes at a price - it only works if we switch our primary development from
>> Gerrit to Github for all the remaining projects. It is a big change to our
>> processes, but I believe we have to go through that transition in order to
>> solve our CI troubles for good.  We started preparing guide and templates
>> to use so that we keep a uniform "look and feel" for all sub-projects, is
>> shall be ready soon.
>>
>> I'd like us to move from "POC" stage to "production", and actively start
>> working on the above, start moving project after project.
>> Let me ask for a final round of thoughts, comments, objections, we are
>> ready to go ahead.
>>
>> It's not going to be easy, but I firmly believe it will greatly improve
>> maintainability of oVirt and reduce overhead that we all struggle with for
>> years.
>>
>
> This is actually pretty easy. We start to work on this last week, and we
> have
> partial CI in place on github and gitlab.
>
> - lint: merged!
> https://github.com/oVirt/vdsm/actions/runs/1526187734
>
> - storage tests:
> https://gerrit.ovirt.org/c/vdsm/+/117882/
> https://github.com/oVirt/vdsm/actions/runs/1526489399
>
> We don't accept pull requests on github yet, but you can open
> pull request for testing, like this:
> https://github.com/oVirt/vdsm/pull/25
> The new CI runs for the pull request.
>
> I also added the same CI pipeline to gitlab:
> https://gitlab.com/nirs/vdsm/-/pipelines/420340180
>
> This will help to avoid locking to github, or backup if github has
> an outage (just happened last week).
>
> The new CI is based on containers, and should be usable locally
> using podman, see:
> https://github.com/oVirt/vdsm/blob/master/tests/start-container
>
> The only thing missing to move to github is a way to trigger OST
> and add OST results when it is done.
>

Update:

- vdsm CI moved to github:
  https://github.com/oVirt/vdsm/actions/runs/1535645571

- vdsm CI in gitlab is almost complete (non-stroage tests missing):
  https://gitlab.com/nirs/vdsm/-/pipelines/421515813/builds

- ovirt-imageio CI moved to gitub:
  https://github.com/oVirt/ovirt-imageio/actions/runs/1535038194

Developers can fork vdsm or ovirt-imageio and test changes on
their forks.

Nir


>
>
>> Thanks,
>> michal
>>
>> On 10. 11. 2021, at 9:17, Sandro Bonazzola  wrote:
>>
>> Hi, here's an update on what has been done so far and how it is going.
>>
>> *COPR*
>> All the oVirt active subprojects are now built on COPR except oVirt
>> Engine Appliance and oVirt Node: I'm still looking into how to build them
>> on COPR.
>>
>> Of those subprojects only the following are not yet built automatically
>> on patch merge event as they have pending patches for enabling the
>> automation:
>> - ovirt-engine-nodejs-modules:
>> https://gerrit.ovirt.org/c/ovirt-engine-nodejs-modules/+/117506
>> - ovirt-engine-ui-extensions:
>> https://gerrit.ovirt.org/c/ovirt-engine-ui-extensions/+/117512
>> - ovirt-web-ui: https://github.com/oVirt/ovirt-web-ui/pull/1532
>>
>> You can see the build status for the whole project here:
>> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/
>> If you are maintaining an oVirt project and you want to enable builds for
>> CentOS Stream 9 or other architectures supported by copr please let me know.
>>
>> So far, the COPR infrastructure seems reliable and working well.
>>
>> *GitHub*
>> The following projects are developed on GitHub only:
>> - ovirt-ansible-collection
>> - ovirt-cockpit-sso
>> - ovirt-web-ui
>> - 

[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

[Michal Skrivanek]

> On 2. 12. 2021, at 12:51, Milan Zamazal  wrote:
> 
> Michal Skrivanek  writes:
> 
>>> On 1. 12. 2021, at 16:57, Nir Soffer  wrote:
>>> 
>>> On Wed, Dec 1, 2021 at 11:38 AM Milan Zamazal  wrote:
 
 Michal Skrivanek  writes:
 
> Hi all,
> so far we haven't encounter any blocking issue with this effort, I
> wanted to propose to decide on oVirt development moving to GitHub,
> COPR and CBS. Recent issue with decommissioning of our CI datacenter
> is a good reminder why we are doing that...
> What do we want to do?
> 1) move "ovirt-master-snapshot" compose to COPR
> it is feasible for all projects except ovirt-node and
> appliance due to COPR limitations, for these two we plan to use a
> self-hosted runner in github env.
> it replaces the "build-artifacts" stdci stage
> 2) move release to CentOS Community Build System to simplify our oVirt 
> releases
> replaces our custom releng-tools process and aligns us better
> with CentOS that is our main (and only) platform we support.
> 3) move development from Gerrit to GitHub
> this is a very visible change and affects every oVirt
> developer. We need a way how to test posted patches and the current
> stdci "check-patch" stage is overly complex and slow to run, we lack
> people for stdci maintenance in general (bluntly, it's a dead
> project). Out of the various options that exist we ended up converging
> to Github. Why? Just because it's the most simple thing to do for us,
> with least amount of effort, least amount of additional people and hw
> resources, with a manageable learning curve. It comes at a price - it
> only works if we switch our primary development from Gerrit to Github
> for all the remaining projects. It is a big change to our processes,
> but I believe we have to go through that transition in order to solve
> our CI troubles for good.  We started preparing guide and templates to
> use so that we keep a uniform "look and feel" for all sub-projects, is
> shall be ready soon.
> 
> I'd like us to move from "POC" stage to "production", and actively
> start working on the above, start moving project after project.
> Let me ask for a final round of thoughts, comments, objections, we are 
> ready to go ahead.
 
 Hi,
 
 the Vdsm maintainers have discussed the possibility of moving Vdsm
 development to GitHub and we consider it a reasonable and feasible
 option.  Although GitHub is not on par with Gerrit as for code reviews,
 having a more reliable common development platform outweighs the
 disadvantages.  There is already an ongoing work on having a fully
 usable Vdsm CI on GitHub.
 
 One thing related to the move is that we would like to retain the
 history of code reviews from Gerrit.  The comments there contain
 valuable information that we wouldn't like to lose.  Is there a way to
 export the public Gerrit contents, once we make a switch to GitHub for
 each particular project, to something that could be reasonably used for
 patch archaeology when needed?
>>> 
>>> I think keeping a readonly instance would be best, one all projects
>>> migrated to github.
>> 
>> 
>>> 
>>> I hope there is a way to export the data to static html so it will be
>>> available forever without running an actual gerrit instance.
>> 
>> no idea...it can definitely be scraped patch after patch..but it's
>> going to be really huge and again, it will keep running, there's no
>> plan to shut it down or anything.
>> gerrit.ovirt.org will stay up 
> 
> And working / properly maintained?  Can you guarantee it will remain
> usable for the relevant purposes?  If yes then it would be indeed the
> best option.

why not? it has been in care of the infra@ovirt team for more than a decade 
now, why would it change?
The reason for our move to github is primarily due to stdci "issues. It's 
hurting our effectiveness (for a long time now) and it will become obsolete by 
a something that doesn't require that much babysitting. So we'll drop it.
But that is no reason to drop useful things. 
For as long as it is helpful it will stay, of course. Beyond that - probably 
not. Same happened to the project's history before - whole history before open 
sourcing is entirely gone, and reviews and review comments from the time when 
we used gerrit.usersys.redhat.com before we fully moved to gerrit.ovirt.org in 
2011/2012 are gone too. At some point it became useless to keep it alive.


> 
>> for as long as it's needed and relevant. If it ever comes to shutting
>> it down I don't think there's going to be anyone caring about the
>> comments
>> 
>>> 
>>> Nir
>>> 
 
> It's not going to be easy, but I firmly believe it will greatly
> improve maintainability of oVirt and reduce overhead that we all
> struggle with for years.
> 
> Thanks,
> michal
> 

[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

[Milan Zamazal]

Michal Skrivanek  writes:

>> On 1. 12. 2021, at 16:57, Nir Soffer  wrote:
>> 
>> On Wed, Dec 1, 2021 at 11:38 AM Milan Zamazal  wrote:
>>> 
>>> Michal Skrivanek  writes:
>>> 
 Hi all,
 so far we haven't encounter any blocking issue with this effort, I
 wanted to propose to decide on oVirt development moving to GitHub,
 COPR and CBS. Recent issue with decommissioning of our CI datacenter
 is a good reminder why we are doing that...
 What do we want to do?
 1) move "ovirt-master-snapshot" compose to COPR
  it is feasible for all projects except ovirt-node and
 appliance due to COPR limitations, for these two we plan to use a
 self-hosted runner in github env.
  it replaces the "build-artifacts" stdci stage
 2) move release to CentOS Community Build System to simplify our oVirt 
 releases
  replaces our custom releng-tools process and aligns us better
 with CentOS that is our main (and only) platform we support.
 3) move development from Gerrit to GitHub
  this is a very visible change and affects every oVirt
 developer. We need a way how to test posted patches and the current
 stdci "check-patch" stage is overly complex and slow to run, we lack
 people for stdci maintenance in general (bluntly, it's a dead
 project). Out of the various options that exist we ended up converging
 to Github. Why? Just because it's the most simple thing to do for us,
 with least amount of effort, least amount of additional people and hw
 resources, with a manageable learning curve. It comes at a price - it
 only works if we switch our primary development from Gerrit to Github
 for all the remaining projects. It is a big change to our processes,
 but I believe we have to go through that transition in order to solve
 our CI troubles for good.  We started preparing guide and templates to
 use so that we keep a uniform "look and feel" for all sub-projects, is
 shall be ready soon.
 
 I'd like us to move from "POC" stage to "production", and actively
 start working on the above, start moving project after project.
 Let me ask for a final round of thoughts, comments, objections, we are 
 ready to go ahead.
>>> 
>>> Hi,
>>> 
>>> the Vdsm maintainers have discussed the possibility of moving Vdsm
>>> development to GitHub and we consider it a reasonable and feasible
>>> option.  Although GitHub is not on par with Gerrit as for code reviews,
>>> having a more reliable common development platform outweighs the
>>> disadvantages.  There is already an ongoing work on having a fully
>>> usable Vdsm CI on GitHub.
>>> 
>>> One thing related to the move is that we would like to retain the
>>> history of code reviews from Gerrit.  The comments there contain
>>> valuable information that we wouldn't like to lose.  Is there a way to
>>> export the public Gerrit contents, once we make a switch to GitHub for
>>> each particular project, to something that could be reasonably used for
>>> patch archaeology when needed?
>> 
>> I think keeping a readonly instance would be best, one all projects
>> migrated to github.
>
>
>> 
>> I hope there is a way to export the data to static html so it will be
>> available forever without running an actual gerrit instance.
>
> no idea...it can definitely be scraped patch after patch..but it's
> going to be really huge and again, it will keep running, there's no
> plan to shut it down or anything.
> gerrit.ovirt.org will stay up 

And working / properly maintained?  Can you guarantee it will remain
usable for the relevant purposes?  If yes then it would be indeed the
best option.

> for as long as it's needed and relevant. If it ever comes to shutting
> it down I don't think there's going to be anyone caring about the
> comments
>
>> 
>> Nir
>> 
>>> 
 It's not going to be easy, but I firmly believe it will greatly
 improve maintainability of oVirt and reduce overhead that we all
 struggle with for years.
 
 Thanks,
 michal
 
> On 10. 11. 2021, at 9:17, Sandro Bonazzola  wrote:
> 
> Hi, here's an update on what has been done so far and how it is going.
> 
> COPR
> All the oVirt active subprojects are now built on COPR except oVirt
> Engine Appliance and oVirt Node: I'm still looking into how to build
> them on COPR.
> 
> Of those subprojects only the following are not yet built
> automatically on patch merge event as they have pending patches for
> enabling the automation:
> - ovirt-engine-nodejs-modules:
> https://gerrit.ovirt.org/c/ovirt-engine-nodejs-modules/+/117506
> -
> ovirt-engine-ui-extensions:
> https://gerrit.ovirt.org/c/ovirt-engine-ui-extensions/+/117512
> 
> - ovirt-web-ui: https://github.com/oVirt/ovirt-web-ui/pull/1532

[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

[Michal Skrivanek]

> On 1. 12. 2021, at 16:57, Nir Soffer  wrote:
> 
> On Wed, Dec 1, 2021 at 11:38 AM Milan Zamazal  wrote:
>> 
>> Michal Skrivanek  writes:
>> 
>>> Hi all,
>>> so far we haven't encounter any blocking issue with this effort, I
>>> wanted to propose to decide on oVirt development moving to GitHub,
>>> COPR and CBS. Recent issue with decommissioning of our CI datacenter
>>> is a good reminder why we are doing that...
>>> What do we want to do?
>>> 1) move "ovirt-master-snapshot" compose to COPR
>>>  it is feasible for all projects except ovirt-node and
>>> appliance due to COPR limitations, for these two we plan to use a
>>> self-hosted runner in github env.
>>>  it replaces the "build-artifacts" stdci stage
>>> 2) move release to CentOS Community Build System to simplify our oVirt 
>>> releases
>>>  replaces our custom releng-tools process and aligns us better
>>> with CentOS that is our main (and only) platform we support.
>>> 3) move development from Gerrit to GitHub
>>>  this is a very visible change and affects every oVirt
>>> developer. We need a way how to test posted patches and the current
>>> stdci "check-patch" stage is overly complex and slow to run, we lack
>>> people for stdci maintenance in general (bluntly, it's a dead
>>> project). Out of the various options that exist we ended up converging
>>> to Github. Why? Just because it's the most simple thing to do for us,
>>> with least amount of effort, least amount of additional people and hw
>>> resources, with a manageable learning curve. It comes at a price - it
>>> only works if we switch our primary development from Gerrit to Github
>>> for all the remaining projects. It is a big change to our processes,
>>> but I believe we have to go through that transition in order to solve
>>> our CI troubles for good.  We started preparing guide and templates to
>>> use so that we keep a uniform "look and feel" for all sub-projects, is
>>> shall be ready soon.
>>> 
>>> I'd like us to move from "POC" stage to "production", and actively
>>> start working on the above, start moving project after project.
>>> Let me ask for a final round of thoughts, comments, objections, we are 
>>> ready to go ahead.
>> 
>> Hi,
>> 
>> the Vdsm maintainers have discussed the possibility of moving Vdsm
>> development to GitHub and we consider it a reasonable and feasible
>> option.  Although GitHub is not on par with Gerrit as for code reviews,
>> having a more reliable common development platform outweighs the
>> disadvantages.  There is already an ongoing work on having a fully
>> usable Vdsm CI on GitHub.
>> 
>> One thing related to the move is that we would like to retain the
>> history of code reviews from Gerrit.  The comments there contain
>> valuable information that we wouldn't like to lose.  Is there a way to
>> export the public Gerrit contents, once we make a switch to GitHub for
>> each particular project, to something that could be reasonably used for
>> patch archaeology when needed?
> 
> I think keeping a readonly instance would be best, one all projects
> migrated to github.


> 
> I hope there is a way to export the data to static html so it will be
> available forever without running an actual gerrit instance.

no idea...it can definitely be scraped patch after patch..but it's going to be 
really huge and again, it will keep running, there's no plan to shut it down or 
anything.
gerrit.ovirt.org will stay up for as long as it's needed and relevant. If it 
ever comes to shutting it down I don't think there's going to be anyone caring 
about the comments

> 
> Nir
> 
>> 
>>> It's not going to be easy, but I firmly believe it will greatly
>>> improve maintainability of oVirt and reduce overhead that we all
>>> struggle with for years.
>>> 
>>> Thanks,
>>> michal
>>> 
 On 10. 11. 2021, at 9:17, Sandro Bonazzola  wrote:
 
 Hi, here's an update on what has been done so far and how it is going.
 
 COPR
 All the oVirt active subprojects are now built on COPR except oVirt
 Engine Appliance and oVirt Node: I'm still looking into how to build
 them on COPR.
 
 Of those subprojects only the following are not yet built
 automatically on patch merge event as they have pending patches for
 enabling the automation:
 - ovirt-engine-nodejs-modules:
 https://gerrit.ovirt.org/c/ovirt-engine-nodejs-modules/+/117506
 -
 ovirt-engine-ui-extensions:
 https://gerrit.ovirt.org/c/ovirt-engine-ui-extensions/+/117512
 
 - ovirt-web-ui: https://github.com/oVirt/ovirt-web-ui/pull/1532
 
 
 You can see the build status for the whole project here:
 https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/
 

[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

[Nir Soffer]

On Wed, Dec 1, 2021 at 11:38 AM Milan Zamazal  wrote:
>
> Michal Skrivanek  writes:
>
> > Hi all,
> > so far we haven't encounter any blocking issue with this effort, I
> > wanted to propose to decide on oVirt development moving to GitHub,
> > COPR and CBS. Recent issue with decommissioning of our CI datacenter
> > is a good reminder why we are doing that...
> > What do we want to do?
> > 1) move "ovirt-master-snapshot" compose to COPR
> >   it is feasible for all projects except ovirt-node and
> > appliance due to COPR limitations, for these two we plan to use a
> > self-hosted runner in github env.
> >   it replaces the "build-artifacts" stdci stage
> > 2) move release to CentOS Community Build System to simplify our oVirt 
> > releases
> >   replaces our custom releng-tools process and aligns us better
> > with CentOS that is our main (and only) platform we support.
> > 3) move development from Gerrit to GitHub
> >   this is a very visible change and affects every oVirt
> > developer. We need a way how to test posted patches and the current
> > stdci "check-patch" stage is overly complex and slow to run, we lack
> > people for stdci maintenance in general (bluntly, it's a dead
> > project). Out of the various options that exist we ended up converging
> > to Github. Why? Just because it's the most simple thing to do for us,
> > with least amount of effort, least amount of additional people and hw
> > resources, with a manageable learning curve. It comes at a price - it
> > only works if we switch our primary development from Gerrit to Github
> > for all the remaining projects. It is a big change to our processes,
> > but I believe we have to go through that transition in order to solve
> > our CI troubles for good.  We started preparing guide and templates to
> > use so that we keep a uniform "look and feel" for all sub-projects, is
> > shall be ready soon.
> >
> > I'd like us to move from "POC" stage to "production", and actively
> > start working on the above, start moving project after project.
> > Let me ask for a final round of thoughts, comments, objections, we are 
> > ready to go ahead.
>
> Hi,
>
> the Vdsm maintainers have discussed the possibility of moving Vdsm
> development to GitHub and we consider it a reasonable and feasible
> option.  Although GitHub is not on par with Gerrit as for code reviews,
> having a more reliable common development platform outweighs the
> disadvantages.  There is already an ongoing work on having a fully
> usable Vdsm CI on GitHub.
>
> One thing related to the move is that we would like to retain the
> history of code reviews from Gerrit.  The comments there contain
> valuable information that we wouldn't like to lose.  Is there a way to
> export the public Gerrit contents, once we make a switch to GitHub for
> each particular project, to something that could be reasonably used for
> patch archaeology when needed?

I think keeping a readonly instance would be best, one all projects
migrated to github.

I hope there is a way to export the data to static html so it will be
available forever without running an actual gerrit instance.

Nir

>
> > It's not going to be easy, but I firmly believe it will greatly
> > improve maintainability of oVirt and reduce overhead that we all
> > struggle with for years.
> >
> > Thanks,
> > michal
> >
> >> On 10. 11. 2021, at 9:17, Sandro Bonazzola  wrote:
> >>
> >> Hi, here's an update on what has been done so far and how it is going.
> >>
> >> COPR
> >> All the oVirt active subprojects are now built on COPR except oVirt
> >> Engine Appliance and oVirt Node: I'm still looking into how to build
> >> them on COPR.
> >>
> >> Of those subprojects only the following are not yet built
> >> automatically on patch merge event as they have pending patches for
> >> enabling the automation:
> >> - ovirt-engine-nodejs-modules:
> >> https://gerrit.ovirt.org/c/ovirt-engine-nodejs-modules/+/117506
> >> -
> >> ovirt-engine-ui-extensions:
> >> https://gerrit.ovirt.org/c/ovirt-engine-ui-extensions/+/117512
> >> 
> >> - ovirt-web-ui: https://github.com/oVirt/ovirt-web-ui/pull/1532
> >> 
> >>
> >> You can see the build status for the whole project here:
> >> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/
> >> 
> >> If you are maintaining an oVirt project and you want to enable
> >> builds for CentOS Stream 9 or other architectures supported by copr
> >> please let me know.
> >>
> >> So far, the COPR infrastructure seems reliable and working well.
> >>
> >> GitHub
> >> The following projects are developed on GitHub only:
> >> - ovirt-ansible-collection
> >> - ovirt-cockpit-sso
> >> - ovirt-web-ui
> >> - python-ovirt-engine-sdk4
> >> - ovirt-engine-sdk-go

[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

[Nir Soffer]

On Wed, Dec 1, 2021 at 11:11 AM Michal Skrivanek <
michal.skriva...@redhat.com> wrote:

> Hi all,
> so far we haven't encounter any blocking issue with this effort, I wanted
> to propose to decide on oVirt development moving  to GitHub, COPR and CBS.
> Recent issue with decommissioning of our CI datacenter is a good reminder
> why we are doing that...
> What do we want to do?
> 1) move "ovirt-master-snapshot" compose to COPR
> it is feasible for all projects except ovirt-node and appliance due to
> COPR limitations, for these two we plan to use a self-hosted runner in
> github env.
> it replaces the "build-artifacts" stdci stage
> 2) move release to CentOS Community Build System to simplify our oVirt
> releases
> replaces our custom releng-tools process and aligns us better with CentOS
> that is our main (and only) platform we support.
> 3) move development from Gerrit to GitHub
> this is a very visible change and affects every oVirt developer. We need a
> way how to test posted patches and the current stdci "check-patch" stage is
> overly complex and slow to run, we lack people for stdci maintenance in
> general (bluntly, it's a dead project). Out of the various options that
> exist we ended up converging to Github. Why? Just because it's the most
> simple thing to do for us, with least amount of effort, least amount of
> additional people and hw resources, with a manageable learning curve. It
> comes at a price - it only works if we switch our primary development from
> Gerrit to Github for all the remaining projects. It is a big change to our
> processes, but I believe we have to go through that transition in order to
> solve our CI troubles for good.  We started preparing guide and templates
> to use so that we keep a uniform "look and feel" for all sub-projects, is
> shall be ready soon.
>
> I'd like us to move from "POC" stage to "production", and actively start
> working on the above, start moving project after project.
> Let me ask for a final round of thoughts, comments, objections, we are
> ready to go ahead.
>
> It's not going to be easy, but I firmly believe it will greatly improve
> maintainability of oVirt and reduce overhead that we all struggle with for
> years.
>

This is actually pretty easy. We start to work on this last week, and we
have
partial CI in place on github and gitlab.

- lint: merged!
https://github.com/oVirt/vdsm/actions/runs/1526187734

- storage tests:
https://gerrit.ovirt.org/c/vdsm/+/117882/
https://github.com/oVirt/vdsm/actions/runs/1526489399

We don't accept pull requests on github yet, but you can open
pull request for testing, like this:
https://github.com/oVirt/vdsm/pull/25
The new CI runs for the pull request.

I also added the same CI pipeline to gitlab:
https://gitlab.com/nirs/vdsm/-/pipelines/420340180

This will help to avoid locking to github, or backup if github has
an outage (just happened last week).

The new CI is based on containers, and should be usable locally
using podman, see:
https://github.com/oVirt/vdsm/blob/master/tests/start-container

The only thing missing to move to github is a way to trigger OST
and add OST results when it is done.

Nir


> Thanks,
> michal
>
> On 10. 11. 2021, at 9:17, Sandro Bonazzola  wrote:
>
> Hi, here's an update on what has been done so far and how it is going.
>
> *COPR*
> All the oVirt active subprojects are now built on COPR except oVirt Engine
> Appliance and oVirt Node: I'm still looking into how to build them on COPR.
>
> Of those subprojects only the following are not yet built automatically on
> patch merge event as they have pending patches for enabling the automation:
> - ovirt-engine-nodejs-modules:
> https://gerrit.ovirt.org/c/ovirt-engine-nodejs-modules/+/117506
> - ovirt-engine-ui-extensions:
> https://gerrit.ovirt.org/c/ovirt-engine-ui-extensions/+/117512
> - ovirt-web-ui: https://github.com/oVirt/ovirt-web-ui/pull/1532
>
> You can see the build status for the whole project here:
> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/
> If you are maintaining an oVirt project and you want to enable builds for
> CentOS Stream 9 or other architectures supported by copr please let me know.
>
> So far, the COPR infrastructure seems reliable and working well.
>
> *GitHub*
> The following projects are developed on GitHub only:
> - ovirt-ansible-collection
> - ovirt-cockpit-sso
> - ovirt-web-ui
> - python-ovirt-engine-sdk4
> - ovirt-engine-sdk-go
>
> Within this list:
> - ovirt-engine-sdk-go is not being built in COPR as the rpm is not needed
> for developing with go and the automation is already handled on GitHub
> actions only.
> - ovirt-cockpit-sso is still triggering jenkins jobs but it's ready to
> drop them as PR are now tested with github actions too and builds are
> handled in COPR.
>
> So far, moving the development to GitHub only seems to be working well and
> I would suggest the maintainers of the oVirt subprojects to consider moving
> to GitHub only as well.
> +Sanja Bonic  

[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

[Milan Zamazal]

Michal Skrivanek  writes:

> Hi all,
> so far we haven't encounter any blocking issue with this effort, I
> wanted to propose to decide on oVirt development moving to GitHub,
> COPR and CBS. Recent issue with decommissioning of our CI datacenter
> is a good reminder why we are doing that...
> What do we want to do?
> 1) move "ovirt-master-snapshot" compose to COPR
>   it is feasible for all projects except ovirt-node and
> appliance due to COPR limitations, for these two we plan to use a
> self-hosted runner in github env.
>   it replaces the "build-artifacts" stdci stage
> 2) move release to CentOS Community Build System to simplify our oVirt 
> releases
>   replaces our custom releng-tools process and aligns us better
> with CentOS that is our main (and only) platform we support.
> 3) move development from Gerrit to GitHub
>   this is a very visible change and affects every oVirt
> developer. We need a way how to test posted patches and the current
> stdci "check-patch" stage is overly complex and slow to run, we lack
> people for stdci maintenance in general (bluntly, it's a dead
> project). Out of the various options that exist we ended up converging
> to Github. Why? Just because it's the most simple thing to do for us,
> with least amount of effort, least amount of additional people and hw
> resources, with a manageable learning curve. It comes at a price - it
> only works if we switch our primary development from Gerrit to Github
> for all the remaining projects. It is a big change to our processes,
> but I believe we have to go through that transition in order to solve
> our CI troubles for good.  We started preparing guide and templates to
> use so that we keep a uniform "look and feel" for all sub-projects, is
> shall be ready soon.
>
> I'd like us to move from "POC" stage to "production", and actively
> start working on the above, start moving project after project.
> Let me ask for a final round of thoughts, comments, objections, we are ready 
> to go ahead.

Hi,

the Vdsm maintainers have discussed the possibility of moving Vdsm
development to GitHub and we consider it a reasonable and feasible
option.  Although GitHub is not on par with Gerrit as for code reviews,
having a more reliable common development platform outweighs the
disadvantages.  There is already an ongoing work on having a fully
usable Vdsm CI on GitHub.

One thing related to the move is that we would like to retain the
history of code reviews from Gerrit.  The comments there contain
valuable information that we wouldn't like to lose.  Is there a way to
export the public Gerrit contents, once we make a switch to GitHub for
each particular project, to something that could be reasonably used for
patch archaeology when needed?

> It's not going to be easy, but I firmly believe it will greatly
> improve maintainability of oVirt and reduce overhead that we all
> struggle with for years.
>
> Thanks,
> michal
>
>> On 10. 11. 2021, at 9:17, Sandro Bonazzola  wrote:
>> 
>> Hi, here's an update on what has been done so far and how it is going.
>> 
>> COPR
>> All the oVirt active subprojects are now built on COPR except oVirt
>> Engine Appliance and oVirt Node: I'm still looking into how to build
>> them on COPR.
>> 
>> Of those subprojects only the following are not yet built
>> automatically on patch merge event as they have pending patches for
>> enabling the automation:
>> - ovirt-engine-nodejs-modules:
>> https://gerrit.ovirt.org/c/ovirt-engine-nodejs-modules/+/117506
>> -
>> ovirt-engine-ui-extensions:
>> https://gerrit.ovirt.org/c/ovirt-engine-ui-extensions/+/117512
>> 
>> - ovirt-web-ui: https://github.com/oVirt/ovirt-web-ui/pull/1532
>> 
>> 
>> You can see the build status for the whole project here:
>> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/
>> 
>> If you are maintaining an oVirt project and you want to enable
>> builds for CentOS Stream 9 or other architectures supported by copr
>> please let me know.
>> 
>> So far, the COPR infrastructure seems reliable and working well.
>> 
>> GitHub
>> The following projects are developed on GitHub only:
>> - ovirt-ansible-collection
>> - ovirt-cockpit-sso
>> - ovirt-web-ui
>> - python-ovirt-engine-sdk4
>> - ovirt-engine-sdk-go
>> 
>> Within this list:
>> - ovirt-engine-sdk-go is not being built in COPR as the rpm is not
>> needed for developing with go and the automation is already handled
>> on GitHub actions only.
>> - ovirt-cockpit-sso is still triggering jenkins jobs but it's ready
>> to drop them as PR are now tested with github actions too and builds
>> are handled in COPR.
>> 
>> So far, moving the development to GitHub only seems to be working
>> well and I would suggest the 

[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

[Michal Skrivanek]

Hi all,
so far we haven't encounter any blocking issue with this effort, I wanted to 
propose to decide on oVirt development moving  to GitHub, COPR and CBS. Recent 
issue with decommissioning of our CI datacenter is a good reminder why we are 
doing that...
What do we want to do?
1) move "ovirt-master-snapshot" compose to COPR
it is feasible for all projects except ovirt-node and appliance due to 
COPR limitations, for these two we plan to use a self-hosted runner in github 
env.
it replaces the "build-artifacts" stdci stage
2) move release to CentOS Community Build System to simplify our oVirt releases
replaces our custom releng-tools process and aligns us better with 
CentOS that is our main (and only) platform we support.
3) move development from Gerrit to GitHub
this is a very visible change and affects every oVirt developer. We 
need a way how to test posted patches and the current stdci "check-patch" stage 
is overly complex and slow to run, we lack people for stdci maintenance in 
general (bluntly, it's a dead project). Out of the various options that exist 
we ended up converging to Github. Why? Just because it's the most simple thing 
to do for us, with least amount of effort, least amount of additional people 
and hw resources, with a manageable learning curve. It comes at a price - it 
only works if we switch our primary development from Gerrit to Github for all 
the remaining projects. It is a big change to our processes, but I believe we 
have to go through that transition in order to solve our CI troubles for good.  
We started preparing guide and templates to use so that we keep a uniform "look 
and feel" for all sub-projects, is shall be ready soon.

I'd like us to move from "POC" stage to "production", and actively start 
working on the above, start moving project after project.
Let me ask for a final round of thoughts, comments, objections, we are ready to 
go ahead.

It's not going to be easy, but I firmly believe it will greatly improve 
maintainability of oVirt and reduce overhead that we all struggle with for 
years.

Thanks,
michal

> On 10. 11. 2021, at 9:17, Sandro Bonazzola  wrote:
> 
> Hi, here's an update on what has been done so far and how it is going.
> 
> COPR
> All the oVirt active subprojects are now built on COPR except oVirt Engine 
> Appliance and oVirt Node: I'm still looking into how to build them on COPR.
> 
> Of those subprojects only the following are not yet built automatically on 
> patch merge event as they have pending patches for enabling the automation:
> - ovirt-engine-nodejs-modules: 
> https://gerrit.ovirt.org/c/ovirt-engine-nodejs-modules/+/117506 
> - 
> ovirt-engine-ui-extensions: 
> https://gerrit.ovirt.org/c/ovirt-engine-ui-extensions/+/117512 
> 
> - ovirt-web-ui: https://github.com/oVirt/ovirt-web-ui/pull/1532 
> 
> 
> You can see the build status for the whole project here: 
> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/ 
> 
> If you are maintaining an oVirt project and you want to enable builds for 
> CentOS Stream 9 or other architectures supported by copr please let me know.
> 
> So far, the COPR infrastructure seems reliable and working well.
> 
> GitHub
> The following projects are developed on GitHub only:
> - ovirt-ansible-collection
> - ovirt-cockpit-sso
> - ovirt-web-ui
> - python-ovirt-engine-sdk4
> - ovirt-engine-sdk-go
> 
> Within this list:
> - ovirt-engine-sdk-go is not being built in COPR as the rpm is not needed for 
> developing with go and the automation is already handled on GitHub actions 
> only.
> - ovirt-cockpit-sso is still triggering jenkins jobs but it's ready to drop 
> them as PR are now tested with github actions too and builds are handled in 
> COPR.
> 
> So far, moving the development to GitHub only seems to be working well and I 
> would suggest the maintainers of the oVirt subprojects to consider moving to 
> GitHub only as well.
> +Sanja Bonic  can help you enabling GitHub actions 
> for your oVirt projects so please ping her if you need help.
> 
> CentOS Community Build
> 
> I'm going to try building the same projects currently being built in COPR 
> also within the CentOS Community Build system in the coming weeks.
> If you are already a CentOS Virtualization SIG member and you want to help 
> with this effort please let me know what you are going to build there so we 
> won't duplicate the work.
> If you are an oVirt project maintainer I would recommend you to join CentOS 
> Virtualization SIG so you'll be independent releasing your package builds.
> 
> 
> 
> 
> Il giorno mar 2 nov 2021 alle ore 12:06 Sandro Bonazzola  > ha scritto:
> Hi, 

[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

[Sandro Bonazzola]

Hi, here's an update on what has been done so far and how it is going.

*COPR*
All the oVirt active subprojects are now built on COPR except oVirt Engine
Appliance and oVirt Node: I'm still looking into how to build them on COPR.

Of those subprojects only the following are not yet built automatically on
patch merge event as they have pending patches for enabling the automation:
- ovirt-engine-nodejs-modules:
https://gerrit.ovirt.org/c/ovirt-engine-nodejs-modules/+/117506
- ovirt-engine-ui-extensions:
https://gerrit.ovirt.org/c/ovirt-engine-ui-extensions/+/117512
- ovirt-web-ui: https://github.com/oVirt/ovirt-web-ui/pull/1532

You can see the build status for the whole project here:
https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/
If you are maintaining an oVirt project and you want to enable builds for
CentOS Stream 9 or other architectures supported by copr please let me know.

So far, the COPR infrastructure seems reliable and working well.

*GitHub*
The following projects are developed on GitHub only:
- ovirt-ansible-collection
- ovirt-cockpit-sso
- ovirt-web-ui
- python-ovirt-engine-sdk4
- ovirt-engine-sdk-go

Within this list:
- ovirt-engine-sdk-go is not being built in COPR as the rpm is not needed
for developing with go and the automation is already handled on GitHub
actions only.
- ovirt-cockpit-sso is still triggering jenkins jobs but it's ready to drop
them as PR are now tested with github actions too and builds are handled in
COPR.

So far, moving the development to GitHub only seems to be working well and
I would suggest the maintainers of the oVirt subprojects to consider moving
to GitHub only as well.
+Sanja Bonic  can help you enabling GitHub actions for
your oVirt projects so please ping her if you need help.

*CentOS Community Build*

I'm going to try building the same projects currently being built in COPR
also within the CentOS Community Build system in the coming weeks.
If you are already a CentOS Virtualization SIG member and you want to help
with this effort please let me know what you are going to build there so we
won't duplicate the work.
If you are an oVirt project maintainer I would recommend you to join CentOS
Virtualization SIG so you'll be independent releasing your package builds.




Il giorno mar 2 nov 2021 alle ore 12:06 Sandro Bonazzola <
sbona...@redhat.com> ha scritto:

> Hi, after researching for a while and discussing part of it with a few
> other developers on IRC and calls, I'd like to update on current efforts.
> Recently I sent several patches to allow building of the merged patches
> (equivalent of build-artifacts stage in oVirt Standard CI) using copr.
> How does this work:
> Within the git repository a Makefile needs to be created in
> `.copr/Makefile`.
> The makefile needs to provide a `srpm` target with the instructions on how
> to generate a .src.rpm.
> That makefile will be executed with something like:  `make -f
> .copr/Makefile srpm outdir="/tmp/outdir/"` on copr infrastructure where the
> outdir will point to the place where the src.rpm will be stored to be sent
> to the mock instances for the different targets (el8, el9, x86_64, aarch64,
> ppc64le).
> An example of the patch providing this makefile can be seen here:
> https://gerrit.ovirt.org/c/ovirt-provider-ovn/+/117396/1/.copr/Makefile
> Please note the src.rpm will be built on Fedora 34 (as of today, 35 may be
> used soon).
>
> On GitHub, a webhook needs to be added in the repository configuration
> pointing to the copr API trigger. An administrator of the github/oVirt
> organization is needed in order to do that.
> I can handle the webhook setup for your project within the oVirt GitHub,
> please ping me as needed.
>
> The result of the latest builds can be displayed on GitHub README as in
> this patch:
> https://gerrit.ovirt.org/c/ovirt-provider-ovn/+/117396/1/README.adoc
> (Asciidoc) or this one
> https://gerrit.ovirt.org/c/vdsm/+/117368/3/README.md (Markdown)
>
> All the builds will be executed only once the patch is merged. The build
> will happen in
> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/
> This is going to replace the existing
> https://resources.ovirt.org/pub/ovirt-master-snapshot  repository
> structure.
> Advantages:
> - builds will be signed
> - composes will be immediately available and not waiting till the next day
> to be in the nightly
>
> On copr, in order to add a package to the compose you need to have admin
> role on
> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/permissions/
> I can handle for you the addition of the package to the compose, ping me
> as needed.
>
> Several packages have been already updated to match this flow, you can see
> current status here:
> https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/
> If you don't see your oVirt subproject there, please help getting it ready.
> I'm going to track status here:
> 

[ovirt-devel] Re: oVirt Community - open discussion around repositories and workflows

[Sandro Bonazzola]

Hi, after researching for a while and discussing part of it with a few
other developers on IRC and calls, I'd like to update on current efforts.
Recently I sent several patches to allow building of the merged patches
(equivalent of build-artifacts stage in oVirt Standard CI) using copr.
How does this work:
Within the git repository a Makefile needs to be created in
`.copr/Makefile`.
The makefile needs to provide a `srpm` target with the instructions on how
to generate a .src.rpm.
That makefile will be executed with something like:  `make -f
.copr/Makefile srpm outdir="/tmp/outdir/"` on copr infrastructure where the
outdir will point to the place where the src.rpm will be stored to be sent
to the mock instances for the different targets (el8, el9, x86_64, aarch64,
ppc64le).
An example of the patch providing this makefile can be seen here:
https://gerrit.ovirt.org/c/ovirt-provider-ovn/+/117396/1/.copr/Makefile
Please note the src.rpm will be built on Fedora 34 (as of today, 35 may be
used soon).

On GitHub, a webhook needs to be added in the repository configuration
pointing to the copr API trigger. An administrator of the github/oVirt
organization is needed in order to do that.
I can handle the webhook setup for your project within the oVirt GitHub,
please ping me as needed.

The result of the latest builds can be displayed on GitHub README as in
this patch:
https://gerrit.ovirt.org/c/ovirt-provider-ovn/+/117396/1/README.adoc
(Asciidoc) or this one https://gerrit.ovirt.org/c/vdsm/+/117368/3/README.md
(Markdown)

All the builds will be executed only once the patch is merged. The build
will happen in
https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/
This is going to replace the existing
https://resources.ovirt.org/pub/ovirt-master-snapshot  repository structure.
Advantages:
- builds will be signed
- composes will be immediately available and not waiting till the next day
to be in the nightly

On copr, in order to add a package to the compose you need to have admin
role on
https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/permissions/
I can handle for you the addition of the package to the compose, ping me as
needed.

Several packages have been already updated to match this flow, you can see
current status here:
https://copr.fedorainfracloud.org/coprs/ovirt/ovirt-master-snapshot/monitor/
If you don't see your oVirt subproject there, please help getting it ready.
I'm going to track status here:
https://github.com/oVirt/ovirt-site/wiki/Building-oVirt-on-COPR (not yet
started, give me a day :-) )

Please note this cover only for the "build-artifacts" stage and is meant to
provide builds of the project on a per patch way.

We are aiming at having all projects built in copr by the end of November
2021.
Once this will be completed we can drop build-artifacts related code from
the repos and free jenkins resources.

For official releases I'm looking into using the CentOS Community Build
System https://cbs.centos.org/koji/.
Maintainers willing to take care of the build and release of their own
packages are welcome to join the CentOS Virtualization SIG
https://wiki.centos.org/SpecialInterestGroup/Virtualization . This should
replace the releng-tools flow we are currently using for shipping releases
on https://resources.ovirt.org/pub/



Il giorno gio 21 ott 2021 alle ore 14:38 Sandro Bonazzola <
sbona...@redhat.com> ha scritto:

> Dear community members,
>
> We would like to take some next steps in improving usability and general
> decision making for our community and are interested in your thoughts and
> suggestions.
>
> The first step is a decision regarding our version control and workflows
> associated with it. Contributions to new features and general improvements
> for oVirt outside of Red Hat have been rare and we would like to hear from
> you if we could simplify this process for you. One suggestion we have is
> that we could simplify the contribution process and improve the contributor
> experience by moving our repositories to GitHub or GitLab.
>
> Currently, our Gerrit (gerrit.ovirt.org) instance is mirrored to GitHub
> and we already have several repositories that are exclusively developed
> there, including their CI running on GitHub Actions, for example:
> * https://github.com/oVirt/go-ovirt-client
> * https://github.com/oVirt/512-byte-vm
>
> There are also various related projects that run on GitLab, such as:
> * https://gitlab.com/qemu-project
> * https://gitlab.com/libvirt
>
> One recent project that moved from Gerrit to GitLab with a very similar
> discussion is mediawiki:
> https://www.mediawiki.org/wiki/GitLab_consultation/Discussion_summary
>
> Once we hear more of your thoughts around this and if the decision falls
> on moving to either GitHub or GitLab, our plan would be to start changing
> existing workflows and CI project by project as it makes sense. Both
> platforms offer similar features. Two key points that stand out for us are
> that:
> * GitLab is