Re: [ovirt-devel] FC27 updates broken for ovirt-4.2

[Dan Horák]

On Tue, 27 Feb 2018 16:26:28 -0500
Cole Robinson <crobi...@redhat.com> wrote:

> On 02/27/2018 08:03 AM, Dan Horák wrote:
> > On Tue, 27 Feb 2018 13:54:06 +0100
> > Viktor Mihajlovski <mihaj...@linux.vnet.ibm.com> wrote:
> > 
> >> On 27.02.2018 13:35, Nir Soffer wrote:
> >>> בתאריך יום ג׳, 27 בפבר׳ 2018, 13:25, מאת Dan Horák
> >>>  ‏<d...@danny.cz>:
> >>>
> >>>> On Tue, 27 Feb 2018 10:13:15 +0100
> >>>> Viktor Mihajlovski <mihaj...@linux.vnet.ibm.com> wrote:
> >>>>
> >>>>> On 27.02.2018 01:26, Nir Soffer wrote:
> >>>>>> בתאריך יום ב׳, 26 בפבר׳ 2018, 22:10, מאת Yaniv Kaul
> >>>>>>  ‏<yk...@redhat.com>:
> >>>>>>
> >>>>>>> On Mon, Feb 26, 2018 at 7:17 PM, Viktor Mihajlovski <
> >>>>>>> mihaj...@linux.vnet.ibm.com> wrote:
> >>>>>>>
> >>>>>>>> I just tried to update the ovirt packages on my FC27 host,
> >>>>>>>> but failed due to https://gerrit.ovirt.org/#/c/87628/
> >>>>>>>>
> >>>>>>>> vdsm now requires libvirt >= 3.10.0-132 but Fedora 27 has
> >>>>>>>> only 3.7.0-4 the moment.
> >>>>>>>>
> >>>>>>>> It's generic Fedora 27, but since I run on s390,
> >>>>>>>> cross-posting to s390 list.
> >>>>>>>>
> >>>>>>>> I guess there's good reason to require libvirt 3.10. Is there
> >>>>>>>> any chance that we can get libvirt updated for Fedora 27?
> >>>>>>>>
> >>>>>>>
> >>>>>>> Perhaps use the virt-preview[1] repo for now?
> >>>>>>>
> >>>>>>
> >>>>>> Yes, we require virt-preview for Fedora. This is why that patch
> >>>>>> did not fail in the CI.
> >>>>> Makes sense, unfortunately virt-preview doesn't contains s390
> >>>>> binaries at this point in time. Would be great if at least
> >>>>> libvirt and qemu could be built for s390.
> >>>>
> >>>> looks like it's even x86_64 only, /me wonders what it would
> >>>> require to offer other arches (aarch64, ppc64le, s390x) as well
> >>>>
> >>>
> >>> If we need to support platform not supported in virt-preview, we
> >>> need to chage the requirement so it is used only on x86_64.
> >>>
> >>> Victor, would you like to send a patch?
> >> I believe there was a good reason to bump the libvirt requirement
> >> in the vdsm package (some bugfix). Ideally, virt-preview should be
> >> build for s390 as well.
> >> If I'm not mistaking, the script
> >> https://github.com/crobinso/build-fedora-virt-preview is used to
> >> build the RPMs and populate the repository.
> >>
> >> Dan, Cole: what would it take to run this on the fedora-390 build
> >> machine?
> > 
> > after a brief look the script needs to be made multi-arch-aware (it
> > hard-codes x86_64 in some places), when it calls mock, and then it
> > needs some HW (we have ppc64le and s390x even now, aarch64 might
> > take a while), overall it looks doable to me. Cole, what do you
> > think?
> > 
> 
> I'm open to the idea in theory but in practice right now the script
> uses mock locally so it's basically tied to the one build machine I
> use which is x86. I have arm32 and aarch64 hardware locally but TBH I
> have very little interest in running a build farm and dealing with
> all the issues of connecting to remote machines, pulling down build
> output, etc. In fact I've been meaning to move virt-preview to copr
> for a long while which is going to tie it even deeper to x86, this
> would make virt-preview easier to enable and let me scrap much of my
> custom code for building/uploading repo contents.

copr would give us ppc64le and probably aarch64 too in addition to x86,
but can't help with s390. We already have build infra internally
covering all arches driven by Jenkins, with plans to move the workloads
to CentOS infra. I'll look whether or how it could be used for
multi-arch virt-preview repos. 

> We could re-implement it using koji scratch builds which have multiple
> arch support nowadays but I did that in the past for x86 and I recall
> feeling it was quite brittle, though I don't remember the details,
> maybe it was just my implementation.

I suspect the problem with koji scratch builds in this case is that
they can't be used in buildroots, which the virt-preview stack requires.


Dan
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] FC27 updates broken for ovirt-4.2

[Dan Horák]

On Tue, 27 Feb 2018 13:54:06 +0100
Viktor Mihajlovski <mihaj...@linux.vnet.ibm.com> wrote:

> On 27.02.2018 13:35, Nir Soffer wrote:
> > בתאריך יום ג׳, 27 בפבר׳ 2018, 13:25, מאת Dan Horák ‏<d...@danny.cz>:
> > 
> >> On Tue, 27 Feb 2018 10:13:15 +0100
> >> Viktor Mihajlovski <mihaj...@linux.vnet.ibm.com> wrote:
> >>
> >>> On 27.02.2018 01:26, Nir Soffer wrote:
> >>>> בתאריך יום ב׳, 26 בפבר׳ 2018, 22:10, מאת Yaniv Kaul
> >>>>  ‏<yk...@redhat.com>:
> >>>>
> >>>>> On Mon, Feb 26, 2018 at 7:17 PM, Viktor Mihajlovski <
> >>>>> mihaj...@linux.vnet.ibm.com> wrote:
> >>>>>
> >>>>>> I just tried to update the ovirt packages on my FC27 host, but
> >>>>>> failed due to https://gerrit.ovirt.org/#/c/87628/
> >>>>>>
> >>>>>> vdsm now requires libvirt >= 3.10.0-132 but Fedora 27 has only
> >>>>>> 3.7.0-4 the moment.
> >>>>>>
> >>>>>> It's generic Fedora 27, but since I run on s390, cross-posting
> >>>>>> to s390 list.
> >>>>>>
> >>>>>> I guess there's good reason to require libvirt 3.10. Is there
> >>>>>> any chance that we can get libvirt updated for Fedora 27?
> >>>>>>
> >>>>>
> >>>>> Perhaps use the virt-preview[1] repo for now?
> >>>>>
> >>>>
> >>>> Yes, we require virt-preview for Fedora. This is why that patch
> >>>> did not fail in the CI.
> >>> Makes sense, unfortunately virt-preview doesn't contains s390
> >>> binaries at this point in time. Would be great if at least
> >>> libvirt and qemu could be built for s390.
> >>
> >> looks like it's even x86_64 only, /me wonders what it would
> >> require to offer other arches (aarch64, ppc64le, s390x) as well
> >>
> > 
> > If we need to support platform not supported in virt-preview, we
> > need to chage the requirement so it is used only on x86_64.
> > 
> > Victor, would you like to send a patch?
> I believe there was a good reason to bump the libvirt requirement in
> the vdsm package (some bugfix). Ideally, virt-preview should be build
> for s390 as well.
> If I'm not mistaking, the script
> https://github.com/crobinso/build-fedora-virt-preview is used to build
> the RPMs and populate the repository.
> 
> Dan, Cole: what would it take to run this on the fedora-390 build
> machine?

after a brief look the script needs to be made multi-arch-aware (it
hard-codes x86_64 in some places), when it calls mock, and then it
needs some HW (we have ppc64le and s390x even now, aarch64 might take
a while), overall it looks doable to me. Cole, what do you think?


Dan
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] FC27 updates broken for ovirt-4.2

[Dan Horák]

On Tue, 27 Feb 2018 10:13:15 +0100
Viktor Mihajlovski  wrote:

> On 27.02.2018 01:26, Nir Soffer wrote:
> > בתאריך יום ב׳, 26 בפבר׳ 2018, 22:10, מאת Yaniv Kaul
> >  ‏:
> > 
> >> On Mon, Feb 26, 2018 at 7:17 PM, Viktor Mihajlovski <
> >> mihaj...@linux.vnet.ibm.com> wrote:
> >>
> >>> I just tried to update the ovirt packages on my FC27 host, but
> >>> failed due to https://gerrit.ovirt.org/#/c/87628/
> >>>
> >>> vdsm now requires libvirt >= 3.10.0-132 but Fedora 27 has only
> >>> 3.7.0-4 the moment.
> >>>
> >>> It's generic Fedora 27, but since I run on s390, cross-posting to
> >>> s390 list.
> >>>
> >>> I guess there's good reason to require libvirt 3.10. Is there any
> >>> chance that we can get libvirt updated for Fedora 27?
> >>>
> >>
> >> Perhaps use the virt-preview[1] repo for now?
> >>
> > 
> > Yes, we require virt-preview for Fedora. This is why that patch did
> > not fail in the CI.
> Makes sense, unfortunately virt-preview doesn't contains s390 binaries
> at this point in time. Would be great if at least libvirt and qemu
> could be built for s390.

looks like it's even x86_64 only, /me wonders what it would require to
offer other arches (aarch64, ppc64le, s390x) as well


Dan

> [...]
> -- 
> Regards,
>  Viktor Mihajlovski
> ___
> s390x mailing list -- s3...@lists.fedoraproject.org
> To unsubscribe send an email to s390x-le...@lists.fedoraproject.org
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Adding s390 support to oVirt

[Dan Horák]

On Thu, 14 Dec 2017 11:24:27 +0100
Viktor Mihajlovski  wrote:

> On 10.12.2017 10:10, Barak Korren wrote:
> [...]
> > 
> > An update for everyone woh may have been watching this thread - we
> > made it work.
> What a nice surprise after returning from a few days off. Big thanks
> to both of you, Barak and Dan.
> > 
> > With Dan's kind help we've attached an s390x VM to oVirt's CI
> > infrastructure. I've then gone ahead and made some code changes to
> > make our CI code play nice on it (So far we just assumed we own the
> > execution slaves and can do what we want on them...). Following that
> > I've gone ahead and added the basic configuration needed to make the
> > oVirt CI system support s390x jobs.
> > 
> > For now we only support using Fedora 26 on s390x. Please let me know
> > if other distributions are desired.
> Lately, I've been using Fedora 27 for the s390x porting, mostly
> because the newer package levels of the key virtualization
> components. If possible, Fedora 27 would be nice.

The Marist College agrees to provide us more guests, so we shouldn't
see capacity issues in the near (and mid) future. F-26 is good enough
for your F-27 builds, because you use mock, right?

> > 
> > The code changes I've made had already been tested and are now
> > pending code review:
> > https://gerrit.ovirt.org/c/85219
> > https://gerrit.ovirt.org/c/85221
> > 
> > Once those patches are merged it will become possible to add s390x
> > jobs to any oVirt project by adding '390x' to the list of
> > architectures targeted by the project in the JJB YAML, as well as
> > setting the 'node_filter' to be 's390x' for that architecture.
> > 
> Looking forward to see this merged. Once the s390x RPMs show up in the
> repository, I will try to do a clean setup of a s390x cluster.


Dan
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Adding s390 support to oVirt

[Dan Horák]

sending once more for the s390x list ...

On Fri, 24 Nov 2017 10:05:06 +0100
Viktor Mihajlovski <mihaj...@linux.vnet.ibm.com> wrote:

> On 21.11.2017 11:26, Dan Horák wrote:
> [...]
> >> qemu s390x emulation does not work with code compiled for z12.
> >> Would a real virtual machine be what you need?
> >> The Fedora team DOES have access to a z13. Not sure how much
> >> resources are available, but can you contact Dan Horak (on cc) if
> >> there is enough spare capacity.
> > 
> > Christian is right, we have a publicly accessible guest running
> > Fedora on the Marist College z13 mainframe. It's currently used by
> > ~5 projects (for example glibc and qemu) as their build and CI
> > host, so adding another project depends how intensive ovirt's usage
> > would be.
> As a first step one could only build the packages needed for the KVM
> host. At this point in time that would be vdsm and ovirt-host, both
> are building rather quickly.
> It should be possible to ensure that only these are built on a s390
> system using appropriate node filters.

I would say there are still 2 options how to build "add-on" packages
that install/run on top of Fedora and it depends whether they create
dependency chains or are standalone.

For standalone packages (= their Buildrequires can be resolved solely
from the Fedora repos) one can use scratch builds koji and grab the
results (= rpms) when the build finishes.

For packages with dependency chains is using an own "builder" machine
required. The mock tool is capable of managing a repo with built rpms,
so it shouldn't be difficult to achieve on our public guest.


Dan
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Adding s390 support to oVirt

[Dan Horák]

On Fri, 24 Nov 2017 10:05:06 +0100
Viktor Mihajlovski <mihaj...@linux.vnet.ibm.com> wrote:

> On 21.11.2017 11:26, Dan Horák wrote:
> [...]
> >> qemu s390x emulation does not work with code compiled for z12.
> >> Would a real virtual machine be what you need?
> >> The Fedora team DOES have access to a z13. Not sure how much
> >> resources are available, but can you contact Dan Horak (on cc) if
> >> there is enough spare capacity.
> > 
> > Christian is right, we have a publicly accessible guest running
> > Fedora on the Marist College z13 mainframe. It's currently used by
> > ~5 projects (for example glibc and qemu) as their build and CI
> > host, so adding another project depends how intensive ovirt's usage
> > would be.
> As a first step one could only build the packages needed for the KVM
> host. At this point in time that would be vdsm and ovirt-host, both
> are building rather quickly.
> It should be possible to ensure that only these are built on a s390
> system using appropriate node filters.

I would say there are still 2 options how to build "add-on" packages
that install/run on top of Fedora and it depends whether they create
dependency chains or are standalone.

For standalone packages (= their Buildrequires can be resolved solely
from the Fedora repos) one can use scratch builds koji and grab the
results (= rpms) when the build finishes.

For packages with dependency chains is using an own "builder" machine
required. The mock tool is capable of managing a repo with built rpms,
so it shouldn't be difficult to achieve on our public guest.


Dan
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Adding s390 support to oVirt

[Dan Horák]

On Mon, 20 Nov 2017 09:24:00 +0100
Martin Sivak  wrote:

> Dan Horak used to be part of this project. Lets see if he still
> remembers that.
> 
> Dan: Can you please tell us how the s390 builds in Fedora are/were
> done when no hw was available?

AFAIK Fedora was always building its packages on real mainframe HW,
using a dedicated LPAR on the Red Hat's machine.


Dan
 
> Martin Sivak
> 
> On Sat, Nov 18, 2017 at 12:14 PM, Barak Korren 
> wrote:
> > On 16 November 2017 at 19:38, Martin Sivak 
> > wrote:
> >>> Koji is very opinionated about how RPMs and specfiles should look,
> >>> AFAIK
> >>
> >> Actually, koji does not care much. Fedora packaging rules are
> >> enforced by package reviewers.
> >>
> >>> More specifically, Koji usually assumes the starting point for the
> >>> build process would be a specfile
> >>
> >> This is correct though.
> >>
> >>> Does fedora have an s390x server associated to it?
> >>
> >> They used to have s390 emulators running for that purpose iirc.
> >
> > I wonder if someone could provide more information about this. Is
> > this done via qemu? Or built-in to mock perhaps? It this exists, I
> > can pave the way to enabling s390x build support on oVirt infra.
> >
> >
> > --
> > Barak Korren
> > RHV DevOps team , RHCE, RHCi
> > Red Hat EMEA
> > redhat.com | TRIED. TESTED. TRUSTED. | redhat.com/trusted
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel

Re: [ovirt-devel] Adding s390 support to oVirt

[Dan Horák]

On Mon, 20 Nov 2017 09:16:48 +0100
Christian Borntraeger  wrote:

> 
> 
> On 11/19/2017 09:53 AM, Barak Korren wrote:
> > On 18 November 2017 at 13:14, Barak Korren 
> > wrote:
> >> On 16 November 2017 at 19:38, Martin Sivak 
> >> wrote:
>  Koji is very opinionated about how RPMs and specfiles should
>  look, AFAIK
> >>>
> >>> Actually, koji does not care much. Fedora packaging rules are
> >>> enforced by package reviewers.
> >>>
>  More specifically, Koji usually assumes the starting point for
>  the build process would be a specfile
> >>>
> >>> This is correct though.
> >>>
>  Does fedora have an s390x server associated to it?
> >>>
> >>> They used to have s390 emulators running for that purpose iirc.
> >>
> >> I wonder if someone could provide more information about this. Is
> >> this done via qemu? Or built-in to mock perhaps? It this exists, I
> >> can pave the way to enabling s390x build support on oVirt infra.
> > 
> > There seems to be a 'qemu-system-s390x.x86_64' package avalable for
> > CentOS7 in EPEL, so we might be able to use that to get some
> > emulated s390x Jenkins slaves up. As none of the oVirt infra team
> > members is currently experienced with using this, we would
> > appreciate so help there with:
> 
> qemu s390x emulation does not work with code compiled for z12.
> Would a real virtual machine be what you need?
> The Fedora team DOES have access to a z13. Not sure how much
> resources are available, but can you contact Dan Horak (on cc) if
> there is enough spare capacity.

Christian is right, we have a publicly accessible guest running Fedora
on the Marist College z13 mainframe. It's currently used by ~5 projects
(for example glibc and qemu) as their build and CI host, so adding
another project depends how intensive ovirt's usage would be.


Dan

> 
> 
> 
> > * Figuring out the right libvirt commands to get an s390x VM running
> > on an x86_64 host.
> > * Figuring out the right way to get an OS image for that VM
> > (Probably using virt-install to run the Fedora installer)
> > * Making sure some JVM is available in those VMs
> > 
> > Once we have that, the rest is quite straight forward.
> > 
> > I've created a Jira ticket to map out, discuss and track the
> > required steps: https://ovirt-jira.atlassian.net/browse/OVIRT-1772
> > 
> > WRT the OS we would use, I can see there are s390x Fedora builds
> > available but no CentOS builds, is there ongoing work to remedy
> > this, we'd rather not have to upgrade our Jenkins slaves every 6
> > months...
> > 
> > As a side note - at this time it seems we would have to manually
> > spin up the s390x VMs with libvirt. It would be awesome if oVirt
> > could get support for foreign architecture emulation at some point.
> > 
> 
___
Devel mailing list
Devel@ovirt.org
http://lists.ovirt.org/mailman/listinfo/devel