Bug#685134: Bug#684909: Bug#685134: [s390-tools] please add patch from qemu

2017-12-12 Thread Christian Ehrhardt
> Would the attached patch be acceptable for debian?
>
[...]
> Regards,
>
> Dimitri.

Hi,
I'm pinging here as qemu 2.11 is likely soon to be released, so qemu
work is ahead anyway.
Therefore I wanted to ask if this found some sort of consensus (as
there was no more reply to Dimitris last patch) and is just not yet
worked on - or - if there is still too much disagreement?



Bug#685134: Bug#684909: Bug#685134: [s390-tools] please add patch from qemu

2017-11-15 Thread Dimitri John Ledkov
On 19 October 2017 at 20:53, Aurelien Jarno  wrote:
> On 2017-10-19 16:31, Philipp Kern wrote:
>> On 10/19/2017 03:06 PM, Michael Tokarev wrote:
>> > Debian has much stricter policy wrt blobs (DFSG),
>> > and debian builds for more architectures (the firmware,
>> > if it is part of qemu-system-s390 package, needs to be
>> > built on all architectures where this binary package
>> > builts, or it needs to be a separate arch-all package).
>>
>> Note that the arch:all autobuilders are amd64. gcc-*-s390x-linux-gnu
>> exists in Debian, although only on i386 and amd64. I don't think there's
>> a policy today that precludes you from forcing users to build arch:all
>> on amd64 for technical reasons.
>

Well, depwait state is valid, no? E.g. i'm sure we have many arch:all
packages that cannot be built on some architectures if e.g. arch:all
source package has an arch:all build dependency that is not
installable on some architectures.

As long as arch:all is buildable on a few common architectures, that's
good enough, no?

> Indeed that's one option to build it, that's for example the solution
> chosen to build slof using gcc-powerpc64-linux-gnu. So far nobody
> complained it's buildable only on amd64, i386, ppc64el and x32.
>

Note there are now two s390 firmwares in qemu: ccw and netboot. The
netboot one needs SLOF sources.

> The other alternative is to build a cross-compiler using binutils-source
> and gcc-source (that requires that the none or elf os is supported for
> this architecture). This has the advantage of ignoring all the flags
> that debhelper tries to push that make a firmware to not build or break.
> That's the solution chosen for example for openbios.
>

Would the attached patch be acceptable for debian?

* It drops stripping roms/SLOF

* Adds Build-Depends-Indep: crossbuild-essential-s390x
(such that it is needed only on the arch:all debian builder, or when
building with -A, meaning it is not needed when doing arch builds,
tested that arch-only build doesn't require this)
(maybe we can add a build profile  or like 
for this too, to exclude this step, and thus enable people to manually
build arch-all & arch-any packages, without the crosstoolchainy bits)

* Adds code to compile s390 firmwares and install them into
qemu-s390-firmware arch:all package
(package name is arbitrary - it did not feel right to call it bios...
any better name suggestions are welcome)

Given that roms/SLOF code is needed by two firmwares, maybe we can
explore collapsing src:qemu-slof into src:qemu too, using strategy
similar to the above one.

Installing s390x cross toolchain, and compiling s390 firmware adds
trivial amount of extra build-time, given how long/big the full
src:qemu build already is.

-- 
Regards,

Dimitri.
diff -ru debian/qemu-2.10.0+dfsg/debian/changelog qemu-2.10.0+dfsg/debian/changelog
--- debian/qemu-2.10.0+dfsg/debian/changelog	2017-10-08 10:51:09.0 +0100
+++ qemu-2.10.0+dfsg/debian/changelog	2017-11-14 20:57:02.0 +
@@ -1,3 +1,10 @@
+qemu (1:2.10.0+dfsg-2.1) UNRELEASED; urgency=medium
+
+  * Non-maintainer upload.
+  * Compile and provide s390 ccw & netboot firmware.
+
+ -- Dimitri John Ledkov   Tue, 14 Nov 2017 20:56:50 +
+
 qemu (1:2.10.0+dfsg-2) unstable; urgency=medium
 
   * update to upstream 2.10.1 point release
diff -ru debian/qemu-2.10.0+dfsg/debian/control qemu-2.10.0+dfsg/debian/control
--- debian/qemu-2.10.0+dfsg/debian/control	2017-10-08 10:51:09.0 +0100
+++ qemu-2.10.0+dfsg/debian/control	2017-11-15 13:01:01.624664678 +
@@ -106,6 +106,7 @@
 ##--enable-netmap todo bsd
 ##--enable-quorum todo needs gcrypt
 ##--enable-xen-pci-passthrough todo
+Build-Depends-Indep: crossbuild-essential-s390x
 Build-Conflicts: oss4-dev
 Standards-Version: 3.9.8
 Homepage: http://www.qemu.org/
@@ -482,3 +483,11 @@
  Please note that old qemu-kvm configuration files (in /etc/kvm/) are
  no longer used.
 
+Package: qemu-s390-firware
+Architecture: all
+Depends: ${misc:Depends}
+Enhances: qemu-system-misc
+Description: QEMU s390 ccw and pxeboot firmware images
+ QEMU is a fast processor emulator. This package provides ccw (virtio) and
+ netboot (pxeboot) firmware for the s390 system emulator.
+
diff -ru debian/qemu-2.10.0+dfsg/debian/control-in qemu-2.10.0+dfsg/debian/control-in
--- debian/qemu-2.10.0+dfsg/debian/control-in	2017-10-08 10:35:23.0 +0100
+++ qemu-2.10.0+dfsg/debian/control-in	2017-11-15 13:01:00.724667246 +
@@ -107,6 +107,7 @@
 ##--enable-netmap todo bsd
 ##--enable-quorum todo needs gcrypt
 ##--enable-xen-pci-passthrough todo
+Build-Depends-Indep: crossbuild-essential-s390x
 Build-Conflicts: oss4-dev
 Standards-Version: 3.9.8
 Homepage: http://www.qemu.org/
@@ -529,6 +530,17 @@
  Please note that old qemu-kvm configuration files (in /etc/kvm/) are
  no longer used.
 
+Package: qemu-s390-firware
+Architecture: all
+Depends: ${misc:Depends}
+:ubuntu:Breaks: qemu-system-s390x (<< 1:2.10+dfsg-3~)
+:ubuntu:Replaces: 

Bug#685134: Bug#684909: Bug#685134: [s390-tools] please add patch from qemu

2017-10-19 Thread Aurelien Jarno
On 2017-10-19 16:31, Philipp Kern wrote:
> On 10/19/2017 03:06 PM, Michael Tokarev wrote:
> > Debian has much stricter policy wrt blobs (DFSG),
> > and debian builds for more architectures (the firmware,
> > if it is part of qemu-system-s390 package, needs to be
> > built on all architectures where this binary package
> > builts, or it needs to be a separate arch-all package).
> 
> Note that the arch:all autobuilders are amd64. gcc-*-s390x-linux-gnu
> exists in Debian, although only on i386 and amd64. I don't think there's
> a policy today that precludes you from forcing users to build arch:all
> on amd64 for technical reasons.

Indeed that's one option to build it, that's for example the solution
chosen to build slof using gcc-powerpc64-linux-gnu. So far nobody
complained it's buildable only on amd64, i386, ppc64el and x32.

The other alternative is to build a cross-compiler using binutils-source
and gcc-source (that requires that the none or elf os is supported for
this architecture). This has the advantage of ignoring all the flags
that debhelper tries to push that make a firmware to not build or break.
That's the solution chosen for example for openbios.

Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net


signature.asc
Description: PGP signature


Bug#685134: Bug#684909: Bug#685134: [s390-tools] please add patch from qemu

2017-10-19 Thread Aurelien Jarno
On 2017-10-19 18:15, Dimitri John Ledkov wrote:
> On 19 October 2017 at 15:31, Philipp Kern  wrote:
> > On 10/19/2017 03:06 PM, Michael Tokarev wrote:
> >> Debian has much stricter policy wrt blobs (DFSG),
> >> and debian builds for more architectures (the firmware,
> >> if it is part of qemu-system-s390 package, needs to be
> >> built on all architectures where this binary package
> >> builts, or it needs to be a separate arch-all package).
> >
> > Note that the arch:all autobuilders are amd64. gcc-*-s390x-linux-gnu
> > exists in Debian, although only on i386 and amd64. I don't think there's
> > a policy today that precludes you from forcing users to build arch:all
> > on amd64 for technical reasons.
> >
> > Kind regards
> > Philipp Kern
> >
> 
> The s390x firmware in question, is built from source, on Ubuntu, on
> every src:qemu upload on the s390x architecture and shipped in an
> arch:s390x package.
> 
> qemu-system-s390x requires to run on the s390x hardware, as it is
> effectively passthrough kvm only, and is not userspace emulated.
> (Does not work on non-s390x machines).

Maybe because Ubuntu decided to build it only on s390x. Debian ships it
built for other architectures and it works perfectly. You can emulate an
s390x with qemu-system-s390x on amd64, arm or mips. This firmware works
too.

Aurelien

-- 
Aurelien Jarno  GPG: 4096R/1DDD8C9B
aurel...@aurel32.net http://www.aurel32.net



Bug#685134: Bug#684909: Bug#685134: [s390-tools] please add patch from qemu

2017-10-19 Thread Dimitri John Ledkov
On 19 October 2017 at 15:31, Philipp Kern  wrote:
> On 10/19/2017 03:06 PM, Michael Tokarev wrote:
>> Debian has much stricter policy wrt blobs (DFSG),
>> and debian builds for more architectures (the firmware,
>> if it is part of qemu-system-s390 package, needs to be
>> built on all architectures where this binary package
>> builts, or it needs to be a separate arch-all package).
>
> Note that the arch:all autobuilders are amd64. gcc-*-s390x-linux-gnu
> exists in Debian, although only on i386 and amd64. I don't think there's
> a policy today that precludes you from forcing users to build arch:all
> on amd64 for technical reasons.
>
> Kind regards
> Philipp Kern
>

The s390x firmware in question, is built from source, on Ubuntu, on
every src:qemu upload on the s390x architecture and shipped in an
arch:s390x package.

qemu-system-s390x requires to run on the s390x hardware, as it is
effectively passthrough kvm only, and is not userspace emulated.
(Does not work on non-s390x machines).

Therefore src:qemu should build s390x specific firmware and ship in
arch:s390x package as if it is an s390x-specific binary.

Stuff about arch:all autobilders etc do not in practice cause any
constraints here since this particular firmware is useless in a non
_s390x.deb.

There are no plans to make qemu-system-s390x able to perform userspace
emulation on non-s390x architectures.

-- 
Regards,

Dimitri.



Bug#685134: Bug#684909: Bug#685134: [s390-tools] please add patch from qemu

2017-10-19 Thread Philipp Kern
On 10/19/2017 03:06 PM, Michael Tokarev wrote:
> Debian has much stricter policy wrt blobs (DFSG),
> and debian builds for more architectures (the firmware,
> if it is part of qemu-system-s390 package, needs to be
> built on all architectures where this binary package
> builts, or it needs to be a separate arch-all package).

Note that the arch:all autobuilders are amd64. gcc-*-s390x-linux-gnu
exists in Debian, although only on i386 and amd64. I don't think there's
a policy today that precludes you from forcing users to build arch:all
on amd64 for technical reasons.

Kind regards
Philipp Kern



signature.asc
Description: OpenPGP digital signature


Bug#685134: Bug#684909: Bug#685134: [s390-tools] please add patch from qemu

2017-10-19 Thread Michael Tokarev
19.10.2017 15:36, Christian Ehrhardt wrote:
> On Thu, Oct 19, 2017 at 2:01 PM, Philipp Kern  > wrote:
> On 02/03/2016 09:27 AM, Viktor Mihajlovski wrote:
> > On 02.02.2016 01:01, Dimitri John Ledkov wrote:
> >> On Fri, 17 Aug 2012 11:21:04 +0200 bastien ROUCARIES
> >> > 
> wrote:
> >>> Please add http://repo.or.cz/w/s390-tools.git 
>  for booting s390 from qemu. it port 
> virtio.
> >>> I have ported the pach queue to recent s390-tools. Not tested only 
> merge without conflict.
> >> Is this still required? I see that qemu-2.5 has support for booting El
> >> Torito .iso images, and it boots Debian off virtio block drives just
> >> fine.
> >>
> >> Granted, it looks like debian packaging strips the s390 firmware file,
> >> and doesn't rebuild it. Maybe that should simply be fixed and that's
> >> it?
> > Right, including the firmware file would be the proper way to enable
> > QEMU for booting on s390.

There's no way to include firmware on debian without actually building it,
and building it, as far as I can see, only works on s390 architecture.
That's why we don't build other firmware stuff - either lack of cross-
compiler on all necessary architectures, or the requiriment to have
actual system of the necessary architecture (virtual or real). It'd be
nice to have it all in one place.

> If I understand it correctly, this is now obsoleted by the fact that
> qemu dropped s390-virtio and runs with their own s390-ccw rom now that's
> built from the qemu source. Is that correct? Can we close the s390-tools
> bug at least? And I suppose arrange for s390-ccw.rom to be shipped from
> the qemu package?
> 
>  
> On Ubuntu there already is: /usr/share/qemu/s390-ccw.img
> which works for me as part package qemu-system-s390x.
> 
> That is part of the suggested contribution in bug 874347 already.
> See #2 in that bug and the suggestion to accept it without an ubuntu: label 
> in d/control-in.

Debian has much stricter policy wrt blobs (DFSG),
and debian builds for more architectures (the firmware,
if it is part of qemu-system-s390 package, needs to be
built on all architectures where this binary package
builts, or it needs to be a separate arch-all package).

I'll take look at all this once again.

It looks like the s390-tools bug can be closed indeed.

Thanks,

/mjt



Bug#685134: Bug#684909: Bug#685134: [s390-tools] please add patch from qemu

2017-10-19 Thread Christian Ehrhardt
On Thu, Oct 19, 2017 at 2:01 PM, Philipp Kern  wrote:

> On 02/03/2016 09:27 AM, Viktor Mihajlovski wrote:
> > On 02.02.2016 01:01, Dimitri John Ledkov wrote:
> >> On Fri, 17 Aug 2012 11:21:04 +0200 bastien ROUCARIES
> >>  wrote:
> >>> Please add http://repo.or.cz/w/s390-tools.git for booting s390 from
> qemu. it port virtio.
> >>> I have ported the pach queue to recent s390-tools. Not tested only
> merge without conflict.
> >> Is this still required? I see that qemu-2.5 has support for booting El
> >> Torito .iso images, and it boots Debian off virtio block drives just
> >> fine.
> >>
> >> Granted, it looks like debian packaging strips the s390 firmware file,
> >> and doesn't rebuild it. Maybe that should simply be fixed and that's
> >> it?
> > Right, including the firmware file would be the proper way to enable
> > QEMU for booting on s390.
>
> If I understand it correctly, this is now obsoleted by the fact that
> qemu dropped s390-virtio and runs with their own s390-ccw rom now that's
> built from the qemu source. Is that correct? Can we close the s390-tools
> bug at least? And I suppose arrange for s390-ccw.rom to be shipped from
> the qemu package?
>
>
On Ubuntu there already is: /usr/share/qemu/s390-ccw.img
which works for me as part package qemu-system-s390x.

That is part of the suggested contribution in bug 874347 already.
See #2 in that bug and the suggestion to accept it without an ubuntu: label
in d/control-in.


-- 
Christian Ehrhardt
Software Engineer, Ubuntu Server
Canonical Ltd