Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Paolo Bonzini]

On 27/03/19 21:27, Philippe Mathieu-Daudé wrote:
> 
> 
> Le mer. 27 mars 2019 17:06, Daniel P. Berrangé  > a écrit :
> 
> On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote:
> > On 27/03/19 16:30, Daniel P. Berrangé wrote:
> > > Perhaps the VM test scripts should do a "HEAD" request for the image
> > > every time to discover if it has been changed on the server, before
> > > honouring the local cache.
> >
> > Another possibility is to first download the shasum from
> > download.patchew.org , and compare
> _that_ against the one that is stored
> > locally, instead of hardcoding it in QEMU's repository.
> 
> Personally I prefer the idea of having the shasum stored in the repo.
> 
> That means that if we update git master to point to a newer image,
> previous stable branches will stick with their original image, rather
> than using a new image that may be incompatible with the stable branch
> 
> Storing hash in git also means that if someone compromised the patchew
> server, they can't cause developer to run compromised images, without
> first also compromising git to change the hash.
> 
> 
> And we can git bisect too, right?

Only if the image is never overwritten with another one with the same
name (which was also the idea behind storing the files by hash on the
server).

Paolo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Philippe Mathieu-Daudé]

Le mer. 27 mars 2019 17:06, Daniel P. Berrangé  a
écrit :

> On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote:
> > On 27/03/19 16:30, Daniel P. Berrangé wrote:
> > > Perhaps the VM test scripts should do a "HEAD" request for the image
> > > every time to discover if it has been changed on the server, before
> > > honouring the local cache.
> >
> > Another possibility is to first download the shasum from
> > download.patchew.org, and compare _that_ against the one that is stored
> > locally, instead of hardcoding it in QEMU's repository.
>
> Personally I prefer the idea of having the shasum stored in the repo.
>
> That means that if we update git master to point to a newer image,
> previous stable branches will stick with their original image, rather
> than using a new image that may be incompatible with the stable branch
>
> Storing hash in git also means that if someone compromised the patchew
> server, they can't cause developer to run compromised images, without
> first also compromising git to change the hash.
>

And we can git bisect too, right?


> Regards,
> Daniel
> --
> |: https://berrange.com  -o-
> https://www.flickr.com/photos/dberrange :|
> |: https://libvirt.org -o-
> https://fstop138.berrange.com :|
> |: https://entangle-photo.org-o-
> https://www.instagram.com/dberrange :|
>

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Wainer dos Santos Moschetta]

On 03/27/2019 01:15 PM, Paolo Bonzini wrote:

On 27/03/19 17:05, Daniel P. Berrangé wrote:

On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote:

On 27/03/19 16:30, Daniel P. Berrangé wrote:

Perhaps the VM test scripts should do a "HEAD" request for the image
every time to discover if it has been changed on the server, before
honouring the local cache.

Another possibility is to first download the shasum from
download.patchew.org, and compare _that_ against the one that is stored
locally, instead of hardcoding it in QEMU's repository.

Personally I prefer the idea of having the shasum stored in the repo.

That means that if we update git master to point to a newer image,
previous stable branches will stick with their original image, rather
than using a new image that may be incompatible with the stable branch

Storing hash in git also means that if someone compromised the patchew
server, they can't cause developer to run compromised images, without
first also compromising git to change the hash.

The two are not mutually exclusive.  We can warn if the hash doesn't
match against the one in QEMU, add a --force option, or whatever.


I'm about to send a patch to make vm-test work with Python3. I can work 
on that image checking mechanism you folks have discussed, unless 
someone is already working on it.


- Wainer



Also, I have now created symlinks by hash at
http://download.patchew.org/by-sha256sum in case someone finds them useful.

Paolo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Paolo Bonzini]

On 27/03/19 19:08, Wainer dos Santos Moschetta wrote:
> 
> 
> On 03/27/2019 01:15 PM, Paolo Bonzini wrote:
>> On 27/03/19 17:05, Daniel P. Berrangé wrote:
>>> On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote:
 On 27/03/19 16:30, Daniel P. Berrangé wrote:
> Perhaps the VM test scripts should do a "HEAD" request for the image
> every time to discover if it has been changed on the server, before
> honouring the local cache.
 Another possibility is to first download the shasum from
 download.patchew.org, and compare _that_ against the one that is stored
 locally, instead of hardcoding it in QEMU's repository.
>>> Personally I prefer the idea of having the shasum stored in the repo.
>>>
>>> That means that if we update git master to point to a newer image,
>>> previous stable branches will stick with their original image, rather
>>> than using a new image that may be incompatible with the stable branch
>>>
>>> Storing hash in git also means that if someone compromised the patchew
>>> server, they can't cause developer to run compromised images, without
>>> first also compromising git to change the hash.
>> The two are not mutually exclusive.  We can warn if the hash doesn't
>> match against the one in QEMU, add a --force option, or whatever.
> 
> I'm about to send a patch to make vm-test work with Python3. I can work
> on that image checking mechanism you folks have discussed, unless
> someone is already working on it.

Absolutely not, go for it!

Paolo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Paolo Bonzini]

On 27/03/19 17:24, Laszlo Ersek wrote:
> On 03/27/19 17:15, Paolo Bonzini wrote:
>> On 27/03/19 17:05, Daniel P. Berrangé wrote:
>>> On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote:
 On 27/03/19 16:30, Daniel P. Berrangé wrote:
> Perhaps the VM test scripts should do a "HEAD" request for the image
> every time to discover if it has been changed on the server, before
> honouring the local cache.

 Another possibility is to first download the shasum from
 download.patchew.org, and compare _that_ against the one that is stored
 locally, instead of hardcoding it in QEMU's repository.
>>>
>>> Personally I prefer the idea of having the shasum stored in the repo.
>>>
>>> That means that if we update git master to point to a newer image,
>>> previous stable branches will stick with their original image, rather
>>> than using a new image that may be incompatible with the stable branch
>>>
>>> Storing hash in git also means that if someone compromised the patchew
>>> server, they can't cause developer to run compromised images, without
>>> first also compromising git to change the hash.
>>
>> The two are not mutually exclusive.  We can warn if the hash doesn't
>> match against the one in QEMU, add a --force option, or whatever.
>>
>> Also, I have now created symlinks by hash at
>> http://download.patchew.org/by-sha256sum in case someone finds them useful.
> 
> Isn't this risky? If someone replaces an image file (keeping its name),
> the old symlink will continue "working", but the hash stated by the
> symlink's name will not match the pointed-to image file.

Well, the idea is that if you use them you also double-check what you
downloaded. :)

Paolo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

On 03/27/19 17:15, Paolo Bonzini wrote:
> On 27/03/19 17:05, Daniel P. Berrangé wrote:
>> On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote:
>>> On 27/03/19 16:30, Daniel P. Berrangé wrote:
 Perhaps the VM test scripts should do a "HEAD" request for the image
 every time to discover if it has been changed on the server, before
 honouring the local cache.
>>>
>>> Another possibility is to first download the shasum from
>>> download.patchew.org, and compare _that_ against the one that is stored
>>> locally, instead of hardcoding it in QEMU's repository.
>>
>> Personally I prefer the idea of having the shasum stored in the repo.
>>
>> That means that if we update git master to point to a newer image,
>> previous stable branches will stick with their original image, rather
>> than using a new image that may be incompatible with the stable branch
>>
>> Storing hash in git also means that if someone compromised the patchew
>> server, they can't cause developer to run compromised images, without
>> first also compromising git to change the hash.
> 
> The two are not mutually exclusive.  We can warn if the hash doesn't
> match against the one in QEMU, add a --force option, or whatever.
> 
> Also, I have now created symlinks by hash at
> http://download.patchew.org/by-sha256sum in case someone finds them useful.

Isn't this risky? If someone replaces an image file (keeping its name),
the old symlink will continue "working", but the hash stated by the
symlink's name will not match the pointed-to image file.

It would be nice to enforce a hash update on upload, or addressing files
by content.

... Are we inventing a git submodule? :)

Laszlo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Paolo Bonzini]

On 27/03/19 17:05, Daniel P. Berrangé wrote:
> On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote:
>> On 27/03/19 16:30, Daniel P. Berrangé wrote:
>>> Perhaps the VM test scripts should do a "HEAD" request for the image
>>> every time to discover if it has been changed on the server, before
>>> honouring the local cache.
>>
>> Another possibility is to first download the shasum from
>> download.patchew.org, and compare _that_ against the one that is stored
>> locally, instead of hardcoding it in QEMU's repository.
> 
> Personally I prefer the idea of having the shasum stored in the repo.
> 
> That means that if we update git master to point to a newer image,
> previous stable branches will stick with their original image, rather
> than using a new image that may be incompatible with the stable branch
> 
> Storing hash in git also means that if someone compromised the patchew
> server, they can't cause developer to run compromised images, without
> first also compromising git to change the hash.

The two are not mutually exclusive.  We can warn if the hash doesn't
match against the one in QEMU, add a --force option, or whatever.

Also, I have now created symlinks by hash at
http://download.patchew.org/by-sha256sum in case someone finds them useful.

Paolo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Daniel P . Berrangé]

On Wed, Mar 27, 2019 at 04:58:23PM +0100, Paolo Bonzini wrote:
> On 27/03/19 16:30, Daniel P. Berrangé wrote:
> > Perhaps the VM test scripts should do a "HEAD" request for the image
> > every time to discover if it has been changed on the server, before
> > honouring the local cache.
> 
> Another possibility is to first download the shasum from
> download.patchew.org, and compare _that_ against the one that is stored
> locally, instead of hardcoding it in QEMU's repository.

Personally I prefer the idea of having the shasum stored in the repo.

That means that if we update git master to point to a newer image,
previous stable branches will stick with their original image, rather
than using a new image that may be incompatible with the stable branch

Storing hash in git also means that if someone compromised the patchew
server, they can't cause developer to run compromised images, without
first also compromising git to change the hash.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Paolo Bonzini]

On 27/03/19 16:30, Daniel P. Berrangé wrote:
> Perhaps the VM test scripts should do a "HEAD" request for the image
> every time to discover if it has been changed on the server, before
> honouring the local cache.

Another possibility is to first download the shasum from
download.patchew.org, and compare _that_ against the one that is stored
locally, instead of hardcoding it in QEMU's repository.

Paolo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

On 03/27/19 16:30, Daniel P. Berrangé wrote:
> On Wed, Mar 27, 2019 at 03:25:19PM +, Peter Maydell wrote:
>> On Wed, 27 Mar 2019 at 14:31, Laszlo Ersek  wrote:
>>> ... Which in turn raises the question: *before* Peter reported the "xz"
>>> failure first, against my series, at
>>>
>>>   https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg05452.html
>>>
>>> what image had Peter been using? Because, as far as I see it, at
>>> that point there was *no* way for the OpenBSD build test to succeed,
>>> *regardless* of my series. The compiler must not have been available.
>>
>> The tests/vm scripts cache the downloaded image. So my setup
>> presumably has the old image cached and is continuing to use it.
>> This is good for speed and avoiding lots of downloads, but it
>> looks like it's had the unfortunate side effect that nobody
>> noticed the breakage because everybody doing tests already
>> had the old image in their local cache and wasn't using the
>> broken new image :-(
> 
> Perhaps the VM test scripts should do a "HEAD" request for the image
> every time to discover if it has been changed on the server, before
> honouring the local cache.

"curl -I" can do that. (Sorry if that's obvious -- I'm mentioning it
because I used "curl -I" myself manually, as part of the earlier
"investigation".)

Thanks
Laszlo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

On 03/27/19 16:25, Peter Maydell wrote:
> On Wed, 27 Mar 2019 at 14:31, Laszlo Ersek  wrote:
>> ... Which in turn raises the question: *before* Peter reported the "xz"
>> failure first, against my series, at
>>
>>   https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg05452.html
>>
>> what image had Peter been using? Because, as far as I see it, at
>> that point there was *no* way for the OpenBSD build test to succeed,
>> *regardless* of my series. The compiler must not have been available.
> 
> The tests/vm scripts cache the downloaded image. So my setup
> presumably has the old image cached and is continuing to use it.
> This is good for speed and avoiding lots of downloads, but it
> looks like it's had the unfortunate side effect that nobody
> noticed the breakage because everybody doing tests already
> had the old image in their local cache and wasn't using the
> broken new image :-(

Mistery solved!

Thank you very much for responding!

Laszlo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Daniel P . Berrangé]

On Wed, Mar 27, 2019 at 03:25:19PM +, Peter Maydell wrote:
> On Wed, 27 Mar 2019 at 14:31, Laszlo Ersek  wrote:
> > ... Which in turn raises the question: *before* Peter reported the "xz"
> > failure first, against my series, at
> >
> >   https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg05452.html
> >
> > what image had Peter been using? Because, as far as I see it, at
> > that point there was *no* way for the OpenBSD build test to succeed,
> > *regardless* of my series. The compiler must not have been available.
> 
> The tests/vm scripts cache the downloaded image. So my setup
> presumably has the old image cached and is continuing to use it.
> This is good for speed and avoiding lots of downloads, but it
> looks like it's had the unfortunate side effect that nobody
> noticed the breakage because everybody doing tests already
> had the old image in their local cache and wasn't using the
> broken new image :-(

Perhaps the VM test scripts should do a "HEAD" request for the image
every time to discover if it has been changed on the server, before
honouring the local cache.

Regards,
Daniel
-- 
|: https://berrange.com  -o-https://www.flickr.com/photos/dberrange :|
|: https://libvirt.org -o-https://fstop138.berrange.com :|
|: https://entangle-photo.org-o-https://www.instagram.com/dberrange :|

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Peter Maydell]

On Wed, 27 Mar 2019 at 14:31, Laszlo Ersek  wrote:
> ... Which in turn raises the question: *before* Peter reported the "xz"
> failure first, against my series, at
>
>   https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg05452.html
>
> what image had Peter been using? Because, as far as I see it, at
> that point there was *no* way for the OpenBSD build test to succeed,
> *regardless* of my series. The compiler must not have been available.

The tests/vm scripts cache the downloaded image. So my setup
presumably has the old image cached and is continuing to use it.
This is good for speed and avoiding lots of downloads, but it
looks like it's had the unfortunate side effect that nobody
noticed the breakage because everybody doing tests already
had the old image in their local cache and wasn't using the
broken new image :-(

thanks
-- PMM

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

On 03/27/19 16:06, Paolo Bonzini wrote:
> On 27/03/19 15:31, Laszlo Ersek wrote:
>> OK, let me be constructive here. Right now, the *current*
>> "tests/vm/openbsd" script is unusable for *any* testing. (My testing did
>> succeed agains the previous image, which is still available under a
>> different name.) Who's going to fix the current image, and who's going
>> to fix the script? And when?
> 
> For now I'll just revert the image to the old one.

Thank you!

Should I propagate Daniel's original request to a Launchpad ticket? For
example:

Please install SDL 2 development packages in the OpenBSD image,
because SDL 1.2 support has been removed in qemu-4.0 -- see
 and
.

Thank you again!
Laszlo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Paolo Bonzini]

On 27/03/19 15:31, Laszlo Ersek wrote:
> OK, let me be constructive here. Right now, the *current*
> "tests/vm/openbsd" script is unusable for *any* testing. (My testing did
> succeed agains the previous image, which is still available under a
> different name.) Who's going to fix the current image, and who's going
> to fix the script? And when?

For now I'll just revert the image to the old one.

Paolo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

(Keeping Fam on the address list, adding Dan & Markus)

On 03/27/19 14:18, Paolo Bonzini wrote:
> On 25/03/19 11:40, Laszlo Ersek wrote:
>
> (1) The image file at
>  has been
> recently uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT")
> by someone unknown to me, and its sha256sum doesn't match the
> sha256sum in the "tests/vm/openbsd" test script.
>
> This is why my earlier attempts at the OpenBSD build test have
> failed.
>
 Can someone include Fam/Paolo/Brad in this thread please? (I don't
 have their emails in my cellphone). Thanks.
>>> Done.
>> - do we have any idea what happened on download.patchew.org (i.e. why
>> the image matching the script was replaced with an image not matching
>> the script)?
>
> The update was requested by Daniel in this thread:
> https://patchew.org/QEMU/20181031025712.18602-1-f...@redhat.com/
>
> Fam did the update, but forgot to update the sha256sum.

Thank you for tracking this down!

But, it's not just the sha256sum that needs an update in
"tests/vm/openbsd". In addition to that, the script refers to the
compiler under the name

  x86_64-unknown-openbsd6.1-gcc-4.9.4

This compiler is not available in the updated OpenBSD disk image, either
pre-installed, or from the network. Like I described here:

  https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg06277.html

the preinstalled compiler ("gcc") is version 4.2.1, and the one that's
available from the network, with "pkg_add", is called differently:

  x86_64-unknown-openbsd6.4-gcc-4.9.4

So, my take is that *both* the script is out-of-sync with the new
OpenBSD image, *and* the OpenBSD image is unusable (for this particular
test) and has never been verified.

... Which in turn raises the question: *before* Peter reported the "xz"
failure first, against my series, at

  https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg05452.html

what image had Peter been using? Because, as far as I see it, at
that point there was *no* way for the OpenBSD build test to succeed,
*regardless* of my series. The compiler must not have been available.

In fact, what OpenBSD image is Peter using *right now*, for the
build-QEMU-on-OpenBSD test? Again, I see no way how that could
succeed right now.

... Honestly, given that the OpenBSD image was broken & out-of-sync to
begin with, I feel it was questionable to block my series (= Phil's
PULL) from 4.0, just because the series placed an *additional*
requirement (namely "xz") on the OpenBSD image, which had *already* been
in sore need of a functional C compiler. :/

---*---

OK, let me be constructive here. Right now, the *current*
"tests/vm/openbsd" script is unusable for *any* testing. (My testing did
succeed agains the previous image, which is still available under a
different name.) Who's going to fix the current image, and who's going
to fix the script? And when?

Regarding my own series, after all this churn, I'll stick with using bz2
in it; the xz->bz2 storage "loss" is minimal, about 1%.

Thanks,
Laszlo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Paolo Bonzini]

On 25/03/19 11:40, Laszlo Ersek wrote:

 (1) The image file at
  has been recently
 uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") by someone
 unknown to me, and its sha256sum doesn't match the sha256sum in the
 "tests/vm/openbsd" test script.

 This is why my earlier attempts at the OpenBSD build test have failed.

>>> Can someone include Fam/Paolo/Brad in this thread please? (I don't have
>>> their emails in my cellphone). Thanks.
>> Done.
> - do we have any idea what happened on download.patchew.org (i.e. why
> the image matching the script was replaced with an image not matching
> the script)?

The update was requested by Daniel in this thread:
https://patchew.org/QEMU/20181031025712.18602-1-f...@redhat.com/

Fam did the update, but forgot to update the sha256sum.

Paolo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

Hi Peter,

can you please comment:

On 03/22/19 10:17, Laszlo Ersek wrote:
> On 03/22/19 08:02, Philippe Mathieu-Daudé wrote:
>> Le ven. 22 mars 2019 00:33, Laszlo Ersek  a écrit :
>>
>>> On 03/21/19 23:32, Laszlo Ersek wrote:
>>>
 Cool, so let me try this. I'm going to download the xz.old file
 manually. Rename it to just xz. It will then match the built-in
 checksum, and will be used as a cached copy. Then I will try building my
 series in *that* ("old") VM.
>>>
>>> Summary:
>>>
>>> (1) The image file at
>>>  has been recently
>>> uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") by someone
>>> unknown to me, and its sha256sum doesn't match the sha256sum in the
>>> "tests/vm/openbsd" test script.
>>>
>>> This is why my earlier attempts at the OpenBSD build test have failed.
>>>
>>
>> Can someone include Fam/Paolo/Brad in this thread please? (I don't have
>> their emails in my cellphone). Thanks.
> 
> Done.

- do we have any idea what happened on download.patchew.org (i.e. why
the image matching the script was replaced with an image not matching
the script)?

>>> And in fact I don't understand how it could work for anyone else -- the
>>> compiler that the "tests/vm/openbsd" script specifies is neither
>>> installed, nor available with "pkg_add", in this image.
>>>
>>>
>>> (2) Against the "old" image
>>> , which indeed
>>> has the expected
>>> sha256sum=8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf,
>>> the build test *does* succeed.
>>>
>>> (
>>>
>>> In order to make use of the old image, it has to be downloaded manually,
>>> then moved/renamed to:
>>>
>>>   $HOME/.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd
>>>
>>> because the last filename component must be the sha1sum of the URL
>>> itself, for the caching mechanism to recognize the compressed image:
>>>
 $ echo -n 'http://download.patchew.org/openbsd-6.1-amd64.img.xz' \
   | sha1sum
 bc4733f6c6e76931702528a515a1bf70eb8baecd  -
>>>
>>> )
>>>
>>> I'm attaching the log of the successful OpenBSD build test, which I
>>> captured with "screen" (see the "BUNZIP2" lines in it, in particular).

- are you satisfied with the OpenBSD build testing I did (using the
"old" image, matching the script)?

Thanks
Laszlo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

On 03/22/19 08:02, Philippe Mathieu-Daudé wrote:
> Le ven. 22 mars 2019 00:33, Laszlo Ersek  a écrit :
> 
>> On 03/21/19 23:32, Laszlo Ersek wrote:
>>
>>> Cool, so let me try this. I'm going to download the xz.old file
>>> manually. Rename it to just xz. It will then match the built-in
>>> checksum, and will be used as a cached copy. Then I will try building my
>>> series in *that* ("old") VM.
>>
>> Summary:
>>
>> (1) The image file at
>>  has been recently
>> uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") by someone
>> unknown to me, and its sha256sum doesn't match the sha256sum in the
>> "tests/vm/openbsd" test script.
>>
>> This is why my earlier attempts at the OpenBSD build test have failed.
>>
> 
> Can someone include Fam/Paolo/Brad in this thread please? (I don't have
> their emails in my cellphone). Thanks.

Done.

Thanks
Laszlo

>> And in fact I don't understand how it could work for anyone else -- the
>> compiler that the "tests/vm/openbsd" script specifies is neither
>> installed, nor available with "pkg_add", in this image.
>>
>>
>> (2) Against the "old" image
>> , which indeed
>> has the expected
>> sha256sum=8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf,
>> the build test *does* succeed.
>>
>> (
>>
>> In order to make use of the old image, it has to be downloaded manually,
>> then moved/renamed to:
>>
>>   $HOME/.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd
>>
>> because the last filename component must be the sha1sum of the URL
>> itself, for the caching mechanism to recognize the compressed image:
>>
>>> $ echo -n 'http://download.patchew.org/openbsd-6.1-amd64.img.xz' \
>>>   | sha1sum
>>> bc4733f6c6e76931702528a515a1bf70eb8baecd  -
>>
>> )
>>
>> I'm attaching the log of the successful OpenBSD build test, which I
>> captured with "screen" (see the "BUNZIP2" lines in it, in particular).
>>
>> Thanks,
>> Laszlo
>>
> 

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Philippe Mathieu-Daudé]

Le ven. 22 mars 2019 00:33, Laszlo Ersek  a écrit :

> On 03/21/19 23:32, Laszlo Ersek wrote:
>
> > Cool, so let me try this. I'm going to download the xz.old file
> > manually. Rename it to just xz. It will then match the built-in
> > checksum, and will be used as a cached copy. Then I will try building my
> > series in *that* ("old") VM.
>
> Summary:
>
> (1) The image file at
>  has been recently
> uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") by someone
> unknown to me, and its sha256sum doesn't match the sha256sum in the
> "tests/vm/openbsd" test script.
>
> This is why my earlier attempts at the OpenBSD build test have failed.
>

Can someone include Fam/Paolo/Brad in this thread please? (I don't have
their emails in my cellphone). Thanks.

And in fact I don't understand how it could work for anyone else -- the
> compiler that the "tests/vm/openbsd" script specifies is neither
> installed, nor available with "pkg_add", in this image.
>
>
> (2) Against the "old" image
> , which indeed
> has the expected
> sha256sum=8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf,
> the build test *does* succeed.
>
> (
>
> In order to make use of the old image, it has to be downloaded manually,
> then moved/renamed to:
>
>   $HOME/.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd
>
> because the last filename component must be the sha1sum of the URL
> itself, for the caching mechanism to recognize the compressed image:
>
> > $ echo -n 'http://download.patchew.org/openbsd-6.1-amd64.img.xz' \
> >   | sha1sum
> > bc4733f6c6e76931702528a515a1bf70eb8baecd  -
>
> )
>
> I'm attaching the log of the successful OpenBSD build test, which I
> captured with "screen" (see the "BUNZIP2" lines in it, in particular).
>
> Thanks,
> Laszlo
>

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

On 03/21/19 23:32, Laszlo Ersek wrote:

> Cool, so let me try this. I'm going to download the xz.old file
> manually. Rename it to just xz. It will then match the built-in
> checksum, and will be used as a cached copy. Then I will try building my
> series in *that* ("old") VM.

Summary:

(1) The image file at
 has been recently
uploaded ("Last-Modified: Wed, 27 Feb 2019 11:48:18 GMT") by someone
unknown to me, and its sha256sum doesn't match the sha256sum in the
"tests/vm/openbsd" test script.

This is why my earlier attempts at the OpenBSD build test have failed.
And in fact I don't understand how it could work for anyone else -- the
compiler that the "tests/vm/openbsd" script specifies is neither
installed, nor available with "pkg_add", in this image.


(2) Against the "old" image
, which indeed
has the expected
sha256sum=8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf,
the build test *does* succeed.

(

In order to make use of the old image, it has to be downloaded manually,
then moved/renamed to:

  $HOME/.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd

because the last filename component must be the sha1sum of the URL
itself, for the caching mechanism to recognize the compressed image:

> $ echo -n 'http://download.patchew.org/openbsd-6.1-amd64.img.xz' \
>   | sha1sum
> bc4733f6c6e76931702528a515a1bf70eb8baecd  -

)

I'm attaching the log of the successful OpenBSD build test, which I
captured with "screen" (see the "BUNZIP2" lines in it, in particular).

Thanks,
Laszlo


screen.log.xz
Description: application/xz

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

On 03/21/19 21:58, Philippe Mathieu-Daudé wrote:
> Le jeu. 21 mars 2019 18:22, Laszlo Ersek  a écrit :

>> Unfortunately, I got the exact same setup error:

>>> ERROR: "x86_64-unknown-openbsd6.1-gcc-4.9.4" either does not exist
>>> or does not work

> $ VERBOSE=1 make
> Or DEBUG=1. If default ./configure && make fails you get shell and can
> run again with nodefault options. You can check basevm.py in tests/.

I used the "interactive shell" commands from "docs/devel/testing.rst",
to look around inside the image.

The preinstalled gcc is this one:

> qemu-openbsd# gcc --version
> gcc (GCC) 4.2.1 20070719

not what the "openbsd" test case refers to.

Let's see what I could install myself:

> qemu-openbsd# cat /etc/installurl
> https://fastly.cdn.openbsd.org/pub/OpenBSD

> qemu-openbsd# pkg_info -Q gcc
> [...]
> gcc-4.9.4p12
> [...]

> qemu-openbsd# pkg_add gcc-4.9.4p12

And then this gives me "x86_64-unknown-openbsd6.4-gcc-4.9.4". Again,
*not* what the "openbsd" script refers to.

Then I look at the "openbsd" script itself, and realize it downloads:

> cimg = 
> self._download_with_cache("http://download.patchew.org/openbsd-6.1-amd64.img.xz;,
> 
> sha256sum='8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf')

Hmmm. Can I perhaps *browse* the download directory on patchew, in
Firefox? I sure can, directory listing is enabled:

> README 25-Aug-2017 12:14  
>  0
> freebsd-11.1-amd64.img.xz  03-Aug-2017 09:21  
>  545422116
> netbsd-7.1-amd64.img.xz03-Aug-2017 09:32  
>  224195948
> openbsd-6.1-amd64.img.xz   27-Feb-2019 11:48  
>  675094624
> openbsd-6.1-amd64.img.xz.old   16-Aug-2017 03:23  
>  622579972
> patchew-db-small.tar.xz15-Mar-2018 10:17  
>   10451488

There it is. "openbsd-6.1-amd64.img.xz.old" has an mtime of "16-Aug-2017
03:23", which slightly precedes commit fdfaa33291eb ("tests: Add OpenBSD
image", 2017-09-22).

But someone must have added that ".old" suffix to the filename
*recently*, and uploaded "openbsd-6.1-amd64.img.xz" in its place -- see
the mtime on "openbsd-6.1-amd64.img.xz": "27-Feb-2019 11:48".

I guess the "openbsd" script should have been updated too, in sync. :/

Bonus question: why doesn't the download abort with an sha256sum
mismatch? Here's what I get on my end, after manually fetching both
images:

> 0a836e5013fd502c2c1e14f3f29b25bb1f1ee946abb27d3ca3596d678a728476  
> openbsd-6.1-amd64.img.xz
> 8c6cedc483e602cfee5e04f0406c64eb99138495e8ca580bc0293bcf0640c1bf  
> openbsd-6.1-amd64.img.xz.old

The "old" image matches the checksum (added in original commit
fdfaa33291eb, see above), but the new one doesn't.

... Aaargh, "basevm.py" only uses the checksum for local caching (i.e.
for identifying stale vs. up-to-date local copies), not for
verification!

Cool, so let me try this. I'm going to download the xz.old file
manually. Rename it to just xz. It will then match the built-in
checksum, and will be used as a cached copy. Then I will try building my
series in *that* ("old") VM.

Thanks
Laszlo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Philippe Mathieu-Daudé]

Le jeu. 21 mars 2019 18:22, Laszlo Ersek  a écrit :

> On 03/21/19 17:55, Laszlo Ersek wrote:
> > On 03/21/19 12:52, Peter Maydell wrote:
> >> On Thu, 21 Mar 2019 at 11:34, Laszlo Ersek  wrote:
> >>>
> >>> Repo:   https://github.com/lersek/qemu.git
> >>> Branch: edk2_build_v3
> >>>
> >>> Version 2, that is:
> >>>
> >>>   [Qemu-devel] [PATCH v2 00/12] bundle edk2 platform firmware with QEMU
> >>>
> >>> was posted at:
> >>>
> >>>   https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04670.html
> >>>   20190313210057.32584-1-lersek@redhat.com">http://mid.mail-archive.com/20190313210057.32584-1-lersek@redhat.com
> >>>
> >>> Updates in v3 are noted on each patch individually, in the Notes
> >>> section. In summary,
> >>>
> >>> - I've picked up feedback tags from the v2 thread (see above),
> >>>
> >>> - I've replaced the xz compression with bz2 compression according to
> the
> >>>   subthread
> >>>
> >>> Re: [Qemu-devel] [PULL 00/12] EDK2 Firmware roms
> >>
> >> Thanks. Could you check that this works in the OpenBSD VM test?
> >> It should just be a matter of
> >>   make -C build vm-build-openbsd
> >
> > Actually I tried to do that before posting v3, following the
> > instructions in "docs/devel/testing.rst":
> >
> >   cd $QEMU_SRC/tests/vm
> >   ./openbsd --build-image --image /mnt/data/tmp/openbsd.img
> >   ./openbsd --debug --image /mnt/data/tmp/openbsd.img \
> > --build-qemu $QEMU_SRC
>

These isn't the line I remember using with vmtest openbsd. I remember
specifying host-cc and python.
I guess you can find it in openbsd.py under tests/vm/.

>
> > The first "openbsd" invocation succeeded, but the second failed, with a
> > complaint about some (cross?)compiler missing.
> >
> > Anyway I'll now retry with the make command you specify above.
>
> Unfortunately, I got the exact same setup error:
>
> > make: Entering directory `.../tmp/qemu-build-opt.d'
> > VM-IMAGE openbsd
> > --2019-03-21 18:14:46--
> http://download.patchew.org/openbsd-6.1-amd64.img.xz
> > Resolving download.patchew.org (download.patchew.org)... 140.211.15.4
> > Connecting to download.patchew.org 
> > (download.patchew.org)|140.211.15.4|:80...
> connected.
> > HTTP request sent, awaiting response... 200 OK
> > Length: 675094624 (644M) [application/octet-stream]
> > Saving to:
> '.../.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd.download'
> >
> >
> 100%[=>]
> 675,094,624 12.0MB/s   in 53s
> >
> > 2019-03-21 18:15:40 (12.1 MB/s) -
> '.../.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd.download'
> saved [675094624/675094624]
> >
> > Extracting the image...
> > .../.cache/qemu-vm/images/openbsd.img.tmp.xz (1/1)
> >   100 %643.8 MiB / 32.0 GiB = 0.020   253 MiB/s   2:09
> > VM-BUILD openbsd
> > Cloning into
> '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot'...
> > done.
> > Checking out files: 100% (6748/6748), done.
> > Submodule 'dtc' (https://git.qemu.org/git/dtc.git) registered for path
> 'dtc'
> > Cloning into
> '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/dtc'...
> > Submodule 'ui/keycodemapdb' (https://git.qemu.org/git/keycodemapdb.git)
> registered for path 'ui/keycodemapdb'
> > Cloning into
> '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/ui/keycodemapdb'...
> > Submodule 'tests/fp/berkeley-softfloat-3' (
> https://github.com/cota/berkeley-softfloat-3) registered for path
> 'tests/fp/berkeley-softfloat-3'
> > Cloning into
> '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/tests/fp/berkeley-softfloat-3'...
> > Submodule 'tests/fp/berkeley-testfloat-3' (
> https://github.com/cota/berkeley-testfloat-3) registered for path
> 'tests/fp/berkeley-testfloat-3'
> > Cloning into
> '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/tests/fp/berkeley-testfloat-3'...
> >
> > ERROR: "x86_64-unknown-openbsd6.1-gcc-4.9.4" either does not exist or
> does not work
> >
> > make: *** [vm-build-openbsd] Error 3
> > make: Leaving directory `.../tmp/qemu-build-opt.d'
>

To reach shell even on failure you have to use smth like:

$ VERBOSE=1 make
Or DEBUG=1. If default ./configure && make fails you get shell and can run
again with nodefault options. You can check basevm.py in tests/.


> Thanks,
> Laszlo
>

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

On 03/21/19 17:55, Laszlo Ersek wrote:
> On 03/21/19 12:52, Peter Maydell wrote:
>> On Thu, 21 Mar 2019 at 11:34, Laszlo Ersek  wrote:
>>>
>>> Repo:   https://github.com/lersek/qemu.git
>>> Branch: edk2_build_v3
>>>
>>> Version 2, that is:
>>>
>>>   [Qemu-devel] [PATCH v2 00/12] bundle edk2 platform firmware with QEMU
>>>
>>> was posted at:
>>>
>>>   https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04670.html
>>>   20190313210057.32584-1-lersek@redhat.com">http://mid.mail-archive.com/20190313210057.32584-1-lersek@redhat.com
>>>
>>> Updates in v3 are noted on each patch individually, in the Notes
>>> section. In summary,
>>>
>>> - I've picked up feedback tags from the v2 thread (see above),
>>>
>>> - I've replaced the xz compression with bz2 compression according to the
>>>   subthread
>>>
>>> Re: [Qemu-devel] [PULL 00/12] EDK2 Firmware roms
>>
>> Thanks. Could you check that this works in the OpenBSD VM test?
>> It should just be a matter of
>>   make -C build vm-build-openbsd
>
> Actually I tried to do that before posting v3, following the
> instructions in "docs/devel/testing.rst":
>
>   cd $QEMU_SRC/tests/vm
>   ./openbsd --build-image --image /mnt/data/tmp/openbsd.img
>   ./openbsd --debug --image /mnt/data/tmp/openbsd.img \
> --build-qemu $QEMU_SRC
>
> The first "openbsd" invocation succeeded, but the second failed, with a
> complaint about some (cross?)compiler missing.
>
> Anyway I'll now retry with the make command you specify above.

Unfortunately, I got the exact same setup error:

> make: Entering directory `.../tmp/qemu-build-opt.d'
> VM-IMAGE openbsd
> --2019-03-21 18:14:46--  http://download.patchew.org/openbsd-6.1-amd64.img.xz
> Resolving download.patchew.org (download.patchew.org)... 140.211.15.4
> Connecting to download.patchew.org (download.patchew.org)|140.211.15.4|:80... 
> connected.
> HTTP request sent, awaiting response... 200 OK
> Length: 675094624 (644M) [application/octet-stream]
> Saving to: 
> '.../.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd.download'
>
> 100%[=>]
>  675,094,624 12.0MB/s   in 53s
>
> 2019-03-21 18:15:40 (12.1 MB/s) - 
> '.../.cache/qemu-vm/download/bc4733f6c6e76931702528a515a1bf70eb8baecd.download'
>  saved [675094624/675094624]
>
> Extracting the image...
> .../.cache/qemu-vm/images/openbsd.img.tmp.xz (1/1)
>   100 %643.8 MiB / 32.0 GiB = 0.020   253 MiB/s   2:09
> VM-BUILD openbsd
> Cloning into 
> '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot'...
> done.
> Checking out files: 100% (6748/6748), done.
> Submodule 'dtc' (https://git.qemu.org/git/dtc.git) registered for path 'dtc'
> Cloning into 
> '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/dtc'...
> Submodule 'ui/keycodemapdb' (https://git.qemu.org/git/keycodemapdb.git) 
> registered for path 'ui/keycodemapdb'
> Cloning into 
> '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/ui/keycodemapdb'...
> Submodule 'tests/fp/berkeley-softfloat-3' 
> (https://github.com/cota/berkeley-softfloat-3) registered for path 
> 'tests/fp/berkeley-softfloat-3'
> Cloning into 
> '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/tests/fp/berkeley-softfloat-3'...
> Submodule 'tests/fp/berkeley-testfloat-3' 
> (https://github.com/cota/berkeley-testfloat-3) registered for path 
> 'tests/fp/berkeley-testfloat-3'
> Cloning into 
> '.../tmp/qemu-build-opt.d/vm-test-N8zknt.tmp/data-35ed0.tar.vroot/tests/fp/berkeley-testfloat-3'...
>
> ERROR: "x86_64-unknown-openbsd6.1-gcc-4.9.4" either does not exist or does 
> not work
>
> make: *** [vm-build-openbsd] Error 3
> make: Leaving directory `.../tmp/qemu-build-opt.d'

Thanks,
Laszlo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Peter Maydell]

On Thu, 21 Mar 2019 at 16:55, Laszlo Ersek  wrote:
> On 03/21/19 12:52, Peter Maydell wrote:
> > Thanks. Could you check that this works in the OpenBSD VM test?
> > It should just be a matter of
> >   make -C build vm-build-openbsd
>
> Actually I tried to do that before posting v3, following the
> instructions in "docs/devel/testing.rst":
>
>   cd $QEMU_SRC/tests/vm
>   ./openbsd --build-image --image /mnt/data/tmp/openbsd.img
>   ./openbsd --debug --image /mnt/data/tmp/openbsd.img \
> --build-qemu $QEMU_SRC

This is in the "manual invocation" subsection of the docs.
You want the "quickstart" subsection which has the same
'make vm-build-whatever' command I suggest.

> Aha! So first I guess I should run at least configure, for setting up an
> out-of-tree build, and then try the "make" command.

You need to configure, yes. I think it should work on an in-tree
build, but I've never tried it as I never use in-tree builds.
(We're probably going to get rid of in-tree build support in 4.1.)

thanks
-- PMM

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

On 03/21/19 12:52, Peter Maydell wrote:
> On Thu, 21 Mar 2019 at 11:34, Laszlo Ersek  wrote:
>>
>> Repo:   https://github.com/lersek/qemu.git
>> Branch: edk2_build_v3
>>
>> Version 2, that is:
>>
>>   [Qemu-devel] [PATCH v2 00/12] bundle edk2 platform firmware with QEMU
>>
>> was posted at:
>>
>>   https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04670.html
>>   20190313210057.32584-1-lersek@redhat.com">http://mid.mail-archive.com/20190313210057.32584-1-lersek@redhat.com
>>
>> Updates in v3 are noted on each patch individually, in the Notes
>> section. In summary,
>>
>> - I've picked up feedback tags from the v2 thread (see above),
>>
>> - I've replaced the xz compression with bz2 compression according to the
>>   subthread
>>
>> Re: [Qemu-devel] [PULL 00/12] EDK2 Firmware roms
> 
> Thanks. Could you check that this works in the OpenBSD VM test?
> It should just be a matter of
>   make -C build vm-build-openbsd

Actually I tried to do that before posting v3, following the
instructions in "docs/devel/testing.rst":

  cd $QEMU_SRC/tests/vm
  ./openbsd --build-image --image /mnt/data/tmp/openbsd.img
  ./openbsd --debug --image /mnt/data/tmp/openbsd.img \
--build-qemu $QEMU_SRC

The first "openbsd" invocation succeeded, but the second failed, with a
complaint about some (cross?)compiler missing.

Anyway I'll now retry with the make command you specify above.

> (with 'build' replaced with whatever your build directory's name is):

Aha! So first I guess I should run at least configure, for setting up an
out-of-tree build, and then try the "make" command.

> it will download the VM image and run it automatically. You'll
> want to be doing this as a user that can use KVM so that we
> do the build in a KVM VM rather than a slow emulated one :-))

Thanks, I'll report back.

Laszlo

Re: [Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Peter Maydell]

On Thu, 21 Mar 2019 at 11:34, Laszlo Ersek  wrote:
>
> Repo:   https://github.com/lersek/qemu.git
> Branch: edk2_build_v3
>
> Version 2, that is:
>
>   [Qemu-devel] [PATCH v2 00/12] bundle edk2 platform firmware with QEMU
>
> was posted at:
>
>   https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04670.html
>   20190313210057.32584-1-lersek@redhat.com">http://mid.mail-archive.com/20190313210057.32584-1-lersek@redhat.com
>
> Updates in v3 are noted on each patch individually, in the Notes
> section. In summary,
>
> - I've picked up feedback tags from the v2 thread (see above),
>
> - I've replaced the xz compression with bz2 compression according to the
>   subthread
>
> Re: [Qemu-devel] [PULL 00/12] EDK2 Firmware roms

Thanks. Could you check that this works in the OpenBSD VM test?
It should just be a matter of
  make -C build vm-build-openbsd

(with 'build' replaced with whatever your build directory's name is):
it will download the VM image and run it automatically. You'll
want to be doing this as a user that can use KVM so that we
do the build in a KVM VM rather than a slow emulated one :-))

thanks
-- PMM

[Qemu-devel] [PATCH for-4.1 v3 00/12] bundle edk2 platform firmware with QEMU

[Laszlo Ersek]

Repo:   https://github.com/lersek/qemu.git
Branch: edk2_build_v3

Version 2, that is:

  [Qemu-devel] [PATCH v2 00/12] bundle edk2 platform firmware with QEMU

was posted at:

  https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg04670.html
  20190313210057.32584-1-lersek@redhat.com">http://mid.mail-archive.com/20190313210057.32584-1-lersek@redhat.com

Updates in v3 are noted on each patch individually, in the Notes
section. In summary,

- I've picked up feedback tags from the v2 thread (see above),

- I've replaced the xz compression with bz2 compression according to the
  subthread

Re: [Qemu-devel] [PULL 00/12] EDK2 Firmware roms

  at:

https://lists.gnu.org/archive/html/qemu-devel/2019-03/msg06016.html

http://mid.mail-archive.com/CAFEAcA-UcX6Brv3O28vcCw31bt=QpvbUwnPBBjhETm=vfci...@mail.gmail.com

I'm pasting a cumulative diffstat, and diff, between v2 and v3 below.
Note that a normal diffstat for this v3 series is at the bottom of the
cover letter, as usual.

>  Makefile   |   6 +++---
>  configure  |   2 +-
>  pc-bios/README |   2 +-
>  pc-bios/edk2-aarch64-code.fd.bz2   | Bin 0 -> 1177603 bytes
>  pc-bios/edk2-aarch64-code.fd.xz| Bin 1146804 -> 0 bytes
>  pc-bios/edk2-arm-code.fd.bz2   | Bin 0 -> 1173662 bytes
>  pc-bios/edk2-arm-code.fd.xz| Bin 1147852 -> 0 bytes
>  pc-bios/edk2-arm-vars.fd.bz2   | Bin 0 -> 263 bytes
>  pc-bios/edk2-arm-vars.fd.xz| Bin 10008 -> 0 bytes
>  pc-bios/edk2-i386-code.fd.bz2  | Bin 0 -> 1688659 bytes
>  pc-bios/edk2-i386-code.fd.xz   | Bin 1674764 -> 0 bytes
>  pc-bios/edk2-i386-secure-code.fd.bz2   | Bin 0 -> 1881979 bytes
>  pc-bios/edk2-i386-secure-code.fd.xz| Bin 1870024 -> 0 bytes
>  pc-bios/edk2-i386-vars.fd.bz2  | Bin 0 -> 190 bytes
>  pc-bios/edk2-i386-vars.fd.xz   | Bin 320 -> 0 bytes
>  pc-bios/edk2-x86_64-code.fd.bz2| Bin 0 -> 1669280 bytes
>  pc-bios/edk2-x86_64-code.fd.xz | Bin 1655276 -> 0 bytes
>  pc-bios/edk2-x86_64-secure-code.fd.bz2 | Bin 0 -> 1901210 bytes
>  pc-bios/edk2-x86_64-secure-code.fd.xz  | Bin 1889024 -> 0 bytes
>  roms/Makefile.edk2 |   6 +++---
>  20 files changed, 8 insertions(+), 8 deletions(-)

> diff --git a/configure b/configure
> index 5061f68653a3..df6e01ada833 100755
> --- a/configure
> +++ b/configure
> @@ -7865,7 +7865,7 @@ for bios_file in \
>  $source_path/pc-bios/*.img \
>  $source_path/pc-bios/openbios-* \
>  $source_path/pc-bios/u-boot.* \
> -$source_path/pc-bios/edk2-*.fd.xz \
> +$source_path/pc-bios/edk2-*.fd.bz2 \
>  $source_path/pc-bios/palcode-*
>  do
>  LINKS="$LINKS pc-bios/$(basename $bios_file)"
> diff --git a/Makefile b/Makefile
> index 6c5c1a1cad6e..f4562092e8dc 100644
> --- a/Makefile
> +++ b/Makefile
> @@ -296,9 +296,9 @@ ui/input-keymap-%.c: $(KEYCODEMAP_GEN) $(KEYCODEMAP_CSV) 
> $(SRC_PATH)/ui/Makefile
>  $(KEYCODEMAP_GEN): .git-submodule-status
>  $(KEYCODEMAP_CSV): .git-submodule-status
>
> -edk2-decompressed = $(basename $(wildcard pc-bios/edk2-*.fd.xz))
> -pc-bios/edk2-%.fd: pc-bios/edk2-%.fd.xz
> - $(call quiet-command,xz -d -c $< > $@,"UNXZ",$<)
> +edk2-decompressed = $(basename $(wildcard pc-bios/edk2-*.fd.bz2))
> +pc-bios/edk2-%.fd: pc-bios/edk2-%.fd.bz2
> + $(call quiet-command,bzip2 -d -c $< > $@,"BUNZIP2",$<)
>
>  # Don't try to regenerate Makefile or configure
>  # We don't generate any of them
> diff --git a/roms/Makefile.edk2 b/roms/Makefile.edk2
> index e5c3036949c8..822c547fec64 100644
> --- a/roms/Makefile.edk2
> +++ b/roms/Makefile.edk2
> @@ -34,11 +34,11 @@ flashdevs := \
>   arm-vars \
>   i386-vars
>
> -all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.xz) \
> +all: $(foreach flashdev,$(flashdevs),../pc-bios/edk2-$(flashdev).fd.bz2) \
>   ../pc-bios/edk2-licenses.txt
>
> -../pc-bios/edk2-%.fd.xz: ../pc-bios/edk2-%.fd
> - xz -9 -e -c $< > $@
> +../pc-bios/edk2-%.fd.bz2: ../pc-bios/edk2-%.fd
> + bzip2 -9 -c $< > $@
>
>  # When the build completes, we need not keep the uncompressed flash device
>  # files.
> diff --git a/pc-bios/README b/pc-bios/README
> index f6acd3209610..807d95dd5418 100644
> --- a/pc-bios/README
> +++ b/pc-bios/README
> @@ -51,7 +51,7 @@
>  - QemuMacDrivers (https://github.com/ozbenh/QemuMacDrivers) is a project to
>provide virtualised drivers for PPC MacOS guests.
>
> -- The "edk2-*.fd.xz" images are platform firmware binaries and matching UEFI
> +- The "edk2-*.fd.bz2" images are platform firmware binaries and matching UEFI
>variable store templates built from the TianoCore community's EFI 
> Development
>Kit II project
>. The images
> diff --git a/pc-bios/edk2-aarch64-code.fd.bz2 
> b/pc-bios/edk2-aarch64-code.fd.bz2
> new file mode 100644
> index ..b213334d9553
> Binary files /dev/null and