subject:"\[Bug 1882671\] Re\: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO

** Tags removed: block-proposed block-proposed-focal verification-needed 
verification-needed-focal
** Tags added: verification-failed verification-failed-focal

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-16 Thread Launchpad Bug Tracker

** Merge proposal linked:
   
https://code.launchpad.net/~paelzer/ubuntu/+source/ipxe/+git/ipxe/+merge/387531

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-16 Thread Launchpad Bug Tracker

** Merge proposal linked:
   
https://code.launchpad.net/~paelzer/ubuntu/+source/ipxe/+git/ipxe/+merge/387521

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

fix as URL =>
https://github.com/ipxe/ipxe/commit/2ae5d4338661b65c63eb5cb1a96e5b803fe7d620

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

I saw your update on refresh - yeah despite feeling sort of safe on the
change as-is this fix seems even better for an SRU.

Let me get that into groovy (there the packaging change made sense, no need to 
turn that back).
And from there SRU it to Focal.


Thank you Michael and Lazlo for the work and discussion on this.

** Changed in: ipxe (Ubuntu)
   Status: Fix Released => In Progress

** No longer affects: ipxe (Ubuntu Eoan)

** Changed in: ipxe (Ubuntu Focal)
   Status: Fix Committed => In Progress

** Changed in: ipxe (Ubuntu Focal)
   Importance: Low => Medium

** Changed in: ipxe (Ubuntu)
   Importance: High => Medium

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

/usr/share/OVMF/OVMF_CODE.fd
  /var/lib/uvtool/libvirt/images/bionic.VARS.fd

makes it run eif ovmf EFI like
-drive 
file=/usr/share/OVMF/OVMF_CODE.fd,if=pflash,format=raw,unit=0,readonly=on 
-drive 
file=/var/lib/uvtool/libvirt/images/bionic.VARS.fd,if=pflash,format=raw,unit=1

Still mgriates well from Bionic -> Focal and Back
B
$ virsh migrate --unsafe --live bionic qemu+ssh://10.225.185.44/system
F
$ virsh migrate --unsafe --live bionic qemu+ssh://10.225.185.35/system
B
$ virsh console bionic
Connected to domain bionic
Escape character is ^]
Ubuntu 18.04.4 LTS bionic ttyS0
bionic login: 

So that also works well.
The only possible option missing to be tested is the real https boot via Focals 
OVMF (which didn't have https enabled), but unless someone comes back 
explaining that there is an odd combination that could have got that going we 
should be good.

Let is sit a bit more time in proposed to be sure and wait for feedback.
Then we can declare it fully verified.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-16 Thread Laszlo Ersek (Red Hat)

Hi Michael, thank you for the fix, and for mentioning it here. I didn't
ignore your comment 32 when I was writing what would become comments 33
and 34 -- I think we must have been writing our comments in parallel,
and I simply didn't see yours.

Christian, now you should be able to resolve this LP ticket without
modifying anything on the packaging side, by backporting upstream iPXE
commit 2ae5d4338.

Thanks!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-15 Thread Christian Ehrhardt 

Regression tests completed, no issues migrating in/out of updates nor
between releases due to changing sizes (That mostly covers the non EFI
roms thou).

I want to also do some more manual tests with EFI guests in that regard.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-15 Thread Laszlo Ersek (Red Hat)

Sorry about the malformed table in comment 33 -- that's not my doing. I
laid it out correctly; Launchpad messed it up by squeezing whitespace.
Here it is again, using dots rather than spaces.

Ubuntu.release..edk2.HTTPS.enabled..iPXE.HTTPS.enabled..iPXE.TPL.regression
--..--..--..---
Bionic..no..no.
Focal...no..yes.yes
Groovy..yes.(bug.1883114)...no.(this.bug)...masked.(this.bug)..

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-15 Thread Laszlo Ersek (Red Hat)

Hello Christian,

For *some* form of UEFI HTTPS boot, you have to enable *at least one* of
the {edk2, iPXE} HTTPS stacks. I'm unfamiliar with the Ubuntu releases,
but my understanding is the following:

Ubuntu release  edk2 HTTPS enabled  iPXE HTTPS enabled  iPXE TPL regression
--  --  --  ---
Bionic  no  no
Focal   no  yes yes
Groovy  yes (bug 1883114)   no (this bug)   masked (this bug)

In Groovy, you can work around the iPXE TPL regression by disabling the
iPXE HTTPS stack (i.e., in the efi-e1000e option ROM). Because, you can
effectively "replace" it with the edk2 HTTPS stack in the platform
firmware (in the OVMF binary), per bug 1883114.

In Focal, if you do the same to iPXE, you can't fall back to the edk2
HTTPS stack in OVMF -- because bug 1883114 is out of scope for Focal,
AIUI.

However, disabling the iPXE HTTPS stack in Focal would not cause a
regression, in my opinion. That's because in Focal you can't boot the
"OVMF + efi-e1000e" combination *at all* -- you don't get far enough in
the boot process to even *attempt* HTTPS boot (or a boot from another
kind of media, for that matter).

Thus in Focal, no form of *UEFI boot* (HTTPS or otherwise) has ever
worked, so there's nothing to regress by disabling the iPXE HTTPS stack
in "efi-e1000e.rom".

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-15 Thread Michael Brown

The TPL manipulation issue in iPXE is fixed as of commit
http://github.com/ipxe/ipxe/commit/2ae5d4338

Building an iPXE ROM with HTTPS enabled will now initialise with no
problems in qemu.

Michael

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-14 Thread Christian Ehrhardt 

Test #3:

 * Extra Test: HTTPS boot a uEFI guest with the efi roms from ipxe-qemu
   with old/new ipxe-qmeu code. This shall ensure that OVMF can really take
   over as-is or if we need bug 1883114 before we can do so.
   Details TBD when I'm doing these tests

I created a q35 guest in libvirt without a disk and set it to run in uEFI mode 
via OVMF.
Starting that without further setup runs into an EFI loader that can't find 
anything to boot.

Start PXE over IPv4
...
Not Found
Start HTTP Boot over IPv4
...
Not Found
-> into interactive boot-failed menu

As I mentioned before in comment #26 Focals EDK2 didn't have HTTPS
enabled yet, only in Groovy.


Therefure using the OVMF of groovy and the ipxe-qemu package from 
Focal-proposed I set up a test.

$ cp ovmf-groovy/usr/share/OVMF/OVMF_VARS.fd test-vars.fd
$ qemu-system-x86_64 -enable-kvm -drive 
if=pflash,format=raw,readonly,file=/home/ubuntu/ovmf-groovy/usr/share/OVMF/OVMF_CODE.fd
 -drive if=pflash,format=raw,file=test-vars.fd  -monitor stdio

We can see that in this OVMF build the OVMF device manager has the
option to enroll TLScerts. But TBH I haven't ever used this setup to
then HTTPS boot through EFI/OVMF.


I found [1] but before going through all the lengths to set this up I wonder 
for further regression testing I wonder if there at all was a way to get HTTPS 
boot working in EFI mode with:
a) https enabled /usr/lib/ipxe/qemu/efi-e1000e.rom
b) not https enabled /usr/share/OVMF/OVMF_CODE.fd


I'm a bit lost in all the rom/boot/https/loader options.
I beg your pardon but @Lazlo do you know if above mentioned way existed and 
might - now that we take https away from (a) - be regressing?
If so which way would this need to be set up to be tested?
Is [1][2] a proper way to exercise this in Focal "using the https in e1000e" or 
would that only work with the HTTPS enabled OVMF of groovy?

[1]: https://en.opensuse.org/UEFI_HTTPBoot_Server_Setup
[2]: 
https://edk2-docs.gitbook.io/getting-started-with-uefi-https-boot-on-edk-ii/introduction

P.S. cross release migration tests still running

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-14 Thread Christian Ehrhardt 

Test #2
 * We pad the rom sizes to be sure, but never the less double check
   migrations between Bionic <-> Focal (known to break on size changes)

Manual size check (can be seen in build log):
OK: efi-e1000e.rom is exactly 524288 bytes as expected
...

Seems ok, a regression test doing cross release migrations with proposed
enabled is running

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-14 Thread Christian Ehrhardt 

Test #1:

 * Test the attached OVMF that triggers the bug:
qemu-system-x86_64 -enable-kvm -monitor stdio -drive 
if=pflash,snapshot=on,format=raw,file=OVMF-14c7ed8b51f6-DEBUG-enabled.fd 
-global e1000.romfile=/usr/lib/ipxe/qemu/efi-e1000.rom -debugcon file:debug.log 
-global isa-debugcon.iobase=0x402

Focal as-is with ipxe-qemu  1.0.0+git-20190109.133f4c4-0ubuntu3
fails:

ASSERT /root/edk2/MdeModulePkg/Core/Dxe/Image/Image.c(1676): Image->Tpl
== gEfiCurrentTpl

Upgrading to proposed:
root@f-ipxe:~# apt install ipxe-qemu=1.0.0+git-20190109.133f4c4-0ubuntu3.1
Reading package lists... Done
Building dependency tree   
Reading state information... Done
The following packages will be upgraded:
  ipxe-qemu
1 upgraded, 0 newly installed, 0 to remove and 28 not upgraded.
Need to get 903 kB of archives.
After this operation, 1749 kB of additional disk space will be used.
Get:1 http://archive.ubuntu.com/ubuntu focal-proposed/main amd64 ipxe-qemu all 
1.0.0+git-20190109.133f4c4-0ubuntu3.1 [903 kB]
Fetched 903 kB in 0s (1963 kB/s)
(Reading database ... 47652 files and directories currently installed.)
Preparing to unpack .../ipxe-qemu_1.0.0+git-20190109.133f4c4-0ubuntu3.1_all.deb 
...
Unpacking ipxe-qemu (1.0.0+git-20190109.133f4c4-0ubuntu3.1) over 
(1.0.0+git-20190109.133f4c4-0ubuntu3) ...
Setting up ipxe-qemu (1.0.0+git-20190109.133f4c4-0ubuntu3.1) ...

With the version from proposed the crash/assert no more happens.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-07 Thread Brian Murray

Hello Vladislav, or anyone else affected,

Accepted ipxe into focal-proposed. The package will build now and be
available at
https://launchpad.net/ubuntu/+source/ipxe/1.0.0+git-20190109.133f4c4-0ubuntu3.1
in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.  Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, what testing has been
performed on the package and change the tag from verification-needed-
focal to verification-done-focal. If it does not fix the bug for you,
please add a comment stating that, and change the tag to verification-
failed-focal. In either case, without details of your testing we will
not be able to proceed.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance for helping!

N.B. The updated package will be released to -updates after the bug(s)
fixed by this package have been verified and the package has been in
-proposed for a minimum of 7 days.

** Changed in: ipxe (Ubuntu Focal)
   Status: Triaged => Fix Committed

** Tags added: verification-needed verification-needed-focal

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-02 Thread Christian Ehrhardt 

We will need quite some time to ensure this isn't breaking things.
The merge proposal was reviewed and I'll upload to Focal-unapproved now.
The intention is to not have me testing in advance and then having a short time
in -proposed. Instead I think for this case it will be helpful to have it in
proposed early, but longer to increase the amount of 3rd-party/people testing
it.

Therefore I have added block-proposed tags right away.

** Changed in: ipxe (Ubuntu Eoan)
Status: Triaged => Won't Fix

** Tags added: block-proposed block-proposed-focal

** Description changed:

[Impact]

* Booting some OVMF through the efi roms in our ipxe-qemu package triggers
a bad ordering of TPL manipulations and eventually gets the boot stuck.

* It was analyzed by Lazlo that this was due to HTTPS being enabled in the
rom itself triggering this fail when gathering entropy. In addition
isn't required to have HTTPS enabled in this part of the roms as the EFI
payload would be able to handle that on it's own (inside OVMF file from
src:edk2 instead of the combined rom like efi-e1000.rom).

* Disabling HTTPS in the efi-*.rom files should have no functional
impact but mitigate the TPL issues we are seeing.

[Test Case]

* I have provided a test OVMF load attached to this bug

https://bugs.launchpad.net/ipxe/+bug/1882671/+attachment/5387066/+files/OVMF-14c7ed8b51f6-DEBUG-enabled.fd

* Start that in qemu like:
$ qemu-system-x86_64 -enable-kvm -monitor stdio -drive
if=pflash,snapshot=on,format=raw,file=/root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd
-global e1000.romfile=/usr/lib/ipxe/qemu/efi-e1000.rom -debugcon
file:debug.log -global isa-debugcon.iobase=0x402

# /usr/lib/ipxe/qemu/efi-e1000.rom is from package ipxe-qemu
# /root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd is the debug build I
attached

Symptoms when failing:
- UI never leaves the "initializing" state
- in the debug.log the bad case asserts with
ASSERT /root/edk2/MdeModulePkg/Core/Dxe/Image/Image.c(1676):
Image->Tpl == gEfiCurrentTpl

* Extra Test: HTTPS boot a uEFI guest with the efi roms from ipxe-qemu
- with old/new ipxe-qmeu code. This shall ensure that OVMF can really take
-over as-is or if we need bug 1883114 before we can do so.
-Details TBD
+ with old/new ipxe-qmeu code. This shall ensure that OVMF can really take
+ over as-is or if we need bug 1883114 before we can do so.
+ Details TBD when I'm doing these tests

+ * We pad the rom sizes to be sure, but never the less double check
+migrations between Bionic <-> Focal (known to break on size changes)

[Regression Potential]

* Messing around with rom options always is a complex topic, but I'd like
to quote Lazlo (who understands all of this better anyway):

"you don't need the iPXE HTTPS implementation, because OVMF already
contains the edk2 HTTPS BOOT feature (assuming you build OVMF with "-D
NETWORK_HTTP_BOOT_ENABLE -D NETWORK_TLS_ENABLE"). The whole point of
CONFIG=qemu is to strip the UEFI build of iPXE to a mere Simple Network
Protocol driver. Such a build provides only the lowest level hardware
abstraction for UEFI, and the rest (IPv[46], TCP/UDP, DHCP, PXE, TFTP,
HTTP(S)) are provided on top by the edk2 implementations of various
standard UEFI interfaces."

* Never the less, if you seek regressions then in the UEFI handling of
HTTPS boot.

[Other Info]

* This happens with some OVMF versions like the one I have provided for
debugging but not with the OVMF delivered as part of Focal. Therefore we
should fix it but priority isn't that high.

* Quote from the discussion: "If you want to run a full-featured iPXE
build on a UEFI machine (including: in an OVMF guest), you still can,
of course; lots of people do that, for good reasons. But that use case
is best served by the *standalone UEFI application* build of iPXE
(produced at "bin-x86_64-efi/ipxe.efi" by "make"). The UEFI *driver*
build of iPXE should be as minimal as possible, in comparison -- just
provide SNP for the desired NIC models."
=> Only the EFI roms are changed, neither the PCI roms nor the standalon
IPXE builds will be modified.
+
+ * It makes sense to keep this longer in -proposed to increase the
+chance of more people testing this. Therefore I'd ask to accept this
+early. I can do my (extended) tests against -proposed and we'd get
+some community coverage.

---

The version of QEMU (4.2.0) packaged for Ubuntu 20.04 hangs indefinitely
at boot if an OVMF bios is used. This happens ONLY with qemu-system-
x86_64. qemu-system-i386 works fine with the latest ia32 OVMF bios.

NOTE[1]: the same identical OVMF bios works fine on QEMU 2.x packaged with
Ubuntu 18.04.
NOTE[2]: reproducing the fatal bug requires *no* operating system:

qemu-system-x86_64

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-01 Thread Christian Ehrhardt 

I have prepared a merge proposals and PPA test builds for Focal/Eoan
E-MP => 
https://code.launchpad.net/~paelzer/ubuntu/+source/ipxe/+git/ipxe/+merge/386647
E-PPA => 
https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4126/+packages
F-MP => 
https://code.launchpad.net/~paelzer/ubuntu/+source/ipxe/+git/ipxe/+merge/386648
F-PPA => 
https://launchpad.net/~ci-train-ppa-service/+archive/ubuntu/4127/+packages

For Eoan/Focal we need to be sure that the OVMF builds from edk2 can
really take over the HTTPS functionality. Because edk2 itself for
Debian/Ubuntu only got enabled later in >=Groovy:

  edk2 (2020.05-2) unstable; urgency=medium
  * Enable https boot support, thanks to Dimitri John Ledkov. LP: #1883114.

This comes down to:
-COMMON_FLAGS = -DNETWORK_HTTP_BOOT_ENABLE=TRUE -DSECURE_BOOT_ENABLE=TRUE
+COMMON_FLAGS = -DNETWORK_HTTP_BOOT_ENABLE=TRUE -DNETWORK_TLS_ENABLE 
-DSECURE_BOOT_ENABLE=TRUE

Therefore once we drop HTTPS from the ipxe-qemu combined efi roms
expecting that OVMF will take over this we need to ensure this can work
without above enabling being available in Eoan/Focal as well.

/me looks for a good way to verify that as I'm unsure if the test
mentioned in bug 1883114 will really proved what we need in regard to
dropping https here. Maybe an actual OVMF boot via HTTPS should be set
up. If there are suggestions for a good way to test that this OVMF-
HTTPS-takeover works as expected I'm open to them.

If it turns out that we need to enable it in edk2/ovmf before we can go
on in ipxe/ipxe-qemu we we can upload ipxe-qemu with a versioned BREAKS
to the older ovmf package (to avoid https is dropped in 'ipxe-qemu', but
not yet enabled in the 'ovmf'). But if needed backporting bug 1883114
becomes a pre-req to SRU this bug here.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-01 Thread Launchpad Bug Tracker

** Merge proposal linked:
   
https://code.launchpad.net/~paelzer/ubuntu/+source/ipxe/+git/ipxe/+merge/386647

** Merge proposal linked:
   
https://code.launchpad.net/~paelzer/ubuntu/+source/ipxe/+git/ipxe/+merge/386648

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-07-01 Thread Christian Ehrhardt 

** Description changed:

  [Impact]
  
-  * Booting some OVMF through the efi roms in our ipxe-qemu package triggers
-a bad ordering of TPL manipulations and eventually gets the boot stuck.
+  * Booting some OVMF through the efi roms in our ipxe-qemu package triggers
+    a bad ordering of TPL manipulations and eventually gets the boot stuck.
  
-  * It was analyzed by Lazlo that this was due to HTTPS being enabled in the 
-rom itself triggering this fail when gathering entropy. In addition 
-isn't required to have HTTPS enabled in this part of the roms as the EFI 
-payload would be able to handle that on it's own (inside OVMF file from 
-src:edk2 instead of the combined rom like efi-e1000.rom).
+  * It was analyzed by Lazlo that this was due to HTTPS being enabled in the
+    rom itself triggering this fail when gathering entropy. In addition
+    isn't required to have HTTPS enabled in this part of the roms as the EFI
+    payload would be able to handle that on it's own (inside OVMF file from
+    src:edk2 instead of the combined rom like efi-e1000.rom).
  
-  * Disabling HTTPS in the efi-*.rom files should have no functional 
-impact but mitigate the TPL issues we are seeing.
+  * Disabling HTTPS in the efi-*.rom files should have no functional
+    impact but mitigate the TPL issues we are seeing.
  
  [Test Case]
  
-  * I have provided a test OVMF load attached to this bug 
+  * I have provided a test OVMF load attached to this bug
  
https://bugs.launchpad.net/ipxe/+bug/1882671/+attachment/5387066/+files/OVMF-14c7ed8b51f6-DEBUG-enabled.fd
  
-  * Start that in qemu like:
-$ qemu-system-x86_64 -enable-kvm -monitor stdio -drive 
if=pflash,snapshot=on,format=raw,file=/root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd
 -global e1000.romfile=/usr/lib/ipxe/qemu/efi-e1000.rom -debugcon 
file:debug.log -global isa-debugcon.iobase=0x402
+  * Start that in qemu like:
+    $ qemu-system-x86_64 -enable-kvm -monitor stdio -drive 
if=pflash,snapshot=on,format=raw,file=/root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd
 -global e1000.romfile=/usr/lib/ipxe/qemu/efi-e1000.rom -debugcon 
file:debug.log -global isa-debugcon.iobase=0x402
  
-   # /usr/lib/ipxe/qemu/efi-e1000.rom is from package ipxe-qemu
-   # /root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd is the debug build I 
- attached
+   # /usr/lib/ipxe/qemu/efi-e1000.rom is from package ipxe-qemu
+   # /root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd is the debug build I
+ attached
  
-   Symptoms when failing:
-   - UI never leaves the "initializing" state
-   - in the debug.log the bad case asserts with
- ASSERT /root/edk2/MdeModulePkg/Core/Dxe/Image/Image.c(1676):
-   Image->Tpl == gEfiCurrentTpl
+   Symptoms when failing:
+   - UI never leaves the "initializing" state
+   - in the debug.log the bad case asserts with
+ ASSERT /root/edk2/MdeModulePkg/Core/Dxe/Image/Image.c(1676):
+   Image->Tpl == gEfiCurrentTpl
  
-  * Extra Test: HTTPS boot a uEFI guest with the efi roms from ipxe-qemu 
-with old/new code
+  * Extra Test: HTTPS boot a uEFI guest with the efi roms from ipxe-qemu
+    with old/new ipxe-qmeu code. This shall ensure that OVMF can really take 
+over as-is or if we need bug 1883114 before we can do so.
+Details TBD
+ 
  
  [Regression Potential]
  
-  * Messing around with rom options always is a complex topic, but I'd like 
-to quote Lazlo (who understands all of this better anyway): 
+  * Messing around with rom options always is a complex topic, but I'd like
+    to quote Lazlo (who understands all of this better anyway):
  
  "you don't need the iPXE HTTPS implementation, because OVMF already
  contains the edk2 HTTPS BOOT feature (assuming you build OVMF with "-D
  NETWORK_HTTP_BOOT_ENABLE -D NETWORK_TLS_ENABLE"). The whole point of
  CONFIG=qemu is to strip the UEFI build of iPXE to a mere Simple Network
  Protocol driver. Such a build provides only the lowest level hardware
  abstraction for UEFI, and the rest (IPv[46], TCP/UDP, DHCP, PXE, TFTP,
  HTTP(S)) are provided on top by the edk2 implementations of various
  standard UEFI interfaces."
  
-  * Never the less, if you seek regressions then in the UEFI handling of 
-HTTPS boot.
- 
+  * Never the less, if you seek regressions then in the UEFI handling of
+    HTTPS boot.
  
  [Other Info]
-  
-  * This happens with some OVMF versions like the one I have provided for 
-debugging but not with the OVMF delivered as part of Focal. Therefore we 
-should fix it but priority isn't that high.
  
-  * Quote from the discussion: "If you want to run a full-featured iPXE 
-build on a UEFI machine (including: in an OVMF guest), you still can, 
-of course; lots of people do that, for good reasons. But that use case 
-is best served by the *standalone UEFI application* build of iPXE 
-(produced at "bin-x86_64-efi/ipxe.efi" by "make"). The UEFI *driver* 
-build of iPXE should be as minimal as possible, in

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-06-30 Thread Launchpad Bug Tracker

This bug was fixed in the package ipxe -
1.0.0+git-20190125.36a4c85-5ubuntu1

---
ipxe (1.0.0+git-20190125.36a4c85-5ubuntu1) groovy; urgency=medium

  * Merge with Debian unstable (LP: #1884758). Remaining changes:
- Split grub integration from ipxe->grub-ipxe.
  - d/control: add package and adapt dependencies
  - d/[grub-]ipxe.install: move some files to grub-ipxe
  - rename d/ipxe.post* to d/grub-ipxe-post*
  [updated to match 1.0.0+git-20190125.36a4c85-5]
- d/util/check-rom-sizes, d/rules: check sizes of generated roms to avoid
  accidentally breaking KVM live migration on updates/fixes.
  [updated for new and dropped rom file names]
- debian/copyright: update copyright information to satisfy lintian
  dep5 checks (LP: 1747071)
- Build ROMs for QEMU with CONFIG=qemu (LP: 1789319)
  [updated to match 1.0.0+git-20190125.36a4c85-5 and be more verbose]
- debian/patches/handle-dhcp-nack.patch: Handle DHCP NAK and send a
  re-discover. (LP 1707999)
- d/p/0005-strip-802.1Q-VLAN-0-priority-tags.patch: Strip 802.1Q VLAN 0
  priority tags; Fixes PXE when VLAN tag is 0. (LP: 1805920)
- d/tree/ipxe/etc/grub.d/20_ipxe: Make grub-ipxe work under UEFI
  (LP: 1811496)
  - Use ipxe.efi under UEFI
  - Save default entry when iPXE is selected
- d/tree/ipxe/etc/grub.d/20_ipxe: Identify ipxe grub menu entry in
  an easier way (LP: 1858374)
  * Dropped changes
- new upstream release 20190109.133f4c4 [superseded by new upstream]
- new upstream release 20180124.fbe8c52d [superseded by new upstream]
- Revert to the former git snapshot 20150424.a25a16d
  [superseded by new upstream]
  - This brings back debian/patches/0002-Don-t-use-libiberty.patch as needed
on the older source.
  - Adapt d/p/0001-rom-change-banner-timeout.diff.patch to former state to
match old source.
  - drop d/p/util-elf2efi-GNU_SOURCE.patch as it was not needed on old
source
- Fix FTBFS with newer perl versions [upstream]
- d/p/0004-fix_no-pie_option.patch: correct -no-pie option
  to build without pie
  [ still carried before ]
- Install ne.rom as pxe-ne2k_isa.rom
  - d/ipxe-qemu.install: Install ne.rom as pxe-ne2k_isa.rom.
  - d/ipxe-qemu.links: compat link for ne.rom
  [ no more needed, was an old xen hvmloader bug ]
- d/ipxe-qemu.links: Add compat links from /usr/share/qemu
  to /usr/lib/ipxe/qemu.
  [ no more needed, transition from trusty ]
- add new rom for vmxnet3 (LP 1737211) [in Debian now]
- Add e1000e firmware for qemu. (closes 884240) [in Debian now]
- d/control: ipxe-qemu breaks qemu-system-x86 <2.11 [no more needed]
- d/p/enable-https.patch: Enable HTTPS support.
  [resolved per rom type in d/rules now]
  * Added changes
- d/rules: only enable https on non EFI roms. This lets EFI handle https
  itself and avoids breakage in TPL manipulations (LP: #1882671)
- d/util/check-rom-sizes: fix if size does exactly match

ipxe (1.0.0+git-20190125.36a4c85-5) unstable; urgency=medium

  * Cleanup src/bin correctly. (closes: #952275)

ipxe (1.0.0+git-20190125.36a4c85-4) unstable; urgency=medium

  * Use new source format instead of own rules.
  * Use debhelper 12.
  * Move ipxe.efi into /boot. (closes: #947267)

ipxe (1.0.0+git-20190125.36a4c85-3) unstable; urgency=medium

  * Combine legacy and EFI rom again. (closes: #947024)

ipxe (1.0.0+git-20190125.36a4c85-2) unstable; urgency=medium

  * Add Vcs information.
  * Include snponly.efi. (closes: #944321)

ipxe (1.0.0+git-20190125.36a4c85-1) unstable; urgency=medium

  * New snapshot. (closes: #832765, #906365)
  * Add e1000e and vmxnet3 firmware for qemu. (closes: #884240, #868124)

 -- Christian Ehrhardt   Tue, 23 Jun
2020 16:23:52 +0200

** Changed in: ipxe (Ubuntu)
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-06-29 Thread Christian Ehrhardt 

Thanks Lazlo, I'll keep the legacy roms as CONFIG=qemu then.
I didn't plan to change this on an SRU anyway, but going forward I wanted to 
adapt this to be correct. Hearing that in RH you also used CONFIG=qemu covering 
*both* is kind of re-assuring to keep it like that for now.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-06-26 Thread Laszlo Ersek (Red Hat)

Christian,

> But it seems for legacy roms like 82540em.rom CONFIG=qemu being set or not 
> doesn't make a
> difference.

(1) That's my understanding as well; from the following original iPXE
commits anyway:

- a15c0d7e868a ("[efi] Allow user experience to be downgraded", 2015-07-22),
- a200ad462e69 ("[build] Add named configuration for qemu", 2015-07-22).

(I played a part in those commits existing.)

(2) However, just to be 100% sure, I recommend *not* changing the
CONFIG= setting for the traditional BIOS build. If you have
had CONFIG=qemu there, then keep it. If you haven't, then don't add it
now.

What matters is that the EFI build be performed with CONFIG=qemu.

(FWIW, last time I looked, in RHEL downstream we used to build iPXE with
CONFIG=qemu covering *both* the traditional BIOS image and the EFI
image. But, I don't think that's enough reason for you to diverge now
from your previous BIOS image settings in Ubuntu. I don't think you
should risk regressions with that. Really, what matters is that the EFI
image be built with CONFIG=qemu.)

Thanks
Laszlo

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

I beg your pardon Lazlo, but one more question on CONFIG=qemu.
Since it was introduced config=QEMU was exported for efi and legacy roms.
But it seems for legacy roms like 82540em.rom CONFIG=qemu being set or not 
doesn't make a difference.

I I just look at src/config/qemu/* vs src/config/ there is a massive
difference and I see it showing up in the build log, so the export has
"some" effect.

Grepping for qemu.*82540em I see

Old:
/tmp/buildlog_ubuntu-focal-amd64.ipxe_1.0.0+git-20190109.133f4c4-0ubuntu3_BUILDING.txt:35050:gcc
  -DARCH=i386 -DPLATFORM=pcbios -DSECUREBOOT=0 -march=i386 -fomit-frame-pointer 
-fstrength-reduce -falign-jumps=1 -falign-loops=1 -falign-functions=1 
-mpreferred-stack-boundary=2 -mregparm=3 -mrtd -freg-struct-return -m32 
-fshort-wchar -Ui386 -Ulinux -DNVALGRIND -Iinclude -I. -Iarch/x86/include 
-Iarch/i386/include -Iarch/i386/include/pcbios -Os -g -ffreestanding -Wall -W 
-Wformat-nonliteral  -fno-stack-protector -fno-dwarf2-cfi-asm -fno-exceptions  
-fno-unwind-tables -fno-asynchronous-unwind-tables -Wno-address 
-Wno-stringop-truncation -fno-PIE -no-pie   -ffunction-sections -fdata-sections 
-include include/compiler.h -DASM_TCHAR='@' -DASM_TCHAR_OPS='@' -DCONFIG=qemu 
-DLOCAL_CONFIG=qemu   -DOBJECT=version -DBUILD_NAME="\"82540em.rom\"" \
/tmp/buildlog_ubuntu-focal-amd64.ipxe_1.0.0+git-20190109.133f4c4-0ubuntu3_BUILDING.txt:47807:gcc
  -DARCH=x86_64 -DPLATFORM=efi -DSECUREBOOT=0 -fstrength-reduce 
-fomit-frame-pointer -falign-jumps=1 -falign-loops=1 -falign-functions=1 -m64 
-mno-mmx -mno-sse -fshort-wchar -Ui386 -Ulinux -DNVALGRIND -fpie -mno-red-zone 
-Iinclude -I. -Iarch/x86/include -Iarch/x86_64/include 
-Iarch/x86_64/include/efi -Os -g -ffreestanding -Wall -W -Wformat-nonliteral  
-fno-stack-protector -fno-dwarf2-cfi-asm -fno-exceptions  -fno-unwind-tables 
-fno-asynchronous-unwind-tables -Wno-address -Wno-stringop-truncation   
-ffunction-sections -fdata-sections -include include/compiler.h -DASM_TCHAR='@' 
-DASM_TCHAR_OPS='@' -DCONFIG=qemu -DLOCAL_CONFIG=qemu   -DOBJECT=version 
-DBUILD_NAME="\"82540em.efidrv\"" \

New:
ipxe_1.0.0+git-20190125.36a4c85-5ubuntu1~ppa1_amd64.build:43490:gcc 
-DARCH=x86_64 -DPLATFORM=efi -DSECUREBOOT=0 -fstrength-reduce 
-fomit-frame-pointer -falign-jumps=1 -falign-loops=1 -falign-functions=1 -m64 
-mno-mmx -mno-sse -fshort-wchar -Ui386 -Ulinux -DNVALGRIND -fpie -mno-red-zone 
-Iinclude -I. -Iarch/x86/include -Iarch/x86_64/include 
-Iarch/x86_64/include/efi -Os -g -ffreestanding -Wall -W -Wformat-nonliteral 
-fno-stack-protector -fno-dwarf2-cfi-asm -fno-exceptions  -fno-unwind-tables 
-fno-asynchronous-unwind-tables -Wno-address -Wno-stringop-truncation  
-ffunction-sections -fdata-sections -include include/compiler.h -DASM_TCHAR='@' 
-DASM_TCHAR_OPS='@' -DCONFIG=qemu -DLOCAL_CONFIG=qemu   -DOBJECT=version 
-DBUILD_NAME="\"82540em.efidrv\"" \

But at least on the output size of 82540em.rom I see no difference.
Hence my question - does CONFIG=qemu have no effect on the legacy roms?
If it has an effect what is the recommended config for them - setting it or not?
(Right now I have no more set CONFIG=qemu for legacy roms, but since it would 
be a change I want to be sure it really is a no-op or at least what is 
recommended).

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

** Description changed:

- The version of QEMU (4.2.0) packaged for Ubuntu 20.04 hangs indefinitely
- at boot if an OVMF bios is used. This happens ONLY with qemu-system-
- x86_64. qemu-system-i386 works fine with the latest ia32 OVMF bios.
+ [Impact]
+ 
+  * Booting some OVMF through the efi roms in our ipxe-qemu package triggers
+a bad ordering of TPL manipulations and eventually gets the boot stuck.
+ 
+  * It was analyzed by Lazlo that this was due to HTTPS being enabled in the 
+rom itself triggering this fail when gathering entropy. In addition 
+isn't required to have HTTPS enabled in this part of the roms as the EFI 
+payload would be able to handle that on it's own (inside OVMF file from 
+src:edk2 instead of the combined rom like efi-e1000.rom).
+ 
+  * Disabling HTTPS in the efi-*.rom files should have no functional 
+impact but mitigate the TPL issues we are seeing.
+ 
+ [Test Case]
+ 
+  * I have provided a test OVMF load attached to this bug 
+ 
https://bugs.launchpad.net/ipxe/+bug/1882671/+attachment/5387066/+files/OVMF-14c7ed8b51f6-DEBUG-enabled.fd
+ 
+  * Start that in qemu like:
+$ qemu-system-x86_64 -enable-kvm -monitor stdio -drive 
if=pflash,snapshot=on,format=raw,file=/root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd
 -global e1000.romfile=/usr/lib/ipxe/qemu/efi-e1000.rom -debugcon 
file:debug.log -global isa-debugcon.iobase=0x402
+ 
+   # /usr/lib/ipxe/qemu/efi-e1000.rom is from package ipxe-qemu
+   # /root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd is the debug build I 
+ attached
+ 
+   Symptoms when failing:
+   - UI never leaves the "initializing" state
+   - in the debug.log the bad case asserts with
+ ASSERT /root/edk2/MdeModulePkg/Core/Dxe/Image/Image.c(1676):
+   Image->Tpl == gEfiCurrentTpl
+ 
+  * Extra Test: HTTPS boot a uEFI guest with the efi roms from ipxe-qemu 
+with old/new code
+ 
+ [Regression Potential]
+ 
+  * Messing around with rom options always is a complex topic, but I'd like 
+to quote Lazlo (who understands all of this better anyway): 
+ 
+ "you don't need the iPXE HTTPS implementation, because OVMF already
+ contains the edk2 HTTPS BOOT feature (assuming you build OVMF with "-D
+ NETWORK_HTTP_BOOT_ENABLE -D NETWORK_TLS_ENABLE"). The whole point of
+ CONFIG=qemu is to strip the UEFI build of iPXE to a mere Simple Network
+ Protocol driver. Such a build provides only the lowest level hardware
+ abstraction for UEFI, and the rest (IPv[46], TCP/UDP, DHCP, PXE, TFTP,
+ HTTP(S)) are provided on top by the edk2 implementations of various
+ standard UEFI interfaces."
+ 
+  * Never the less, if you seek regressions then in the UEFI handling of 
+HTTPS boot.
+ 
+ 
+ [Other Info]
+  
+  * This happens with some OVMF versions like the one I have provided for 
+debugging but not with the OVMF delivered as part of Focal. Therefore we 
+should fix it but priority isn't that high.
+ 
+  * Quote from the discussion: "If you want to run a full-featured iPXE 
+build on a UEFI machine (including: in an OVMF guest), you still can, 
+of course; lots of people do that, for good reasons. But that use case 
+is best served by the *standalone UEFI application* build of iPXE 
+(produced at "bin-x86_64-efi/ipxe.efi" by "make"). The UEFI *driver* 
+build of iPXE should be as minimal as possible, in comparison -- just 
+provide SNP for the desired NIC models."
+=> Only the EFI roms are changed, neither the PCI roms nor the standalon 
+IPXE builds will be modified.
+ 
+ 
+ ---
+ 
+ 
+ The version of QEMU (4.2.0) packaged for Ubuntu 20.04 hangs indefinitely at 
boot if an OVMF bios is used. This happens ONLY with qemu-system-x86_64. 
qemu-system-i386 works fine with the latest ia32 OVMF bios.
  
  NOTE[1]: the same identical OVMF bios works fine on QEMU 2.x packaged with 
Ubuntu 18.04.
  NOTE[2]: reproducing the fatal bug requires *no* operating system:
  
-qemu-system-x86_64 -bios OVMF-pure-efi.fd
+    qemu-system-x86_64 -bios OVMF-pure-efi.fd
  
  On its window QEMU gets stuck at the very first stage:
-"Guest has not initialized the display (yet)."
+    "Guest has not initialized the display (yet)."
  
  NOTE[3]: QEMU gets stuck no matter if KVM is used or not.
  
  NOTE[4]: By adding the `-d int` option it is possible to observe that
  QEMU is, apparently, stuck in an endless loop of interrupts. For the
  first few seconds, registers' values vary quickly, but at some point
  they reach a final value, while the interrupt counter increments:
  
-   2568: v=68 e= i=0 cpl=0 IP=0038:07f1d225 pc=07f1d225 
SP=0030:07f0c8d0 env->regs[R_EAX]=
+   2568: v=68 e= i=0 cpl=0 IP=0038:07f1d225 pc=07f1d225 
SP=0030:07f0c8d0 env->regs[R_EAX]=
  RAX= RBX=07f0c920 RCX= 
RDX=0001
  RSI=06d18798 RDI=8664 RBP= 
RSP=07f0c8d0
  R8

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

Following [1] I was building my test OVMF as guided by Lazlo.
That way I was able to use the combined e1000 EFI of the Ubuntu packages vs the 
debug OVMF build.

Using that I can confirm the behavior (Bionic working, Focal failing).

$ qemu-system-x86_64 -enable-kvm -monitor stdio -drive
if=pflash,snapshot=on,format=raw,file=/root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd
-global e1000.romfile=/usr/lib/ipxe/qemu/efi-e1000.rom -debugcon
file:debug.log -global isa-debugcon.iobase=0x402

#/usr/lib/ipxe/qemu/efi-e1000.rom is from package ipxe-qemu
#/root/edk2/Build/OvmfX64/DEBUG_GCC5/FV/OVMF.fd is the debug build, I'll attach 
that file here to eas repo in other places

Symptoms when failing:
- UI never leaves the "initializing" state
- in the debug.log the bad case asserts with
  ASSERT /root/edk2/MdeModulePkg/Core/Dxe/Image/Image.c(1676): Image->Tpl == 
gEfiCurrentTpl

That it seems to work fine with the ovmf build that is in Focal
0~20191122.bd85bf54-2ubuntu3 makes the SRU of this less urgent IMHO. And
also resolves some of my remaining confusion since I've known that EFI
boots in general work - and it seems (at least in this test POV) - that
only a newer or different built OVMF triggers the issue.

[1]: https://wiki.ubuntu.com/UEFI/EDK2#Set_up_build_environment

** Attachment added: "OVMF build with debug to verify the bug"
   
https://bugs.launchpad.net/ubuntu/+source/ipxe/+bug/1882671/+attachment/5387066/+files/OVMF-14c7ed8b51f6-DEBUG-enabled.fd

** Changed in: ipxe (Ubuntu)
   Status: Confirmed => In Progress

** Changed in: ipxe (Ubuntu Focal)
   Status: New => Triaged

** Changed in: ipxe (Ubuntu Eoan)
   Status: New => Triaged

** Changed in: ipxe (Ubuntu Eoan)
   Importance: Undecided => Low

** Changed in: ipxe (Ubuntu Focal)
   Importance: Undecided => Low

** Changed in: ipxe (Ubuntu)
   Importance: Undecided => High

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-06-25 Thread Laszlo Ersek (Red Hat)

I used

qemu-system-x86_64 \
  -enable-kvm \
  -monitor stdio \
  -drive if=pflash,snapshot=on,format=raw,file=OVMF.fd \
  -global e1000.romfile=82540em.combined.rom \
  -debugcon file:debug.log \
  -global isa-debugcon.iobase=0x402

where "OVMF.fd" was built from edk2 at then-master (14c7ed8b51f6 -- see
comment 7), and "82540em.combined.rom" is the combined oprom (with HTTPS
enabled in the EFI driver too) built from the problematic iPXE
version(s).

OVMF was built with "-b DEBUG". ("-b RELEASE" would remove the ASSERT()s
from the firmware modules.)

"debug.log" captures the firmware debug output. That's the file that
ends with the ASSERT failure seen in comment 4.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

Thank you Lazlo, going forward that is the process that I have it execute then.
On the SRU I'll "only" disable HTTPS on EFI roms and we can take a look if 
nothing else stops working but this case here would be fixed.

--

One question thou - asking for help to be able to make this an SRU at
soem point.

Vladislav initially reported the test could just be "qemu -bios ovmf".
I tried this in various Ubuntu releases like:
  qemu-system-x86_64 -bios /usr/share/ovmf/OVMF.fd
But without any other setup that properly gives the edk tianocore logo followed 
by some PXE tries and failing to boot. After the timeouts all passed it 
eventually gives me the UEFI shell.

I'd have hoped this behaves differently in Bionic (working) and Focal
(failing) per the initial report - but it seems to be the same.

My point is that I fail to recreate the bug in the existing releases,
which is a requirement (to be able to verify the fix) for the SRU.

@Vladislav (or @Lazlo since you said you can reproduce it) - is there
now a better way known how to trigger this.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-06-25 Thread Launchpad Bug Tracker

** Merge proposal linked:
   
https://code.launchpad.net/~paelzer/ubuntu/+source/ipxe/+git/ipxe/+merge/386372

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

2020-06-24 Thread Laszlo Ersek (Red Hat)

Christian,

what you describe seems to be exactly what I propose. Namely:
- build "82540em.rom" with HTTPS enabled,
- build "82540em.efirom" with CONFIG=qemu, and HTTPS disabled,
- create a combined option ROM image from the above two, using "catrom.pl".

Thanks
Laszlo

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

@Lazlo - are combined roms breaking your suggestion to "just disable https in 
efi roms"?
In the build for the efi roms it uses this at some point:
  src/util/catrom.pl src/bin-i386-pcbios/82540em.rom 
src/bin-x86_64-efi/82540em.efirom > src/bin-combined/82540em.efirom

So the *efi* file in ipxe-qemu will be a combined rom (since 2013).
Of these the legacy one will have HTTPS enabled and the EFI part won't.
Would you expect that this is a problem for the suggested workaround (I don't 
but would be glad if you would let me know what you think about it)?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

Note: the bisection result of d8c500b7945e ("[efi] Drop to
TPL_APPLICATION when gathering entropy" is in <=133f4c4 but > fbe8c52d
which means for Ubuntu releases that would be affected >=Disco.

** Also affects: ipxe (Ubuntu Focal)
   Importance: Undecided
   Status: New

** Also affects: ipxe (Ubuntu Eoan)
   Importance: Undecided
   Status: New

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled

The above was an FYI, but is should be fine as outlined by Lazlo this
isn't needed as since ipxe 1.0.0+git-20180124.fbe8c52d-0ubuntu4 we use
CONFIG=qemu and in comment #7 he explained that in this case "totally
don't need (or even *use*) the iPXE HTTPS infrastructure (the entropy
gathering that trips the ASSERT seems spurious to me, with CONFIG=qemu).
With CONFIG=qemu, iPXE provides the UEFI SNP (Simple Network Protocol)
interface on top of the e1000 NIC, and the crypto stuff (if any) is done
by the platform firmware (edk2 / OVMF)"

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1882671

Title:
  unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS
  enabled

To manage notifications about this bug go to:
https://bugs.launchpad.net/ipxe/+bug/1882671/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1882671] Re: unbalanced UEFI TPL manipulations in iPXE with DOWNLOAD_PROTO_HTTPS enabled