[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Steve Langasek]

** Changed in: shim (Ubuntu Xenial)
   Status: Fix Committed => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Mathieu Trudel-Lapierre]

The latest zesty d-i image (20101020ubuntu487) build on 2016-11-07
should include the right shim already.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

Re: [Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Steve Langasek]

On Fri, Nov 11, 2016 at 07:24:13PM -, Jason Gerard DeRose wrote:
> I recall that d-i needed to be rebuilt for a new shim to be properly
> represented in an ISO... has this happened yet?

It has not.  OTOH this is the same exact binary that is currently in the
zesty release, so it should be possible to test a daily image there
(provided we get a d-i rebuild in zesty).

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Jason Gerard DeRose]

Steve,

I'm not sure whether it was a truly representative test, but it works
fine with latest Xenial daily desktop ISO under QEMU + OVMF, and these
dailies do have xenial-proposed enabled.

(I'm testing the 2016 ISO, sha1sum
0ed4db8dad7142837ce9175e2b9617c4dd93a326.)

I recall that d-i needed to be rebuilt for a new shim to be properly
represented in an ISO... has this happened yet?

Thanks!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Steve Langasek]

Hello Jason, or anyone else affected,

Accepted shim into xenial-proposed. The package will build now and be
available at
https://launchpad.net/ubuntu/+source/shim/0.9+1474479173.6c180c6-0ubuntu1
in a few hours, and then in the -proposed repository.

Please help us by testing this new package.  See
https://wiki.ubuntu.com/Testing/EnableProposed for documentation on how
to enable and use -proposed.Your feedback will aid us getting this
update out to other Ubuntu users.

If this package fixes the bug for you, please add a comment to this bug,
mentioning the version of the package you tested, and change the tag
from verification-needed to verification-done. If it does not fix the
bug for you, please add a comment stating that, and change the tag to
verification-failed.  In either case, details of your testing will help
us make a better decision.

Further information regarding the verification process can be found at
https://wiki.ubuntu.com/QATeam/PerformingSRUVerification .  Thank you in
advance!

** Changed in: shim (Ubuntu Xenial)
   Status: New => Fix Committed

** Tags added: verification-needed

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Mathew Hodson]

** No longer affects: grub2 (Ubuntu)

** No longer affects: debian-cd (Ubuntu)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Mathieu Trudel-Lapierre]

Turns out we didn't need grub2 for this case since we reverted to the
"old" shim.

Zesty now has the new shim and we'll proceed with the SRUs shortly.

** Changed in: grub2 (Ubuntu)
   Status: In Progress => Invalid

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Jason Gerard DeRose]

And I sanity checked the server ISO on the same slew of UEFI hardware...
no issues found.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Jason Gerard DeRose]

Okay, the 20161005.1 ISOs seem to have done the trick. Tested the
desktop and server ISOs under QEMU+OVMF, plus tested the desktop ISO on
a slew of UEFI hardware. No issues encountered shim-wise.

I'll test the server ISO on UEFI hardware shortly, but there are a few
other things I need to finish up first.

Big thanks to everyone who helped on this!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Jason Gerard DeRose]

Hmmm, today's yakkety-desktop-amd64.iso
(sha1:494bc027be3d29c494eb17d057dcc51cdfc6f50b) is seemingly still using
the broken shim package?

I'm guessing there's something special about how the shim package gets
onto the ISO as it doesn't seem to be listed in yakkety-desktop-
amd64.manifest?

But for whatever reason, I'm still getting the same exception when
debugging with `-serial stdio`:

 X64 Exception Type - 0D(#GP - General Protection)  CPU Apic ID -  

RIP  - 7E64D5BA, CS  - 0038, RFLAGS - 00010202
ExceptionData - 
RAX  - AFAFAFAFAFAFAFAF, RCX - 7F1C5820, RDX - 7F1C5820
RBX  - 7F132198, RSP - 7FB1BA40, RBP - 7FB1BAF0
RSI  - 7E6DBD9A, RDI - 7E62CFBA
R8   - 0004, R9  - , R10 - 0020
R11  - 0002, R12 - 7EEB34B8, R13 - 7EEB34C0
R14  - 7FB33620, R15 - 7EDD6018
DS   - 0030, ES  - 0030, FS  - 0030
GS   - 0030, SS  - 0030
CR0  - 8033, CR2 - , CR3 - 7FABA000
CR4  - 0668, CR8 - 
DR0  - , DR1 - , DR2 - 
DR3  - , DR6 - 0FF0, DR7 - 0400
GDTR - 7FAA8698 0047, LDTR - 
IDTR - 7F5E4018 0FFF,   TR - 
FXSAVE_STATE - 7FB1B6A0
 Find PE image (No PDB)  (ImageBase=7E62D000, 
EntryPoint=7E64A000) 

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Launchpad Bug Tracker]

This bug was fixed in the package shim -
0.9+1465500757.14a5905.is.0.8-0ubuntu2

---
shim (0.9+1465500757.14a5905.is.0.8-0ubuntu2) wily; urgency=medium

  * Revert to shim 0.8 for now; which at least doesn't crash if fallback.efi
is absent. (LP: #1624096)
- This effectively reverts shim to 0.8-0ubuntu2.

 -- Mathieu Trudel-Lapierre   Mon, 03 Oct 2016
14:32:28 -0400

** Changed in: shim (Ubuntu)
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Mathieu Trudel-Lapierre]

We're still waiting for shim to be signed by Microsoft. I don't expect
issues with a FFE for the new shim, since it fixes some important bugs.
If it doesn't make it though, we can provide the new shim as a stable
release update.

Given that we're very close to release however, it seems like it's time
to do a revert for now.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Jason Gerard DeRose]

@Mathieu - I was on vacation last week, so I wasn't in the loop on IRC.

What's the current status of this? Does it seem feasible that the fixed
shim package can be signed (and FFE'd) in time for 16.10? Or are we
already at the point where reverting to the shim 0.8-0ubuntu2 package
from Xenial (with whatever needed version trickery) is the only
realistic hope for fixing this?

If there's anything I can do to help, please don't hesitate to ask!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[sierdzio]

Apparently this affects me as well, with Kubuntu 16.10 (beta2 and all
subsequent daily builds), on a self-assembled PC (no QEMU, no laptop).
If it helps in anything, I'm using Asus Z170P motherboard.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Colin Watson]

** Changed in: shim (Ubuntu)
   Status: Fix Released => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[pereze]

** Changed in: shim (Ubuntu)
   Status: In Progress => Fix Released

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Ubuntu QA Website]

This bug has been reported on the Ubuntu ISO testing tracker.

A list of all reports related to this bug can be found here:
http://iso.qa.ubuntu.com/qatracker/reports/bugs/1624096

** Tags added: iso-testing

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Jason Gerard DeRose]

@@Mathieu yeah, if you can get me a custom ISO with an updated shim
package (doesn't need to be signed, I'm not using secure boot), then
I'll giving a thorough testing under QEMU and on all the UEFI hardware I
have access too (6 different laptops, 3 different desktops).

Thanks!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Mathieu Trudel-Lapierre]

As Laszlo mentioned, this can affect other systems than QEMU. I
definitely can't boot the ISO on my thinkpad when shim debugging is
enabled.

Then, as discussed, fallback.efi shouldn't be on the ISO. It's clearly
not going to work due to the way shim is designed. Given that, we don't
need a debian-cd task to install fallback...

I'm working on preparing the shim update since yesterday. I can get you
a working shim if necessary for testing for a custom remastered CD
image.

** Changed in: grub2 (Ubuntu)
   Status: New => Triaged

** Changed in: debian-cd (Ubuntu)
   Status: Triaged => Invalid

** Changed in: grub2 (Ubuntu)
   Importance: Medium => High

** Changed in: shim (Ubuntu)
   Status: Triaged => In Progress

** Changed in: grub2 (Ubuntu)
   Status: Triaged => In Progress

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Laszlo Ersek (Red Hat)]

@Jason -- your script looks alright to me. Can you attach the OVMF debug
log captured with the script? (Although, if the debug mask configured at
build time in the DSC files don't enable the DEBUG_VERBOSE bit, I won't
see everything in the log that I would like to see.)

More importantly, can you upload your test ISO image (with the shim bug
fixed, or worked around) somewhere? If you don't want to expose the URL
publicly, feel free to send it to me in a private email, or in a private
Launchpad message. (I vaguely recall that such a thing exists.)

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Jason Gerard DeRose]

@Laszlo - darn, no luck. Following your recommendations, PXE booting is
no longer attempted, but I still end up at Shell> and the installer
doesn't launch.

I'm attaching the isolated test script I'm using ATM. If I try it with
the latest Yakkety desktop daily ISO, I hit the above hardware exception
(as expected because of the issue in `shim`). However, if I try it with
the same ISO modified to include /EFI/BOOT/fallback.efi, it goes
directly to Shell> rather than launching the installer.

Please let me know if you spot any goofs in my script or can think of
anything else to try.

(Note: my test script doesn't have any -net devices at all, but my image
mastering tools do, so that's why I know that PXE booting wasn't being
tried any more.)

@Mathieu - under the assumption that there is more to this than just the
issue in `shim`, or at least that the presence of fallback.efi can't
fully work-around it, do you have any suggestions as to where I should
go looking for other things that have changed between the Yakkety and
Xenial ISOs, things that might be interacting with the `shim` bug in odd
ways?

Also, I should make it clear why this bug is critical to System76: our
imaging mastering tools use QEMU + OVMF to create our UEFI images, so
this is something we absolutely need to find some solution for in order
to ship 16.10. Because (most likely) we'll need 16.10 to initially ship
Kaby Lake :D

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Jason Gerard DeRose]

Oops, forgot to attach my test script :P

** Attachment added: "test-yakkety-guest.sh"
   
https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1624096/+attachment/4745643/+files/test-yakkety-guest.sh

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Jason Gerard DeRose]

@Laszlo - thank you very much for the detailed explanation!

Sounds like your tips, plus /EFI/BOOT/fallback.efi being present on the
ISO, should be enough for me to work-around this issue. I'll let you
know how it goes.

Thanks again!

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Laszlo Ersek (Red Hat)]

@Jason, there are two separate topics in your question.

First, controlling the boot order from the QEMU command line (i.e.,
filtering and/or reordering the persistent UEFI boot options that (a)
exist from earlier in the varstore, plus (b) OVMF's platform BDS
regenerates at every boot).

For this, you have to use the

-device ,bootindex=N

propertiey, which in turn necessitates the modern, separate notation for
backend/frontend.

For example, for network devices you have to spell out

-netdev ,id=netdev0,... \
-device virtio-net-pci,netdev=netdev0,bootindex=2

For disks, for example with the virtio-blk-pci frontend, it requires

-drive if=none,id=drive0,file=ZZZ,... \
-device virtio-blk-pci,drive=drive0,bootindex=1

The various shorthands like "-net nic", "-hda", "-drive if=virtio" don't
allow you to specify the bootindex=N property, and therefore are
unsuitable for OVMF. (At least if you want to control the boot order
from the QEMU command line.)

So, in this specific case, assuming you have one QCOW2 system disk
(created with qemu-img) that you want to install Ubuntu to, plus the
installer ISO you want to install from, I would recommend:

-drive if=pflash,readonly,format=raw,file=PATH_TO_OVMF_CODE_FD \
-drive if=pflash,format=raw,file=PATH_TO_PRIVATE_VARSTORE \
\
-debugcon file:ovmf.debug.log \
-global isa-debugcon.iobase=0x402 \
\
-chardev stdio,signal=off,mux=on,id=char0 \
-mon chardev=char0,mode=readline,default \
-serial chardev:char0 \
\
-device virtio-scsi-pci,id=scsi0 \
\
-drive id=sysdisk,if=none,format=qcow2,discard=on,cache=writeback,file=... \
-device scsi-hd,drive=sysdisk,bus=scsi0.0,bootindex=1 \
\
-drive id=installer,if=none,format=raw,file=... \
-device scsi-cd,drive=installer,bus=scsi0.0,bootindex=2 \

This will (a) capture the OVMF log; (b) give you access to both the QEMU
monitor and the guest's serial console -- switch between them with [C-a
c]; (c) create a virtio-scsi disk and CD-ROM for the guest, with the
(target) system disk and the installer ISO, respectively; (d) assign
bootindex=1 to the system disk, and bootindex=2 to the installer ISO.

The upshot is that when you first boot the VM, the installer ISO will be
launched (because the system disk is still empty), but after
installation, the VM will boot off of the system disk.

If there is a (QEMU default, or manually configured) virtual NIC in the
VM as well, then PXE boot will *not* be attempted. The reason is that
you assign a bootindex to at least one device, but no bootindex is
assigned to the NIC. This will cause OVMF to filter out any UEFI boot
options (created manually or automatically) that would refer to the NIC.

If the yakkety installer still doesn't boot with the above command line
snippet (*and* with the shim bug fixed or worked around), then I'd say
the installer ISO is malformed in some other way.


The second topic is why the shim bug doesn't hit hard on some physical systems. 
For this, consider how EFI_FILE_PROTOCOL.Close() works -- it releases the 
entire container structure that contains EFI_FILE_PROTOCOL. When you call 
FileProtocol->Close() next, using the same pointer -- i.e., use-after-free --, 
then the Close function pointer is read from freed storage.

As I mentioned earler, due to OVMF setting bit #3 (value 8) in
PcdDebugPropertyMask, memory that is freed gets scrubbed with the byte
value 0xAF. (Funnily enough, this hex value comes from the name of
Andrew Fish, the inventor of EFI.) So when you read the Close function
pointer from an EFI_FILE_PROTOCOL instance that has been closed
(released) already, you get 0xAFAFAFAFAFAFAFAF -- that's why you see
such an instruction pointer (RIP) in the register dump above.

Now, when shim executes the use-after-free (= the second close) on a
UEFI system that does *not* do the memory scrubbing on free, then all
the earlier contents of the freed EFI_FILE_PROTOCOL instance are likely
still in place. Hence the call probably corrupts memory elsewhere, but
it does not blow up at once. (Which is actually much worse bug
behavior.)

This is why you don't see any direct symptoms on physical machines:
memory scrubbing on free is a debugging feature, and none of the
physical firmwares in question enable it apparently. The upstream shim
commit that fixes the regression also mentions "This issue only affects
certain systems".

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Laszlo Ersek (Red Hat)]

Also, I should mention in passing -- again -- that Launchpad is
completely retarded for truncating comments in the full bug view. It
doesn't offer any option to see both the full bug and full comments. How
stupid is that?! Are people who take the time to explain things in
detail really considered "verbose"? Do their comments really deserve to
be abbreviated in the full bug view? "Your comment is too long, so users
who care about it should click another link, and *replace* the full bug
view with a sole comment".

Screw you Launchpad.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Jason Gerard DeRose]

@Mathieu - I've been doing some quick experiments with fallback.efi on
the ISO, and I'm not sure that alone will fix things.

The problem is that when fallback.efi is present, the installer isn't
launching. Instead, OVMF tries to PXE boot (which in my test environment
fails because I don't have the needed DHCP/TFTP setup), then OVMF falls
back to Shell>

I tried adding /EFI/BOOT/fallback.efi to both the latest Yakkety daily
ISO and the 16.04.1 ISO. In both cases, the installer doesn't boot, I
end up at Shell>

So although fallback.efi can work around the X64 Exception in the
Yakkety version of shim, it still doesn't give you a bootable installer.
If there's something obvious I'm missing, please let me know!

Also, do you have any idea why this faulty shim code path is taken when
running under QEMU + OVMF, but does not seem to be taken when running on
physical hardware?

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Mathieu Trudel-Lapierre]

Given that it will take a bit of time to get a new shim signed; we'll
also need to ship fallback.efi on the CD (and it makes sense to do this
anyway), and on disk in general. I've added the tasks for debian-cd and
grub2 to do so.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Mathieu Trudel-Lapierre]

Triaging, this is my problem.

In my defense, I don't think the regression was known at the point I
took that snapshot :)

** Changed in: shim (Ubuntu)
   Status: Confirmed => Triaged

** Changed in: shim (Ubuntu)
   Importance: Undecided => High

** Changed in: shim (Ubuntu)
 Assignee: (unassigned) => Mathieu Trudel-Lapierre (cyphermox)

** Also affects: grub2 (Ubuntu)
   Importance: Undecided
   Status: New

** Also affects: debian-cd (Ubuntu)
   Importance: Undecided
   Status: New

** Changed in: debian-cd (Ubuntu)
   Status: New => Triaged

** Changed in: debian-cd (Ubuntu)
   Importance: Undecided => High

** Changed in: debian-cd (Ubuntu)
 Assignee: (unassigned) => Mathieu Trudel-Lapierre (cyphermox)

** Changed in: grub2 (Ubuntu)
 Assignee: (unassigned) => Mathieu Trudel-Lapierre (cyphermox)

** Changed in: grub2 (Ubuntu)
   Importance: Undecided => Medium

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/debian-cd/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs

[Bug 1624096] Re: yakkety: backport (or rebase to) fix eliminating a double-close in shim

[Laszlo Ersek (Red Hat)]

@Jason -- according to the upstream fix (7052e7530755) that Yakkety is
currently missing, the upstream regression comes from upstream commit
4794822.

That commit (i.e., the regression) is between the 0.9 release and
14a5905. According to
,
the previous ubuntu shim version was "0.8-0ubuntu2", while the most
recent one is "0.9+1465500757.14a5905-0ubuntu1" (including the
regression but not its fix). So, in Ubuntu, it was the latest shim
rebase (dated "Tue, 26 Jul 2016 16:48:32 -0400") that introduced the
bug.

-- 
You received this bug notification because you are a member of Ubuntu
Bugs, which is subscribed to Ubuntu.
https://bugs.launchpad.net/bugs/1624096

Title:
  yakkety: backport (or rebase to) fix eliminating a double-close in
  shim

To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/shim/+bug/1624096/+subscriptions

-- 
ubuntu-bugs mailing list
ubuntu-bugs@lists.ubuntu.com
https://lists.ubuntu.com/mailman/listinfo/ubuntu-bugs