On 12/10/2025 1:10 PM, Tony Camuso wrote:
On 11/24/2025 7:34 PM, Bagas Sanjaya wrote:
On Thu, Nov 13, 2025 at 09:37:14AM -0500, Tony Camuso wrote:
The tboot->shutdown_entry is effectively bios code and CET needs to be
disabled before calling it.
It resolves TBOOT shutdown failure bug, reported on the SLES (SUSE Linux
Enterprise Server) 16.0 OS. OS power off, called by the "init 0" command,
was failing, due to activated Intel Control-Flow Enforcement Technology
(CET).
Disabling CET has allowed to execute OS and TBOOT shutdown properly.
Are ``systemctl poweroff`` and ``shutdown -P`` are also affected?
Confused...
Yes, all shutdown methods on kernels launched with tboot, on systems that
expose the CPU ibt flag to kernels v6.12+ will cause the stack trace appended
below.
The stack trace demonstrates that CET enforcement collides with legacy
BIOS shutdown code that lacks ENDBR markers. The kernel BUG at
cet.c:102 is a direct result of CET being active when jumping into
tboot->shutdown_entry.
Legacy BIOS/tboot code without ENDBR now traps, requiring CET to be disabled
around that call.
The patch:
Prevents CET from falsely trapping on non-CET BIOS code.
Need to clarify further:
The patch prevents CET from trapping when tboot invokes the BIOS-provided
shutdown_entry routine, which lacks ENDBR instructions.
tboot side:
In tboot_shutdown(), the kernel switches to the tboot page tables and
then calls:
shutdown = (void(*)(void))(unsigned long)tboot->shutdown_entry;
shutdown();
That shutdown_entry pointer comes from the tboot structure, populated
at boot.
In the tboot project directory, see include/tboot.h
In the kernel, see include/linux/tboot.h
BIOS side:
The actual routine behind shutdown_entry is implemented in BIOS/firmware.
It’s not compiled with CET/IBT support, so it lacks the required ENDBR64
instruction at its entry point
When CET is still enabled, the CPU enforces IBT. Jumping into that BIOS
routine without ENDBR triggers a #CP (control protection exception),
which is what the stack trace shows.
So it is the BIOS shutdown_entry function itself that causes the trap,
but only because tboot is handing control to it while CET is active.
What happens:
From the stack trace
Missing ENDBR: 0x8041d0
kernel BUG at arch/x86/kernel/cet.c:102!
RIP: 0010:0x8041d0
This shows the CPU trapping on entry into the shutdown routine at address
0x8041d0, which is the pointer stored in tboot->shutdown_entry
The shutdown_entry field is explicitly documented as the physical address
of the BIOS shutdown routine. This structure is populated by tboot at boot.
Maintains system stability during shutdown.
Preserves CET protection elsewhere, only disabling it for the
narrow window where legacy firmware must run.
[ 169.420078] reboot: Power down
[ 169.427516] Missing ENDBR: 0x8041d0
[ 169.431128] ------------[ cut here ]------------
[ 169.435805] kernel BUG at arch/x86/kernel/cet.c:102!
[ 169.440840] Oops: invalid opcode: 0000 [#1] SMP NOPTI
[ 169.445966] CPU: 0 UID: 0 PID: 3354 Comm: poweroff Kdump: loaded Not tainted
6.12.0-124.8.1.el10_1.x86_64 #1 PREEMPT(voluntary)
[ 169.457580] Hardware name: Dell Inc. PowerEdge R570/03TJR3, BIOS 1.2.1
01/23/2025
[ 169.465113] RIP: 0010:exc_control_protection+0x18c/0x190
[ 169.470490] Code: 1c ff 45 31 c9 49 89 d8 b9 09 00 00 00 48 8b 93 80 00 00 00 be
63 00 00 00 48 c7 c7 a4 85 e5 a4 e8 79 92 30 ff e9 02 ff ff ff <0f> 0b 66 90 90
90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 66 0f
[ 169.489292] RSP: 0018:ff55b3cba167fa88 EFLAGS: 00010002
[ 169.494581] RAX: 0000000000000017 RBX: ff55b3cba167faa8 RCX: 00000000ffff7fff
[ 169.501765] RDX: 0000000000000000 RSI: 0000000000000003 RDI: 0000000000000001
[ 169.508949] RBP: 0000000000000003 R08: 0000000000000000 R09: ffffffffa59e2b08
[ 169.516132] R10: ffffffffa5922ac8 R11: 0000000000000003 R12: 0000000000000000
[ 169.523316] R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
[ 169.530514] FS: 00007f5f9a122140(0000) GS:ff39175c2de00000(0000)
knlGS:0000000000000000
[ 169.538659] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[ 169.544454] CR2: 0000559386dc5320 CR3: 000000010fbe2000 CR4: 0000000000f71ef0
[ 169.551651] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[ 169.558835] DR3: 0000000000000000 DR6: 00000000fffe07f0 DR7: 0000000000000400
[ 169.566019] PKRU: 55555554
[ 169.568784] Call Trace:
[ 169.571295] <TASK>
[ 169.573458] ? show_trace_log_lvl+0x1b0/0x2f0
[ 169.577880] ? show_trace_log_lvl+0x1b0/0x2f0
[ 169.582301] ? asm_exc_control_protection+0x26/0x30
[ 169.587244] ? exc_control_protection+0x18c/0x190
[ 169.592011] ? __die_body.cold+0x8/0x12
[ 169.595910] ? die+0x2e/0x50
[ 169.598863] ? do_trap+0xca/0x110
[ 169.602243] ? do_error_trap+0x65/0x80
[ 169.606049] ? exc_control_protection+0x18c/0x190
[ 169.610816] ? exc_invalid_op+0x50/0x70
[ 169.614715] ? exc_control_protection+0x18c/0x190
[ 169.619482] ? asm_exc_invalid_op+0x1a/0x20
[ 169.623728] ? exc_control_protection+0x18c/0x190
[ 169.628496] ? exc_control_protection+0x14f/0x190
[ 169.633263] asm_exc_control_protection+0x26/0x30
[ 169.638030] RIP: 0010:0x8041d0
[ 169.641142] Code: Unable to access opcode bytes at 0x8041a6.
[ 169.646857] RSP: 0018:ff55b3cba167fb50 EFLAGS: 00010007
[ 169.652144] RAX: 00000000008041d0 RBX: 0000000000000000 RCX: 0000000000000005
[ 169.659341] RDX: 00c6e8a7c0000000 RSI: 0000000000000001 RDI: ffffffffff1ff000
[ 169.666525] RBP: 0000000000000005 R08: 0000000000000000 R09: 000000000000ffff
[ 169.673709] R10: 0000000000000000 R11: ffffffffffff0000 R12: 0000000000002001
[ 169.680906] R13: ffffffffa5ae02c8 R14: 00000000ffffffff R15: 0000000000000000
[ 169.688091] ? tboot_shutdown+0x5b/0x140
[ 169.692084] ? tboot_sleep+0x12c/0x140
[ 169.695890] ? acpi_os_enter_sleep+0x2b/0x60
[ 169.700221] ? acpi_hw_legacy_sleep+0x140/0x1c0
[ 169.704816] ? acpi_power_off+0x16/0x40
[ 169.708715] ? sys_off_notify+0x48/0x70
[ 169.712615] ? notifier_call_chain+0x5a/0xd0
[ 169.716943] ? atomic_notifier_call_chain+0x32/0x50
[ 169.721885] ? do_kernel_power_off+0x3e/0x50
[ 169.726213] ? native_machine_power_off+0x21/0x40
[ 169.730983] ? __do_sys_reboot+0x1d2/0x240
[ 169.735151] ? do_syscall_64+0x7d/0x160
[ 169.739053] ? syscall_exit_work+0xf3/0x120
[ 169.743302] ? syscall_exit_to_user_mode+0x32/0x190
[ 169.748243] ? do_syscall_64+0x89/0x160
[ 169.752143] ? __count_memcg_events+0xdf/0x170
[ 169.756645] ? handle_mm_fault+0x256/0x370
[ 169.760813] ? do_user_addr_fault+0x347/0x640
[ 169.765235] ? exc_page_fault+0x73/0x160
[ 169.769228] ? entry_SYSCALL_64_after_hwframe+0x76/0x7e
_______________________________________________
tboot-devel mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/tboot-devel
