Author: Zachary Yedidia Date: 2026-07-06T20:56:00-07:00 New Revision: b068ae9e376b1e3d721064e140e638cad076d0bc
URL: https://github.com/llvm/llvm-project/commit/b068ae9e376b1e3d721064e140e638cad076d0bc DIFF: https://github.com/llvm/llvm-project/commit/b068ae9e376b1e3d721064e140e638cad076d0bc.diff LOG: [LFI][X86] Add X86 LFI target and system instruction rewrites (#189569) This PR introduces an x86-64 backend for Lightweight Fault Isolation (LFI), similar to the one being developed for AArch64. LFI is a compiler-based mechanism that enables efficient in-process sandboxing. See the RFC from last fall for details. This PR adds the `x86_64_lfi` target (similar to `aarch64_lfi`), sets up reserved registers, and implements some initial rewrites for system instructions (system calls and TLS accesses). The rewrites are done at the MC level, using the `MCLFIRewriter` infrastructure. I have updated the documentation to describe the x86-64 sandboxing scheme and to list rewrites that will be implemented in future PRs (to keep each individual PR small). For performance and compatibility reasons, the plan is currently to use bundling for maintaining control-flow integrity in the sandbox, which is not yet supported. For now we are setting up the rewrites without using bundling, but we can also use a CFI mechanism based on shadow stack+endbr in order to have something usable while bundling is in progress. Added: llvm/lib/Target/X86/MCTargetDesc/X86MCLFIRewriter.cpp llvm/lib/Target/X86/MCTargetDesc/X86MCLFIRewriter.h llvm/test/CodeGen/X86/lfi-sibcall.ll llvm/test/MC/X86/LFI/abi-note.s llvm/test/MC/X86/LFI/syscall.s llvm/test/MC/X86/LFI/thread-pointer-errors.s llvm/test/MC/X86/LFI/thread-pointer.s Modified: clang/lib/Basic/Targets/X86.cpp llvm/docs/LFI.rst llvm/include/llvm/TargetParser/Triple.h llvm/lib/MC/MCLFI.cpp llvm/lib/Target/X86/MCTargetDesc/CMakeLists.txt llvm/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.cpp llvm/lib/Target/X86/X86ISelLowering.cpp llvm/lib/Target/X86/X86ISelLoweringCall.cpp llvm/lib/Target/X86/X86RegisterInfo.cpp llvm/lib/Target/X86/X86Subtarget.h llvm/lib/TargetParser/Triple.cpp Removed: ################################################################################ diff --git a/clang/lib/Basic/Targets/X86.cpp b/clang/lib/Basic/Targets/X86.cpp index 9545367044555..b2728386aed30 100644 --- a/clang/lib/Basic/Targets/X86.cpp +++ b/clang/lib/Basic/Targets/X86.cpp @@ -531,6 +531,10 @@ void X86TargetInfo::getTargetDefines(const LangOptions &Opts, DefineStd(Builder, "i386", Opts); } + if (getTriple().isLFI()) { + Builder.defineMacro("__LFI__"); + } + Builder.defineMacro("__SEG_GS"); Builder.defineMacro("__SEG_FS"); Builder.defineMacro("__seg_gs", "__attribute__((address_space(256)))"); diff --git a/llvm/docs/LFI.rst b/llvm/docs/LFI.rst index 74cfe02c76f21..cf54e4e120026 100644 --- a/llvm/docs/LFI.rst +++ b/llvm/docs/LFI.rst @@ -38,10 +38,9 @@ runtime), responsible for initializing the sandbox region, loading the program, and servicing system call requests, or other forms of runtime calls. LFI uses an architecture-specific sandboxing scheme based on the general -technique of Software-Based Fault Isolation (SFI). Initial support for LFI in -LLVM is focused on the AArch64 platform, with x86-64 support planned for the -future. The initial version of LFI for AArch64 is designed to support the -Armv8.1 AArch64 architecture. +technique of Software-Based Fault Isolation (SFI). LLVM currently supports LFI +for the AArch64 and X86-64 platforms. The AArch64 version is designed to +support the Armv8.1 AArch64 architecture. See `https://github.com/lfi-project <https://github.com/lfi-project/>`__ for details about the LFI project and additional software needed to run LFI @@ -50,11 +49,11 @@ programs. Compiler Requirements +++++++++++++++++++++ -When building for the ``aarch64_lfi`` target, the compiler must restrict use of -the instruction set to a subset of instructions, which are known to be safe -from a sandboxing perspective. To do this, we apply a set of simple rewrites at -the assembly language level to transform standard native AArch64 assembly into -LFI-compatible AArch64 assembly. +When building for an LFI target (``aarch64_lfi`` or ``x86_64_lfi``), the +compiler must restrict use of the instruction set to a subset of instructions, +which are known to be safe from a sandboxing perspective. To do this, we apply a +set of simple rewrites at the assembly language level to transform standard +native assembly into LFI-compatible assembly. These rewrites (also called "expansions") are applied at the very end of the LLVM compilation pipeline (during the assembler step). This allows the rewrites @@ -451,6 +450,118 @@ In certain cases, guards may be hoisted outside of loops. | | | +-----------------------+-------------------------------+ +X86-64 +++++++ + +The X86-64 LFI target is ``x86_64_lfi``. + +Reserved Registers +================== + +The X86-64 LFI target reserves the following registers: + +* ``r14``: always holds the sandbox base address. Also used as the runtime call + table pointer (the runtime call table is stored at the sandbox base). +* ``gs``: always holds the sandbox base address (used as a segment register for + memory access sandboxing). +* ``rsp``: always holds an address within the sandbox. +* ``r15``: context register (see `Context Register`_). +* ``r11``: scratch register. + +Assembly Rewrites +================= + +Terminology +~~~~~~~~~~~ + +In the following assembly rewrites, some shorthand is used. + +* ``%rN`` or ``%eN``: refers to any general-purpose non-reserved register. +* ``{a,b,c}``: matches any of ``a``, ``b``, or ``c``. + +Control flow +~~~~~~~~~~~~ + +**Note**: these rewrites have not been implemented. + +Memory accesses +~~~~~~~~~~~~~~~ + +**Note**: these rewrites have not been implemented. + +String instructions +~~~~~~~~~~~~~~~~~~~ + +**Note**: these rewrites have not been implemented. + +Stack modification +~~~~~~~~~~~~~~~~~~ + +**Note**: these rewrites have not been implemented. + +System instructions +~~~~~~~~~~~~~~~~~~~ + +System calls are rewritten into a sequence that loads the return address into +the scratch register and jumps to the runtime call handler. The runtime call +handler table is stored at the address pointed to by ``r14``. The ``r11`` +register stores the return address (marked by the label ``.Ltmp`` in the +block below). + ++-------------------+-------------------------------+ +| Original | Rewritten | ++-------------------+-------------------------------+ +| .. code-block:: | .. code-block:: | +| | | +| syscall | leaq .Ltmp(%rip), %r11 | +| | jmpq *-8(%r14) | +| | .Ltmp: | +| | | ++-------------------+-------------------------------+ + +Thread pointer +~~~~~~~~~~~~~~ + +Thread pointer accesses via the ``%fs`` segment (used for TLS) are rewritten to +use the virtual thread pointer from the context register (``r15``) at offset 16 +(see `Context Register`_). The rewrite handles any load or store instruction +with an ``%fs``-segment memory operand. ``Op`` represents any such instruction. + ++--------------------------------------+----------------------------------------+ +| Original | Rewritten | ++--------------------------------------+----------------------------------------+ +| .. code-block:: | .. code-block:: | +| | | +| Op %fs:0, %rD | Op 16(%r15), %rD | +| | | ++--------------------------------------+----------------------------------------+ +| .. code-block:: | .. code-block:: | +| | | +| Op %fs:(%rX), %rD | movq 16(%r15), %rD | +| | Op (%rD, %rX), %rD | +| | | ++--------------------------------------+----------------------------------------+ +| .. code-block:: | .. code-block:: | +| | | +| Op %rS, %fs:(%rX) | movq 16(%r15), %r11 | +| | Op %rS, (%r11, %rX) | +| | | ++--------------------------------------+----------------------------------------+ +| .. code-block:: | .. code-block:: | +| | | +| Op %fs:N(%rX, %rY, S), %rD | movq 16(%r15), %r11 | +| | leaq (%r11, %rX), %r11 | +| | Op N(%r11, %rY, S), %rD | +| | | ++--------------------------------------+----------------------------------------+ +| .. code-block:: | .. code-block:: | +| | | +| Op %rS, %fs:N(%rX, %rY, S) | movq 16(%r15), %r11 | +| | leaq (%r11, %rX), %r11 | +| | Op %rS, N(%r11, %rY, S) | +| | | ++--------------------------------------+----------------------------------------+ + References ++++++++++ diff --git a/llvm/include/llvm/TargetParser/Triple.h b/llvm/include/llvm/TargetParser/Triple.h index 4cf54022c5c32..827da879f80b4 100644 --- a/llvm/include/llvm/TargetParser/Triple.h +++ b/llvm/include/llvm/TargetParser/Triple.h @@ -159,6 +159,8 @@ class Triple { AArch64SubArch_arm64ec, AArch64SubArch_lfi, + X86_64SubArch_lfi, + KalimbaSubArch_v3, KalimbaSubArch_v4, KalimbaSubArch_v5, @@ -916,8 +918,10 @@ class Triple { /// Tests whether the target is LFI. bool isLFI() const { - return getArch() == Triple::aarch64 && - getSubArch() == Triple::AArch64SubArch_lfi; + return (getArch() == Triple::aarch64 && + getSubArch() == Triple::AArch64SubArch_lfi) || + (getArch() == Triple::x86_64 && + getSubArch() == Triple::X86_64SubArch_lfi); } /// Tests whether the target supports the EHABI exception diff --git a/llvm/lib/MC/MCLFI.cpp b/llvm/lib/MC/MCLFI.cpp index a5c65540044b6..7b149f4b53b4b 100644 --- a/llvm/lib/MC/MCLFI.cpp +++ b/llvm/lib/MC/MCLFI.cpp @@ -61,6 +61,10 @@ void emitLFINoteSection(MCStreamer &Streamer, MCContext &Ctx) { NoteName = ".note.LFI.ABI.aarch64"; NoteArch = "aarch64"; break; + case Triple::x86_64: + NoteName = ".note.LFI.ABI.x86_64"; + NoteArch = "x86_64"; + break; default: reportFatalUsageError("Unsupported architecture for LFI"); } diff --git a/llvm/lib/Target/X86/MCTargetDesc/CMakeLists.txt b/llvm/lib/Target/X86/MCTargetDesc/CMakeLists.txt index f2e7d43fc17f6..94d11cb1573be 100644 --- a/llvm/lib/Target/X86/MCTargetDesc/CMakeLists.txt +++ b/llvm/lib/Target/X86/MCTargetDesc/CMakeLists.txt @@ -9,6 +9,7 @@ add_llvm_component_library(LLVMX86Desc X86MCTargetDesc.cpp X86MCAsmInfo.cpp X86MCCodeEmitter.cpp + X86MCLFIRewriter.cpp X86MachObjectWriter.cpp X86MnemonicTables.cpp X86ELFObjectWriter.cpp diff --git a/llvm/lib/Target/X86/MCTargetDesc/X86MCLFIRewriter.cpp b/llvm/lib/Target/X86/MCTargetDesc/X86MCLFIRewriter.cpp new file mode 100644 index 0000000000000..73dcbeaea918e --- /dev/null +++ b/llvm/lib/Target/X86/MCTargetDesc/X86MCLFIRewriter.cpp @@ -0,0 +1,248 @@ +//===- X86MCLFIRewriter.cpp -------------------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// +// +// This file implements the X86MCLFIRewriter class, which rewrites X86-64 +// instructions for LFI (Lightweight Fault Isolation) sandboxing. +// +//===----------------------------------------------------------------------===// + +#include "X86MCLFIRewriter.h" +#include "X86BaseInfo.h" +#include "X86MCTargetDesc.h" +#include "llvm/MC/MCContext.h" +#include "llvm/MC/MCExpr.h" +#include "llvm/MC/MCInst.h" +#include "llvm/MC/MCStreamer.h" +#include "llvm/MC/MCSubtargetInfo.h" + +using namespace llvm; + +// LFI reserved registers. +static constexpr MCRegister LFIBaseReg = X86::R14; +static constexpr MCRegister LFIScratchReg = X86::R11; +static constexpr MCRegister LFITPReg = X86::R15; + +// Byte offset into the context register file (pointed to by R15) where the +// thread pointer is stored. +static constexpr int TPOffset = 16; + +static bool isSyscall(const MCInst &Inst) { + return Inst.getOpcode() == X86::SYSCALL; +} + +// Find the index of the memory operand if it has an %fs segment override. +// Returns -1 if there is no memory operand or no %fs override. +static int findFSMemOperand(const MCInst &Inst, const MCInstrInfo &InstInfo) { + const MCInstrDesc &Desc = InstInfo.get(Inst.getOpcode()); + int MemRefIdx = X86II::getMemoryOperandNo(Desc.TSFlags); + if (MemRefIdx < 0) + return -1; + int MemIdx = MemRefIdx + X86II::getOperandBias(Desc); + const MCOperand &Seg = Inst.getOperand(MemIdx + X86::AddrSegmentReg); + if (Seg.isReg() && Seg.getReg() == X86::FS) + return MemIdx; + return -1; +} + +// Return true if the instruction reads from Reg. +static bool readsRegister(const MCInst &Inst, const MCInstrDesc &Desc, + MCRegister Reg, const MCRegisterInfo &RI) { + for (unsigned I = Desc.getNumDefs(), E = Inst.getNumOperands(); I < E; ++I) { + const MCOperand &Op = Inst.getOperand(I); + if (Op.isReg() && Op.getReg() && RI.regsOverlap(Op.getReg(), Reg)) + return true; + } + for (MCPhysReg Use : Desc.implicit_uses()) + if (RI.regsOverlap(Use, Reg)) + return true; + return false; +} + +// Return true if Reg is absent or a 64-bit general-purpose register. +static bool isGR64OrNone(MCRegister Reg) { + return Reg == X86::NoRegister || + X86MCRegisterClasses[X86::GR64RegClassID].contains(Reg); +} + +// syscall +// -> +// leaq .Ltmp(%rip), %r11 +// jmpq *(%r14) +// .Ltmp: +void X86::X86MCLFIRewriter::rewriteSyscall(const MCInst &Inst, MCStreamer &Out, + const MCSubtargetInfo &STI) { + MCSymbol *Symbol = Out.getContext().createTempSymbol(); + + // leaq .Ltmp(%rip), %r11 + MCInst Lea; + Lea.setOpcode(X86::LEA64r); + Lea.addOperand(MCOperand::createReg(LFIScratchReg)); + Lea.addOperand(MCOperand::createReg(X86::RIP)); + Lea.addOperand(MCOperand::createImm(1)); + Lea.addOperand(MCOperand::createReg(X86::NoRegister)); + Lea.addOperand( + MCOperand::createExpr(MCSymbolRefExpr::create(Symbol, Out.getContext()))); + Lea.addOperand(MCOperand::createReg(X86::NoRegister)); + Out.emitInstruction(Lea, STI); + + // jmpq *(%r14) + MCInst Jmp; + Jmp.setOpcode(X86::JMP64m); + Jmp.addOperand(MCOperand::createReg(LFIBaseReg)); + Jmp.addOperand(MCOperand::createImm(1)); + Jmp.addOperand(MCOperand::createReg(X86::NoRegister)); + Jmp.addOperand(MCOperand::createImm(-8)); + Jmp.addOperand(MCOperand::createReg(X86::NoRegister)); + Out.emitInstruction(Jmp, STI); + + Out.emitLabel(Symbol); +} + +// Emit: movq TPOffset(%r15), %Reg +static void emitTPLoad(MCRegister Reg, MCStreamer &Out, + const MCSubtargetInfo &STI) { + MCInst Mov; + Mov.setOpcode(X86::MOV64rm); + Mov.addOperand(MCOperand::createReg(Reg)); + Mov.addOperand(MCOperand::createReg(LFITPReg)); + Mov.addOperand(MCOperand::createImm(1)); + Mov.addOperand(MCOperand::createReg(X86::NoRegister)); + Mov.addOperand(MCOperand::createImm(TPOffset)); + Mov.addOperand(MCOperand::createReg(X86::NoRegister)); + Out.emitInstruction(Mov, STI); +} + +bool X86::X86MCLFIRewriter::isFSAccess(const MCInst &Inst) { + return (mayLoad(Inst) || mayStore(Inst)) && + findFSMemOperand(Inst, *InstInfo) >= 0; +} + +// Rewrite %fs-segment memory accesses to use the virtual thread pointer stored +// at TPOffset(%r15). The actual memory access is currently unsandboxed because +// load/store sandboxing is not yet supported. Example rewrites: +// +// movq %fs:0, %rax +// -> +// movq 16(%r15), %rax +// +// movq %fs:(%rdi), %rax +// -> +// movq 16(%r15), %rax +// movq (%rax, %rdi), %rax +// +// movq %fs:8(%rdi, %rsi, 2), %rax +// -> +// movq 16(%r15), %rax +// leaq (%rax, %rdi), %rax +// movq 8(%rax, %rsi, 2), %rax +void X86::X86MCLFIRewriter::rewriteFSAccess(const MCInst &Inst, MCStreamer &Out, + const MCSubtargetInfo &STI) { + int MemIdx = findFSMemOperand(Inst, *InstInfo); + assert(MemIdx >= 0); + + MCRegister BaseReg = Inst.getOperand(MemIdx + X86::AddrBaseReg).getReg(); + MCRegister IndexReg = Inst.getOperand(MemIdx + X86::AddrIndexReg).getReg(); + bool HasBase = BaseReg != X86::NoRegister; + bool HasIndex = IndexReg != X86::NoRegister; + bool HasDisp = !Inst.getOperand(MemIdx + X86::AddrDisp).isImm() || + Inst.getOperand(MemIdx + X86::AddrDisp).getImm() != 0; + + // %fs:0 -> TPOffset(%r15) + if (!HasBase && !HasIndex && !HasDisp) { + MCInst Modified(Inst); + Modified.getOperand(MemIdx + X86::AddrBaseReg).setReg(LFITPReg); + Modified.getOperand(MemIdx + X86::AddrDisp).setImm(TPOffset); + Modified.getOperand(MemIdx + X86::AddrSegmentReg).setReg(X86::NoRegister); + return Out.emitInstruction(Modified, STI); + } + + if (!isGR64OrNone(BaseReg) || !isGR64OrNone(IndexReg) || + BaseReg == X86::RSP || BaseReg == X86::RIP) + return error(Inst, "unsupported addressing mode for %fs access"); + + const MCInstrDesc &Desc = InstInfo->get(Inst.getOpcode()); + + // Reuse operand 0 as the TP temporary when the instruction writes it without + // also reading it, otherwise use %r11. + MCRegister TPDest = LFIScratchReg; + if (MemIdx > 0 && Inst.getOperand(0).isReg()) { + MCRegister DestReg = Inst.getOperand(0).getReg(); + if (Desc.getNumDefs() > 0 && + X86MCRegisterClasses[X86::GR64RegClassID].contains(DestReg) && + !readsRegister(Inst, Desc, DestReg, *RegInfo)) + TPDest = DestReg; + } + + if (TPDest == LFIScratchReg && + readsRegister(Inst, Desc, LFIScratchReg, *RegInfo)) + return error(Inst, "%fs access reads reserved register %r11"); + + emitTPLoad(TPDest, Out, STI); + + // Both slots occupied: the compute base via lea. For example: + // + // movq %fs:8(%rdi,%rsi,2), %rax + // -> + // movq 16(%r15), %rax + // leaq (%rax,%rdi), %rax + // movq 8(%rax,%rsi,2), %rax + if (HasBase && HasIndex) { + MCInst Lea; + Lea.setOpcode(X86::LEA64r); + Lea.addOperand(MCOperand::createReg(TPDest)); + Lea.addOperand(MCOperand::createReg(TPDest)); + Lea.addOperand(MCOperand::createImm(1)); + Lea.addOperand(MCOperand::createReg(BaseReg)); + Lea.addOperand(MCOperand::createImm(0)); + Lea.addOperand(MCOperand::createReg(X86::NoRegister)); + Out.emitInstruction(Lea, STI); + } + + // Emit the access with TPDest as the new base, and the original base + // (offset from %fs) as the new index. For example: + // + // movq %fs:(%rdi), %rax + // -> + // movq 16(%r15), %rax + // movq (%rax,%rdi), %rax + MCInst Modified(Inst); + Modified.getOperand(MemIdx + X86::AddrBaseReg).setReg(TPDest); + if (HasBase && !HasIndex) + Modified.getOperand(MemIdx + X86::AddrIndexReg).setReg(BaseReg); + Modified.getOperand(MemIdx + X86::AddrSegmentReg).setReg(X86::NoRegister); + Out.emitInstruction(Modified, STI); +} + +void X86::X86MCLFIRewriter::doRewriteInst(const MCInst &Inst, MCStreamer &Out, + const MCSubtargetInfo &STI) { + if (mayModifyRegister(Inst, LFIBaseReg) || mayModifyRegister(Inst, LFITPReg)) + return error(Inst, "illegal modification of reserved LFI register"); + + if (isSyscall(Inst)) + return rewriteSyscall(Inst, Out, STI); + + if (isFSAccess(Inst)) + return rewriteFSAccess(Inst, Out, STI); + + // Pass through all other instructions unchanged. + Out.emitInstruction(Inst, STI); +} + +bool X86::X86MCLFIRewriter::rewriteInst(const MCInst &Inst, MCStreamer &Out, + const MCSubtargetInfo &STI) { + // The guard prevents rewrite-recursion when we emit instructions from inside + // the rewriter (such instructions should not be rewritten). + if (!Enabled || Guard) + return false; + Guard = true; + + doRewriteInst(Inst, Out, STI); + + Guard = false; + return true; +} diff --git a/llvm/lib/Target/X86/MCTargetDesc/X86MCLFIRewriter.h b/llvm/lib/Target/X86/MCTargetDesc/X86MCLFIRewriter.h new file mode 100644 index 0000000000000..d74f875311d94 --- /dev/null +++ b/llvm/lib/Target/X86/MCTargetDesc/X86MCLFIRewriter.h @@ -0,0 +1,54 @@ +//===- X86MCLFIRewriter.h ---------------------------------------*- C++ -*-===// +// +// Part of the LLVM Project, under the Apache License v2.0 with LLVM Exceptions. +// See https://llvm.org/LICENSE.txt for license information. +// SPDX-License-Identifier: Apache-2.0 WITH LLVM-exception +// +//===----------------------------------------------------------------------===// +// +// This file declares the X86MCLFIRewriter class, the X86 specific +// subclass of MCLFIRewriter. +// +//===----------------------------------------------------------------------===// +#ifndef LLVM_LIB_TARGET_X86_MCTARGETDESC_X86MCLFIREWRITER_H +#define LLVM_LIB_TARGET_X86_MCTARGETDESC_X86MCLFIREWRITER_H + +#include "llvm/MC/MCInstrInfo.h" +#include "llvm/MC/MCLFIRewriter.h" +#include "llvm/MC/MCRegisterInfo.h" + +namespace llvm { +class MCContext; +class MCInst; +class MCStreamer; +class MCSubtargetInfo; + +namespace X86 { + +class X86MCLFIRewriter : public MCLFIRewriter { +public: + X86MCLFIRewriter(MCContext &Ctx, std::unique_ptr<MCRegisterInfo> &&RI, + std::unique_ptr<MCInstrInfo> &&II) + : MCLFIRewriter(Ctx, std::move(RI), std::move(II)) {} + + bool rewriteInst(const MCInst &Inst, MCStreamer &Out, + const MCSubtargetInfo &STI) override; + +private: + /// Recursion guard to prevent infinite loops when emitting instructions. + bool Guard = false; + + void doRewriteInst(const MCInst &Inst, MCStreamer &Out, + const MCSubtargetInfo &STI); + + void rewriteSyscall(const MCInst &Inst, MCStreamer &Out, + const MCSubtargetInfo &STI); + + bool isFSAccess(const MCInst &Inst); + void rewriteFSAccess(const MCInst &Inst, MCStreamer &Out, + const MCSubtargetInfo &STI); +}; + +} // namespace X86 +} // namespace llvm +#endif // LLVM_LIB_TARGET_X86_MCTARGETDESC_X86MCLFIREWRITER_H diff --git a/llvm/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.cpp b/llvm/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.cpp index f156f3b7b89fc..5f50e197ec3da 100644 --- a/llvm/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.cpp +++ b/llvm/lib/Target/X86/MCTargetDesc/X86MCTargetDesc.cpp @@ -16,6 +16,7 @@ #include "X86BaseInfo.h" #include "X86IntelInstPrinter.h" #include "X86MCAsmInfo.h" +#include "X86MCLFIRewriter.h" #include "X86TargetStreamer.h" #include "llvm-c/Visibility.h" #include "llvm/ADT/APInt.h" @@ -787,6 +788,14 @@ static MCInstrAnalysis *createX86MCInstrAnalysis(const MCInstrInfo *Info) { return new X86_MC::X86MCInstrAnalysis(Info); } +static MCLFIRewriter * +createX86MCLFIRewriter(MCContext &Ctx, + std::unique_ptr<MCRegisterInfo> &&RegInfo, + std::unique_ptr<MCInstrInfo> &&InstInfo) { + return new X86::X86MCLFIRewriter(Ctx, std::move(RegInfo), + std::move(InstInfo)); +} + // Force static initialization. extern "C" LLVM_C_ABI void LLVMInitializeX86TargetMC() { for (Target *T : {&getTheX86_32Target(), &getTheX86_64Target()}) { @@ -809,6 +818,9 @@ extern "C" LLVM_C_ABI void LLVMInitializeX86TargetMC() { // Register the code emitter. TargetRegistry::RegisterMCCodeEmitter(*T, createX86MCCodeEmitter); + // Register the LFI rewriter. + TargetRegistry::RegisterMCLFIRewriter(*T, createX86MCLFIRewriter); + // Register the obj target streamer. TargetRegistry::RegisterObjectTargetStreamer(*T, createX86ObjectTargetStreamer); diff --git a/llvm/lib/Target/X86/X86ISelLowering.cpp b/llvm/lib/Target/X86/X86ISelLowering.cpp index 9209134a055c6..0dd8d05e25230 100644 --- a/llvm/lib/Target/X86/X86ISelLowering.cpp +++ b/llvm/lib/Target/X86/X86ISelLowering.cpp @@ -19998,6 +19998,13 @@ X86TargetLowering::LowerGlobalTLSAddress(SDValue Op, SelectionDAG &DAG) const { if (Subtarget.isTargetELF()) { TLSModel::Model model = DAG.getTarget().getTLSModel(GV); + // LFI does not support dlopen, so all TLS can be statically resolved. We + // automatically upgrade dynamic models to InitialExec because the dynamic + // TLS sequences are slower than necessary and interact poorly with LFI + // rewriting combined with rewrites from linker relaxation. + if (Subtarget.isLFI()) + if (model == TLSModel::GeneralDynamic || model == TLSModel::LocalDynamic) + model = TLSModel::InitialExec; switch (model) { case TLSModel::GeneralDynamic: if (Subtarget.is64Bit()) { diff --git a/llvm/lib/Target/X86/X86ISelLoweringCall.cpp b/llvm/lib/Target/X86/X86ISelLoweringCall.cpp index c42d81cd0d886..819e0a023c1c5 100644 --- a/llvm/lib/Target/X86/X86ISelLoweringCall.cpp +++ b/llvm/lib/Target/X86/X86ISelLoweringCall.cpp @@ -3007,6 +3007,11 @@ bool X86TargetLowering::isEligibleForSiblingCallOpt( if (IsCalleeWin64 != IsCallerWin64) return false; + // Do not optimize vararg calls with 6 arguments for LFI since LFI reserves + // %r11, meaning there will not be enough registers available. + if (Subtarget.isLFI() && ArgLocs.size() > 5) + return false; + // If we are using a GOT, don't generate sibling calls to non-local, // default-visibility symbols. Tail calling such a symbol requires using a GOT // relocation, which forces early binding of the symbol. This breaks code that diff --git a/llvm/lib/Target/X86/X86RegisterInfo.cpp b/llvm/lib/Target/X86/X86RegisterInfo.cpp index c84e0f441a459..c4fbf40919677 100644 --- a/llvm/lib/Target/X86/X86RegisterInfo.cpp +++ b/llvm/lib/Target/X86/X86RegisterInfo.cpp @@ -673,6 +673,16 @@ BitVector X86RegisterInfo::getReservedRegs(const MachineFunction &MF) const { Reserved.set(*AI); } + // Reserve registers for LFI sandboxing. + if (MF.getSubtarget<X86Subtarget>().isLFI()) { + for (MCRegAliasIterator AI(X86::R11, this, true); AI.isValid(); ++AI) + Reserved.set(*AI); + for (MCRegAliasIterator AI(X86::R14, this, true); AI.isValid(); ++AI) + Reserved.set(*AI); + for (MCRegAliasIterator AI(X86::R15, this, true); AI.isValid(); ++AI) + Reserved.set(*AI); + } + assert(checkAllSuperRegsMarked(Reserved, {X86::SIL, X86::DIL, X86::BPL, X86::SPL, X86::SIH, X86::DIH, X86::BPH, X86::SPH})); diff --git a/llvm/lib/Target/X86/X86Subtarget.h b/llvm/lib/Target/X86/X86Subtarget.h index c01874e24ed34..6cfea56457910 100644 --- a/llvm/lib/Target/X86/X86Subtarget.h +++ b/llvm/lib/Target/X86/X86Subtarget.h @@ -309,6 +309,8 @@ class X86Subtarget final : public X86GenSubtargetInfo { bool isTargetMCU() const { return TargetTriple.isOSIAMCU(); } bool isTargetFuchsia() const { return TargetTriple.isOSFuchsia(); } + bool isLFI() const { return TargetTriple.isLFI(); } + bool isTargetWindowsMSVC() const { return TargetTriple.isWindowsMSVCEnvironment(); } diff --git a/llvm/lib/TargetParser/Triple.cpp b/llvm/lib/TargetParser/Triple.cpp index 603227e891d4b..e63159056e03a 100644 --- a/llvm/lib/TargetParser/Triple.cpp +++ b/llvm/lib/TargetParser/Triple.cpp @@ -193,6 +193,10 @@ StringRef Triple::getArchName(ArchType Kind, SubArchType SubArch) { if (SubArch == AArch64SubArch_lfi) return "aarch64_lfi"; break; + case Triple::x86_64: + if (SubArch == X86_64SubArch_lfi) + return "x86_64_lfi"; + break; case Triple::spirv: switch (SubArch) { case Triple::SPIRVSubArch_v10: @@ -812,7 +816,7 @@ Triple::ArchType Triple::parseArch(StringRef ArchName) { .Cases({"i386", "i486", "i586", "i686"}, Triple::x86) // FIXME: Do we need to support these? .Cases({"i786", "i886", "i986"}, Triple::x86) - .Cases({"amd64", "x86_64", "x86_64h"}, Triple::x86_64) + .Cases({"amd64", "x86_64", "x86_64h", "x86_64_lfi"}, Triple::x86_64) .Cases({"powerpc", "powerpcspe", "ppc", "ppc32"}, Triple::ppc) .Cases({"powerpcle", "ppcle", "ppc32le"}, Triple::ppcle) .Cases({"powerpc64", "ppu", "ppc64"}, Triple::ppc64) @@ -1072,6 +1076,9 @@ static Triple::SubArchType parseSubArch(StringRef SubArchName) { if (SubArchName == "aarch64_lfi") return Triple::AArch64SubArch_lfi; + if (SubArchName == "x86_64_lfi") + return Triple::X86_64SubArch_lfi; + if (SubArchName.starts_with("spirv")) return StringSwitch<Triple::SubArchType>(SubArchName) .EndsWith("v1.0", Triple::SPIRVSubArch_v10) diff --git a/llvm/test/CodeGen/X86/lfi-sibcall.ll b/llvm/test/CodeGen/X86/lfi-sibcall.ll new file mode 100644 index 0000000000000..84be6594a2836 --- /dev/null +++ b/llvm/test/CodeGen/X86/lfi-sibcall.ll @@ -0,0 +1,21 @@ +; RUN: llc < %s -mtriple=x86_64_lfi | FileCheck %s + +; LFI reserves %r11. For indirect vararg tail calls that consume all six +; argument GPRs, the function pointer would normally be loaded into %r11. With +; %r11 reserved there is no free register for the call target, so sibling-call +; optimization must be disabled. + +define void @caller6_indirect_vararg(ptr %fn, i64 %a, i64 %b, i64 %c, i64 %d, i64 %e, i64 %f) { +; CHECK-LABEL: caller6_indirect_vararg: +; CHECK: callq * +; CHECK-NOT: TAILCALL + tail call void (i64, i64, i64, i64, i64, i64, ...) %fn(i64 %a, i64 %b, i64 %c, i64 %d, i64 %e, i64 %f) + ret void +} + +define void @caller5_indirect_vararg(ptr %fn, i64 %a, i64 %b, i64 %c, i64 %d, i64 %e) { +; CHECK-LABEL: caller5_indirect_vararg: +; CHECK: jmpq *{{.*}} # TAILCALL + tail call void (i64, i64, i64, i64, i64, ...) %fn(i64 %a, i64 %b, i64 %c, i64 %d, i64 %e) + ret void +} diff --git a/llvm/test/MC/X86/LFI/abi-note.s b/llvm/test/MC/X86/LFI/abi-note.s new file mode 100644 index 0000000000000..92fb457873a63 --- /dev/null +++ b/llvm/test/MC/X86/LFI/abi-note.s @@ -0,0 +1,15 @@ +// RUN: llvm-mc -triple x86_64_lfi %s | FileCheck %s +// RUN: llvm-mc -filetype=obj -triple x86_64_lfi %s | llvm-readelf -S - | FileCheck %s --check-prefix=ELF + +// CHECK: .section .note.LFI.ABI.x86_64,"aG",@note,.note.LFI.ABI.x86_64,comdat +// CHECK-NEXT: .long 4 +// CHECK-NEXT: .long 7 +// CHECK-NEXT: .long 1 +// CHECK-NEXT: .ascii "LFI" +// CHECK-NEXT: .byte 0 +// CHECK-NEXT: .p2align 2, 0x0 +// CHECK-NEXT: .ascii "x86_64" +// CHECK-NEXT: .byte 0 +// CHECK-NEXT: .p2align 2, 0x0 + +// ELF: .note.LFI.ABI.x86_64 NOTE {{.*}} AG diff --git a/llvm/test/MC/X86/LFI/syscall.s b/llvm/test/MC/X86/LFI/syscall.s new file mode 100644 index 0000000000000..b184bd481df17 --- /dev/null +++ b/llvm/test/MC/X86/LFI/syscall.s @@ -0,0 +1,6 @@ +// RUN: llvm-mc -triple x86_64_lfi %s | FileCheck %s + +syscall +// CHECK: leaq .Ltmp0(%rip), %r11 +// CHECK-NEXT: jmpq *-8(%r14) +// CHECK-NEXT: .Ltmp0: diff --git a/llvm/test/MC/X86/LFI/thread-pointer-errors.s b/llvm/test/MC/X86/LFI/thread-pointer-errors.s new file mode 100644 index 0000000000000..99cacd69fedee --- /dev/null +++ b/llvm/test/MC/X86/LFI/thread-pointer-errors.s @@ -0,0 +1,19 @@ +// RUN: not llvm-mc -triple x86_64_lfi %s 2>&1 | FileCheck %s + +movq %fs:(%rsp), %rax +// CHECK: error: unsupported addressing mode for %fs access + +movq %fs:(%rsp,%rcx,1), %rax +// CHECK: error: unsupported addressing mode for %fs access + +movq %fs:foo(%rip), %rax +// CHECK: error: unsupported addressing mode for %fs access + +movl %fs:(%edi), %eax +// CHECK: error: unsupported addressing mode for %fs access + +cmpq %fs:(%rdi), %r11 +// CHECK: error: %fs access reads reserved register %r11 + +movq %r11, %fs:(%rdi) +// CHECK: error: %fs access reads reserved register %r11 diff --git a/llvm/test/MC/X86/LFI/thread-pointer.s b/llvm/test/MC/X86/LFI/thread-pointer.s new file mode 100644 index 0000000000000..bc03b1e601af7 --- /dev/null +++ b/llvm/test/MC/X86/LFI/thread-pointer.s @@ -0,0 +1,96 @@ +// RUN: llvm-mc -triple x86_64_lfi %s | FileCheck %s + +movq %fs:0, %rax +// CHECK: movq 16(%r15), %rax + +movq %fs:0, %rdi +// CHECK: movq 16(%r15), %rdi + +movq %fs:0, %rcx +// CHECK: movq 16(%r15), %rcx + +addq %fs:0, %rax +// CHECK: addq 16(%r15), %rax + +movq %fs:(%rdi), %rax +// CHECK: movq 16(%r15), %rax +// CHECK-NEXT: movq (%rax,%rdi), %rax + +movq %fs:(%rcx), %rdx +// CHECK: movq 16(%r15), %rdx +// CHECK-NEXT: movq (%rdx,%rcx), %rdx + +// base == dest, falls back to %r11 +movq %fs:(%rax), %rax +// CHECK: movq 16(%r15), %r11 +// CHECK-NEXT: movq (%r11,%rax), %rax + +movq %rax, %fs:(%rdi) +// CHECK: movq 16(%r15), %r11 +// CHECK-NEXT: movq %rax, (%r11,%rdi) + +movq %fs:8(%rdi,%rsi,2), %rax +// CHECK: movq 16(%r15), %rax +// CHECK-NEXT: leaq (%rax,%rdi), %rax +// CHECK-NEXT: movq 8(%rax,%rsi,2), %rax + +movq %fs:(%rax,%rbx,4), %rcx +// CHECK: movq 16(%r15), %rcx +// CHECK-NEXT: leaq (%rcx,%rax), %rcx +// CHECK-NEXT: movq (%rcx,%rbx,4), %rcx + +movq %rax, %fs:8(%rdi,%rsi,2) +// CHECK: movq 16(%r15), %r11 +// CHECK-NEXT: leaq (%r11,%rdi), %r11 +// CHECK-NEXT: movq %rax, 8(%r11,%rsi,2) + +movq %fs:foo, %rax +// CHECK: movq 16(%r15), %rax +// CHECK-NEXT: movq foo(%rax), %rax + +movq %fs:foo@TPOFF, %rax +// CHECK: movq 16(%r15), %rax +// CHECK-NEXT: movq foo@TPOFF(%rax), %rax + +movq %fs:foo(%rdi), %rax +// CHECK: movq 16(%r15), %rax +// CHECK-NEXT: movq foo(%rax,%rdi), %rax + +movq %fs:foo@TPOFF(%rdi,%rsi,2), %rax +// CHECK: movq 16(%r15), %rax +// CHECK-NEXT: leaq (%rax,%rdi), %rax +// CHECK-NEXT: movq foo@TPOFF(%rax,%rsi,2), %rax + +movq %rcx, %fs:foo +// CHECK: movq 16(%r15), %r11 +// CHECK-NEXT: movq %rcx, foo(%r11) + +cmpq %fs:(%rdi), %rax +// CHECK: movq 16(%r15), %r11 +// CHECK-NEXT: cmpq (%r11,%rdi), %rax + +cmpq %fs:(%rbx,%rcx), %rax +// CHECK: movq 16(%r15), %r11 +// CHECK-NEXT: leaq (%r11,%rbx), %r11 +// CHECK-NEXT: cmpq (%r11,%rcx), %rax + +cmpq %fs:8(%rdi,%rsi,2), %rax +// CHECK: movq 16(%r15), %r11 +// CHECK-NEXT: leaq (%r11,%rdi), %r11 +// CHECK-NEXT: cmpq 8(%r11,%rsi,2), %rax + +subq %fs:(%rdi), %rax +// CHECK: movq 16(%r15), %r11 +// CHECK-NEXT: subq (%r11,%rdi), %rax + +cmoveq %fs:(%rdi), %rax +// CHECK: movq 16(%r15), %r11 +// CHECK-NEXT: cmoveq (%r11,%rdi), %rax + +andnq %fs:(%rdi), %rax, %rbx +// CHECK: movq 16(%r15), %rbx +// CHECK-NEXT: andnq (%rbx,%rdi), %rax, %rbx + +andnq %fs:(%rdi), %rax, %rax +// CHECK: movq 16(%r15), %r11 +// CHECK-NEXT: andnq (%r11,%rdi), %rax, %rax _______________________________________________ cfe-commits mailing list [email protected] https://lists.llvm.org/cgi-bin/mailman/listinfo/cfe-commits
