Re: [Xen-devel] [PATCH v1 00/27] x86: PIE support and option to extend KASLR randomization
On Thu, Oct 12, 2017 at 9:28 AM, Tom Lendackywrote: > On 10/12/2017 10:34 AM, Thomas Garnier wrote: >> >> On Wed, Oct 11, 2017 at 2:34 PM, Tom Lendacky >> wrote: >>> >>> On 10/11/2017 3:30 PM, Thomas Garnier wrote: Changes: - patch v1: - Simplify ftrace implementation. - Use gcc mstack-protector-guard-reg=%gs with PIE when possible. - rfc v3: - Use --emit-relocs instead of -pie to reduce dynamic relocation space on mapped memory. It also simplifies the relocation process. - Move the start the module section next to the kernel. Remove the need for -mcmodel=large on modules. Extends module space from 1 to 2G maximum. - Support for XEN PVH as 32-bit relocations can be ignored with --emit-relocs. - Support for GOT relocations previously done automatically with -pie. - Remove need for dynamic PLT in modules. - Support dymamic GOT for modules. - rfc v2: - Add support for global stack cookie while compiler default to fs without mcmodel=kernel - Change patch 7 to correctly jump out of the identity mapping on kexec load preserve. These patches make the changes necessary to build the kernel as Position Independent Executable (PIE) on x86_64. A PIE kernel can be relocated below the top 2G of the virtual address space. It allows to optionally extend the KASLR randomization range from 1G to 3G. >>> >>> >>> Hi Thomas, >>> >>> I've applied your patches so that I can verify that SME works with PIE. >>> Unfortunately, I'm running into build warnings and errors when I enable >>> PIE. >>> >>> With CONFIG_STACK_VALIDATION=y I receive lots of messages like this: >>> >>>drivers/scsi/libfc/fc_exch.o: warning: objtool: >>> fc_destroy_exch_mgr()+0x0: call without frame pointer save/setup >>> >>> Disabling CONFIG_STACK_VALIDATION suppresses those. >> >> >> I ran into that, I plan to fix it in the next iteration. >> >>> >>> But near the end of the build, I receive errors like this: >>> >>>arch/x86/kernel/setup.o: In function `dump_kernel_offset': >>>.../arch/x86/kernel/setup.c:801:(.text+0x32): relocation truncated to >>> fit: R_X86_64_32S against symbol `_text' defined in .text section in >>> .tmp_vmlinux1 >>>. >>>. about 10 more of the above type messages >>>. >>>make: *** [vmlinux] Error 1 >>>Error building kernel, exiting >>> >>> Are there any config options that should or should not be enabled when >>> building with PIE enabled? Is there a compiler requirement for PIE (I'm >>> using gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5))? >> >> >> I never ran into these ones and I tested compilers older and newer. >> What was your exact configuration? > > > I'll send you the config in a separate email. > > Thanks, > Tom Thanks for your feedback (Tom and Markus). The issue was linked to using a modern gcc with a modern linker, I managed to repro and fix it on my current version. I will create a v1.5 for Kees Cook to keep on one of his branch for few weeks so I can collect as much feedback from 0day. After that I will send v2. > > >> >>> >>> Thanks, >>> Tom >>> Thanks a lot to Ard Biesheuvel & Kees Cook on their feedback on compiler changes, PIE support and KASLR in general. Thanks to Roland McGrath on his feedback for using -pie versus --emit-relocs and details on compiler code generation. The patches: - 1-3, 5-1#, 17-18: Change in assembly code to be PIE compliant. - 4: Add a new _ASM_GET_PTR macro to fetch a symbol address generically. - 14: Adapt percpu design to work correctly when PIE is enabled. - 15: Provide an option to default visibility to hidden except for key symbols. It removes errors between compilation units. - 16: Adapt relocation tool to handle PIE binary correctly. - 19: Add support for global cookie. - 20: Support ftrace with PIE (used on Ubuntu config). - 21: Fix incorrect address marker on dump_pagetables. - 22: Add option to move the module section just after the kernel. - 23: Adapt module loading to support PIE with dynamic GOT. - 24: Make the GOT read-only. - 25: Add the CONFIG_X86_PIE option (off by default). - 26: Adapt relocation tool to generate a 64-bit relocation table. - 27: Add the CONFIG_RANDOMIZE_BASE_LARGE option to increase relocation range from 1G to 3G (off by default). Performance/Size impact: Size of vmlinux (Default configuration): File size: - PIE disabled: +0.31% - PIE enabled: -3.210% (less relocations) .text section: - PIE disabled: +0.000644% - PIE enabled: +0.837%
Re: [Xen-devel] [PATCH v1 00/27] x86: PIE support and option to extend KASLR randomization
On 10/12/2017 10:34 AM, Thomas Garnier wrote: On Wed, Oct 11, 2017 at 2:34 PM, Tom Lendackywrote: On 10/11/2017 3:30 PM, Thomas Garnier wrote: Changes: - patch v1: - Simplify ftrace implementation. - Use gcc mstack-protector-guard-reg=%gs with PIE when possible. - rfc v3: - Use --emit-relocs instead of -pie to reduce dynamic relocation space on mapped memory. It also simplifies the relocation process. - Move the start the module section next to the kernel. Remove the need for -mcmodel=large on modules. Extends module space from 1 to 2G maximum. - Support for XEN PVH as 32-bit relocations can be ignored with --emit-relocs. - Support for GOT relocations previously done automatically with -pie. - Remove need for dynamic PLT in modules. - Support dymamic GOT for modules. - rfc v2: - Add support for global stack cookie while compiler default to fs without mcmodel=kernel - Change patch 7 to correctly jump out of the identity mapping on kexec load preserve. These patches make the changes necessary to build the kernel as Position Independent Executable (PIE) on x86_64. A PIE kernel can be relocated below the top 2G of the virtual address space. It allows to optionally extend the KASLR randomization range from 1G to 3G. Hi Thomas, I've applied your patches so that I can verify that SME works with PIE. Unfortunately, I'm running into build warnings and errors when I enable PIE. With CONFIG_STACK_VALIDATION=y I receive lots of messages like this: drivers/scsi/libfc/fc_exch.o: warning: objtool: fc_destroy_exch_mgr()+0x0: call without frame pointer save/setup Disabling CONFIG_STACK_VALIDATION suppresses those. I ran into that, I plan to fix it in the next iteration. But near the end of the build, I receive errors like this: arch/x86/kernel/setup.o: In function `dump_kernel_offset': .../arch/x86/kernel/setup.c:801:(.text+0x32): relocation truncated to fit: R_X86_64_32S against symbol `_text' defined in .text section in .tmp_vmlinux1 . . about 10 more of the above type messages . make: *** [vmlinux] Error 1 Error building kernel, exiting Are there any config options that should or should not be enabled when building with PIE enabled? Is there a compiler requirement for PIE (I'm using gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5))? I never ran into these ones and I tested compilers older and newer. What was your exact configuration? I'll send you the config in a separate email. Thanks, Tom Thanks, Tom Thanks a lot to Ard Biesheuvel & Kees Cook on their feedback on compiler changes, PIE support and KASLR in general. Thanks to Roland McGrath on his feedback for using -pie versus --emit-relocs and details on compiler code generation. The patches: - 1-3, 5-1#, 17-18: Change in assembly code to be PIE compliant. - 4: Add a new _ASM_GET_PTR macro to fetch a symbol address generically. - 14: Adapt percpu design to work correctly when PIE is enabled. - 15: Provide an option to default visibility to hidden except for key symbols. It removes errors between compilation units. - 16: Adapt relocation tool to handle PIE binary correctly. - 19: Add support for global cookie. - 20: Support ftrace with PIE (used on Ubuntu config). - 21: Fix incorrect address marker on dump_pagetables. - 22: Add option to move the module section just after the kernel. - 23: Adapt module loading to support PIE with dynamic GOT. - 24: Make the GOT read-only. - 25: Add the CONFIG_X86_PIE option (off by default). - 26: Adapt relocation tool to generate a 64-bit relocation table. - 27: Add the CONFIG_RANDOMIZE_BASE_LARGE option to increase relocation range from 1G to 3G (off by default). Performance/Size impact: Size of vmlinux (Default configuration): File size: - PIE disabled: +0.31% - PIE enabled: -3.210% (less relocations) .text section: - PIE disabled: +0.000644% - PIE enabled: +0.837% Size of vmlinux (Ubuntu configuration): File size: - PIE disabled: -0.201% - PIE enabled: -0.082% .text section: - PIE disabled: same - PIE enabled: +1.319% Size of vmlinux (Default configuration + ORC): File size: - PIE enabled: -3.167% .text section: - PIE enabled: +0.814% Size of vmlinux (Ubuntu configuration + ORC): File size: - PIE enabled: -3.167% .text section: - PIE enabled: +1.26% The size increase is mainly due to not having access to the 32-bit signed relocation that can be used with mcmodel=kernel. A small part is due to reduced optimization for PIE code. This bug [1] was opened with gcc to provide a better code generation for kernel PIE. Hackbench (50% and 1600% on thread/process for pipe/sockets): - PIE disabled: no significant change (avg +0.1% on latest test). - PIE enabled: between -0.50% to +0.86% in average (default and Ubuntu
Re: [Xen-devel] [PATCH v1 00/27] x86: PIE support and option to extend KASLR randomization
On 2017.10.12 at 08:34 -0700, Thomas Garnier wrote: > On Wed, Oct 11, 2017 at 2:34 PM, Tom Lendackywrote: > > On 10/11/2017 3:30 PM, Thomas Garnier wrote: > >> Changes: > >> - patch v1: > >> - Simplify ftrace implementation. > >> - Use gcc mstack-protector-guard-reg=%gs with PIE when possible. > >> - rfc v3: > >> - Use --emit-relocs instead of -pie to reduce dynamic relocation space > >> on > >> mapped memory. It also simplifies the relocation process. > >> - Move the start the module section next to the kernel. Remove the > >> need for > >> -mcmodel=large on modules. Extends module space from 1 to 2G maximum. > >> - Support for XEN PVH as 32-bit relocations can be ignored with > >> --emit-relocs. > >> - Support for GOT relocations previously done automatically with -pie. > >> - Remove need for dynamic PLT in modules. > >> - Support dymamic GOT for modules. > >> - rfc v2: > >> - Add support for global stack cookie while compiler default to fs > >> without > >> mcmodel=kernel > >> - Change patch 7 to correctly jump out of the identity mapping on > >> kexec load > >> preserve. > >> > >> These patches make the changes necessary to build the kernel as Position > >> Independent Executable (PIE) on x86_64. A PIE kernel can be relocated below > >> the top 2G of the virtual address space. It allows to optionally extend the > >> KASLR randomization range from 1G to 3G. > > > > Hi Thomas, > > > > I've applied your patches so that I can verify that SME works with PIE. > > Unfortunately, I'm running into build warnings and errors when I enable > > PIE. > > > > With CONFIG_STACK_VALIDATION=y I receive lots of messages like this: > > > > drivers/scsi/libfc/fc_exch.o: warning: objtool: > > fc_destroy_exch_mgr()+0x0: call without frame pointer save/setup > > > > Disabling CONFIG_STACK_VALIDATION suppresses those. > > I ran into that, I plan to fix it in the next iteration. > > > > > But near the end of the build, I receive errors like this: > > > > arch/x86/kernel/setup.o: In function `dump_kernel_offset': > > .../arch/x86/kernel/setup.c:801:(.text+0x32): relocation truncated to > > fit: R_X86_64_32S against symbol `_text' defined in .text section in > > .tmp_vmlinux1 > > . > > . about 10 more of the above type messages > > . > > make: *** [vmlinux] Error 1 > > Error building kernel, exiting > > > > Are there any config options that should or should not be enabled when > > building with PIE enabled? Is there a compiler requirement for PIE (I'm > > using gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5))? > > I never ran into these ones and I tested compilers older and newer. > What was your exact configuration? I get with gcc trunk and CONFIG_RANDOMIZE_BASE_LARGE=y: ... MODPOST vmlinux.o ld: failed to convert GOTPCREL relocation; relink with --no-relax and after adding --no-relax to vmlinux_link() in scripts/link-vmlinux.sh: MODPOST vmlinux.o virt/kvm/vfio.o: In function `kvm_vfio_update_coherency.isra.4': vfio.c:(.text+0x63): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `vfio_external_check_extension' virt/kvm/vfio.o: In function `kvm_vfio_destroy': vfio.c:(.text+0xf7): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `vfio_group_set_kvm' vfio.c:(.text+0x10a): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `vfio_group_put_external_user' virt/kvm/vfio.o: In function `kvm_vfio_set_attr': vfio.c:(.text+0x2bc): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `vfio_external_group_match_file' vfio.c:(.text+0x307): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `vfio_group_set_kvm' vfio.c:(.text+0x31a): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `vfio_group_put_external_user' vfio.c:(.text+0x3b9): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `vfio_group_get_external_user' vfio.c:(.text+0x462): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `vfio_group_set_kvm' vfio.c:(.text+0x4bd): relocation truncated to fit: R_X86_64_PLT32 against undefined symbol `vfio_group_put_external_user' make: *** [Makefile:1000: vmlinux] Error 1 Works fine with CONFIG_RANDOMIZE_BASE_LARGE unset. -- Markus config.gz Description: application/gunzip ___ Xen-devel mailing list Xen-devel@lists.xen.org https://lists.xen.org/xen-devel
Re: [Xen-devel] [PATCH v1 00/27] x86: PIE support and option to extend KASLR randomization
On Wed, Oct 11, 2017 at 2:34 PM, Tom Lendackywrote: > On 10/11/2017 3:30 PM, Thomas Garnier wrote: >> Changes: >> - patch v1: >> - Simplify ftrace implementation. >> - Use gcc mstack-protector-guard-reg=%gs with PIE when possible. >> - rfc v3: >> - Use --emit-relocs instead of -pie to reduce dynamic relocation space on >> mapped memory. It also simplifies the relocation process. >> - Move the start the module section next to the kernel. Remove the need >> for >> -mcmodel=large on modules. Extends module space from 1 to 2G maximum. >> - Support for XEN PVH as 32-bit relocations can be ignored with >> --emit-relocs. >> - Support for GOT relocations previously done automatically with -pie. >> - Remove need for dynamic PLT in modules. >> - Support dymamic GOT for modules. >> - rfc v2: >> - Add support for global stack cookie while compiler default to fs >> without >> mcmodel=kernel >> - Change patch 7 to correctly jump out of the identity mapping on kexec >> load >> preserve. >> >> These patches make the changes necessary to build the kernel as Position >> Independent Executable (PIE) on x86_64. A PIE kernel can be relocated below >> the top 2G of the virtual address space. It allows to optionally extend the >> KASLR randomization range from 1G to 3G. > > Hi Thomas, > > I've applied your patches so that I can verify that SME works with PIE. > Unfortunately, I'm running into build warnings and errors when I enable > PIE. > > With CONFIG_STACK_VALIDATION=y I receive lots of messages like this: > > drivers/scsi/libfc/fc_exch.o: warning: objtool: fc_destroy_exch_mgr()+0x0: > call without frame pointer save/setup > > Disabling CONFIG_STACK_VALIDATION suppresses those. I ran into that, I plan to fix it in the next iteration. > > But near the end of the build, I receive errors like this: > > arch/x86/kernel/setup.o: In function `dump_kernel_offset': > .../arch/x86/kernel/setup.c:801:(.text+0x32): relocation truncated to fit: > R_X86_64_32S against symbol `_text' defined in .text section in .tmp_vmlinux1 > . > . about 10 more of the above type messages > . > make: *** [vmlinux] Error 1 > Error building kernel, exiting > > Are there any config options that should or should not be enabled when > building with PIE enabled? Is there a compiler requirement for PIE (I'm > using gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5))? I never ran into these ones and I tested compilers older and newer. What was your exact configuration? > > Thanks, > Tom > >> >> Thanks a lot to Ard Biesheuvel & Kees Cook on their feedback on compiler >> changes, PIE support and KASLR in general. Thanks to Roland McGrath on his >> feedback for using -pie versus --emit-relocs and details on compiler code >> generation. >> >> The patches: >> - 1-3, 5-1#, 17-18: Change in assembly code to be PIE compliant. >> - 4: Add a new _ASM_GET_PTR macro to fetch a symbol address generically. >> - 14: Adapt percpu design to work correctly when PIE is enabled. >> - 15: Provide an option to default visibility to hidden except for key >> symbols. >> It removes errors between compilation units. >> - 16: Adapt relocation tool to handle PIE binary correctly. >> - 19: Add support for global cookie. >> - 20: Support ftrace with PIE (used on Ubuntu config). >> - 21: Fix incorrect address marker on dump_pagetables. >> - 22: Add option to move the module section just after the kernel. >> - 23: Adapt module loading to support PIE with dynamic GOT. >> - 24: Make the GOT read-only. >> - 25: Add the CONFIG_X86_PIE option (off by default). >> - 26: Adapt relocation tool to generate a 64-bit relocation table. >> - 27: Add the CONFIG_RANDOMIZE_BASE_LARGE option to increase relocation >> range >> from 1G to 3G (off by default). >> >> Performance/Size impact: >> >> Size of vmlinux (Default configuration): >> File size: >> - PIE disabled: +0.31% >> - PIE enabled: -3.210% (less relocations) >> .text section: >> - PIE disabled: +0.000644% >> - PIE enabled: +0.837% >> >> Size of vmlinux (Ubuntu configuration): >> File size: >> - PIE disabled: -0.201% >> - PIE enabled: -0.082% >> .text section: >> - PIE disabled: same >> - PIE enabled: +1.319% >> >> Size of vmlinux (Default configuration + ORC): >> File size: >> - PIE enabled: -3.167% >> .text section: >> - PIE enabled: +0.814% >> >> Size of vmlinux (Ubuntu configuration + ORC): >> File size: >> - PIE enabled: -3.167% >> .text section: >> - PIE enabled: +1.26% >> >> The size increase is mainly due to not having access to the 32-bit signed >> relocation that can be used with mcmodel=kernel. A small part is due to >> reduced >> optimization for PIE code. This bug [1] was opened with gcc to provide a >> better >> code generation for kernel PIE. >> >> Hackbench (50% and 1600% on thread/process for pipe/sockets): >>
Re: [Xen-devel] [PATCH v1 00/27] x86: PIE support and option to extend KASLR randomization
On 10/11/2017 3:30 PM, Thomas Garnier wrote: > Changes: > - patch v1: > - Simplify ftrace implementation. > - Use gcc mstack-protector-guard-reg=%gs with PIE when possible. > - rfc v3: > - Use --emit-relocs instead of -pie to reduce dynamic relocation space on > mapped memory. It also simplifies the relocation process. > - Move the start the module section next to the kernel. Remove the need > for > -mcmodel=large on modules. Extends module space from 1 to 2G maximum. > - Support for XEN PVH as 32-bit relocations can be ignored with > --emit-relocs. > - Support for GOT relocations previously done automatically with -pie. > - Remove need for dynamic PLT in modules. > - Support dymamic GOT for modules. > - rfc v2: > - Add support for global stack cookie while compiler default to fs without > mcmodel=kernel > - Change patch 7 to correctly jump out of the identity mapping on kexec > load > preserve. > > These patches make the changes necessary to build the kernel as Position > Independent Executable (PIE) on x86_64. A PIE kernel can be relocated below > the top 2G of the virtual address space. It allows to optionally extend the > KASLR randomization range from 1G to 3G. Hi Thomas, I've applied your patches so that I can verify that SME works with PIE. Unfortunately, I'm running into build warnings and errors when I enable PIE. With CONFIG_STACK_VALIDATION=y I receive lots of messages like this: drivers/scsi/libfc/fc_exch.o: warning: objtool: fc_destroy_exch_mgr()+0x0: call without frame pointer save/setup Disabling CONFIG_STACK_VALIDATION suppresses those. But near the end of the build, I receive errors like this: arch/x86/kernel/setup.o: In function `dump_kernel_offset': .../arch/x86/kernel/setup.c:801:(.text+0x32): relocation truncated to fit: R_X86_64_32S against symbol `_text' defined in .text section in .tmp_vmlinux1 . . about 10 more of the above type messages . make: *** [vmlinux] Error 1 Error building kernel, exiting Are there any config options that should or should not be enabled when building with PIE enabled? Is there a compiler requirement for PIE (I'm using gcc version 5.4.0 20160609 (Ubuntu 5.4.0-6ubuntu1~16.04.5))? Thanks, Tom > > Thanks a lot to Ard Biesheuvel & Kees Cook on their feedback on compiler > changes, PIE support and KASLR in general. Thanks to Roland McGrath on his > feedback for using -pie versus --emit-relocs and details on compiler code > generation. > > The patches: > - 1-3, 5-1#, 17-18: Change in assembly code to be PIE compliant. > - 4: Add a new _ASM_GET_PTR macro to fetch a symbol address generically. > - 14: Adapt percpu design to work correctly when PIE is enabled. > - 15: Provide an option to default visibility to hidden except for key > symbols. > It removes errors between compilation units. > - 16: Adapt relocation tool to handle PIE binary correctly. > - 19: Add support for global cookie. > - 20: Support ftrace with PIE (used on Ubuntu config). > - 21: Fix incorrect address marker on dump_pagetables. > - 22: Add option to move the module section just after the kernel. > - 23: Adapt module loading to support PIE with dynamic GOT. > - 24: Make the GOT read-only. > - 25: Add the CONFIG_X86_PIE option (off by default). > - 26: Adapt relocation tool to generate a 64-bit relocation table. > - 27: Add the CONFIG_RANDOMIZE_BASE_LARGE option to increase relocation > range > from 1G to 3G (off by default). > > Performance/Size impact: > > Size of vmlinux (Default configuration): > File size: > - PIE disabled: +0.31% > - PIE enabled: -3.210% (less relocations) > .text section: > - PIE disabled: +0.000644% > - PIE enabled: +0.837% > > Size of vmlinux (Ubuntu configuration): > File size: > - PIE disabled: -0.201% > - PIE enabled: -0.082% > .text section: > - PIE disabled: same > - PIE enabled: +1.319% > > Size of vmlinux (Default configuration + ORC): > File size: > - PIE enabled: -3.167% > .text section: > - PIE enabled: +0.814% > > Size of vmlinux (Ubuntu configuration + ORC): > File size: > - PIE enabled: -3.167% > .text section: > - PIE enabled: +1.26% > > The size increase is mainly due to not having access to the 32-bit signed > relocation that can be used with mcmodel=kernel. A small part is due to > reduced > optimization for PIE code. This bug [1] was opened with gcc to provide a > better > code generation for kernel PIE. > > Hackbench (50% and 1600% on thread/process for pipe/sockets): > - PIE disabled: no significant change (avg +0.1% on latest test). > - PIE enabled: between -0.50% to +0.86% in average (default and Ubuntu > config). > > slab_test (average of 10 runs): > - PIE disabled: no significant change (-2% on latest run, likely noise). > - PIE enabled: between -1% and +0.8% on latest runs. > > Kernbench (average of 10 Half and Optimal runs): >