Re: [PATCH 15/56] KVM: arm64: Add build rules for separate VHE/nVHE object files
Hi Eric, On Tue, 18 May 2021 12:48:41 +0100, Auger Eric wrote: > > Hi David, Marc, > [...] > >> --- a/scripts/kallsyms.c > >> +++ b/scripts/kallsyms.c > >> @@ -109,6 +109,7 @@ static bool is_ignored_symbol(const char *name, char > >> type) > >>".LASANPC", /* s390 kasan local symbols */ > >>"__crc_", /* modversions */ > >>"__efistub_", /* arm64 EFI stub namespace */ > >> + "__kvm_nvhe_", /* arm64 non-VHE KVM namespace */ > > The addition of this line seems to have introduced errors on the > > 'vmlinux symtab matches kallsyms' perf test (perf test -v 1) which fails > > on aarch64 for all __kvm_nvhe_ prefixed symbols, like > > > > ERR : : __kvm_nvhe___invalid not on kallsyms > > ERR : : __kvm_nvhe___do_hyp_init not on kallsyms > > ERR : : __kvm_nvhe___kvm_handle_stub_hvc not on kallsyms > > ERR : : __kvm_nvhe_reset not on kallsyms > > ../.. > > > > I understand we willingly hided those symbols from /proc/kallsyms. Do > > you confirm the right fix is to upgrade the perf test suite accordingly? > > Were you eventually able to reproduce? Unfortunately not, I always end-up with the test failing even if I add the symbols back. I must be doing something wrong... M. -- Without deviation from the norm, progress is not possible. ___ kvmarm mailing list kvmarm@lists.cs.columbia.edu https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 15/56] KVM: arm64: Add build rules for separate VHE/nVHE object files
Hi David, Marc,
On 5/4/21 4:47 PM, Auger Eric wrote:
> Hi David, Marc,
>
> On 8/5/20 7:56 PM, Marc Zyngier wrote:
>> From: David Brazdil
>>
>> Add new folders arch/arm64/kvm/hyp/{vhe,nvhe} and Makefiles for building code
>> that runs in EL2 under VHE/nVHE KVM, repsectivelly. Add an include folder for
>> hyp-specific header files which will include code common to VHE/nVHE.
>>
>> Build nVHE code with -D__KVM_NVHE_HYPERVISOR__, VHE code with
>> -D__KVM_VHE_HYPERVISOR__.
>>
>> Under nVHE compile each source file into a `.hyp.tmp.o` object first, then
>> prefix all its symbols with "__kvm_nvhe_" using `objcopy` and produce
>> a `.hyp.o`. Suffixes were chosen so that it would be possible for VHE and
>> nVHE
>> to share some source files, but compiled with different CFLAGS.
>>
>> The nVHE ELF symbol prefix is added to kallsyms.c as ignored. EL2-only
>> symbols
>> will never appear in EL1 stack traces.
>>
>> Due to symbol prefixing, add a section in image-vars.h for aliases of symbols
>> that are defined in nVHE EL2 and accessed by kernel in EL1 or vice versa.
>>
>> Signed-off-by: David Brazdil
>> Signed-off-by: Marc Zyngier
>> Link: https://lore.kernel.org/r/20200625131420.71444-4-dbraz...@google.com
>> ---
>> arch/arm64/kernel/image-vars.h | 14 +
>> arch/arm64/kvm/hyp/Makefile | 10 +++---
>> arch/arm64/kvm/hyp/nvhe/Makefile | 34
>> arch/arm64/kvm/hyp/vhe/Makefile | 17
>> scripts/kallsyms.c | 1 +
>> 5 files changed, 73 insertions(+), 3 deletions(-)
>> create mode 100644 arch/arm64/kvm/hyp/nvhe/Makefile
>> create mode 100644 arch/arm64/kvm/hyp/vhe/Makefile
>>
>> diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h
>> index be0a63ffed23..3dc27da47712 100644
>> --- a/arch/arm64/kernel/image-vars.h
>> +++ b/arch/arm64/kernel/image-vars.h
>> @@ -51,4 +51,18 @@ __efistub__ctype = _ctype;
>>
>> #endif
>>
>> +#ifdef CONFIG_KVM
>> +
>> +/*
>> + * KVM nVHE code has its own symbol namespace prefixed with __kvm_nvhe_, to
>> + * separate it from the kernel proper. The following symbols are legally
>> + * accessed by it, therefore provide aliases to make them linkable.
>> + * Do not include symbols which may not be safely accessed under hypervisor
>> + * memory mappings.
>> + */
>> +
>> +#define KVM_NVHE_ALIAS(sym) __kvm_nvhe_##sym = sym;
>> +
>> +#endif /* CONFIG_KVM */
>> +
>> #endif /* __ARM64_KERNEL_IMAGE_VARS_H */
>> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
>> index 5d8357ddc234..9c5dfe6ff80b 100644
>> --- a/arch/arm64/kvm/hyp/Makefile
>> +++ b/arch/arm64/kvm/hyp/Makefile
>> @@ -3,10 +3,14 @@
>> # Makefile for Kernel-based Virtual Machine module, HYP part
>> #
>>
>> -ccflags-y += -fno-stack-protector -DDISABLE_BRANCH_PROFILING \
>> -$(DISABLE_STACKLEAK_PLUGIN)
>> +incdir := $(srctree)/$(src)/include
>> +subdir-asflags-y := -I$(incdir)
>> +subdir-ccflags-y := -I$(incdir) \
>> +-fno-stack-protector\
>> +-DDISABLE_BRANCH_PROFILING \
>> +$(DISABLE_STACKLEAK_PLUGIN)
>>
>> -obj-$(CONFIG_KVM) += hyp.o
>> +obj-$(CONFIG_KVM) += hyp.o nvhe/
>> obj-$(CONFIG_KVM_INDIRECT_VECTORS) += smccc_wa.o
>>
>> hyp-y := vgic-v3-sr.o timer-sr.o aarch32.o vgic-v2-cpuif-proxy.o
>> sysreg-sr.o \
>> diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile
>> b/arch/arm64/kvm/hyp/nvhe/Makefile
>> new file mode 100644
>> index ..955f4188e00f
>> --- /dev/null
>> +++ b/arch/arm64/kvm/hyp/nvhe/Makefile
>> @@ -0,0 +1,34 @@
>> +# SPDX-License-Identifier: GPL-2.0
>> +#
>> +# Makefile for Kernel-based Virtual Machine module, HYP/nVHE part
>> +#
>> +
>> +asflags-y := -D__KVM_NVHE_HYPERVISOR__
>> +ccflags-y := -D__KVM_NVHE_HYPERVISOR__
>> +
>> +obj-y :=
>> +
>> +obj-y := $(patsubst %.o,%.hyp.o,$(obj-y))
>> +extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))
>> +
>> +$(obj)/%.hyp.tmp.o: $(src)/%.c FORCE
>> +$(call if_changed_rule,cc_o_c)
>> +$(obj)/%.hyp.tmp.o: $(src)/%.S FORCE
>> +$(call if_changed_rule,as_o_S)
>> +$(obj)/%.hyp.o: $(obj)/%.hyp.tmp.o FORCE
>> +$(call if_changed,hypcopy)
>> +
>> +quiet_cmd_hypcopy = HYPCOPY $@
>> + cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@
>> +
>> +# KVM nVHE code is run at a different exception code with a different map,
>> so
>> +# compiler instrumentation that inserts callbacks or checks into the code
>> may
>> +# cause crashes. Just disable it.
>> +GCOV_PROFILE:= n
>> +KASAN_SANITIZE := n
>> +UBSAN_SANITIZE := n
>> +KCOV_INSTRUMENT := n
>> +
>> +# Skip objtool checking for this directory because nVHE code is compiled
>> with
>> +# non-standard build rules.
>> +OBJECT_FILES_NON_STANDARD := y
>> diff --git a/arch/arm64/kvm/hyp/vhe/Makefile
>> b/arch/arm64/kvm/hyp/vhe/Makefile
>> new file mode 100644
>> index ..e04375546081
>> --- /dev/null
>> +++
Re: [PATCH 15/56] KVM: arm64: Add build rules for separate VHE/nVHE object files
Hi Marc,
On 5/5/21 8:03 PM, Marc Zyngier wrote:
> Hi Eric,
>
> On Tue, 04 May 2021 15:47:36 +0100,
> Auger Eric wrote:
>>
>> Hi David, Marc,
>>
>> On 8/5/20 7:56 PM, Marc Zyngier wrote:
>>> From: David Brazdil
>>>
>>> Add new folders arch/arm64/kvm/hyp/{vhe,nvhe} and Makefiles for building
>>> code
>>> that runs in EL2 under VHE/nVHE KVM, repsectivelly. Add an include folder
>>> for
>>> hyp-specific header files which will include code common to VHE/nVHE.
>>>
>>> Build nVHE code with -D__KVM_NVHE_HYPERVISOR__, VHE code with
>>> -D__KVM_VHE_HYPERVISOR__.
>>>
>>> Under nVHE compile each source file into a `.hyp.tmp.o` object first, then
>>> prefix all its symbols with "__kvm_nvhe_" using `objcopy` and produce
>>> a `.hyp.o`. Suffixes were chosen so that it would be possible for VHE and
>>> nVHE
>>> to share some source files, but compiled with different CFLAGS.
>>>
>>> The nVHE ELF symbol prefix is added to kallsyms.c as ignored. EL2-only
>>> symbols
>>> will never appear in EL1 stack traces.
>>>
>>> Due to symbol prefixing, add a section in image-vars.h for aliases of
>>> symbols
>>> that are defined in nVHE EL2 and accessed by kernel in EL1 or vice versa.
>>>
>>> Signed-off-by: David Brazdil
>>> Signed-off-by: Marc Zyngier
>>> Link: https://lore.kernel.org/r/20200625131420.71444-4-dbraz...@google.com
>>> ---
>>> arch/arm64/kernel/image-vars.h | 14 +
>>> arch/arm64/kvm/hyp/Makefile | 10 +++---
>>> arch/arm64/kvm/hyp/nvhe/Makefile | 34
>>> arch/arm64/kvm/hyp/vhe/Makefile | 17
>>> scripts/kallsyms.c | 1 +
>>> 5 files changed, 73 insertions(+), 3 deletions(-)
>>> create mode 100644 arch/arm64/kvm/hyp/nvhe/Makefile
>>> create mode 100644 arch/arm64/kvm/hyp/vhe/Makefile
>
> [...]
>
>>> diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
>>> index 6dc3078649fa..0096cd965332 100644
>>> --- a/scripts/kallsyms.c
>>> +++ b/scripts/kallsyms.c
>>> @@ -109,6 +109,7 @@ static bool is_ignored_symbol(const char *name, char
>>> type)
>>> ".LASANPC", /* s390 kasan local symbols */
>>> "__crc_", /* modversions */
>>> "__efistub_", /* arm64 EFI stub namespace */
>>> + "__kvm_nvhe_", /* arm64 non-VHE KVM namespace */
>> The addition of this line seems to have introduced errors on the
>> 'vmlinux symtab matches kallsyms' perf test (perf test -v 1) which fails
>> on aarch64 for all __kvm_nvhe_ prefixed symbols, like
>>
>> ERR : : __kvm_nvhe___invalid not on kallsyms
>> ERR : : __kvm_nvhe___do_hyp_init not on kallsyms
>> ERR : : __kvm_nvhe___kvm_handle_stub_hvc not on kallsyms
>> ERR : : __kvm_nvhe_reset not on kallsyms
>> ../..
>>
>> I understand we willingly hided those symbols from /proc/kallsyms. Do
>> you confirm the right fix is to upgrade the perf test suite accordingly?
>
> Hmmm. This test always fail here, no matter whether I have this line
> or not:
sorry Marc I missed your reply,
>
>
> maz@big-leg-emma:~$ sudo perf_5.10 test -v 1
> 1: vmlinux symtab matches kallsyms :
> --- start ---
> test child forked, pid 664
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> /proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
> Looking at the vmlinux_path (8 entries long)
> symsrc__init: cannot get elf header.
> symsrc__init: cannot get elf header.
> Couldn't find a vmlinux that matches the kernel running on this machine,
> skipping test
> test child finished with -2
> end
> vmlinux symtab matches kallsyms: Skip
>
>
> Rookie question: How do you provide a kernel to the test framework?
On my end I compiled and installed a kernel and compiled perf from that
same kernel. Should work. I don't know how to point to a specific vmlinux
Thanks
Eric
>
> Thanks,
>
> M.
>
___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 15/56] KVM: arm64: Add build rules for separate VHE/nVHE object files
Hi Eric,
On Tue, 04 May 2021 15:47:36 +0100,
Auger Eric wrote:
>
> Hi David, Marc,
>
> On 8/5/20 7:56 PM, Marc Zyngier wrote:
> > From: David Brazdil
> >
> > Add new folders arch/arm64/kvm/hyp/{vhe,nvhe} and Makefiles for building
> > code
> > that runs in EL2 under VHE/nVHE KVM, repsectivelly. Add an include folder
> > for
> > hyp-specific header files which will include code common to VHE/nVHE.
> >
> > Build nVHE code with -D__KVM_NVHE_HYPERVISOR__, VHE code with
> > -D__KVM_VHE_HYPERVISOR__.
> >
> > Under nVHE compile each source file into a `.hyp.tmp.o` object first, then
> > prefix all its symbols with "__kvm_nvhe_" using `objcopy` and produce
> > a `.hyp.o`. Suffixes were chosen so that it would be possible for VHE and
> > nVHE
> > to share some source files, but compiled with different CFLAGS.
> >
> > The nVHE ELF symbol prefix is added to kallsyms.c as ignored. EL2-only
> > symbols
> > will never appear in EL1 stack traces.
> >
> > Due to symbol prefixing, add a section in image-vars.h for aliases of
> > symbols
> > that are defined in nVHE EL2 and accessed by kernel in EL1 or vice versa.
> >
> > Signed-off-by: David Brazdil
> > Signed-off-by: Marc Zyngier
> > Link: https://lore.kernel.org/r/20200625131420.71444-4-dbraz...@google.com
> > ---
> > arch/arm64/kernel/image-vars.h | 14 +
> > arch/arm64/kvm/hyp/Makefile | 10 +++---
> > arch/arm64/kvm/hyp/nvhe/Makefile | 34
> > arch/arm64/kvm/hyp/vhe/Makefile | 17
> > scripts/kallsyms.c | 1 +
> > 5 files changed, 73 insertions(+), 3 deletions(-)
> > create mode 100644 arch/arm64/kvm/hyp/nvhe/Makefile
> > create mode 100644 arch/arm64/kvm/hyp/vhe/Makefile
[...]
> > diff --git a/scripts/kallsyms.c b/scripts/kallsyms.c
> > index 6dc3078649fa..0096cd965332 100644
> > --- a/scripts/kallsyms.c
> > +++ b/scripts/kallsyms.c
> > @@ -109,6 +109,7 @@ static bool is_ignored_symbol(const char *name, char
> > type)
> > ".LASANPC", /* s390 kasan local symbols */
> > "__crc_", /* modversions */
> > "__efistub_", /* arm64 EFI stub namespace */
> > + "__kvm_nvhe_", /* arm64 non-VHE KVM namespace */
> The addition of this line seems to have introduced errors on the
> 'vmlinux symtab matches kallsyms' perf test (perf test -v 1) which fails
> on aarch64 for all __kvm_nvhe_ prefixed symbols, like
>
> ERR : : __kvm_nvhe___invalid not on kallsyms
> ERR : : __kvm_nvhe___do_hyp_init not on kallsyms
> ERR : : __kvm_nvhe___kvm_handle_stub_hvc not on kallsyms
> ERR : : __kvm_nvhe_reset not on kallsyms
> ../..
>
> I understand we willingly hided those symbols from /proc/kallsyms. Do
> you confirm the right fix is to upgrade the perf test suite accordingly?
Hmmm. This test always fail here, no matter whether I have this line
or not:
maz@big-leg-emma:~$ sudo perf_5.10 test -v 1
1: vmlinux symtab matches kallsyms :
--- start ---
test child forked, pid 664
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
/proc/{kallsyms,modules} inconsistency while looking for "[bpf]" module!
Looking at the vmlinux_path (8 entries long)
symsrc__init: cannot get elf header.
symsrc__init: cannot get elf header.
Couldn't find a vmlinux that matches the kernel running on this machine,
skipping test
test child finished with -2
end
vmlinux symtab matches kallsyms: Skip
Rookie question: How do you provide a kernel to the test framework?
Thanks,
M.
--
Without deviation from the norm, progress is not possible.
___
kvmarm mailing list
kvmarm@lists.cs.columbia.edu
https://lists.cs.columbia.edu/mailman/listinfo/kvmarm
Re: [PATCH 15/56] KVM: arm64: Add build rules for separate VHE/nVHE object files
Hi David, Marc,
On 8/5/20 7:56 PM, Marc Zyngier wrote:
> From: David Brazdil
>
> Add new folders arch/arm64/kvm/hyp/{vhe,nvhe} and Makefiles for building code
> that runs in EL2 under VHE/nVHE KVM, repsectivelly. Add an include folder for
> hyp-specific header files which will include code common to VHE/nVHE.
>
> Build nVHE code with -D__KVM_NVHE_HYPERVISOR__, VHE code with
> -D__KVM_VHE_HYPERVISOR__.
>
> Under nVHE compile each source file into a `.hyp.tmp.o` object first, then
> prefix all its symbols with "__kvm_nvhe_" using `objcopy` and produce
> a `.hyp.o`. Suffixes were chosen so that it would be possible for VHE and nVHE
> to share some source files, but compiled with different CFLAGS.
>
> The nVHE ELF symbol prefix is added to kallsyms.c as ignored. EL2-only symbols
> will never appear in EL1 stack traces.
>
> Due to symbol prefixing, add a section in image-vars.h for aliases of symbols
> that are defined in nVHE EL2 and accessed by kernel in EL1 or vice versa.
>
> Signed-off-by: David Brazdil
> Signed-off-by: Marc Zyngier
> Link: https://lore.kernel.org/r/20200625131420.71444-4-dbraz...@google.com
> ---
> arch/arm64/kernel/image-vars.h | 14 +
> arch/arm64/kvm/hyp/Makefile | 10 +++---
> arch/arm64/kvm/hyp/nvhe/Makefile | 34
> arch/arm64/kvm/hyp/vhe/Makefile | 17
> scripts/kallsyms.c | 1 +
> 5 files changed, 73 insertions(+), 3 deletions(-)
> create mode 100644 arch/arm64/kvm/hyp/nvhe/Makefile
> create mode 100644 arch/arm64/kvm/hyp/vhe/Makefile
>
> diff --git a/arch/arm64/kernel/image-vars.h b/arch/arm64/kernel/image-vars.h
> index be0a63ffed23..3dc27da47712 100644
> --- a/arch/arm64/kernel/image-vars.h
> +++ b/arch/arm64/kernel/image-vars.h
> @@ -51,4 +51,18 @@ __efistub__ctype = _ctype;
>
> #endif
>
> +#ifdef CONFIG_KVM
> +
> +/*
> + * KVM nVHE code has its own symbol namespace prefixed with __kvm_nvhe_, to
> + * separate it from the kernel proper. The following symbols are legally
> + * accessed by it, therefore provide aliases to make them linkable.
> + * Do not include symbols which may not be safely accessed under hypervisor
> + * memory mappings.
> + */
> +
> +#define KVM_NVHE_ALIAS(sym) __kvm_nvhe_##sym = sym;
> +
> +#endif /* CONFIG_KVM */
> +
> #endif /* __ARM64_KERNEL_IMAGE_VARS_H */
> diff --git a/arch/arm64/kvm/hyp/Makefile b/arch/arm64/kvm/hyp/Makefile
> index 5d8357ddc234..9c5dfe6ff80b 100644
> --- a/arch/arm64/kvm/hyp/Makefile
> +++ b/arch/arm64/kvm/hyp/Makefile
> @@ -3,10 +3,14 @@
> # Makefile for Kernel-based Virtual Machine module, HYP part
> #
>
> -ccflags-y += -fno-stack-protector -DDISABLE_BRANCH_PROFILING \
> - $(DISABLE_STACKLEAK_PLUGIN)
> +incdir := $(srctree)/$(src)/include
> +subdir-asflags-y := -I$(incdir)
> +subdir-ccflags-y := -I$(incdir) \
> + -fno-stack-protector\
> + -DDISABLE_BRANCH_PROFILING \
> + $(DISABLE_STACKLEAK_PLUGIN)
>
> -obj-$(CONFIG_KVM) += hyp.o
> +obj-$(CONFIG_KVM) += hyp.o nvhe/
> obj-$(CONFIG_KVM_INDIRECT_VECTORS) += smccc_wa.o
>
> hyp-y := vgic-v3-sr.o timer-sr.o aarch32.o vgic-v2-cpuif-proxy.o sysreg-sr.o
> \
> diff --git a/arch/arm64/kvm/hyp/nvhe/Makefile
> b/arch/arm64/kvm/hyp/nvhe/Makefile
> new file mode 100644
> index ..955f4188e00f
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/nvhe/Makefile
> @@ -0,0 +1,34 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for Kernel-based Virtual Machine module, HYP/nVHE part
> +#
> +
> +asflags-y := -D__KVM_NVHE_HYPERVISOR__
> +ccflags-y := -D__KVM_NVHE_HYPERVISOR__
> +
> +obj-y :=
> +
> +obj-y := $(patsubst %.o,%.hyp.o,$(obj-y))
> +extra-y := $(patsubst %.hyp.o,%.hyp.tmp.o,$(obj-y))
> +
> +$(obj)/%.hyp.tmp.o: $(src)/%.c FORCE
> + $(call if_changed_rule,cc_o_c)
> +$(obj)/%.hyp.tmp.o: $(src)/%.S FORCE
> + $(call if_changed_rule,as_o_S)
> +$(obj)/%.hyp.o: $(obj)/%.hyp.tmp.o FORCE
> + $(call if_changed,hypcopy)
> +
> +quiet_cmd_hypcopy = HYPCOPY $@
> + cmd_hypcopy = $(OBJCOPY) --prefix-symbols=__kvm_nvhe_ $< $@
> +
> +# KVM nVHE code is run at a different exception code with a different map, so
> +# compiler instrumentation that inserts callbacks or checks into the code may
> +# cause crashes. Just disable it.
> +GCOV_PROFILE := n
> +KASAN_SANITIZE := n
> +UBSAN_SANITIZE := n
> +KCOV_INSTRUMENT := n
> +
> +# Skip objtool checking for this directory because nVHE code is compiled with
> +# non-standard build rules.
> +OBJECT_FILES_NON_STANDARD := y
> diff --git a/arch/arm64/kvm/hyp/vhe/Makefile b/arch/arm64/kvm/hyp/vhe/Makefile
> new file mode 100644
> index ..e04375546081
> --- /dev/null
> +++ b/arch/arm64/kvm/hyp/vhe/Makefile
> @@ -0,0 +1,17 @@
> +# SPDX-License-Identifier: GPL-2.0
> +#
> +# Makefile for Kernel-based Virtual Machine module, HYP/nVHE part
> +#
> +
> +asflags-y :=
