Re: Linux 4.9.93

[Sami Tolvanen]

On Mon, Apr 9, 2018 at 10:02 AM Greg Hackmann  wrote:
> Sami, what are the plans for upstreaming this work?

CFI is a clang-specific feature that depends on LTO. Based on the earlier
LTO discussion, we decided to collect some more evidence that clang's LTO
doesn't actually break anything in the kernel before sending out the next
revision of these patches. I will look into upstreaming this code again
once we have sufficient test coverage, hopefully by the end of the year.

Sami

Re: Linux 4.9.93

[Sami Tolvanen]

On Mon, Apr 9, 2018 at 10:02 AM Greg Hackmann  wrote:
> Sami, what are the plans for upstreaming this work?

CFI is a clang-specific feature that depends on LTO. Based on the earlier
LTO discussion, we decided to collect some more evidence that clang's LTO
doesn't actually break anything in the kernel before sending out the next
revision of these patches. I will look into upstreaming this code again
once we have sufficient test coverage, hopefully by the end of the year.

Sami

Re: Linux 4.9.93

[Greg Hackmann]

On 04/09/2018 04:41 AM, Greg KH wrote:
> On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
>> On Mon, 9 Apr 2018 11:49:37 +0200
>> Ard Biesheuvel  wrote:
>>
>>> On 9 April 2018 at 11:30, Greg KH  wrote:
 On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
> Hi,
>
> After this patchset, a kernel built with CFI fails. Disabling
> UNMAP_KERNEL_AT_EL0 fix the issue obviously.  
>>>
>>> How does one 'build a kernel with CFI' for arm64?
>>
>> From Google work on Android-4.9
>>
>> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
>>
>> I am not sure what is the plan on their side to upstream (Greg?), but 
>> definitely
>> useful to isolate actual issues.
> 
> Ah, yeah, if you are taking anything from the Android tree, all bets are
> off :)
> 
> Greg Hackmann can probably answer the questions about CFI and these
> patches, as he's working on merging this stable release into the
> android-4.9 branch this week.  I'll let him deal with this...
> 
> thanks,
> 
> greg k-h
> 

I've added Sami Tolvanen, who's handling CFI in the kernel.

Sami, what are the plans for upstreaming this work?

Re: Linux 4.9.93

[Greg Hackmann]

On 04/09/2018 04:41 AM, Greg KH wrote:
> On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
>> On Mon, 9 Apr 2018 11:49:37 +0200
>> Ard Biesheuvel  wrote:
>>
>>> On 9 April 2018 at 11:30, Greg KH  wrote:
 On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
> Hi,
>
> After this patchset, a kernel built with CFI fails. Disabling
> UNMAP_KERNEL_AT_EL0 fix the issue obviously.  
>>>
>>> How does one 'build a kernel with CFI' for arm64?
>>
>> From Google work on Android-4.9
>>
>> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
>>
>> I am not sure what is the plan on their side to upstream (Greg?), but 
>> definitely
>> useful to isolate actual issues.
> 
> Ah, yeah, if you are taking anything from the Android tree, all bets are
> off :)
> 
> Greg Hackmann can probably answer the questions about CFI and these
> patches, as he's working on merging this stable release into the
> android-4.9 branch this week.  I'll let him deal with this...
> 
> thanks,
> 
> greg k-h
> 

I've added Sami Tolvanen, who's handling CFI in the kernel.

Sami, what are the plans for upstreaming this work?

Re: Linux 4.9.93

[Greg KH]

On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
> On Mon, 9 Apr 2018 11:49:37 +0200
> Ard Biesheuvel  wrote:
> 
> > On 9 April 2018 at 11:30, Greg KH  wrote:
> > > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
> > >> Hi,
> > >>
> > >> After this patchset, a kernel built with CFI fails. Disabling
> > >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.  
> > 
> > How does one 'build a kernel with CFI' for arm64?
> 
> From Google work on Android-4.9
> 
> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> 
> I am not sure what is the plan on their side to upstream (Greg?), but 
> definitely
> useful to isolate actual issues.

Ah, yeah, if you are taking anything from the Android tree, all bets are
off :)

Greg Hackmann can probably answer the questions about CFI and these
patches, as he's working on merging this stable release into the
android-4.9 branch this week.  I'll let him deal with this...

thanks,

greg k-h

Re: Linux 4.9.93

[Greg KH]

On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
> On Mon, 9 Apr 2018 11:49:37 +0200
> Ard Biesheuvel  wrote:
> 
> > On 9 April 2018 at 11:30, Greg KH  wrote:
> > > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
> > >> Hi,
> > >>
> > >> After this patchset, a kernel built with CFI fails. Disabling
> > >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.  
> > 
> > How does one 'build a kernel with CFI' for arm64?
> 
> From Google work on Android-4.9
> 
> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> 
> I am not sure what is the plan on their side to upstream (Greg?), but 
> definitely
> useful to isolate actual issues.

Ah, yeah, if you are taking anything from the Android tree, all bets are
off :)

Greg Hackmann can probably answer the questions about CFI and these
patches, as he's working on merging this stable release into the
android-4.9 branch this week.  I'll let him deal with this...

thanks,

greg k-h

Re: Linux 4.9.93

[Jean-Baptiste Theou]

On Mon, 9 Apr 2018 11:07:41 +0100
Mark Rutland  wrote:

> On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
> > On Mon, 9 Apr 2018 11:49:37 +0200
> > Ard Biesheuvel  wrote:
> >   
> > > On 9 April 2018 at 11:30, Greg KH  wrote:  
> > > > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> > > >> Hi,
> > > >>
> > > >> After this patchset, a kernel built with CFI fails. Disabling
> > > >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> > > 
> > > How does one 'build a kernel with CFI' for arm64?  
> > 
> > From Google work on Android-4.9
> > 
> > https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> > 
> > I am not sure what is the plan on their side to upstream (Greg?), but 
> > definitely
> > useful to isolate actual issues.
> >   
> > > > Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> > > > android-common-4.9?  
> > 
> > It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
> > 
> > My take is that CFI doesn't like 
> > 
> >  * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t 
> > swapper)
> > 
> > and 
> > 
> > remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
> > 
> > Maybe just flag this function to not use CFI? I remember that Sami Tolvanen 
> > did
> > similar changes.  
> 
> From a quick scan, it looks like CFI uses shadow memory for function
> prologues. Since we're taking the PA of a function pointer, presumably
> this no longer maps to valid shadow.
> 
> I'd expect the same to apply to uses of cpu_replace_ttbr1(), but it
> looks like the only user of that is marked as __init, and that patch
> adds __nocfi to __init functions.
> 
> So you probably need to mark kpti_install_ng_mappings() as __nocfi.
>  
> > I know it's a bit out of context since CFI support for ARM64 is not 
> > upstream yet,
> > but unfortunate that an stable patchset trigger such failures.  
> 
> This is simply the nature of out-of-tree code.
> 
> In future, it would be very helpful if you could provide context for
> out-of-tree patches in the initial report.
> 

I can pass the initial CFI failures by tagging the function with __nocfi, but 
still face issues down the road.
That said, it's out of tree, so my problem.

Will investigate.

Thanks a lot for the quick support.

Best regards

> Thanks,
> Mark.

Re: Linux 4.9.93

[Jean-Baptiste Theou]

On Mon, 9 Apr 2018 11:07:41 +0100
Mark Rutland  wrote:

> On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
> > On Mon, 9 Apr 2018 11:49:37 +0200
> > Ard Biesheuvel  wrote:
> >   
> > > On 9 April 2018 at 11:30, Greg KH  wrote:  
> > > > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> > > >> Hi,
> > > >>
> > > >> After this patchset, a kernel built with CFI fails. Disabling
> > > >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> > > 
> > > How does one 'build a kernel with CFI' for arm64?  
> > 
> > From Google work on Android-4.9
> > 
> > https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> > 
> > I am not sure what is the plan on their side to upstream (Greg?), but 
> > definitely
> > useful to isolate actual issues.
> >   
> > > > Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> > > > android-common-4.9?  
> > 
> > It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
> > 
> > My take is that CFI doesn't like 
> > 
> >  * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t 
> > swapper)
> > 
> > and 
> > 
> > remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
> > 
> > Maybe just flag this function to not use CFI? I remember that Sami Tolvanen 
> > did
> > similar changes.  
> 
> From a quick scan, it looks like CFI uses shadow memory for function
> prologues. Since we're taking the PA of a function pointer, presumably
> this no longer maps to valid shadow.
> 
> I'd expect the same to apply to uses of cpu_replace_ttbr1(), but it
> looks like the only user of that is marked as __init, and that patch
> adds __nocfi to __init functions.
> 
> So you probably need to mark kpti_install_ng_mappings() as __nocfi.
>  
> > I know it's a bit out of context since CFI support for ARM64 is not 
> > upstream yet,
> > but unfortunate that an stable patchset trigger such failures.  
> 
> This is simply the nature of out-of-tree code.
> 
> In future, it would be very helpful if you could provide context for
> out-of-tree patches in the initial report.
> 

I can pass the initial CFI failures by tagging the function with __nocfi, but 
still face issues down the road.
That said, it's out of tree, so my problem.

Will investigate.

Thanks a lot for the quick support.

Best regards

> Thanks,
> Mark.

Re: Linux 4.9.93

[Jean-Baptiste Theou]

On Mon, 9 Apr 2018 12:25:07 +0200
Ard Biesheuvel  wrote:

> > On 9 Apr 2018, at 11:57, Jean-Baptiste Theou  wrote:
> > 
> > On Mon, 9 Apr 2018 11:49:37 +0200
> > Ard Biesheuvel  wrote:
> >   
> >>> On 9 April 2018 at 11:30, Greg KH  wrote:  
>  On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
>  Hi,
>  
>  After this patchset, a kernel built with CFI fails. Disabling
>  UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> >> 
> >> How does one 'build a kernel with CFI' for arm64?  
> > 
> > From Google work on Android-4.9
> > 
> > https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> > 
> > I am not sure what is the plan on their side to upstream (Greg?), but 
> > definitely
> > useful to isolate actual issues.
> >   
> >>   
> >>> 
> >>> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> >>> android-common-4.9?  
> > 
> > It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
> > 
> > My take is that CFI doesn't like 
> > 
> > * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t 
> > swapper)
> > 
> > and 
> > 
> > remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
> > 
> > Maybe just flag this function to not use CFI? I remember that Sami Tolvanen 
> > did
> > similar changes.
> > 
> > I know it's a bit out of context since CFI support for ARM64 is not 
> > upstream yet,
> > but unfortunate that an stable patchset trigger such failures.
> >   
> 
> I am sorry but if you are implying that we should have tested these patches 
> against the out of tree CFI code, I have to disappoint you: that is simply 
> not upstream’s job, and if the Google engineers merged this into their v4.9 
> tree without proper testing, may I suggest that you report it to them instead?
> 
> OTOH, if that is not what you are implying, please ignore the rant :-)
> 

To be perfectly honest, I forgot that CFI wasn't an upstream feature. Indeed, I 
don't expect upstream test farm to run out of tree.
The real answer here is to upstream CFI ;-)

Thanks a lot

Best regards

> 
> > Thanks a lot
> > 
> > Best regards
> >   
> >>>   
>  Wondering if there is one of the test suite used on the review patchset 
>  that covers the CFI usecase.
>  
>  Best regards,
>  
>  [0.249191] CPU features: detected feature: GIC system register CPU 
>  interface
>  [0.256391] CPU features: detected feature: Privileged Access Never
>  [0.262719] CPU features: detected feature: User Access Override
>  [0.268791] CPU features: detected feature: 32-bit EL0 Support
>  [0.274683] CPU features: detected feature: Kernel page table 
>  isolation (KPTI)
>  [0.282166] CFI failure:
>  [0.282169] CFI failure:
>  [0.282172] CFI failure:
>  [0.282173] CFI failure:
>  [0.282175] CFI failure:
>  [0.282176] CFI failure:
>  [0.282177] CFI failure:
>  [0.282178] CFI failure:
>  [0.282188] [ cut here ]
>  [0.282189] [ cut here ]
>  [0.282190] [ cut here ]
>  [0.282191] [ cut here ]
>  [0.282193] [ cut here ]
>  [0.282196] kernel BUG at kernel/cfi.c:32!
>  [0.282198] [ cut here ]
>  [0.282201] kernel BUG at kernel/cfi.c:32!
>  [0.282202] [ cut here ]
>  [0.282204] kernel BUG at kernel/cfi.c:32!
>  [0.282207] kernel BUG at kernel/cfi.c:32!
>  [0.282209] kernel BUG at kernel/cfi.c:32!
>  [0.282211] kernel BUG at kernel/cfi.c:32!
>  [0.282214] kernel BUG at kernel/cfi.c:32!
>  [0.282215] [ cut here ]
>  [0.282216] kernel BUG at kernel/cfi.c:32!
>  [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>  [0.282224] Modules linked in:
>  [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ 
>  #39
>  [0.282232] Hardware name: 
>  [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
>  [0.282250] PC is at __cfi_check_fail+0x14/0x1c
>  [0.282253] LR is at __cfi_check_fail+0x14/0x1c
>  [0.282255] pc : [] lr : [] 
>  pstate: 60c00085
>  [0.282256] sp : fffbb30cfc30
>  [0.282259] x29: fffbb30cfc30 x28: ff93b6415000
>  [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000
>  [0.282264] x25: ff93b5ce6000 x24: ff93b6419000
>  [0.282266] x23: ff93b65c1000 x22: ff93b65c4000
>  [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0
>  [0.282271] x19:  x18: 002c
>  [0.282274] x17: 000fd054 x16: 

Re: Linux 4.9.93

[Jean-Baptiste Theou]

On Mon, 9 Apr 2018 12:25:07 +0200
Ard Biesheuvel  wrote:

> > On 9 Apr 2018, at 11:57, Jean-Baptiste Theou  wrote:
> > 
> > On Mon, 9 Apr 2018 11:49:37 +0200
> > Ard Biesheuvel  wrote:
> >   
> >>> On 9 April 2018 at 11:30, Greg KH  wrote:  
>  On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
>  Hi,
>  
>  After this patchset, a kernel built with CFI fails. Disabling
>  UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> >> 
> >> How does one 'build a kernel with CFI' for arm64?  
> > 
> > From Google work on Android-4.9
> > 
> > https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> > 
> > I am not sure what is the plan on their side to upstream (Greg?), but 
> > definitely
> > useful to isolate actual issues.
> >   
> >>   
> >>> 
> >>> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> >>> android-common-4.9?  
> > 
> > It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
> > 
> > My take is that CFI doesn't like 
> > 
> > * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t 
> > swapper)
> > 
> > and 
> > 
> > remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
> > 
> > Maybe just flag this function to not use CFI? I remember that Sami Tolvanen 
> > did
> > similar changes.
> > 
> > I know it's a bit out of context since CFI support for ARM64 is not 
> > upstream yet,
> > but unfortunate that an stable patchset trigger such failures.
> >   
> 
> I am sorry but if you are implying that we should have tested these patches 
> against the out of tree CFI code, I have to disappoint you: that is simply 
> not upstream’s job, and if the Google engineers merged this into their v4.9 
> tree without proper testing, may I suggest that you report it to them instead?
> 
> OTOH, if that is not what you are implying, please ignore the rant :-)
> 

To be perfectly honest, I forgot that CFI wasn't an upstream feature. Indeed, I 
don't expect upstream test farm to run out of tree.
The real answer here is to upstream CFI ;-)

Thanks a lot

Best regards

> 
> > Thanks a lot
> > 
> > Best regards
> >   
> >>>   
>  Wondering if there is one of the test suite used on the review patchset 
>  that covers the CFI usecase.
>  
>  Best regards,
>  
>  [0.249191] CPU features: detected feature: GIC system register CPU 
>  interface
>  [0.256391] CPU features: detected feature: Privileged Access Never
>  [0.262719] CPU features: detected feature: User Access Override
>  [0.268791] CPU features: detected feature: 32-bit EL0 Support
>  [0.274683] CPU features: detected feature: Kernel page table 
>  isolation (KPTI)
>  [0.282166] CFI failure:
>  [0.282169] CFI failure:
>  [0.282172] CFI failure:
>  [0.282173] CFI failure:
>  [0.282175] CFI failure:
>  [0.282176] CFI failure:
>  [0.282177] CFI failure:
>  [0.282178] CFI failure:
>  [0.282188] [ cut here ]
>  [0.282189] [ cut here ]
>  [0.282190] [ cut here ]
>  [0.282191] [ cut here ]
>  [0.282193] [ cut here ]
>  [0.282196] kernel BUG at kernel/cfi.c:32!
>  [0.282198] [ cut here ]
>  [0.282201] kernel BUG at kernel/cfi.c:32!
>  [0.282202] [ cut here ]
>  [0.282204] kernel BUG at kernel/cfi.c:32!
>  [0.282207] kernel BUG at kernel/cfi.c:32!
>  [0.282209] kernel BUG at kernel/cfi.c:32!
>  [0.282211] kernel BUG at kernel/cfi.c:32!
>  [0.282214] kernel BUG at kernel/cfi.c:32!
>  [0.282215] [ cut here ]
>  [0.282216] kernel BUG at kernel/cfi.c:32!
>  [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>  [0.282224] Modules linked in:
>  [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ 
>  #39
>  [0.282232] Hardware name: 
>  [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
>  [0.282250] PC is at __cfi_check_fail+0x14/0x1c
>  [0.282253] LR is at __cfi_check_fail+0x14/0x1c
>  [0.282255] pc : [] lr : [] 
>  pstate: 60c00085
>  [0.282256] sp : fffbb30cfc30
>  [0.282259] x29: fffbb30cfc30 x28: ff93b6415000
>  [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000
>  [0.282264] x25: ff93b5ce6000 x24: ff93b6419000
>  [0.282266] x23: ff93b65c1000 x22: ff93b65c4000
>  [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0
>  [0.282271] x19:  x18: 002c
>  [0.282274] x17: 000fd054 x16: 
>  [0.282276] x15: ff93b65ec000 x14: 000c
>  [0.282279] x13: 

Re: Linux 4.9.93

[Ard Biesheuvel]

> On 9 Apr 2018, at 11:57, Jean-Baptiste Theou  wrote:
> 
> On Mon, 9 Apr 2018 11:49:37 +0200
> Ard Biesheuvel  wrote:
> 
>>> On 9 April 2018 at 11:30, Greg KH  wrote:
 On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
 Hi,
 
 After this patchset, a kernel built with CFI fails. Disabling
 UNMAP_KERNEL_AT_EL0 fix the issue obviously.  
>> 
>> How does one 'build a kernel with CFI' for arm64?
> 
> From Google work on Android-4.9
> 
> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> 
> I am not sure what is the plan on their side to upstream (Greg?), but 
> definitely
> useful to isolate actual issues.
> 
>> 
>>> 
>>> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
>>> android-common-4.9?
> 
> It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
> 
> My take is that CFI doesn't like 
> 
> * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)
> 
> and 
> 
> remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
> 
> Maybe just flag this function to not use CFI? I remember that Sami Tolvanen 
> did
> similar changes.
> 
> I know it's a bit out of context since CFI support for ARM64 is not upstream 
> yet,
> but unfortunate that an stable patchset trigger such failures.
> 

I am sorry but if you are implying that we should have tested these patches 
against the out of tree CFI code, I have to disappoint you: that is simply not 
upstream’s job, and if the Google engineers merged this into their v4.9 tree 
without proper testing, may I suggest that you report it to them instead?

OTOH, if that is not what you are implying, please ignore the rant :-)


> Thanks a lot
> 
> Best regards
> 
>>> 
 Wondering if there is one of the test suite used on the review patchset 
 that covers the CFI usecase.
 
 Best regards,
 
 [0.249191] CPU features: detected feature: GIC system register CPU 
 interface
 [0.256391] CPU features: detected feature: Privileged Access Never
 [0.262719] CPU features: detected feature: User Access Override
 [0.268791] CPU features: detected feature: 32-bit EL0 Support
 [0.274683] CPU features: detected feature: Kernel page table isolation 
 (KPTI)
 [0.282166] CFI failure:
 [0.282169] CFI failure:
 [0.282172] CFI failure:
 [0.282173] CFI failure:
 [0.282175] CFI failure:
 [0.282176] CFI failure:
 [0.282177] CFI failure:
 [0.282178] CFI failure:
 [0.282188] [ cut here ]
 [0.282189] [ cut here ]
 [0.282190] [ cut here ]
 [0.282191] [ cut here ]
 [0.282193] [ cut here ]
 [0.282196] kernel BUG at kernel/cfi.c:32!
 [0.282198] [ cut here ]
 [0.282201] kernel BUG at kernel/cfi.c:32!
 [0.282202] [ cut here ]
 [0.282204] kernel BUG at kernel/cfi.c:32!
 [0.282207] kernel BUG at kernel/cfi.c:32!
 [0.282209] kernel BUG at kernel/cfi.c:32!
 [0.282211] kernel BUG at kernel/cfi.c:32!
 [0.282214] kernel BUG at kernel/cfi.c:32!
 [0.282215] [ cut here ]
 [0.282216] kernel BUG at kernel/cfi.c:32!
 [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
 [0.282224] Modules linked in:
 [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ 
 #39
 [0.282232] Hardware name: 
 [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
 [0.282250] PC is at __cfi_check_fail+0x14/0x1c
 [0.282253] LR is at __cfi_check_fail+0x14/0x1c
 [0.282255] pc : [] lr : [] pstate: 
 60c00085
 [0.282256] sp : fffbb30cfc30
 [0.282259] x29: fffbb30cfc30 x28: ff93b6415000
 [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000
 [0.282264] x25: ff93b5ce6000 x24: ff93b6419000
 [0.282266] x23: ff93b65c1000 x22: ff93b65c4000
 [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0
 [0.282271] x19:  x18: 002c
 [0.282274] x17: 000fd054 x16: 
 [0.282276] x15: ff93b65ec000 x14: 000c
 [0.282279] x13: 0004 x12: 
 [0.282281] x11:  x10: 01440144
 [0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
 [0.282286] x7 :  x6 : fffbbac75b60
 [0.282288] x5 :  x4 : 
 [0.282290] x3 : 3a657275 x2 : 
 [0.282292] x1 :  x0 : 000c
 [0.282294]

Re: Linux 4.9.93

[Ard Biesheuvel]

> On 9 Apr 2018, at 11:57, Jean-Baptiste Theou  wrote:
> 
> On Mon, 9 Apr 2018 11:49:37 +0200
> Ard Biesheuvel  wrote:
> 
>>> On 9 April 2018 at 11:30, Greg KH  wrote:
 On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
 Hi,
 
 After this patchset, a kernel built with CFI fails. Disabling
 UNMAP_KERNEL_AT_EL0 fix the issue obviously.  
>> 
>> How does one 'build a kernel with CFI' for arm64?
> 
> From Google work on Android-4.9
> 
> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> 
> I am not sure what is the plan on their side to upstream (Greg?), but 
> definitely
> useful to isolate actual issues.
> 
>> 
>>> 
>>> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
>>> android-common-4.9?
> 
> It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
> 
> My take is that CFI doesn't like 
> 
> * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)
> 
> and 
> 
> remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
> 
> Maybe just flag this function to not use CFI? I remember that Sami Tolvanen 
> did
> similar changes.
> 
> I know it's a bit out of context since CFI support for ARM64 is not upstream 
> yet,
> but unfortunate that an stable patchset trigger such failures.
> 

I am sorry but if you are implying that we should have tested these patches 
against the out of tree CFI code, I have to disappoint you: that is simply not 
upstream’s job, and if the Google engineers merged this into their v4.9 tree 
without proper testing, may I suggest that you report it to them instead?

OTOH, if that is not what you are implying, please ignore the rant :-)


> Thanks a lot
> 
> Best regards
> 
>>> 
 Wondering if there is one of the test suite used on the review patchset 
 that covers the CFI usecase.
 
 Best regards,
 
 [0.249191] CPU features: detected feature: GIC system register CPU 
 interface
 [0.256391] CPU features: detected feature: Privileged Access Never
 [0.262719] CPU features: detected feature: User Access Override
 [0.268791] CPU features: detected feature: 32-bit EL0 Support
 [0.274683] CPU features: detected feature: Kernel page table isolation 
 (KPTI)
 [0.282166] CFI failure:
 [0.282169] CFI failure:
 [0.282172] CFI failure:
 [0.282173] CFI failure:
 [0.282175] CFI failure:
 [0.282176] CFI failure:
 [0.282177] CFI failure:
 [0.282178] CFI failure:
 [0.282188] [ cut here ]
 [0.282189] [ cut here ]
 [0.282190] [ cut here ]
 [0.282191] [ cut here ]
 [0.282193] [ cut here ]
 [0.282196] kernel BUG at kernel/cfi.c:32!
 [0.282198] [ cut here ]
 [0.282201] kernel BUG at kernel/cfi.c:32!
 [0.282202] [ cut here ]
 [0.282204] kernel BUG at kernel/cfi.c:32!
 [0.282207] kernel BUG at kernel/cfi.c:32!
 [0.282209] kernel BUG at kernel/cfi.c:32!
 [0.282211] kernel BUG at kernel/cfi.c:32!
 [0.282214] kernel BUG at kernel/cfi.c:32!
 [0.282215] [ cut here ]
 [0.282216] kernel BUG at kernel/cfi.c:32!
 [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
 [0.282224] Modules linked in:
 [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ 
 #39
 [0.282232] Hardware name: 
 [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
 [0.282250] PC is at __cfi_check_fail+0x14/0x1c
 [0.282253] LR is at __cfi_check_fail+0x14/0x1c
 [0.282255] pc : [] lr : [] pstate: 
 60c00085
 [0.282256] sp : fffbb30cfc30
 [0.282259] x29: fffbb30cfc30 x28: ff93b6415000
 [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000
 [0.282264] x25: ff93b5ce6000 x24: ff93b6419000
 [0.282266] x23: ff93b65c1000 x22: ff93b65c4000
 [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0
 [0.282271] x19:  x18: 002c
 [0.282274] x17: 000fd054 x16: 
 [0.282276] x15: ff93b65ec000 x14: 000c
 [0.282279] x13: 0004 x12: 
 [0.282281] x11:  x10: 01440144
 [0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
 [0.282286] x7 :  x6 : fffbbac75b60
 [0.282288] x5 :  x4 : 
 [0.282290] x3 : 3a657275 x2 : 
 [0.282292] x1 :  x0 : 000c
 [0.282294]
 [0.282294] PC: 0xff93b3f03d50:
 [0.282308] 3d50  b9001ac8 

Re: Linux 4.9.93

[Mark Rutland]

On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
> On Mon, 9 Apr 2018 11:49:37 +0200
> Ard Biesheuvel  wrote:
> 
> > On 9 April 2018 at 11:30, Greg KH  wrote:
> > > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
> > >> Hi,
> > >>
> > >> After this patchset, a kernel built with CFI fails. Disabling
> > >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.  
> > 
> > How does one 'build a kernel with CFI' for arm64?
> 
> From Google work on Android-4.9
> 
> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> 
> I am not sure what is the plan on their side to upstream (Greg?), but 
> definitely
> useful to isolate actual issues.
> 
> > > Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> > > android-common-4.9?
> 
> It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
> 
> My take is that CFI doesn't like 
> 
>  * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)
> 
> and 
> 
> remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
> 
> Maybe just flag this function to not use CFI? I remember that Sami Tolvanen 
> did
> similar changes.

>From a quick scan, it looks like CFI uses shadow memory for function
prologues. Since we're taking the PA of a function pointer, presumably
this no longer maps to valid shadow.

I'd expect the same to apply to uses of cpu_replace_ttbr1(), but it
looks like the only user of that is marked as __init, and that patch
adds __nocfi to __init functions.

So you probably need to mark kpti_install_ng_mappings() as __nocfi.
 
> I know it's a bit out of context since CFI support for ARM64 is not upstream 
> yet,
> but unfortunate that an stable patchset trigger such failures.

This is simply the nature of out-of-tree code.

In future, it would be very helpful if you could provide context for
out-of-tree patches in the initial report.

Thanks,
Mark.

Re: Linux 4.9.93

[Mark Rutland]

On Mon, Apr 09, 2018 at 06:57:51PM +0900, Jean-Baptiste Theou wrote:
> On Mon, 9 Apr 2018 11:49:37 +0200
> Ard Biesheuvel  wrote:
> 
> > On 9 April 2018 at 11:30, Greg KH  wrote:
> > > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
> > >> Hi,
> > >>
> > >> After this patchset, a kernel built with CFI fails. Disabling
> > >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.  
> > 
> > How does one 'build a kernel with CFI' for arm64?
> 
> From Google work on Android-4.9
> 
> https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232
> 
> I am not sure what is the plan on their side to upstream (Greg?), but 
> definitely
> useful to isolate actual issues.
> 
> > > Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> > > android-common-4.9?
> 
> It's a "clean 4.9.93" + whatever is needed for Clang/CFI support
> 
> My take is that CFI doesn't like 
> 
>  * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)
> 
> and 
> 
> remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);
> 
> Maybe just flag this function to not use CFI? I remember that Sami Tolvanen 
> did
> similar changes.

>From a quick scan, it looks like CFI uses shadow memory for function
prologues. Since we're taking the PA of a function pointer, presumably
this no longer maps to valid shadow.

I'd expect the same to apply to uses of cpu_replace_ttbr1(), but it
looks like the only user of that is marked as __init, and that patch
adds __nocfi to __init functions.

So you probably need to mark kpti_install_ng_mappings() as __nocfi.
 
> I know it's a bit out of context since CFI support for ARM64 is not upstream 
> yet,
> but unfortunate that an stable patchset trigger such failures.

This is simply the nature of out-of-tree code.

In future, it would be very helpful if you could provide context for
out-of-tree patches in the initial report.

Thanks,
Mark.

Re: Linux 4.9.93

[Jean-Baptiste Theou]

On Mon, 9 Apr 2018 11:49:37 +0200
Ard Biesheuvel  wrote:

> On 9 April 2018 at 11:30, Greg KH  wrote:
> > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
> >> Hi,
> >>
> >> After this patchset, a kernel built with CFI fails. Disabling
> >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.  
> 
> How does one 'build a kernel with CFI' for arm64?

From Google work on Android-4.9

https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232

I am not sure what is the plan on their side to upstream (Greg?), but definitely
useful to isolate actual issues.

> 
> >
> > Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> > android-common-4.9?

It's a "clean 4.9.93" + whatever is needed for Clang/CFI support

My take is that CFI doesn't like 

 * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)

and 

remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);

Maybe just flag this function to not use CFI? I remember that Sami Tolvanen did
similar changes.

I know it's a bit out of context since CFI support for ARM64 is not upstream 
yet,
but unfortunate that an stable patchset trigger such failures.

Thanks a lot

Best regards

> >  
> >> Wondering if there is one of the test suite used on the review patchset 
> >> that covers the CFI usecase.
> >>
> >> Best regards,
> >>
> >> [0.249191] CPU features: detected feature: GIC system register CPU 
> >> interface
> >> [0.256391] CPU features: detected feature: Privileged Access Never
> >> [0.262719] CPU features: detected feature: User Access Override
> >> [0.268791] CPU features: detected feature: 32-bit EL0 Support
> >> [0.274683] CPU features: detected feature: Kernel page table isolation 
> >> (KPTI)
> >> [0.282166] CFI failure:
> >> [0.282169] CFI failure:
> >> [0.282172] CFI failure:
> >> [0.282173] CFI failure:
> >> [0.282175] CFI failure:
> >> [0.282176] CFI failure:
> >> [0.282177] CFI failure:
> >> [0.282178] CFI failure:
> >> [0.282188] [ cut here ]
> >> [0.282189] [ cut here ]
> >> [0.282190] [ cut here ]
> >> [0.282191] [ cut here ]
> >> [0.282193] [ cut here ]
> >> [0.282196] kernel BUG at kernel/cfi.c:32!
> >> [0.282198] [ cut here ]
> >> [0.282201] kernel BUG at kernel/cfi.c:32!
> >> [0.282202] [ cut here ]
> >> [0.282204] kernel BUG at kernel/cfi.c:32!
> >> [0.282207] kernel BUG at kernel/cfi.c:32!
> >> [0.282209] kernel BUG at kernel/cfi.c:32!
> >> [0.282211] kernel BUG at kernel/cfi.c:32!
> >> [0.282214] kernel BUG at kernel/cfi.c:32!
> >> [0.282215] [ cut here ]
> >> [0.282216] kernel BUG at kernel/cfi.c:32!
> >> [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
> >> [0.282224] Modules linked in:
> >> [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ 
> >> #39
> >> [0.282232] Hardware name: 
> >> [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
> >> [0.282250] PC is at __cfi_check_fail+0x14/0x1c
> >> [0.282253] LR is at __cfi_check_fail+0x14/0x1c
> >> [0.282255] pc : [] lr : [] pstate: 
> >> 60c00085
> >> [0.282256] sp : fffbb30cfc30
> >> [0.282259] x29: fffbb30cfc30 x28: ff93b6415000
> >> [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000
> >> [0.282264] x25: ff93b5ce6000 x24: ff93b6419000
> >> [0.282266] x23: ff93b65c1000 x22: ff93b65c4000
> >> [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0
> >> [0.282271] x19:  x18: 002c
> >> [0.282274] x17: 000fd054 x16: 
> >> [0.282276] x15: ff93b65ec000 x14: 000c
> >> [0.282279] x13: 0004 x12: 
> >> [0.282281] x11:  x10: 01440144
> >> [0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
> >> [0.282286] x7 :  x6 : fffbbac75b60
> >> [0.282288] x5 :  x4 : 
> >> [0.282290] x3 : 3a657275 x2 : 
> >> [0.282292] x1 :  x0 : 000c
> >> [0.282294]
> >> [0.282294] PC: 0xff93b3f03d50:
> >> [0.282308] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
> >> aa1603e0 f90007e8
> >> [0.282315] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
> >> 913ee400 94533cc7
> >> [0.282322] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
> >> f940406b 2a0203e8
> >> [0.282329] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
> >> a9bf7bfd 910003fd
> >> [0.282330]
> >> [0.282330] LR: 0xff93b3f03d50:
> >> [0.282336] 

Re: Linux 4.9.93

[Jean-Baptiste Theou]

On Mon, 9 Apr 2018 11:49:37 +0200
Ard Biesheuvel  wrote:

> On 9 April 2018 at 11:30, Greg KH  wrote:
> > On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:  
> >> Hi,
> >>
> >> After this patchset, a kernel built with CFI fails. Disabling
> >> UNMAP_KERNEL_AT_EL0 fix the issue obviously.  
> 
> How does one 'build a kernel with CFI' for arm64?

From Google work on Android-4.9

https://android.googlesource.com/kernel/common/+/00a195e7c0752ff5d65c9caadfbcc226270ca232

I am not sure what is the plan on their side to upstream (Greg?), but definitely
useful to isolate actual issues.

> 
> >
> > Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> > android-common-4.9?

It's a "clean 4.9.93" + whatever is needed for Clang/CFI support

My take is that CFI doesn't like 

 * void __kpti_install_ng_mappings(int cpu, int num_cpus, phys_addr_t swapper)

and 

remap_fn = (void *)__pa_symbol(idmap_kpti_install_ng_mappings);

Maybe just flag this function to not use CFI? I remember that Sami Tolvanen did
similar changes.

I know it's a bit out of context since CFI support for ARM64 is not upstream 
yet,
but unfortunate that an stable patchset trigger such failures.

Thanks a lot

Best regards

> >  
> >> Wondering if there is one of the test suite used on the review patchset 
> >> that covers the CFI usecase.
> >>
> >> Best regards,
> >>
> >> [0.249191] CPU features: detected feature: GIC system register CPU 
> >> interface
> >> [0.256391] CPU features: detected feature: Privileged Access Never
> >> [0.262719] CPU features: detected feature: User Access Override
> >> [0.268791] CPU features: detected feature: 32-bit EL0 Support
> >> [0.274683] CPU features: detected feature: Kernel page table isolation 
> >> (KPTI)
> >> [0.282166] CFI failure:
> >> [0.282169] CFI failure:
> >> [0.282172] CFI failure:
> >> [0.282173] CFI failure:
> >> [0.282175] CFI failure:
> >> [0.282176] CFI failure:
> >> [0.282177] CFI failure:
> >> [0.282178] CFI failure:
> >> [0.282188] [ cut here ]
> >> [0.282189] [ cut here ]
> >> [0.282190] [ cut here ]
> >> [0.282191] [ cut here ]
> >> [0.282193] [ cut here ]
> >> [0.282196] kernel BUG at kernel/cfi.c:32!
> >> [0.282198] [ cut here ]
> >> [0.282201] kernel BUG at kernel/cfi.c:32!
> >> [0.282202] [ cut here ]
> >> [0.282204] kernel BUG at kernel/cfi.c:32!
> >> [0.282207] kernel BUG at kernel/cfi.c:32!
> >> [0.282209] kernel BUG at kernel/cfi.c:32!
> >> [0.282211] kernel BUG at kernel/cfi.c:32!
> >> [0.282214] kernel BUG at kernel/cfi.c:32!
> >> [0.282215] [ cut here ]
> >> [0.282216] kernel BUG at kernel/cfi.c:32!
> >> [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
> >> [0.282224] Modules linked in:
> >> [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ 
> >> #39
> >> [0.282232] Hardware name: 
> >> [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
> >> [0.282250] PC is at __cfi_check_fail+0x14/0x1c
> >> [0.282253] LR is at __cfi_check_fail+0x14/0x1c
> >> [0.282255] pc : [] lr : [] pstate: 
> >> 60c00085
> >> [0.282256] sp : fffbb30cfc30
> >> [0.282259] x29: fffbb30cfc30 x28: ff93b6415000
> >> [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000
> >> [0.282264] x25: ff93b5ce6000 x24: ff93b6419000
> >> [0.282266] x23: ff93b65c1000 x22: ff93b65c4000
> >> [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0
> >> [0.282271] x19:  x18: 002c
> >> [0.282274] x17: 000fd054 x16: 
> >> [0.282276] x15: ff93b65ec000 x14: 000c
> >> [0.282279] x13: 0004 x12: 
> >> [0.282281] x11:  x10: 01440144
> >> [0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
> >> [0.282286] x7 :  x6 : fffbbac75b60
> >> [0.282288] x5 :  x4 : 
> >> [0.282290] x3 : 3a657275 x2 : 
> >> [0.282292] x1 :  x0 : 000c
> >> [0.282294]
> >> [0.282294] PC: 0xff93b3f03d50:
> >> [0.282308] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
> >> aa1603e0 f90007e8
> >> [0.282315] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
> >> 913ee400 94533cc7
> >> [0.282322] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
> >> f940406b 2a0203e8
> >> [0.282329] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
> >> a9bf7bfd 910003fd
> >> [0.282330]
> >> [0.282330] LR: 0xff93b3f03d50:
> >> [0.282336] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 

Re: Linux 4.9.93

[Mark Rutland]

On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> Hi,

Hi,

> After this patchset, a kernel built with CFI fails. Disabling
> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> 
> Wondering if there is one of the test suite used on the review
> patchset that covers the CFI usecase.

IIUC, you're saying that after my v4.9.y meltdown backport, your own
backport of an out-of-tree patchset does not function.

There's no support for CFI in upsteam (at least as of v4.16), nor in
v4.9.93. I have no idea as to how KPTI interacts with an out-of-tree
patchset, so could you elaborate on the issue?

I've not seen any CFI patches on the linux-arm-kernel mailing list, so
it's not clear to me how this interacts with KPTI.

Which patches do you have applied that implement CFI? Which compiler are
you using?

[...]

> [0.282196] kernel BUG at kernel/cfi.c:32!

This file doesn't exit in v4.9.93...

> [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39

... and here I see you have local changes applied.

It would be best if you tried to get CFI supported upstream before
trying to backport it. That would be best discussed on the
linux-arm-kernel mailing list.

Thanks,
Mark.

Re: Linux 4.9.93

[Mark Rutland]

On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> Hi,

Hi,

> After this patchset, a kernel built with CFI fails. Disabling
> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> 
> Wondering if there is one of the test suite used on the review
> patchset that covers the CFI usecase.

IIUC, you're saying that after my v4.9.y meltdown backport, your own
backport of an out-of-tree patchset does not function.

There's no support for CFI in upsteam (at least as of v4.16), nor in
v4.9.93. I have no idea as to how KPTI interacts with an out-of-tree
patchset, so could you elaborate on the issue?

I've not seen any CFI patches on the linux-arm-kernel mailing list, so
it's not clear to me how this interacts with KPTI.

Which patches do you have applied that implement CFI? Which compiler are
you using?

[...]

> [0.282196] kernel BUG at kernel/cfi.c:32!

This file doesn't exit in v4.9.93...

> [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39

... and here I see you have local changes applied.

It would be best if you tried to get CFI supported upstream before
trying to backport it. That would be best discussed on the
linux-arm-kernel mailing list.

Thanks,
Mark.

Re: Linux 4.9.93

[Ard Biesheuvel]

On 9 April 2018 at 11:30, Greg KH  wrote:
> On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
>> Hi,
>>
>> After this patchset, a kernel built with CFI fails. Disabling
>> UNMAP_KERNEL_AT_EL0 fix the issue obviously.

How does one 'build a kernel with CFI' for arm64?

>
> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> android-common-4.9?
>
>> Wondering if there is one of the test suite used on the review patchset that 
>> covers the CFI usecase.
>>
>> Best regards,
>>
>> [0.249191] CPU features: detected feature: GIC system register CPU 
>> interface
>> [0.256391] CPU features: detected feature: Privileged Access Never
>> [0.262719] CPU features: detected feature: User Access Override
>> [0.268791] CPU features: detected feature: 32-bit EL0 Support
>> [0.274683] CPU features: detected feature: Kernel page table isolation 
>> (KPTI)
>> [0.282166] CFI failure:
>> [0.282169] CFI failure:
>> [0.282172] CFI failure:
>> [0.282173] CFI failure:
>> [0.282175] CFI failure:
>> [0.282176] CFI failure:
>> [0.282177] CFI failure:
>> [0.282178] CFI failure:
>> [0.282188] [ cut here ]
>> [0.282189] [ cut here ]
>> [0.282190] [ cut here ]
>> [0.282191] [ cut here ]
>> [0.282193] [ cut here ]
>> [0.282196] kernel BUG at kernel/cfi.c:32!
>> [0.282198] [ cut here ]
>> [0.282201] kernel BUG at kernel/cfi.c:32!
>> [0.282202] [ cut here ]
>> [0.282204] kernel BUG at kernel/cfi.c:32!
>> [0.282207] kernel BUG at kernel/cfi.c:32!
>> [0.282209] kernel BUG at kernel/cfi.c:32!
>> [0.282211] kernel BUG at kernel/cfi.c:32!
>> [0.282214] kernel BUG at kernel/cfi.c:32!
>> [0.282215] [ cut here ]
>> [0.282216] kernel BUG at kernel/cfi.c:32!
>> [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>> [0.282224] Modules linked in:
>> [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
>> [0.282232] Hardware name: 
>> [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
>> [0.282250] PC is at __cfi_check_fail+0x14/0x1c
>> [0.282253] LR is at __cfi_check_fail+0x14/0x1c
>> [0.282255] pc : [] lr : [] pstate: 
>> 60c00085
>> [0.282256] sp : fffbb30cfc30
>> [0.282259] x29: fffbb30cfc30 x28: ff93b6415000
>> [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000
>> [0.282264] x25: ff93b5ce6000 x24: ff93b6419000
>> [0.282266] x23: ff93b65c1000 x22: ff93b65c4000
>> [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0
>> [0.282271] x19:  x18: 002c
>> [0.282274] x17: 000fd054 x16: 
>> [0.282276] x15: ff93b65ec000 x14: 000c
>> [0.282279] x13: 0004 x12: 
>> [0.282281] x11:  x10: 01440144
>> [0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
>> [0.282286] x7 :  x6 : fffbbac75b60
>> [0.282288] x5 :  x4 : 
>> [0.282290] x3 : 3a657275 x2 : 
>> [0.282292] x1 :  x0 : 000c
>> [0.282294]
>> [0.282294] PC: 0xff93b3f03d50:
>> [0.282308] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
>> aa1603e0 f90007e8
>> [0.282315] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
>> 913ee400 94533cc7
>> [0.282322] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
>> f940406b 2a0203e8
>> [0.282329] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
>> a9bf7bfd 910003fd
>> [0.282330]
>> [0.282330] LR: 0xff93b3f03d50:
>> [0.282336] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
>> aa1603e0 f90007e8
>> [0.282343] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
>> 913ee400 94533cc7
>> [0.282350] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
>> f940406b 2a0203e8
>> [0.282357] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
>> a9bf7bfd 910003fd
>> [0.282358]
>> [0.282358] SP: 0xfffbb30cfbf0:
>> [0.282365] fbf0  b3f03d90 ff93 b30cfc30 fffb b3f03d90 ff93 
>> 60c00085 
>> [0.282372] fc10  b6415000 ff93 b642fa00 ff93   
>> b3f03d90 ff93
>> [0.282378] fc30  b30cfc70 fffb b3d458c0 ff93 0080  
>> 0001 
>> [0.282385] fc50  b65c4000 ff93 b64420f0 ff93 8180e3e0  
>> 0002 
>> [0.282387] Process migration/2 (pid: 25, stack limit = 
>> 0xfffbb30cc000)
>> [0.282389] Call trace:
>> [0.282391] Exception stack(0xfffbb30cfb00 to 0xfffbb30cfc30)
>> [   

Re: Linux 4.9.93

[Ard Biesheuvel]

On 9 April 2018 at 11:30, Greg KH  wrote:
> On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
>> Hi,
>>
>> After this patchset, a kernel built with CFI fails. Disabling
>> UNMAP_KERNEL_AT_EL0 fix the issue obviously.

How does one 'build a kernel with CFI' for arm64?

>
> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> android-common-4.9?
>
>> Wondering if there is one of the test suite used on the review patchset that 
>> covers the CFI usecase.
>>
>> Best regards,
>>
>> [0.249191] CPU features: detected feature: GIC system register CPU 
>> interface
>> [0.256391] CPU features: detected feature: Privileged Access Never
>> [0.262719] CPU features: detected feature: User Access Override
>> [0.268791] CPU features: detected feature: 32-bit EL0 Support
>> [0.274683] CPU features: detected feature: Kernel page table isolation 
>> (KPTI)
>> [0.282166] CFI failure:
>> [0.282169] CFI failure:
>> [0.282172] CFI failure:
>> [0.282173] CFI failure:
>> [0.282175] CFI failure:
>> [0.282176] CFI failure:
>> [0.282177] CFI failure:
>> [0.282178] CFI failure:
>> [0.282188] [ cut here ]
>> [0.282189] [ cut here ]
>> [0.282190] [ cut here ]
>> [0.282191] [ cut here ]
>> [0.282193] [ cut here ]
>> [0.282196] kernel BUG at kernel/cfi.c:32!
>> [0.282198] [ cut here ]
>> [0.282201] kernel BUG at kernel/cfi.c:32!
>> [0.282202] [ cut here ]
>> [0.282204] kernel BUG at kernel/cfi.c:32!
>> [0.282207] kernel BUG at kernel/cfi.c:32!
>> [0.282209] kernel BUG at kernel/cfi.c:32!
>> [0.282211] kernel BUG at kernel/cfi.c:32!
>> [0.282214] kernel BUG at kernel/cfi.c:32!
>> [0.282215] [ cut here ]
>> [0.282216] kernel BUG at kernel/cfi.c:32!
>> [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>> [0.282224] Modules linked in:
>> [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
>> [0.282232] Hardware name: 
>> [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
>> [0.282250] PC is at __cfi_check_fail+0x14/0x1c
>> [0.282253] LR is at __cfi_check_fail+0x14/0x1c
>> [0.282255] pc : [] lr : [] pstate: 
>> 60c00085
>> [0.282256] sp : fffbb30cfc30
>> [0.282259] x29: fffbb30cfc30 x28: ff93b6415000
>> [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000
>> [0.282264] x25: ff93b5ce6000 x24: ff93b6419000
>> [0.282266] x23: ff93b65c1000 x22: ff93b65c4000
>> [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0
>> [0.282271] x19:  x18: 002c
>> [0.282274] x17: 000fd054 x16: 
>> [0.282276] x15: ff93b65ec000 x14: 000c
>> [0.282279] x13: 0004 x12: 
>> [0.282281] x11:  x10: 01440144
>> [0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000
>> [0.282286] x7 :  x6 : fffbbac75b60
>> [0.282288] x5 :  x4 : 
>> [0.282290] x3 : 3a657275 x2 : 
>> [0.282292] x1 :  x0 : 000c
>> [0.282294]
>> [0.282294] PC: 0xff93b3f03d50:
>> [0.282308] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
>> aa1603e0 f90007e8
>> [0.282315] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
>> 913ee400 94533cc7
>> [0.282322] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
>> f940406b 2a0203e8
>> [0.282329] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
>> a9bf7bfd 910003fd
>> [0.282330]
>> [0.282330] LR: 0xff93b3f03d50:
>> [0.282336] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
>> aa1603e0 f90007e8
>> [0.282343] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
>> 913ee400 94533cc7
>> [0.282350] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
>> f940406b 2a0203e8
>> [0.282357] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
>> a9bf7bfd 910003fd
>> [0.282358]
>> [0.282358] SP: 0xfffbb30cfbf0:
>> [0.282365] fbf0  b3f03d90 ff93 b30cfc30 fffb b3f03d90 ff93 
>> 60c00085 
>> [0.282372] fc10  b6415000 ff93 b642fa00 ff93   
>> b3f03d90 ff93
>> [0.282378] fc30  b30cfc70 fffb b3d458c0 ff93 0080  
>> 0001 
>> [0.282385] fc50  b65c4000 ff93 b64420f0 ff93 8180e3e0  
>> 0002 
>> [0.282387] Process migration/2 (pid: 25, stack limit = 
>> 0xfffbb30cc000)
>> [0.282389] Call trace:
>> [0.282391] Exception stack(0xfffbb30cfb00 to 0xfffbb30cfc30)
>> [0.282395] fb00: 

Re: Linux 4.9.93

[Marc Zyngier]

On 09/04/18 10:30, Greg KH wrote:
> On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
>> Hi,
>>
>> After this patchset, a kernel built with CFI fails. Disabling
>> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> 
> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> android-common-4.9?
> 
>> Wondering if there is one of the test suite used on the review patchset that 
>> covers the CFI usecase.
>>
>> Best regards,
>>
>> [0.249191] CPU features: detected feature: GIC system register CPU 
>> interface
>> [0.256391] CPU features: detected feature: Privileged Access Never
>> [0.262719] CPU features: detected feature: User Access Override
>> [0.268791] CPU features: detected feature: 32-bit EL0 Support
>> [0.274683] CPU features: detected feature: Kernel page table isolation 
>> (KPTI)
>> [0.282166] CFI failure:
>> [0.282169] CFI failure:
>> [0.282172] CFI failure:
>> [0.282173] CFI failure:
>> [0.282175] CFI failure:
>> [0.282176] CFI failure:
>> [0.282177] CFI failure:
>> [0.282178] CFI failure:
>> [0.282188] [ cut here ]
>> [0.282189] [ cut here ]
>> [0.282190] [ cut here ]
>> [0.282191] [ cut here ]
>> [0.282193] [ cut here ]
>> [0.282196] kernel BUG at kernel/cfi.c:32!
>> [0.282198] [ cut here ]
>> [0.282201] kernel BUG at kernel/cfi.c:32!
>> [0.282202] [ cut here ]
>> [0.282204] kernel BUG at kernel/cfi.c:32!
>> [0.282207] kernel BUG at kernel/cfi.c:32!
>> [0.282209] kernel BUG at kernel/cfi.c:32!
>> [0.282211] kernel BUG at kernel/cfi.c:32!
>> [0.282214] kernel BUG at kernel/cfi.c:32!
>> [0.282215] [ cut here ]
>> [0.282216] kernel BUG at kernel/cfi.c:32!
>> [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>> [0.282224] Modules linked in:
>> [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
>> [0.282232] Hardware name: 
>> [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
>> [0.282250] PC is at __cfi_check_fail+0x14/0x1c
>> [0.282253] LR is at __cfi_check_fail+0x14/0x1c
>> [0.282255] pc : [] lr : [] pstate: 
>> 60c00085
>> [0.282256] sp : fffbb30cfc30
>> [0.282259] x29: fffbb30cfc30 x28: ff93b6415000 
>> [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000 
>> [0.282264] x25: ff93b5ce6000 x24: ff93b6419000 
>> [0.282266] x23: ff93b65c1000 x22: ff93b65c4000 
>> [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0 
>> [0.282271] x19:  x18: 002c 
>> [0.282274] x17: 000fd054 x16:  
>> [0.282276] x15: ff93b65ec000 x14: 000c 
>> [0.282279] x13: 0004 x12:  
>> [0.282281] x11:  x10: 01440144 
>> [0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000 
>> [0.282286] x7 :  x6 : fffbbac75b60 
>> [0.282288] x5 :  x4 :  
>> [0.282290] x3 : 3a657275 x2 :  
>> [0.282292] x1 :  x0 : 000c 
>> [0.282294] 
>> [0.282294] PC: 0xff93b3f03d50:
>> [0.282308] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
>> aa1603e0 f90007e8
>> [0.282315] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
>> 913ee400 94533cc7
>> [0.282322] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
>> f940406b 2a0203e8
>> [0.282329] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
>> a9bf7bfd 910003fd
>> [0.282330] 
>> [0.282330] LR: 0xff93b3f03d50:
>> [0.282336] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
>> aa1603e0 f90007e8
>> [0.282343] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
>> 913ee400 94533cc7
>> [0.282350] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
>> f940406b 2a0203e8
>> [0.282357] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
>> a9bf7bfd 910003fd
>> [0.282358] 
>> [0.282358] SP: 0xfffbb30cfbf0:
>> [0.282365] fbf0  b3f03d90 ff93 b30cfc30 fffb b3f03d90 ff93 
>> 60c00085 
>> [0.282372] fc10  b6415000 ff93 b642fa00 ff93   
>> b3f03d90 ff93
>> [0.282378] fc30  b30cfc70 fffb b3d458c0 ff93 0080  
>> 0001 
>> [0.282385] fc50  b65c4000 ff93 b64420f0 ff93 8180e3e0  
>> 0002 
>> [0.282387] Process migration/2 (pid: 25, stack limit = 
>> 0xfffbb30cc000)
>> [0.282389] Call trace:
>> [0.282391] Exception stack(0xfffbb30cfb00 to 0xfffbb30cfc30)
>> [0.282395] fb00: 000c   

Re: Linux 4.9.93

[Marc Zyngier]

On 09/04/18 10:30, Greg KH wrote:
> On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
>> Hi,
>>
>> After this patchset, a kernel built with CFI fails. Disabling
>> UNMAP_KERNEL_AT_EL0 fix the issue obviously.
> 
> Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
> android-common-4.9?
> 
>> Wondering if there is one of the test suite used on the review patchset that 
>> covers the CFI usecase.
>>
>> Best regards,
>>
>> [0.249191] CPU features: detected feature: GIC system register CPU 
>> interface
>> [0.256391] CPU features: detected feature: Privileged Access Never
>> [0.262719] CPU features: detected feature: User Access Override
>> [0.268791] CPU features: detected feature: 32-bit EL0 Support
>> [0.274683] CPU features: detected feature: Kernel page table isolation 
>> (KPTI)
>> [0.282166] CFI failure:
>> [0.282169] CFI failure:
>> [0.282172] CFI failure:
>> [0.282173] CFI failure:
>> [0.282175] CFI failure:
>> [0.282176] CFI failure:
>> [0.282177] CFI failure:
>> [0.282178] CFI failure:
>> [0.282188] [ cut here ]
>> [0.282189] [ cut here ]
>> [0.282190] [ cut here ]
>> [0.282191] [ cut here ]
>> [0.282193] [ cut here ]
>> [0.282196] kernel BUG at kernel/cfi.c:32!
>> [0.282198] [ cut here ]
>> [0.282201] kernel BUG at kernel/cfi.c:32!
>> [0.282202] [ cut here ]
>> [0.282204] kernel BUG at kernel/cfi.c:32!
>> [0.282207] kernel BUG at kernel/cfi.c:32!
>> [0.282209] kernel BUG at kernel/cfi.c:32!
>> [0.282211] kernel BUG at kernel/cfi.c:32!
>> [0.282214] kernel BUG at kernel/cfi.c:32!
>> [0.282215] [ cut here ]
>> [0.282216] kernel BUG at kernel/cfi.c:32!
>> [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
>> [0.282224] Modules linked in:
>> [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
>> [0.282232] Hardware name: 
>> [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
>> [0.282250] PC is at __cfi_check_fail+0x14/0x1c
>> [0.282253] LR is at __cfi_check_fail+0x14/0x1c
>> [0.282255] pc : [] lr : [] pstate: 
>> 60c00085
>> [0.282256] sp : fffbb30cfc30
>> [0.282259] x29: fffbb30cfc30 x28: ff93b6415000 
>> [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000 
>> [0.282264] x25: ff93b5ce6000 x24: ff93b6419000 
>> [0.282266] x23: ff93b65c1000 x22: ff93b65c4000 
>> [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0 
>> [0.282271] x19:  x18: 002c 
>> [0.282274] x17: 000fd054 x16:  
>> [0.282276] x15: ff93b65ec000 x14: 000c 
>> [0.282279] x13: 0004 x12:  
>> [0.282281] x11:  x10: 01440144 
>> [0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000 
>> [0.282286] x7 :  x6 : fffbbac75b60 
>> [0.282288] x5 :  x4 :  
>> [0.282290] x3 : 3a657275 x2 :  
>> [0.282292] x1 :  x0 : 000c 
>> [0.282294] 
>> [0.282294] PC: 0xff93b3f03d50:
>> [0.282308] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
>> aa1603e0 f90007e8
>> [0.282315] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
>> 913ee400 94533cc7
>> [0.282322] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
>> f940406b 2a0203e8
>> [0.282329] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
>> a9bf7bfd 910003fd
>> [0.282330] 
>> [0.282330] LR: 0xff93b3f03d50:
>> [0.282336] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
>> aa1603e0 f90007e8
>> [0.282343] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
>> 913ee400 94533cc7
>> [0.282350] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
>> f940406b 2a0203e8
>> [0.282357] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
>> a9bf7bfd 910003fd
>> [0.282358] 
>> [0.282358] SP: 0xfffbb30cfbf0:
>> [0.282365] fbf0  b3f03d90 ff93 b30cfc30 fffb b3f03d90 ff93 
>> 60c00085 
>> [0.282372] fc10  b6415000 ff93 b642fa00 ff93   
>> b3f03d90 ff93
>> [0.282378] fc30  b30cfc70 fffb b3d458c0 ff93 0080  
>> 0001 
>> [0.282385] fc50  b65c4000 ff93 b64420f0 ff93 8180e3e0  
>> 0002 
>> [0.282387] Process migration/2 (pid: 25, stack limit = 
>> 0xfffbb30cc000)
>> [0.282389] Call trace:
>> [0.282391] Exception stack(0xfffbb30cfb00 to 0xfffbb30cfc30)
>> [0.282395] fb00: 000c   

Re: Linux 4.9.93

[Greg KH]

On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> Hi,
> 
> After this patchset, a kernel built with CFI fails. Disabling
> UNMAP_KERNEL_AT_EL0 fix the issue obviously.

Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
android-common-4.9?

> Wondering if there is one of the test suite used on the review patchset that 
> covers the CFI usecase.
> 
> Best regards,
> 
> [0.249191] CPU features: detected feature: GIC system register CPU 
> interface
> [0.256391] CPU features: detected feature: Privileged Access Never
> [0.262719] CPU features: detected feature: User Access Override
> [0.268791] CPU features: detected feature: 32-bit EL0 Support
> [0.274683] CPU features: detected feature: Kernel page table isolation 
> (KPTI)
> [0.282166] CFI failure:
> [0.282169] CFI failure:
> [0.282172] CFI failure:
> [0.282173] CFI failure:
> [0.282175] CFI failure:
> [0.282176] CFI failure:
> [0.282177] CFI failure:
> [0.282178] CFI failure:
> [0.282188] [ cut here ]
> [0.282189] [ cut here ]
> [0.282190] [ cut here ]
> [0.282191] [ cut here ]
> [0.282193] [ cut here ]
> [0.282196] kernel BUG at kernel/cfi.c:32!
> [0.282198] [ cut here ]
> [0.282201] kernel BUG at kernel/cfi.c:32!
> [0.282202] [ cut here ]
> [0.282204] kernel BUG at kernel/cfi.c:32!
> [0.282207] kernel BUG at kernel/cfi.c:32!
> [0.282209] kernel BUG at kernel/cfi.c:32!
> [0.282211] kernel BUG at kernel/cfi.c:32!
> [0.282214] kernel BUG at kernel/cfi.c:32!
> [0.282215] [ cut here ]
> [0.282216] kernel BUG at kernel/cfi.c:32!
> [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
> [0.282224] Modules linked in:
> [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
> [0.282232] Hardware name: 
> [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
> [0.282250] PC is at __cfi_check_fail+0x14/0x1c
> [0.282253] LR is at __cfi_check_fail+0x14/0x1c
> [0.282255] pc : [] lr : [] pstate: 
> 60c00085
> [0.282256] sp : fffbb30cfc30
> [0.282259] x29: fffbb30cfc30 x28: ff93b6415000 
> [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000 
> [0.282264] x25: ff93b5ce6000 x24: ff93b6419000 
> [0.282266] x23: ff93b65c1000 x22: ff93b65c4000 
> [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0 
> [0.282271] x19:  x18: 002c 
> [0.282274] x17: 000fd054 x16:  
> [0.282276] x15: ff93b65ec000 x14: 000c 
> [0.282279] x13: 0004 x12:  
> [0.282281] x11:  x10: 01440144 
> [0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000 
> [0.282286] x7 :  x6 : fffbbac75b60 
> [0.282288] x5 :  x4 :  
> [0.282290] x3 : 3a657275 x2 :  
> [0.282292] x1 :  x0 : 000c 
> [0.282294] 
> [0.282294] PC: 0xff93b3f03d50:
> [0.282308] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
> aa1603e0 f90007e8
> [0.282315] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
> 913ee400 94533cc7
> [0.282322] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
> f940406b 2a0203e8
> [0.282329] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
> a9bf7bfd 910003fd
> [0.282330] 
> [0.282330] LR: 0xff93b3f03d50:
> [0.282336] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
> aa1603e0 f90007e8
> [0.282343] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
> 913ee400 94533cc7
> [0.282350] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
> f940406b 2a0203e8
> [0.282357] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
> a9bf7bfd 910003fd
> [0.282358] 
> [0.282358] SP: 0xfffbb30cfbf0:
> [0.282365] fbf0  b3f03d90 ff93 b30cfc30 fffb b3f03d90 ff93 
> 60c00085 
> [0.282372] fc10  b6415000 ff93 b642fa00 ff93   
> b3f03d90 ff93
> [0.282378] fc30  b30cfc70 fffb b3d458c0 ff93 0080  
> 0001 
> [0.282385] fc50  b65c4000 ff93 b64420f0 ff93 8180e3e0  
> 0002 
> [0.282387] Process migration/2 (pid: 25, stack limit = 0xfffbb30cc000)
> [0.282389] Call trace:
> [0.282391] Exception stack(0xfffbb30cfb00 to 0xfffbb30cfc30)
> [0.282395] fb00: 000c   
> 3a657275
> [0.282397] fb20:   fffbbac75b60 
> 
> [0.282400] fb40: 

Re: Linux 4.9.93

[Greg KH]

On Mon, Apr 09, 2018 at 06:05:34PM +0900, Jean-Baptiste Theou wrote:
> Hi,
> 
> After this patchset, a kernel built with CFI fails. Disabling
> UNMAP_KERNEL_AT_EL0 fix the issue obviously.

Is this a "clean" 4.9.93 tree or a "4.9.93 merged into
android-common-4.9?

> Wondering if there is one of the test suite used on the review patchset that 
> covers the CFI usecase.
> 
> Best regards,
> 
> [0.249191] CPU features: detected feature: GIC system register CPU 
> interface
> [0.256391] CPU features: detected feature: Privileged Access Never
> [0.262719] CPU features: detected feature: User Access Override
> [0.268791] CPU features: detected feature: 32-bit EL0 Support
> [0.274683] CPU features: detected feature: Kernel page table isolation 
> (KPTI)
> [0.282166] CFI failure:
> [0.282169] CFI failure:
> [0.282172] CFI failure:
> [0.282173] CFI failure:
> [0.282175] CFI failure:
> [0.282176] CFI failure:
> [0.282177] CFI failure:
> [0.282178] CFI failure:
> [0.282188] [ cut here ]
> [0.282189] [ cut here ]
> [0.282190] [ cut here ]
> [0.282191] [ cut here ]
> [0.282193] [ cut here ]
> [0.282196] kernel BUG at kernel/cfi.c:32!
> [0.282198] [ cut here ]
> [0.282201] kernel BUG at kernel/cfi.c:32!
> [0.282202] [ cut here ]
> [0.282204] kernel BUG at kernel/cfi.c:32!
> [0.282207] kernel BUG at kernel/cfi.c:32!
> [0.282209] kernel BUG at kernel/cfi.c:32!
> [0.282211] kernel BUG at kernel/cfi.c:32!
> [0.282214] kernel BUG at kernel/cfi.c:32!
> [0.282215] [ cut here ]
> [0.282216] kernel BUG at kernel/cfi.c:32!
> [0.282218] Internal error: Oops - BUG: 0 [#1] PREEMPT SMP
> [0.282224] Modules linked in:
> [0.282230] CPU: 2 PID: 25 Comm: migration/2 Not tainted 4.9.93-perf+ #39
> [0.282232] Hardware name: 
> [0.282235] task: fffbb3b36580 task.stack: fffbb30cc000
> [0.282250] PC is at __cfi_check_fail+0x14/0x1c
> [0.282253] LR is at __cfi_check_fail+0x14/0x1c
> [0.282255] pc : [] lr : [] pstate: 
> 60c00085
> [0.282256] sp : fffbb30cfc30
> [0.282259] x29: fffbb30cfc30 x28: ff93b6415000 
> [0.282261] x27: 0013b65c1000 x26: ff93b5ce6000 
> [0.282264] x25: ff93b5ce6000 x24: ff93b6419000 
> [0.282266] x23: ff93b65c1000 x22: ff93b65c4000 
> [0.282268] x21: 9d12f8172cb2f296 x20: 8180e3e0 
> [0.282271] x19:  x18: 002c 
> [0.282274] x17: 000fd054 x16:  
> [0.282276] x15: ff93b65ec000 x14: 000c 
> [0.282279] x13: 0004 x12:  
> [0.282281] x11:  x10: 01440144 
> [0.282283] x9 : 260822e8751d5000 x8 : 260822e8751d5000 
> [0.282286] x7 :  x6 : fffbbac75b60 
> [0.282288] x5 :  x4 :  
> [0.282290] x3 : 3a657275 x2 :  
> [0.282292] x1 :  x0 : 000c 
> [0.282294] 
> [0.282294] PC: 0xff93b3f03d50:
> [0.282308] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
> aa1603e0 f90007e8
> [0.282315] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
> 913ee400 94533cc7
> [0.282322] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
> f940406b 2a0203e8
> [0.282329] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
> a9bf7bfd 910003fd
> [0.282330] 
> [0.282330] LR: 0xff93b3f03d50:
> [0.282336] 3d50  b9001ac8 f94002c8 370ffec8 17be d421 1400 
> aa1603e0 f90007e8
> [0.282343] 3d70  94536017 f94007e8 17e2 a9bf7bfd 910003fd d000d100 
> 913ee400 94533cc7
> [0.282350] 3d90  d421 1400 b0013788 2a1f03e0 f901c51f d65f03c0 
> f940406b 2a0203e8
> [0.282357] 3db0  2a0103e9 aa0003ea b48b f9000145 f94000cb b40001ab 
> a9bf7bfd 910003fd
> [0.282358] 
> [0.282358] SP: 0xfffbb30cfbf0:
> [0.282365] fbf0  b3f03d90 ff93 b30cfc30 fffb b3f03d90 ff93 
> 60c00085 
> [0.282372] fc10  b6415000 ff93 b642fa00 ff93   
> b3f03d90 ff93
> [0.282378] fc30  b30cfc70 fffb b3d458c0 ff93 0080  
> 0001 
> [0.282385] fc50  b65c4000 ff93 b64420f0 ff93 8180e3e0  
> 0002 
> [0.282387] Process migration/2 (pid: 25, stack limit = 0xfffbb30cc000)
> [0.282389] Call trace:
> [0.282391] Exception stack(0xfffbb30cfb00 to 0xfffbb30cfc30)
> [0.282395] fb00: 000c   
> 3a657275
> [0.282397] fb20:   fffbbac75b60 
> 
> [0.282400] fb40: 

Re: Linux 4.9.93

[Greg KH]

diff --git a/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 
b/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt
index caf297bee1fb..c28d4eb83b76 100644
--- a/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt
+++ b/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt
@@ -35,6 +35,15 @@ Optional properties:
 - ti,palmas-enable-dvfs2: Enable DVFS2. Configure pins for DVFS2 mode.
Selection primary or secondary function associated to GPADC_START
and SYSEN2 pin/pad for DVFS2 interface
+- ti,palmas-override-powerhold: This is applicable for PMICs for which
+   GPIO7 is configured in POWERHOLD mode which has higher priority
+   over DEV_ON bit and keeps the PMIC supplies on even after the DEV_ON
+   bit is turned off. This property enables driver to over ride the
+   POWERHOLD value to GPIO7 so as to turn off the PMIC in power off
+   scenarios. So for GPIO7 if ti,palmas-override-powerhold is set
+   then the GPIO_7 field should never be muxed to anything else.
+   It should be set to POWERHOLD by default and only in case of
+   power off scenarios the driver will over ride the mux value.
 
 This binding uses the following generic properties as defined in
 pinctrl-bindings.txt:
diff --git a/Makefile b/Makefile
index 3ab3b8203bf6..f5cf4159fc20 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,6 @@
 VERSION = 4
 PATCHLEVEL = 9
-SUBLEVEL = 92
+SUBLEVEL = 93
 EXTRAVERSION =
 NAME = Roaring Lionus
 
diff --git a/arch/arm/boot/dts/am335x-pepper.dts 
b/arch/arm/boot/dts/am335x-pepper.dts
index 42b62f54e4b7..30e2f8770aaf 100644
--- a/arch/arm/boot/dts/am335x-pepper.dts
+++ b/arch/arm/boot/dts/am335x-pepper.dts
@@ -139,7 +139,7 @@
 _codec {
status = "okay";
 
-   reset-gpios = < 16 GPIO_ACTIVE_LOW>;
+   gpio-reset = < 16 GPIO_ACTIVE_LOW>;
AVDD-supply = <_reg>;
IOVDD-supply = <_reg>;
DRVDD-supply = <_reg>;
diff --git a/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi 
b/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi
index 6df7829a2c15..78bee26361f1 100644
--- a/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi
+++ b/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi
@@ -204,6 +204,7 @@
interrupt-controller;
 
ti,system-power-controller;
+   ti,palmas-override-powerhold;
 
tps659038_pmic {
compatible = "ti,tps659038-pmic";
diff --git a/arch/arm/boot/dts/am57xx-idk-common.dtsi 
b/arch/arm/boot/dts/am57xx-idk-common.dtsi
index db858fff4e18..1cc62727e43a 100644
--- a/arch/arm/boot/dts/am57xx-idk-common.dtsi
+++ b/arch/arm/boot/dts/am57xx-idk-common.dtsi
@@ -57,6 +57,7 @@
#interrupt-cells = <2>;
interrupt-controller;
ti,system-power-controller;
+   ti,palmas-override-powerhold;
 
tps659038_pmic {
compatible = "ti,tps659038-pmic";
diff --git a/arch/arm/boot/dts/dra7-evm.dts b/arch/arm/boot/dts/dra7-evm.dts
index 132f2be10889..56311fd34f81 100644
--- a/arch/arm/boot/dts/dra7-evm.dts
+++ b/arch/arm/boot/dts/dra7-evm.dts
@@ -398,6 +398,8 @@
tps659038: tps659038@58 {
compatible = "ti,tps659038";
reg = <0x58>;
+   ti,palmas-override-powerhold;
+   ti,system-power-controller;
 
tps659038_pmic {
compatible = "ti,tps659038-pmic";
diff --git a/arch/arm/boot/dts/omap3-n900.dts b/arch/arm/boot/dts/omap3-n900.dts
index 6003b29c0fc0..4d448f145ed1 100644
--- a/arch/arm/boot/dts/omap3-n900.dts
+++ b/arch/arm/boot/dts/omap3-n900.dts
@@ -510,7 +510,7 @@
tlv320aic3x: tlv320aic3x@18 {
compatible = "ti,tlv320aic3x";
reg = <0x18>;
-   reset-gpios = < 28 GPIO_ACTIVE_LOW>; /* 60 */
+   gpio-reset = < 28 GPIO_ACTIVE_HIGH>; /* 60 */
ai3x-gpio-func = <
0 /* AIC3X_GPIO1_FUNC_DISABLED */
5 /* AIC3X_GPIO2_FUNC_DIGITAL_MIC_INPUT */
@@ -527,7 +527,7 @@
tlv320aic3x_aux: tlv320aic3x@19 {
compatible = "ti,tlv320aic3x";
reg = <0x19>;
-   reset-gpios = < 28 GPIO_ACTIVE_LOW>; /* 60 */
+   gpio-reset = < 28 GPIO_ACTIVE_HIGH>; /* 60 */
 
AVDD-supply = <>;
DRVDD-supply = <>;
diff --git a/arch/arm/vfp/vfpmodule.c b/arch/arm/vfp/vfpmodule.c
index da0b33deba6d..5629d7580973 100644
--- a/arch/arm/vfp/vfpmodule.c
+++ b/arch/arm/vfp/vfpmodule.c
@@ -648,7 +648,7 @@ int vfp_restore_user_hwstate(struct user_vfp __user *ufp,
  */
 static int vfp_dying_cpu(unsigned int cpu)
 {
-   vfp_force_reload(cpu, current_thread_info());
+   vfp_current_hw_state[cpu] = NULL;
return 0;
 }
 
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 7769c2e27788..c8471cf46cbb 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -733,6 +733,18 

Re: Linux 4.9.93

[Greg KH]

diff --git a/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt 
b/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt
index caf297bee1fb..c28d4eb83b76 100644
--- a/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt
+++ b/Documentation/devicetree/bindings/pinctrl/pinctrl-palmas.txt
@@ -35,6 +35,15 @@ Optional properties:
 - ti,palmas-enable-dvfs2: Enable DVFS2. Configure pins for DVFS2 mode.
Selection primary or secondary function associated to GPADC_START
and SYSEN2 pin/pad for DVFS2 interface
+- ti,palmas-override-powerhold: This is applicable for PMICs for which
+   GPIO7 is configured in POWERHOLD mode which has higher priority
+   over DEV_ON bit and keeps the PMIC supplies on even after the DEV_ON
+   bit is turned off. This property enables driver to over ride the
+   POWERHOLD value to GPIO7 so as to turn off the PMIC in power off
+   scenarios. So for GPIO7 if ti,palmas-override-powerhold is set
+   then the GPIO_7 field should never be muxed to anything else.
+   It should be set to POWERHOLD by default and only in case of
+   power off scenarios the driver will over ride the mux value.
 
 This binding uses the following generic properties as defined in
 pinctrl-bindings.txt:
diff --git a/Makefile b/Makefile
index 3ab3b8203bf6..f5cf4159fc20 100644
--- a/Makefile
+++ b/Makefile
@@ -1,6 +1,6 @@
 VERSION = 4
 PATCHLEVEL = 9
-SUBLEVEL = 92
+SUBLEVEL = 93
 EXTRAVERSION =
 NAME = Roaring Lionus
 
diff --git a/arch/arm/boot/dts/am335x-pepper.dts 
b/arch/arm/boot/dts/am335x-pepper.dts
index 42b62f54e4b7..30e2f8770aaf 100644
--- a/arch/arm/boot/dts/am335x-pepper.dts
+++ b/arch/arm/boot/dts/am335x-pepper.dts
@@ -139,7 +139,7 @@
 _codec {
status = "okay";
 
-   reset-gpios = < 16 GPIO_ACTIVE_LOW>;
+   gpio-reset = < 16 GPIO_ACTIVE_LOW>;
AVDD-supply = <_reg>;
IOVDD-supply = <_reg>;
DRVDD-supply = <_reg>;
diff --git a/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi 
b/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi
index 6df7829a2c15..78bee26361f1 100644
--- a/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi
+++ b/arch/arm/boot/dts/am57xx-beagle-x15-common.dtsi
@@ -204,6 +204,7 @@
interrupt-controller;
 
ti,system-power-controller;
+   ti,palmas-override-powerhold;
 
tps659038_pmic {
compatible = "ti,tps659038-pmic";
diff --git a/arch/arm/boot/dts/am57xx-idk-common.dtsi 
b/arch/arm/boot/dts/am57xx-idk-common.dtsi
index db858fff4e18..1cc62727e43a 100644
--- a/arch/arm/boot/dts/am57xx-idk-common.dtsi
+++ b/arch/arm/boot/dts/am57xx-idk-common.dtsi
@@ -57,6 +57,7 @@
#interrupt-cells = <2>;
interrupt-controller;
ti,system-power-controller;
+   ti,palmas-override-powerhold;
 
tps659038_pmic {
compatible = "ti,tps659038-pmic";
diff --git a/arch/arm/boot/dts/dra7-evm.dts b/arch/arm/boot/dts/dra7-evm.dts
index 132f2be10889..56311fd34f81 100644
--- a/arch/arm/boot/dts/dra7-evm.dts
+++ b/arch/arm/boot/dts/dra7-evm.dts
@@ -398,6 +398,8 @@
tps659038: tps659038@58 {
compatible = "ti,tps659038";
reg = <0x58>;
+   ti,palmas-override-powerhold;
+   ti,system-power-controller;
 
tps659038_pmic {
compatible = "ti,tps659038-pmic";
diff --git a/arch/arm/boot/dts/omap3-n900.dts b/arch/arm/boot/dts/omap3-n900.dts
index 6003b29c0fc0..4d448f145ed1 100644
--- a/arch/arm/boot/dts/omap3-n900.dts
+++ b/arch/arm/boot/dts/omap3-n900.dts
@@ -510,7 +510,7 @@
tlv320aic3x: tlv320aic3x@18 {
compatible = "ti,tlv320aic3x";
reg = <0x18>;
-   reset-gpios = < 28 GPIO_ACTIVE_LOW>; /* 60 */
+   gpio-reset = < 28 GPIO_ACTIVE_HIGH>; /* 60 */
ai3x-gpio-func = <
0 /* AIC3X_GPIO1_FUNC_DISABLED */
5 /* AIC3X_GPIO2_FUNC_DIGITAL_MIC_INPUT */
@@ -527,7 +527,7 @@
tlv320aic3x_aux: tlv320aic3x@19 {
compatible = "ti,tlv320aic3x";
reg = <0x19>;
-   reset-gpios = < 28 GPIO_ACTIVE_LOW>; /* 60 */
+   gpio-reset = < 28 GPIO_ACTIVE_HIGH>; /* 60 */
 
AVDD-supply = <>;
DRVDD-supply = <>;
diff --git a/arch/arm/vfp/vfpmodule.c b/arch/arm/vfp/vfpmodule.c
index da0b33deba6d..5629d7580973 100644
--- a/arch/arm/vfp/vfpmodule.c
+++ b/arch/arm/vfp/vfpmodule.c
@@ -648,7 +648,7 @@ int vfp_restore_user_hwstate(struct user_vfp __user *ufp,
  */
 static int vfp_dying_cpu(unsigned int cpu)
 {
-   vfp_force_reload(cpu, current_thread_info());
+   vfp_current_hw_state[cpu] = NULL;
return 0;
 }
 
diff --git a/arch/arm64/Kconfig b/arch/arm64/Kconfig
index 7769c2e27788..c8471cf46cbb 100644
--- a/arch/arm64/Kconfig
+++ b/arch/arm64/Kconfig
@@ -733,6 +733,18