Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
* Ingo Molnar wrote: > To quantify it: I just performed a test build of a Linux distro kernel config > (Fedora x86-64), and counted the number of callsites that use 'asm goto' > functionality with the v4.15 kernel (including drivers). > > The results: > > Linux distro | !CONFIG_TRACING > - > total # of functions : 191,567 | 184,443 > total # of instructions : 14,251,355 | 13,526,112 > - > total # of spin_lock*() calls: 25,246 | 25,177 > total # of mutex_lock*() calls : 13,062 | 12,861 > total # of kmalloc*() calls :5,148 | 5,118 > - > total # of 'asm goto' usage sites: 34,851 | 31,059 > total # of 'asm goto' using functions: 18,209 | 16,089 > - > percent of kernel functions using 'asm goto' : 9.5% |8.7% > - Here's the size stats of kernel/sched/built-in.o for the same distro config: optimized | no asm goto - total # of functions : 765 |764 total # of instructions : 46,830 | 47,051 I.e. asm goto support reduces scheduler size by ~0.5%, which is a major generated code size reduction. This doesn't count the live branch patching performance advantages: many of those asm goto usage sites are in hot paths, so the performance impact of it is much larger than that: easily a couple of percentage points in scheduler intensive benchmarks, as Peter mentioned. For example here's a thread context switch benchmark comparison on a modern x86 system running a v4.15 kernel: $ perf stat --repeat 20 --sync --null perf bench sched messaging -t -g 25 no asm goto: 0.136778505 seconds time elapsed ( stddev: +- 0.55% ) asm goto optimized: 0.133773904 seconds time elapsed ( stddev: +- 0.51% ) The asm goto enabled kernel is ~2.25% faster in this benchmark, and the performance penalty of not having asm goto support will only increase in the future. i.e. it very much makes sense to implement asm goto support not just for compatibility reasons, but for performance reasons as well. Thanks, Ingo
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
On Wed, Feb 14, 2018 at 09:31:22AM -0800, Kees Cook wrote: > On Wed, Feb 14, 2018 at 1:08 AM, Peter Zijlstra wrote: > > If you ever get to the point where it makes sense to build a kernel with > > LLVM [...] > > Just to speak to this point in particular: it makes sense to build > kernels with LLVM right now. It's already happened; millions of Pixel > 2 phones are already running with Clang-built kernels. And even if > you're only interested in x86, that works too. Only if you really don't care about performance. And as should be evident from the rest of this thread, you soon won't be able to build it anymore.
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
On Wed, Feb 14, 2018 at 7:31 AM, James Y Knight wrote: > > IMO, inline asm is, generally, a valuable feature to provide in the > compiler as an escape hatch, and asm goto is a relatively sane > extension of it. Side note: one thing that limits "asm goto" in gcc is the fact that you can't have outputs. If clang people are looking at "asm goto", the _syntax_ is actually very straightforward, and there's really absolutely no point in trying to make up some other interface. But extending on what gcc does, and allowing outputs (possibly valid in the fall-through case only, not in the cases where it jumps away to a label) would be a big improvement on what gcc does. At that point, we'd start to pester the gcc people to do the clang thing, rather than the other way around. Because that's only fair ;) Linus
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
On Wed, Feb 14, 2018 at 2:34 AM, Peter Zijlstra wrote: > > So it's far more than just tracepoints. We use it all over the kernel to > do runtime branch patching. Side note: I have a patch to the user access code to do "asm goto" for the exception handling too. In fact, the infrastructure is there already, but because we supported older gcc versions that didn't have asm goto, I never applied the final patch to actually make use of it. See "unsafe_put_user()", which currently does if (unlikely(__pu_err)) goto err_label; rather than just have the exception table itself directly reference "err_label", and not needing any code at all in the regular path. I suspect there are other things like that, where the lack of "asm goto" means that it's not worth the pain of having two code paths. Once we update to a newer gcc requirement, those suddenly will also very naturally use asm goto. The other asm use that the clang people might want to look at is the "condition codes as return value" one. That one we currently handle with CC_SET/CC_OUT, and it's somewhat more amenable to those kinds of games, so missing it isn't as nasty. But it *is* a useful "modern" extension of gcc asms. Maybe clang already supports cflags output, I haven't checked. Linus
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
On Wed, Feb 14, 2018 at 1:08 AM, Peter Zijlstra wrote: > If you ever get to the point where it makes sense to build a kernel with > LLVM [...] Just to speak to this point in particular: it makes sense to build kernels with LLVM right now. It's already happened; millions of Pixel 2 phones are already running with Clang-built kernels. And even if you're only interested in x86, that works too. -Kees -- Kees Cook Pixel Security
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
On Wed, Feb 14, 2018 at 3:31 PM, James Y Knight wrote: > I'd be definitely in favor having clang support asm goto. I wouldn't > want to exclude having other conversations about how to more directly > provide compiler features that the linux kernel could use, ALSO, but I > do not think that conversation should block implementing asm-goto. > > IMO, inline asm is, generally, a valuable feature to provide in the > compiler as an escape hatch, and asm goto is a relatively sane > extension of it. Supporting outgoing edges from an inline asm block is > a reasonable thing for users to desire, and as far as anyone's said so > far, seems like it ought to be fairly easily implementable, without > causing bad side-effects in the compiler. > > Of course, we generally do want to minimize the need for users to use > inline asm, by providing appropriate compiler support for the features > people would otherwise be forced to implement using asm. But I don't > see that as really any more important for asm-goto than any other > inline asm. There will always be a desire for escape hatches, to do > weird and unique things which aren't supported directly in the > compiler. (Also, the kernel is a pretty special case in terms of its > requirements, it seems exceedingly unlikely that we could ever provide > enough compiler support to let it eliminate inline asm.). > If clang implements asm goto, could you also pretty please make it better than GCC and support output constraints in the same asm statements? This would give us a very clean, short, and fast implementation for __get_user() and __put_user(), and it would also help put pressure on GCC to implement the same feature.
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
I'd be definitely in favor having clang support asm goto. I wouldn't want to exclude having other conversations about how to more directly provide compiler features that the linux kernel could use, ALSO, but I do not think that conversation should block implementing asm-goto. IMO, inline asm is, generally, a valuable feature to provide in the compiler as an escape hatch, and asm goto is a relatively sane extension of it. Supporting outgoing edges from an inline asm block is a reasonable thing for users to desire, and as far as anyone's said so far, seems like it ought to be fairly easily implementable, without causing bad side-effects in the compiler. Of course, we generally do want to minimize the need for users to use inline asm, by providing appropriate compiler support for the features people would otherwise be forced to implement using asm. But I don't see that as really any more important for asm-goto than any other inline asm. There will always be a desire for escape hatches, to do weird and unique things which aren't supported directly in the compiler. (Also, the kernel is a pretty special case in terms of its requirements, it seems exceedingly unlikely that we could ever provide enough compiler support to let it eliminate inline asm.). On Wed, Feb 14, 2018 at 3:33 AM, Yatsina, Marina wrote: > > Hi Kees, > > When I raised the question of whether we want to add support for "asm goto" > in llvm I got some feedback from the community that "asm goto" might not be > the best solution for the problem it was invented for (optimizing support for > tracepoints), so I stopped perusing this issue. > I'm CC-ing the developers that participated in the original thread and a few > developers that might be interested in adding support of "asm goto". > I'm also adding the llvm-dev mailing list, in case there are additional > parties interested in voicing their opinion. > > I hope this will give this issue a push forward and we will find a solution > that will not prevent llvm from compiling linux kernel. > > Thanks, > Marina > > -Original Message- > From: Kees Cook [mailto:keesc...@google.com] > Sent: Wednesday, February 14, 2018 02:29 > To: David Woodhouse ; Chandler Carruth > ; Yatsina, Marina > Cc: Guenter Roeck ; X86 ML ; LKML > ; Alan Cox ; Rik > van Riel ; Andi Kleen ; Josh Poimboeuf > ; Tom Lendacky ; Peter Zijlstra > ; Linus Torvalds ; Jiri > Kosina ; Andy Lutomirski ; Hansen, > Dave ; Tim Chen ; Greg > Kroah-Hartman ; Paul Turner ; > Stephen Hines ; Nick Desaulniers > Subject: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang > support) > > On Tue, Feb 13, 2018 at 4:10 PM, David Woodhouse wrote: > > We also need to resolve the asm-goto thing. > > Yes, this is becoming much more urgent, assuming we'll be raising the minimum > GCC version soon and drop support for lacking asm-goto... > > Do you happen to know who the right people are to include to move the > discussion forward? I know various kernel folks that are passionate about it, > but I'm still getting to know who to talk with from llvm. > > I see an earlier thread here: > http://lists.llvm.org/pipermail/llvm-dev/2017-April/111748.html > > It seems to end there? I'm still coming up to speed on it, so I'm likely > missing other context. > > -Kees > > -- > Kees Cook > Pixel Security > - > Intel Israel (74) Limited > > This e-mail and any attachments may contain confidential material for > the sole use of the intended recipient(s). Any review or distribution > by others is strictly prohibited. If you are not the intended > recipient, please contact the sender and delete all copies.
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
* Peter Zijlstra wrote: > On Wed, Feb 14, 2018 at 09:52:59AM +, Yatsina, Marina wrote: > > Hi Peter, > > > > When I started the original thread last year I was in favor of adding > > "asm goto" and didn't understand why it wasn't done by that time. The > > feedback I got is that this feature (optimizing tracepoints) is very > > useful and that we do want it in llvm, but perhaps there's a cleaner > > way of implementing than "asm goto". An alternative suggestion arose > > as well. > > So it's far more than just tracepoints. We use it all over the kernel to > do runtime branch patching. > > One example is avoiding the scheduler preemption callbacks if we know > there are no users. This shaves a few % off a context switch > micro-bench. > > But it is really _all_ over the place. To quantify it: I just performed a test build of a Linux distro kernel config (Fedora x86-64), and counted the number of callsites that use 'asm goto' functionality with the v4.15 kernel (including drivers). The results: Linux distro | !CONFIG_TRACING - total # of functions : 191,567 | 184,443 total # of instructions : 14,251,355 | 13,526,112 - total # of spin_lock*() calls: 25,246 | 25,177 total # of mutex_lock*() calls : 13,062 | 12,861 total # of kmalloc*() calls :5,148 | 5,118 - total # of 'asm goto' usage sites: 34,851 | 31,059 total # of 'asm goto' using functions: 18,209 | 16,089 - percent of kernel functions using 'asm goto' : 9.5% |8.7% - ( Note: I added the 'tracing disabled' column only to demonstrate how widely 'asm goto' is used beyond tracing - but since *all* major Linux distributions have tracing enabled it's the first column that matters in practice. ) So 'asm goto' is a major, major compiler feature the Linux kernel relies on: in fact in the Linux kernel there's more 'asm goto' usage sites than there are spin-lock critical sections! Thanks, Ingo
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
On Wed, 14 Feb 2018, Peter Zijlstra wrote: > On Wed, Feb 14, 2018 at 08:41:43AM +, Chandler Carruth wrote: > > Marina, Kees: > > > > I think the combination of LKML and the LLVM developer's list is probably > > too large a set of folks to have a (somewhat) lengthy exploration of > > options and how to go about this and find a good path forward. > > As one of the original authors of the jump-label implementation I'm > saying there are no options. We're not going to do a second parallel > implementation. If you want to build the kernel, you get to implement > asm-goto, teh end. Amen to that. As x86 maintainer I'm really interested and willing to support llvm/clang for x86 and the kernel in general, but asm-goto is not longer negotiable. asm-goto is in the kernel since Oct 2010 and the relevant llvm/clang bug #9295 is open since Feb 2011. All clang/llvm folks came up with in those 7 years is occasional handwaving about better solutions. We really try to support old tools and tools which lack features for quite a long time, but that's not a free pass for eternity. That said, I'm not willing to wait another 7 years and I'm going to pull the plug on compilers w/o asm-goto support soon whether llvm/clang gets their act together or not. Hint: GCC is not the only major compiler which supports asm-goto. Thanks, tglx
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
On Wed, Feb 14, 2018 at 09:52:59AM +, Yatsina, Marina wrote: > Hi Peter, > > When I started the original thread last year I was in favor of adding > "asm goto" and didn't understand why it wasn't done by that time. The > feedback I got is that this feature (optimizing tracepoints) is very > useful and that we do want it in llvm, but perhaps there's a cleaner > way of implementing than "asm goto". An alternative suggestion arose > as well. So it's far more than just tracepoints. We use it all over the kernel to do runtime branch patching. One example is avoiding the scheduler preemption callbacks if we know there are no users. This shaves a few % off a context switch micro-bench. But it is really _all_ over the place. > I'm sure you can provide a lot of background for the decisions of why > "asm goto" was chosen and which other alternatives were considered, as > you were the one to implement this. I have very little memories from back then, but it was mostly us asking for label addresses in asm and them giving us asm-goto. Using asm we can build our own primitives, and I realize the llvm community doesn't like asm much, but then again, we treat C like a glorified assembler and don't like our compilers too smart :-) > Anyway, I think we should consider the alternatives and not take "asm > goto" as a given. Far too late for that, 7+ years ago when we did this was the time to talk about alternatives, now we have this code base. So we have the two jump_label things: static __always_inline bool arch_static_branch(struct static_key *key, bool branch) { asm_volatile_goto("1:" ".byte " __stringify(STATIC_KEY_INIT_NOP) "\n\t" ".pushsection __jump_table, \"aw\" \n\t" _ASM_ALIGN "\n\t" _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t" ".popsection \n\t" : : "i" (key), "i" (branch) : : l_yes); return false; l_yes: return true; } static __always_inline bool arch_static_branch_jump(struct static_key *key, bool branch) { asm_volatile_goto("1:" ".byte 0xe9\n\t .long %l[l_yes] - 2f\n\t" "2:\n\t" ".pushsection __jump_table, \"aw\" \n\t" _ASM_ALIGN "\n\t" _ASM_PTR "1b, %l[l_yes], %c0 + %c1 \n\t" ".popsection \n\t" : : "i" (key), "i" (branch) : : l_yes); return false; l_yes: return true; } Where we emit either a 5 byte jump or a 5 byte nop and write a special section with meta-data for the branch point. You could possibly capture all that with a built-in, but would have to exactly match our meta-data section and then we'd still be up some creek without no paddle when we need to change it. But we also have: static __always_inline __pure bool _static_cpu_has(u16 bit) { asm_volatile_goto("1: jmp 6f\n" "2:\n" ".skip -(((5f-4f) - (2b-1b)) > 0) * " "((5f-4f) - (2b-1b)),0x90\n" "3:\n" ".section .altinstructions,\"a\"\n" " .long 1b - .\n" /* src offset */ " .long 4f - .\n" /* repl offset */ " .word %P[always]\n" /* always replace */ " .byte 3b - 1b\n" /* src len */ " .byte 5f - 4f\n" /* repl len */ " .byte 3b - 2b\n" /* pad len */ ".previous\n" ".section .altinstr_replacement,\"ax\"\n" "4: jmp %l[t_no]\n" "5:\n" ".previous\n" ".section .altinstructions,\"a\"\n" " .long 1b - .\n" /* src offset */ " .long 0\n" /* no replacement */ " .word %P[feature]\n" /* feature bit */ " .byte 3b - 1b\n" /* src len */ " .byte 0\n" /* repl len */ " .byte 0\n" /* pad len */ ".previous\n" ".section .altinstr_aux,\"ax\"\n" "6:\n" " testb %[bitnum],%[cap_byte]\n" " jnz %l[t_yes]\n" " jmp %l[t_no]\n" ".previous\n" : : [feature] "i" (bit), [always] "i" (X86_FEATURE_ALWAYS), [bitnum] "i" (1 << (bit & 7)), [cap_byte] "m" (((const char *)boot_cpu_data.x86_capability)[bit >> 3]) : : t_yes, t_no); t_yes: return true; t_no: return false; } Which does something similar, but with a completely different meta-data section and a different pre-patch fallback path. But we also do things like: #define __GEN_RMWcc(fullop, var, cc, clobbers, ...) \ do {
RE: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
I believe we do have the infrastructure in llvm to implement a working patch. The "asm goto" was introduced as a solution to a problem (optimizing tracepoints/ nop patching). When we started the discussion in llvm about adding this feature, questions arose if we can design a better solution for this problem than "asm goto". You can find a more about the discussion in the llvm community here: http://lists.llvm.org/pipermail/llvm-dev/2017-April/111748.html * This is the middle of the thread, and there are more messages later on. Thanks, Marina -Original Message- From: Greg Kroah-Hartman [mailto:gre...@linux-foundation.org] Sent: Wednesday, February 14, 2018 11:53 To: Chandler Carruth Cc: Yatsina, Marina ; Kees Cook ; David Woodhouse ; Kreitzer, David L ; Grischenko, Andrei L ; r...@google.com; LLVM Developers ; eh...@mozilla.com; Tayree, Coby ; Matthias Braun ; Dean Michael Berris ; James Y Knight ; Guenter Roeck ; X86 ML ; LKML ; Alan Cox ; Rik van Riel ; Andi Kleen ; Josh Poimboeuf ; Tom Lendacky ; Peter Zijlstra ; Linus Torvalds ; Jiri Kosina ; Andy Lutomirski ; Hansen, Dave ; Tim Chen ; Paul Turner ; Stephen Hines ; Nick Desaulniers Subject: Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support) On Wed, Feb 14, 2018 at 08:41:43AM +, Chandler Carruth wrote: > Marina, Kees: > > I think the combination of LKML and the LLVM developer's list is > probably too large a set of folks to have a (somewhat) lengthy > exploration of options and how to go about this and find a good path forward. Why is there any need to discuss "options" here? What is preventing the implementation of "asm goto" in llvm? Is it the lack of a working patch, or something else? thanks, greg k-h - Intel Israel (74) Limited This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
RE: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
Hi Peter, When I started the original thread last year I was in favor of adding "asm goto" and didn't understand why it wasn't done by that time. The feedback I got is that this feature (optimizing tracepoints) is very useful and that we do want it in llvm, but perhaps there's a cleaner way of implementing than "asm goto". An alternative suggestion arose as well. I'm sure you can provide a lot of background for the decisions of why "asm goto" was chosen and which other alternatives were considered, as you were the one to implement this. This may shed a lot of light on the alternative suggestions that arose in my thread in the llvm community. I think that including GCC folks in the "small team" discussion Chandler suggested might be a very good idea, so if you have some contacts, please send them to Chandler. Anyway, I think we should consider the alternatives and not take "asm goto" as a given, even though this is the implementation of this feature for the past 7 years and it works well. I'm not saying linux kernel needs to rush and adopt an alternative solution ASAP, but we cannot ignore that it exists and may (or may not) be the direction of the future (for both gcc and llvm). Thanks, Marina -Original Message- From: Peter Zijlstra [mailto:pet...@infradead.org] Sent: Wednesday, February 14, 2018 11:09 To: Yatsina, Marina Cc: Kees Cook ; David Woodhouse ; Chandler Carruth ; Kreitzer, David L ; Grischenko, Andrei L ; r...@google.com; LLVM Developers ; eh...@mozilla.com; Tayree, Coby ; Matthias Braun ; Dean Michael Berris ; James Y Knight ; Guenter Roeck ; X86 ML ; LKML ; Alan Cox ; Rik van Riel ; Andi Kleen ; Josh Poimboeuf ; Tom Lendacky ; Linus Torvalds ; Jiri Kosina ; Andy Lutomirski ; Hansen, Dave ; Tim Chen ; Greg Kroah-Hartman ; Paul Turner ; Stephen Hines ; Nick Desaulniers Subject: Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support) On Wed, Feb 14, 2018 at 08:33:39AM +, Yatsina, Marina wrote: > I hope this will give this issue a push forward and we will find a > solution that will not prevent llvm from compiling linux kernel. That solution is asm-goto. We (linux kernel) are not going to support something like it. It's going to be asm-goto or bust. The feature is over 7 years old by now, deal with it. If you ever get to the point where it makes sense to build a kernel with LLVM, then you can bike-shed with the GCC team on new features, until then tough luck. - Intel Israel (74) Limited This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
On Wed, Feb 14, 2018 at 08:41:43AM +, Chandler Carruth wrote: > Marina, Kees: > > I think the combination of LKML and the LLVM developer's list is probably > too large a set of folks to have a (somewhat) lengthy exploration of > options and how to go about this and find a good path forward. Why is there any need to discuss "options" here? What is preventing the implementation of "asm goto" in llvm? Is it the lack of a working patch, or something else? thanks, greg k-h
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
On Wed, Feb 14, 2018 at 08:41:43AM +, Chandler Carruth wrote: > Marina, Kees: > > I think the combination of LKML and the LLVM developer's list is probably > too large a set of folks to have a (somewhat) lengthy exploration of > options and how to go about this and find a good path forward. As one of the original authors of the jump-label implementation I'm saying there are no options. We're not going to do a second parallel implementation. If you want to build the kernel, you get to implement asm-goto, teh end.
Re: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
On Wed, Feb 14, 2018 at 08:33:39AM +, Yatsina, Marina wrote: > I hope this will give this issue a push forward and we will find a > solution that will not prevent llvm from compiling linux kernel. That solution is asm-goto. We (linux kernel) are not going to support something like it. It's going to be asm-goto or bust. The feature is over 7 years old by now, deal with it. If you ever get to the point where it makes sense to build a kernel with LLVM, then you can bike-shed with the GCC team on new features, until then tough luck.
RE: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support)
Hi Kees, When I raised the question of whether we want to add support for "asm goto" in llvm I got some feedback from the community that "asm goto" might not be the best solution for the problem it was invented for (optimizing support for tracepoints), so I stopped perusing this issue. I'm CC-ing the developers that participated in the original thread and a few developers that might be interested in adding support of "asm goto". I'm also adding the llvm-dev mailing list, in case there are additional parties interested in voicing their opinion. I hope this will give this issue a push forward and we will find a solution that will not prevent llvm from compiling linux kernel. Thanks, Marina -Original Message- From: Kees Cook [mailto:keesc...@google.com] Sent: Wednesday, February 14, 2018 02:29 To: David Woodhouse ; Chandler Carruth ; Yatsina, Marina Cc: Guenter Roeck ; X86 ML ; LKML ; Alan Cox ; Rik van Riel ; Andi Kleen ; Josh Poimboeuf ; Tom Lendacky ; Peter Zijlstra ; Linus Torvalds ; Jiri Kosina ; Andy Lutomirski ; Hansen, Dave ; Tim Chen ; Greg Kroah-Hartman ; Paul Turner ; Stephen Hines ; Nick Desaulniers Subject: clang asm-goto support (Was Re: [PATCH v2] x86/retpoline: Add clang support) On Tue, Feb 13, 2018 at 4:10 PM, David Woodhouse wrote: > We also need to resolve the asm-goto thing. Yes, this is becoming much more urgent, assuming we'll be raising the minimum GCC version soon and drop support for lacking asm-goto... Do you happen to know who the right people are to include to move the discussion forward? I know various kernel folks that are passionate about it, but I'm still getting to know who to talk with from llvm. I see an earlier thread here: http://lists.llvm.org/pipermail/llvm-dev/2017-April/111748.html It seems to end there? I'm still coming up to speed on it, so I'm likely missing other context. -Kees -- Kees Cook Pixel Security - Intel Israel (74) Limited This e-mail and any attachments may contain confidential material for the sole use of the intended recipient(s). Any review or distribution by others is strictly prohibited. If you are not the intended recipient, please contact the sender and delete all copies.