On Thu, Feb 22, 2018 at 03:12:53PM +0100, Daniel Borkmann wrote: > Implement a retpoline [0] for the BPF tail call JIT'ing that converts > the indirect jump via jmp %rax that is used to make the long jump into > another JITed BPF image. Since this is subject to speculative execution, > we need to control the transient instruction sequence here as well > when CONFIG_RETPOLINE is set, and direct it into a pause + lfence loop. > The latter aligns also with what gcc / clang emits (e.g. [1]). > > JIT dump after patch: > > # bpftool p d x i 1 > 0: (18) r2 = map[id:1] > 2: (b7) r3 = 0 > 3: (85) call bpf_tail_call#12 > 4: (b7) r0 = 2 > 5: (95) exit > > With CONFIG_RETPOLINE: > > # bpftool p d j i 1 > [...] > 33: cmp %edx,0x24(%rsi) > 36: jbe 0x0000000000000072 |* > 38: mov 0x24(%rbp),%eax > 3e: cmp $0x20,%eax > 41: ja 0x0000000000000072 | > 43: add $0x1,%eax > 46: mov %eax,0x24(%rbp) > 4c: mov 0x90(%rsi,%rdx,8),%rax > 54: test %rax,%rax > 57: je 0x0000000000000072 | > 59: mov 0x28(%rax),%rax > 5d: add $0x25,%rax > 61: callq 0x000000000000006d |+ > 66: pause | > 68: lfence | > 6b: jmp 0x0000000000000066 | > 6d: mov %rax,(%rsp) | > 71: retq | > 72: mov $0x2,%eax > [...] > > * relative fall-through jumps in error case > + retpoline for indirect jump > > Without CONFIG_RETPOLINE: > > # bpftool p d j i 1 > [...] > 33: cmp %edx,0x24(%rsi) > 36: jbe 0x0000000000000063 |* > 38: mov 0x24(%rbp),%eax > 3e: cmp $0x20,%eax > 41: ja 0x0000000000000063 | > 43: add $0x1,%eax > 46: mov %eax,0x24(%rbp) > 4c: mov 0x90(%rsi,%rdx,8),%rax > 54: test %rax,%rax > 57: je 0x0000000000000063 | > 59: mov 0x28(%rax),%rax > 5d: add $0x25,%rax > 61: jmpq *%rax |- > 63: mov $0x2,%eax > [...] > > * relative fall-through jumps in error case > - plain indirect jump as before > > [0] https://support.google.com/faqs/answer/7625886 > [1] > https://github.com/gcc-mirror/gcc/commit/a31e654fa107be968b802786d747e962c2fcdb2b > > Signed-off-by: Daniel Borkmann <dan...@iogearbox.net> > --- > v1 -> v2: > - Moved into nospec-branch.h as suggested by Eric, thanks!
Applied to bpf tree, Thanks Daniel.
