Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[David Ahern]

On 5/2/17 10:58 AM, Andrey Konovalov wrote:
> Do you have a patch that I could test?

not yet.

> 
> I also reported another issue recently, that might also be related to this 
> one:
> https://groups.google.com/forum/#!topic/syzkaller/Rt0pgY4wfiw

different problem. I can still trigger this one with the reproducer you sent

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[David Ahern]

On 5/2/17 10:58 AM, Andrey Konovalov wrote:
> Do you have a patch that I could test?

not yet.

> 
> I also reported another issue recently, that might also be related to this 
> one:
> https://groups.google.com/forum/#!topic/syzkaller/Rt0pgY4wfiw

different problem. I can still trigger this one with the reproducer you sent

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[Andrey Konovalov]

On Tue, May 2, 2017 at 4:44 AM, David Ahern  wrote:
> On 4/26/17 9:15 AM, Andrey Konovalov wrote:
>> +David
>>
>> I've enabled CONFIG_DEBUG_OBJECTS_RCU_HEAD and this is what I get.
>>
>> Apparently the rcu warning is related to the fib6_del_route bug I've
>> been trying to reproduce:
>> https://groups.google.com/forum/#!msg/syzkaller/3SS80JbVPKA/2tfIAcW7DwAJ
>>
>> Adding David, who provided the fix:
>> https://patchwork.ozlabs.org/patch/754913/
>>
>> I've managed to extract a reproducer, attached together with the
>> .config that I used.
>>
>> On commit 5a7ad1146caa895ad718a534399e38bd2ba721b7 (4.11-rc8) with
>> David's patch applied.
>>
>> [ cut here ]
>> WARNING: CPU: 1 PID: 5911 at lib/debugobjects.c:289
>> debug_print_object+0x175/0x210
>> ODEBUG: activate active (active state 1) object type: rcu_head hint:
>> (null)
>> Modules linked in:
>> CPU: 1 PID: 5911 Comm: a.out Not tainted 4.11.0-rc8+ #271
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:16
>>  dump_stack+0x192/0x22d lib/dump_stack.c:52
>>  __warn+0x19f/0x1e0 kernel/panic.c:549
>>  warn_slowpath_fmt+0xe0/0x120 kernel/panic.c:564
>>  debug_print_object+0x175/0x210 lib/debugobjects.c:286
>>  debug_object_activate+0x574/0x7e0 lib/debugobjects.c:442
>>  debug_rcu_head_queue kernel/rcu/rcu.h:75
>>  __call_rcu.constprop.76+0xff/0x9c0 kernel/rcu/tree.c:3229
>>  call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3288
>>  rt6_rcu_free net/ipv6/ip6_fib.c:158
>>  rt6_release+0x1ea/0x290 net/ipv6/ip6_fib.c:188
>>  fib6_del_route net/ipv6/ip6_fib.c:1461
>
> I think I got to the bottom of this one.
>
> With your config, ip6_tunnel is compiled in.
>
> The program runs in a very tight loop, calling 'unshare -n' and then
> spawns 2 sets of 14 threads running random ioctl calls. The networking
> sequence:
>
> 1. New network namespace created via unshare -n
> - ip6tnl0 device is created in down state
>
> 2. address added to ip6tnl0 (equivalent to ip -6 addr add dev ip6tnl0
> fd00::bb/1)
> - the host route is created and inserted into FIB
>
> 3. ip6tnl0 is brought up - starts DAD on the address
>
> 4. exit namespace
> - teardown / cleanup sequence starts
> - lo teardown appears to happen BEFORE teardown of ip6tunl0
>   + removes host route from FIB
>   + host route added to rcu callback list: call_rcu(>dst.rcu_head,
> dst_rcu_free);
>   + rcu callback has not run yet, so rt is NOT on the gc list so it has
> NOT been marked obsolete
>
> 5. worker_thread runs addrconf_dad_completed
> - calls ipv6_ifa_notify which inserts the host route
>
> All of that happens very quickly. The result is that a route that has
> been deleted and added to the RCU list is re-inserted into the FIB. What
> happens next depends on order -- in this case the exit namespace
> eventually gets to cleaning up ip6tnl0 which removes the host route from
> the FIB, calls the rcu function for cleanup -- and triggers the double
> rcu trace.
>
> I have a hack that flags this sequence and prevents the re-insertion
> following DAD. That allows the command to run until it consumes all 2G
> of memory the VM has -- about 600+ iterations without triggering any
> stack traces.

Hi David,

Thanks for looking into this!

Do you have a patch that I could test?

I also reported another issue recently, that might also be related to this one:
https://groups.google.com/forum/#!topic/syzkaller/Rt0pgY4wfiw

Thanks!

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[Andrey Konovalov]

On Tue, May 2, 2017 at 4:44 AM, David Ahern  wrote:
> On 4/26/17 9:15 AM, Andrey Konovalov wrote:
>> +David
>>
>> I've enabled CONFIG_DEBUG_OBJECTS_RCU_HEAD and this is what I get.
>>
>> Apparently the rcu warning is related to the fib6_del_route bug I've
>> been trying to reproduce:
>> https://groups.google.com/forum/#!msg/syzkaller/3SS80JbVPKA/2tfIAcW7DwAJ
>>
>> Adding David, who provided the fix:
>> https://patchwork.ozlabs.org/patch/754913/
>>
>> I've managed to extract a reproducer, attached together with the
>> .config that I used.
>>
>> On commit 5a7ad1146caa895ad718a534399e38bd2ba721b7 (4.11-rc8) with
>> David's patch applied.
>>
>> [ cut here ]
>> WARNING: CPU: 1 PID: 5911 at lib/debugobjects.c:289
>> debug_print_object+0x175/0x210
>> ODEBUG: activate active (active state 1) object type: rcu_head hint:
>> (null)
>> Modules linked in:
>> CPU: 1 PID: 5911 Comm: a.out Not tainted 4.11.0-rc8+ #271
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:16
>>  dump_stack+0x192/0x22d lib/dump_stack.c:52
>>  __warn+0x19f/0x1e0 kernel/panic.c:549
>>  warn_slowpath_fmt+0xe0/0x120 kernel/panic.c:564
>>  debug_print_object+0x175/0x210 lib/debugobjects.c:286
>>  debug_object_activate+0x574/0x7e0 lib/debugobjects.c:442
>>  debug_rcu_head_queue kernel/rcu/rcu.h:75
>>  __call_rcu.constprop.76+0xff/0x9c0 kernel/rcu/tree.c:3229
>>  call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3288
>>  rt6_rcu_free net/ipv6/ip6_fib.c:158
>>  rt6_release+0x1ea/0x290 net/ipv6/ip6_fib.c:188
>>  fib6_del_route net/ipv6/ip6_fib.c:1461
>
> I think I got to the bottom of this one.
>
> With your config, ip6_tunnel is compiled in.
>
> The program runs in a very tight loop, calling 'unshare -n' and then
> spawns 2 sets of 14 threads running random ioctl calls. The networking
> sequence:
>
> 1. New network namespace created via unshare -n
> - ip6tnl0 device is created in down state
>
> 2. address added to ip6tnl0 (equivalent to ip -6 addr add dev ip6tnl0
> fd00::bb/1)
> - the host route is created and inserted into FIB
>
> 3. ip6tnl0 is brought up - starts DAD on the address
>
> 4. exit namespace
> - teardown / cleanup sequence starts
> - lo teardown appears to happen BEFORE teardown of ip6tunl0
>   + removes host route from FIB
>   + host route added to rcu callback list: call_rcu(>dst.rcu_head,
> dst_rcu_free);
>   + rcu callback has not run yet, so rt is NOT on the gc list so it has
> NOT been marked obsolete
>
> 5. worker_thread runs addrconf_dad_completed
> - calls ipv6_ifa_notify which inserts the host route
>
> All of that happens very quickly. The result is that a route that has
> been deleted and added to the RCU list is re-inserted into the FIB. What
> happens next depends on order -- in this case the exit namespace
> eventually gets to cleaning up ip6tnl0 which removes the host route from
> the FIB, calls the rcu function for cleanup -- and triggers the double
> rcu trace.
>
> I have a hack that flags this sequence and prevents the re-insertion
> following DAD. That allows the command to run until it consumes all 2G
> of memory the VM has -- about 600+ iterations without triggering any
> stack traces.

Hi David,

Thanks for looking into this!

Do you have a patch that I could test?

I also reported another issue recently, that might also be related to this one:
https://groups.google.com/forum/#!topic/syzkaller/Rt0pgY4wfiw

Thanks!

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[David Ahern]

On 4/26/17 9:15 AM, Andrey Konovalov wrote:
> +David
> 
> I've enabled CONFIG_DEBUG_OBJECTS_RCU_HEAD and this is what I get.
> 
> Apparently the rcu warning is related to the fib6_del_route bug I've
> been trying to reproduce:
> https://groups.google.com/forum/#!msg/syzkaller/3SS80JbVPKA/2tfIAcW7DwAJ
> 
> Adding David, who provided the fix:
> https://patchwork.ozlabs.org/patch/754913/
> 
> I've managed to extract a reproducer, attached together with the
> .config that I used.
> 
> On commit 5a7ad1146caa895ad718a534399e38bd2ba721b7 (4.11-rc8) with
> David's patch applied.
> 
> [ cut here ]
> WARNING: CPU: 1 PID: 5911 at lib/debugobjects.c:289
> debug_print_object+0x175/0x210
> ODEBUG: activate active (active state 1) object type: rcu_head hint:
> (null)
> Modules linked in:
> CPU: 1 PID: 5911 Comm: a.out Not tainted 4.11.0-rc8+ #271
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16
>  dump_stack+0x192/0x22d lib/dump_stack.c:52
>  __warn+0x19f/0x1e0 kernel/panic.c:549
>  warn_slowpath_fmt+0xe0/0x120 kernel/panic.c:564
>  debug_print_object+0x175/0x210 lib/debugobjects.c:286
>  debug_object_activate+0x574/0x7e0 lib/debugobjects.c:442
>  debug_rcu_head_queue kernel/rcu/rcu.h:75
>  __call_rcu.constprop.76+0xff/0x9c0 kernel/rcu/tree.c:3229
>  call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3288
>  rt6_rcu_free net/ipv6/ip6_fib.c:158
>  rt6_release+0x1ea/0x290 net/ipv6/ip6_fib.c:188
>  fib6_del_route net/ipv6/ip6_fib.c:1461

I think I got to the bottom of this one.

With your config, ip6_tunnel is compiled in.

The program runs in a very tight loop, calling 'unshare -n' and then
spawns 2 sets of 14 threads running random ioctl calls. The networking
sequence:

1. New network namespace created via unshare -n
- ip6tnl0 device is created in down state

2. address added to ip6tnl0 (equivalent to ip -6 addr add dev ip6tnl0
fd00::bb/1)
- the host route is created and inserted into FIB

3. ip6tnl0 is brought up - starts DAD on the address

4. exit namespace
- teardown / cleanup sequence starts
- lo teardown appears to happen BEFORE teardown of ip6tunl0
  + removes host route from FIB
  + host route added to rcu callback list: call_rcu(>dst.rcu_head,
dst_rcu_free);
  + rcu callback has not run yet, so rt is NOT on the gc list so it has
NOT been marked obsolete

5. worker_thread runs addrconf_dad_completed
- calls ipv6_ifa_notify which inserts the host route

All of that happens very quickly. The result is that a route that has
been deleted and added to the RCU list is re-inserted into the FIB. What
happens next depends on order -- in this case the exit namespace
eventually gets to cleaning up ip6tnl0 which removes the host route from
the FIB, calls the rcu function for cleanup -- and triggers the double
rcu trace.

I have a hack that flags this sequence and prevents the re-insertion
following DAD. That allows the command to run until it consumes all 2G
of memory the VM has -- about 600+ iterations without triggering any
stack traces.

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[David Ahern]

On 4/26/17 9:15 AM, Andrey Konovalov wrote:
> +David
> 
> I've enabled CONFIG_DEBUG_OBJECTS_RCU_HEAD and this is what I get.
> 
> Apparently the rcu warning is related to the fib6_del_route bug I've
> been trying to reproduce:
> https://groups.google.com/forum/#!msg/syzkaller/3SS80JbVPKA/2tfIAcW7DwAJ
> 
> Adding David, who provided the fix:
> https://patchwork.ozlabs.org/patch/754913/
> 
> I've managed to extract a reproducer, attached together with the
> .config that I used.
> 
> On commit 5a7ad1146caa895ad718a534399e38bd2ba721b7 (4.11-rc8) with
> David's patch applied.
> 
> [ cut here ]
> WARNING: CPU: 1 PID: 5911 at lib/debugobjects.c:289
> debug_print_object+0x175/0x210
> ODEBUG: activate active (active state 1) object type: rcu_head hint:
> (null)
> Modules linked in:
> CPU: 1 PID: 5911 Comm: a.out Not tainted 4.11.0-rc8+ #271
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  __dump_stack lib/dump_stack.c:16
>  dump_stack+0x192/0x22d lib/dump_stack.c:52
>  __warn+0x19f/0x1e0 kernel/panic.c:549
>  warn_slowpath_fmt+0xe0/0x120 kernel/panic.c:564
>  debug_print_object+0x175/0x210 lib/debugobjects.c:286
>  debug_object_activate+0x574/0x7e0 lib/debugobjects.c:442
>  debug_rcu_head_queue kernel/rcu/rcu.h:75
>  __call_rcu.constprop.76+0xff/0x9c0 kernel/rcu/tree.c:3229
>  call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3288
>  rt6_rcu_free net/ipv6/ip6_fib.c:158
>  rt6_release+0x1ea/0x290 net/ipv6/ip6_fib.c:188
>  fib6_del_route net/ipv6/ip6_fib.c:1461

I think I got to the bottom of this one.

With your config, ip6_tunnel is compiled in.

The program runs in a very tight loop, calling 'unshare -n' and then
spawns 2 sets of 14 threads running random ioctl calls. The networking
sequence:

1. New network namespace created via unshare -n
- ip6tnl0 device is created in down state

2. address added to ip6tnl0 (equivalent to ip -6 addr add dev ip6tnl0
fd00::bb/1)
- the host route is created and inserted into FIB

3. ip6tnl0 is brought up - starts DAD on the address

4. exit namespace
- teardown / cleanup sequence starts
- lo teardown appears to happen BEFORE teardown of ip6tunl0
  + removes host route from FIB
  + host route added to rcu callback list: call_rcu(>dst.rcu_head,
dst_rcu_free);
  + rcu callback has not run yet, so rt is NOT on the gc list so it has
NOT been marked obsolete

5. worker_thread runs addrconf_dad_completed
- calls ipv6_ifa_notify which inserts the host route

All of that happens very quickly. The result is that a route that has
been deleted and added to the RCU list is re-inserted into the FIB. What
happens next depends on order -- in this case the exit namespace
eventually gets to cleaning up ip6tnl0 which removes the host route from
the FIB, calls the rcu function for cleanup -- and triggers the double
rcu trace.

I have a hack that flags this sequence and prevents the re-insertion
following DAD. That allows the command to run until it consumes all 2G
of memory the VM has -- about 600+ iterations without triggering any
stack traces.

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[Paul E. McKenney]

On Wed, Apr 26, 2017 at 04:45:51PM +0200, Andrey Konovalov wrote:
> On Wed, Apr 26, 2017 at 3:59 PM, Paul E. McKenney
>  wrote:
> > On Wed, Apr 26, 2017 at 02:34:15PM +0200, Andrey Konovalov wrote:
> >> Hi,
> >>
> >> I've got the following error report while fuzzing the kernel with 
> >> syzkaller.
> >>
> >> On commit 5a7ad1146caa895ad718a534399e38bd2ba721b7 (4.11-rc8).
> >>
> >> Unfortunately it's not reproducible.
> >>
> >> I'm not sure whether is is an issue with rcu or ipv6.
> >>
> >> ==
> >> BUG: KASAN: use-after-free in __call_rcu.constprop.77+0x13be/0x1640
> >
> > Does building with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y show any splats?
> > (Yeah, kind of a stupid question if it is not reproducible, but had
> > to ask!)
> 
> Hi Paul,
> 
> I'll try enabling this config.
> 
> In the meantime, while I was trying to reproduce this issue, I got this 
> warning:
> 
> [ cut here ]
> WARNING: CPU: 0 PID: 4590 at kernel/rcu/tree.c:2919
> rcu_do_batch.isra.65+0x845/0xbd0
> Modules linked in:
> CPU: 0 PID: 4590 Comm: syz-executor Not tainted 4.11.0-rc8+ #270
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  
>  __dump_stack lib/dump_stack.c:16
>  dump_stack+0x192/0x22d lib/dump_stack.c:52
>  __warn+0x19f/0x1e0 kernel/panic.c:549
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
>  rcu_do_batch.isra.65+0x845/0xbd0 kernel/rcu/tree.c:2919
>  invoke_rcu_callbacks kernel/rcu/tree.c:3142
>  __rcu_process_callbacks kernel/rcu/tree.c:3109
>  rcu_process_callbacks+0x23f/0x810 kernel/rcu/tree.c:3126
>  __do_softirq+0x253/0x78b kernel/softirq.c:284
>  invoke_softirq kernel/softirq.c:364
>  irq_exit+0x149/0x180 kernel/softirq.c:405
>  exiting_irq ./arch/x86/include/asm/apic.h:657
>  smp_apic_timer_interrupt+0x76/0xa0 arch/x86/kernel/apic/apic.c:962
>  apic_timer_interrupt+0x89/0x90
> RIP: 0010:do_anonymous_page mm/memory.c:2962
> RIP: 0010:handle_pte_fault mm/memory.c:3721
> RIP: 0010:__handle_mm_fault+0xdab/0x1c00 mm/memory.c:3841
> RSP: 0018:88006a02f7d0 EFLAGS: 0246 ORIG_RAX: ff10
> RAX: 0005dab0 RBX: 11000d405f00 RCX: 11000d28cd3f
> RDX: dc00 RSI: 8025 RDI: 8800694669f8
> RBP: 88006a02f9a8 R08: 014280ca R09: 11000d405eb0
> R10: 88006a02f660 R11: dc00 R12: 88006a02f8e0
> R13: 88006a02f980 R14: 8800694669b0 R15: 880069466a00
>  
>  handle_mm_fault+0x1aa/0x450 mm/memory.c:3878
>  faultin_page mm/gup.c:408
>  __get_user_pages+0x606/0x14a0 mm/gup.c:607
>  populate_vma_page_range+0xd9/0x100 mm/gup.c:1062
>  __mm_populate+0x278/0x540 mm/gup.c:1112
>  mm_populate ./include/linux/mm.h:2132
>  vm_mmap_pgoff+0x258/0x280 mm/util.c:314
>  SYSC_mmap_pgoff mm/mmap.c:1503
>  SyS_mmap_pgoff+0x22c/0x5e0 mm/mmap.c:1461
>  SYSC_mmap arch/x86/kernel/sys_x86_64.c:96
>  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:87
>  entry_SYSCALL_64_fastpath+0x1a/0xa9 arch/x86/entry/entry_64.S:204
> RIP: 0033:0x4458e9
> RSP: 002b:7f3a15ce4b58 EFLAGS: 0282 ORIG_RAX: 0009
> RAX: ffda RBX: 7f3a15ce5700 RCX: 004458e9
> RDX: 0002 RSI: 009b8000 RDI: 2000
> RBP:  R08:  R09: 8000
> R10: 8032 R11: 0282 R12: 
> R13:  R14: 7f3a15ce59c0 R15: 7f3a15ce5700
> ---[ end trace e36457085170396e ]---

This is current mainline, somewhere after v4.11-rc1 and linus/master?

If so, this warning indicates that RCU's callback list has been corrupted.
This could be a bug in RCU, and if this was 4.12-rc1 and later, my
first guess would be upcoming changes in RCU callback handling.  But I
am pretty sure that you are not running 4.12-rc1.

So I will instead guess that you are double-call_rcu()-ing (sort of like
double-free, except with call_rcu() and friends instead of kfree()).
Or, alternatively, the call_rcu() counterpart to use-after-free.

Thanx, Paul

> >> kernel/rcu/tree.c:3269 at addr 88003b842280
> >> Write of size 8 by task kworker/u10:1/180
> >> CPU: 2 PID: 180 Comm: kworker/u10:1 Not tainted 4.11.0-rc8+ #270
> >> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
> >> 01/01/2011
> >> Workqueue: events_unbound call_usermodehelper_exec_work
> >> Call Trace:
> >>  __dump_stack lib/dump_stack.c:16 [inline]
> >>  dump_stack+0x192/0x22d lib/dump_stack.c:52
> >>  kasan_object_err+0x1c/0x70 mm/kasan/report.c:164
> >>  print_address_description mm/kasan/report.c:202 [inline]
> >>  kasan_report_error mm/kasan/report.c:291 [inline]
> >>  kasan_report+0x252/0x510 mm/kasan/report.c:347
> >>  __asan_report_store8_noabort+0x17/0x20 mm/kasan/report.c:373
> >>  __call_rcu.constprop.77+0x13be/0x1640 kernel/rcu/tree.c:3269
> >>  call_rcu_sched+0x12/0x20 

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[Paul E. McKenney]

On Wed, Apr 26, 2017 at 04:45:51PM +0200, Andrey Konovalov wrote:
> On Wed, Apr 26, 2017 at 3:59 PM, Paul E. McKenney
>  wrote:
> > On Wed, Apr 26, 2017 at 02:34:15PM +0200, Andrey Konovalov wrote:
> >> Hi,
> >>
> >> I've got the following error report while fuzzing the kernel with 
> >> syzkaller.
> >>
> >> On commit 5a7ad1146caa895ad718a534399e38bd2ba721b7 (4.11-rc8).
> >>
> >> Unfortunately it's not reproducible.
> >>
> >> I'm not sure whether is is an issue with rcu or ipv6.
> >>
> >> ==
> >> BUG: KASAN: use-after-free in __call_rcu.constprop.77+0x13be/0x1640
> >
> > Does building with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y show any splats?
> > (Yeah, kind of a stupid question if it is not reproducible, but had
> > to ask!)
> 
> Hi Paul,
> 
> I'll try enabling this config.
> 
> In the meantime, while I was trying to reproduce this issue, I got this 
> warning:
> 
> [ cut here ]
> WARNING: CPU: 0 PID: 4590 at kernel/rcu/tree.c:2919
> rcu_do_batch.isra.65+0x845/0xbd0
> Modules linked in:
> CPU: 0 PID: 4590 Comm: syz-executor Not tainted 4.11.0-rc8+ #270
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Call Trace:
>  
>  __dump_stack lib/dump_stack.c:16
>  dump_stack+0x192/0x22d lib/dump_stack.c:52
>  __warn+0x19f/0x1e0 kernel/panic.c:549
>  warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
>  rcu_do_batch.isra.65+0x845/0xbd0 kernel/rcu/tree.c:2919
>  invoke_rcu_callbacks kernel/rcu/tree.c:3142
>  __rcu_process_callbacks kernel/rcu/tree.c:3109
>  rcu_process_callbacks+0x23f/0x810 kernel/rcu/tree.c:3126
>  __do_softirq+0x253/0x78b kernel/softirq.c:284
>  invoke_softirq kernel/softirq.c:364
>  irq_exit+0x149/0x180 kernel/softirq.c:405
>  exiting_irq ./arch/x86/include/asm/apic.h:657
>  smp_apic_timer_interrupt+0x76/0xa0 arch/x86/kernel/apic/apic.c:962
>  apic_timer_interrupt+0x89/0x90
> RIP: 0010:do_anonymous_page mm/memory.c:2962
> RIP: 0010:handle_pte_fault mm/memory.c:3721
> RIP: 0010:__handle_mm_fault+0xdab/0x1c00 mm/memory.c:3841
> RSP: 0018:88006a02f7d0 EFLAGS: 0246 ORIG_RAX: ff10
> RAX: 0005dab0 RBX: 11000d405f00 RCX: 11000d28cd3f
> RDX: dc00 RSI: 8025 RDI: 8800694669f8
> RBP: 88006a02f9a8 R08: 014280ca R09: 11000d405eb0
> R10: 88006a02f660 R11: dc00 R12: 88006a02f8e0
> R13: 88006a02f980 R14: 8800694669b0 R15: 880069466a00
>  
>  handle_mm_fault+0x1aa/0x450 mm/memory.c:3878
>  faultin_page mm/gup.c:408
>  __get_user_pages+0x606/0x14a0 mm/gup.c:607
>  populate_vma_page_range+0xd9/0x100 mm/gup.c:1062
>  __mm_populate+0x278/0x540 mm/gup.c:1112
>  mm_populate ./include/linux/mm.h:2132
>  vm_mmap_pgoff+0x258/0x280 mm/util.c:314
>  SYSC_mmap_pgoff mm/mmap.c:1503
>  SyS_mmap_pgoff+0x22c/0x5e0 mm/mmap.c:1461
>  SYSC_mmap arch/x86/kernel/sys_x86_64.c:96
>  SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:87
>  entry_SYSCALL_64_fastpath+0x1a/0xa9 arch/x86/entry/entry_64.S:204
> RIP: 0033:0x4458e9
> RSP: 002b:7f3a15ce4b58 EFLAGS: 0282 ORIG_RAX: 0009
> RAX: ffda RBX: 7f3a15ce5700 RCX: 004458e9
> RDX: 0002 RSI: 009b8000 RDI: 2000
> RBP:  R08:  R09: 8000
> R10: 8032 R11: 0282 R12: 
> R13:  R14: 7f3a15ce59c0 R15: 7f3a15ce5700
> ---[ end trace e36457085170396e ]---

This is current mainline, somewhere after v4.11-rc1 and linus/master?

If so, this warning indicates that RCU's callback list has been corrupted.
This could be a bug in RCU, and if this was 4.12-rc1 and later, my
first guess would be upcoming changes in RCU callback handling.  But I
am pretty sure that you are not running 4.12-rc1.

So I will instead guess that you are double-call_rcu()-ing (sort of like
double-free, except with call_rcu() and friends instead of kfree()).
Or, alternatively, the call_rcu() counterpart to use-after-free.

Thanx, Paul

> >> kernel/rcu/tree.c:3269 at addr 88003b842280
> >> Write of size 8 by task kworker/u10:1/180
> >> CPU: 2 PID: 180 Comm: kworker/u10:1 Not tainted 4.11.0-rc8+ #270
> >> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 
> >> 01/01/2011
> >> Workqueue: events_unbound call_usermodehelper_exec_work
> >> Call Trace:
> >>  __dump_stack lib/dump_stack.c:16 [inline]
> >>  dump_stack+0x192/0x22d lib/dump_stack.c:52
> >>  kasan_object_err+0x1c/0x70 mm/kasan/report.c:164
> >>  print_address_description mm/kasan/report.c:202 [inline]
> >>  kasan_report_error mm/kasan/report.c:291 [inline]
> >>  kasan_report+0x252/0x510 mm/kasan/report.c:347
> >>  __asan_report_store8_noabort+0x17/0x20 mm/kasan/report.c:373
> >>  __call_rcu.constprop.77+0x13be/0x1640 kernel/rcu/tree.c:3269
> >>  call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3288
> >>  

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[Andrey Konovalov]

On Wed, Apr 26, 2017 at 3:59 PM, Paul E. McKenney
 wrote:
> On Wed, Apr 26, 2017 at 02:34:15PM +0200, Andrey Konovalov wrote:
>> Hi,
>>
>> I've got the following error report while fuzzing the kernel with syzkaller.
>>
>> On commit 5a7ad1146caa895ad718a534399e38bd2ba721b7 (4.11-rc8).
>>
>> Unfortunately it's not reproducible.
>>
>> I'm not sure whether is is an issue with rcu or ipv6.
>>
>> ==
>> BUG: KASAN: use-after-free in __call_rcu.constprop.77+0x13be/0x1640
>
> Does building with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y show any splats?
> (Yeah, kind of a stupid question if it is not reproducible, but had
> to ask!)

Hi Paul,

I'll try enabling this config.

In the meantime, while I was trying to reproduce this issue, I got this warning:

[ cut here ]
WARNING: CPU: 0 PID: 4590 at kernel/rcu/tree.c:2919
rcu_do_batch.isra.65+0x845/0xbd0
Modules linked in:
CPU: 0 PID: 4590 Comm: syz-executor Not tainted 4.11.0-rc8+ #270
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 
 __dump_stack lib/dump_stack.c:16
 dump_stack+0x192/0x22d lib/dump_stack.c:52
 __warn+0x19f/0x1e0 kernel/panic.c:549
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
 rcu_do_batch.isra.65+0x845/0xbd0 kernel/rcu/tree.c:2919
 invoke_rcu_callbacks kernel/rcu/tree.c:3142
 __rcu_process_callbacks kernel/rcu/tree.c:3109
 rcu_process_callbacks+0x23f/0x810 kernel/rcu/tree.c:3126
 __do_softirq+0x253/0x78b kernel/softirq.c:284
 invoke_softirq kernel/softirq.c:364
 irq_exit+0x149/0x180 kernel/softirq.c:405
 exiting_irq ./arch/x86/include/asm/apic.h:657
 smp_apic_timer_interrupt+0x76/0xa0 arch/x86/kernel/apic/apic.c:962
 apic_timer_interrupt+0x89/0x90
RIP: 0010:do_anonymous_page mm/memory.c:2962
RIP: 0010:handle_pte_fault mm/memory.c:3721
RIP: 0010:__handle_mm_fault+0xdab/0x1c00 mm/memory.c:3841
RSP: 0018:88006a02f7d0 EFLAGS: 0246 ORIG_RAX: ff10
RAX: 0005dab0 RBX: 11000d405f00 RCX: 11000d28cd3f
RDX: dc00 RSI: 8025 RDI: 8800694669f8
RBP: 88006a02f9a8 R08: 014280ca R09: 11000d405eb0
R10: 88006a02f660 R11: dc00 R12: 88006a02f8e0
R13: 88006a02f980 R14: 8800694669b0 R15: 880069466a00
 
 handle_mm_fault+0x1aa/0x450 mm/memory.c:3878
 faultin_page mm/gup.c:408
 __get_user_pages+0x606/0x14a0 mm/gup.c:607
 populate_vma_page_range+0xd9/0x100 mm/gup.c:1062
 __mm_populate+0x278/0x540 mm/gup.c:1112
 mm_populate ./include/linux/mm.h:2132
 vm_mmap_pgoff+0x258/0x280 mm/util.c:314
 SYSC_mmap_pgoff mm/mmap.c:1503
 SyS_mmap_pgoff+0x22c/0x5e0 mm/mmap.c:1461
 SYSC_mmap arch/x86/kernel/sys_x86_64.c:96
 SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:87
 entry_SYSCALL_64_fastpath+0x1a/0xa9 arch/x86/entry/entry_64.S:204
RIP: 0033:0x4458e9
RSP: 002b:7f3a15ce4b58 EFLAGS: 0282 ORIG_RAX: 0009
RAX: ffda RBX: 7f3a15ce5700 RCX: 004458e9
RDX: 0002 RSI: 009b8000 RDI: 2000
RBP:  R08:  R09: 8000
R10: 8032 R11: 0282 R12: 
R13:  R14: 7f3a15ce59c0 R15: 7f3a15ce5700
---[ end trace e36457085170396e ]---

>
> Thanx, Paul
>
>> kernel/rcu/tree.c:3269 at addr 88003b842280
>> Write of size 8 by task kworker/u10:1/180
>> CPU: 2 PID: 180 Comm: kworker/u10:1 Not tainted 4.11.0-rc8+ #270
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Workqueue: events_unbound call_usermodehelper_exec_work
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:16 [inline]
>>  dump_stack+0x192/0x22d lib/dump_stack.c:52
>>  kasan_object_err+0x1c/0x70 mm/kasan/report.c:164
>>  print_address_description mm/kasan/report.c:202 [inline]
>>  kasan_report_error mm/kasan/report.c:291 [inline]
>>  kasan_report+0x252/0x510 mm/kasan/report.c:347
>>  __asan_report_store8_noabort+0x17/0x20 mm/kasan/report.c:373
>>  __call_rcu.constprop.77+0x13be/0x1640 kernel/rcu/tree.c:3269
>>  call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3288
>>  free_pid+0x446/0x5d0 kernel/pid.c:293
>>  __change_pid+0x2a1/0x3d0 kernel/pid.c:411
>>  detach_pid+0x1f/0x30 kernel/pid.c:416
>>  __unhash_process kernel/exit.c:74 [inline]
>>  __exit_signal kernel/exit.c:155 [inline]
>>  release_task+0xbb0/0x1d90 kernel/exit.c:199
>>  wait_task_zombie kernel/exit.c:1230 [inline]
>>  wait_consider_task+0x11fe/0x3410 kernel/exit.c:1458
>>  do_wait_thread kernel/exit.c:1521 [inline]
>>  do_wait+0x3ea/0x8e0 kernel/exit.c:1592
>>  SYSC_wait4 kernel/exit.c:1720 [inline]
>>  SyS_wait4+0x208/0x340 kernel/exit.c:1689
>>  call_usermodehelper_exec_sync kernel/kmod.c:292 [inline]
>>  call_usermodehelper_exec_work+0x1a7/0x2b0 kernel/kmod.c:329
>>  process_one_work+0x9f7/0x1580 kernel/workqueue.c:2097
>>  worker_thread+0x1df/0x14b0 kernel/workqueue.c:2231
>>  

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[Andrey Konovalov]

On Wed, Apr 26, 2017 at 3:59 PM, Paul E. McKenney
 wrote:
> On Wed, Apr 26, 2017 at 02:34:15PM +0200, Andrey Konovalov wrote:
>> Hi,
>>
>> I've got the following error report while fuzzing the kernel with syzkaller.
>>
>> On commit 5a7ad1146caa895ad718a534399e38bd2ba721b7 (4.11-rc8).
>>
>> Unfortunately it's not reproducible.
>>
>> I'm not sure whether is is an issue with rcu or ipv6.
>>
>> ==
>> BUG: KASAN: use-after-free in __call_rcu.constprop.77+0x13be/0x1640
>
> Does building with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y show any splats?
> (Yeah, kind of a stupid question if it is not reproducible, but had
> to ask!)

Hi Paul,

I'll try enabling this config.

In the meantime, while I was trying to reproduce this issue, I got this warning:

[ cut here ]
WARNING: CPU: 0 PID: 4590 at kernel/rcu/tree.c:2919
rcu_do_batch.isra.65+0x845/0xbd0
Modules linked in:
CPU: 0 PID: 4590 Comm: syz-executor Not tainted 4.11.0-rc8+ #270
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
Call Trace:
 
 __dump_stack lib/dump_stack.c:16
 dump_stack+0x192/0x22d lib/dump_stack.c:52
 __warn+0x19f/0x1e0 kernel/panic.c:549
 warn_slowpath_null+0x2c/0x40 kernel/panic.c:584
 rcu_do_batch.isra.65+0x845/0xbd0 kernel/rcu/tree.c:2919
 invoke_rcu_callbacks kernel/rcu/tree.c:3142
 __rcu_process_callbacks kernel/rcu/tree.c:3109
 rcu_process_callbacks+0x23f/0x810 kernel/rcu/tree.c:3126
 __do_softirq+0x253/0x78b kernel/softirq.c:284
 invoke_softirq kernel/softirq.c:364
 irq_exit+0x149/0x180 kernel/softirq.c:405
 exiting_irq ./arch/x86/include/asm/apic.h:657
 smp_apic_timer_interrupt+0x76/0xa0 arch/x86/kernel/apic/apic.c:962
 apic_timer_interrupt+0x89/0x90
RIP: 0010:do_anonymous_page mm/memory.c:2962
RIP: 0010:handle_pte_fault mm/memory.c:3721
RIP: 0010:__handle_mm_fault+0xdab/0x1c00 mm/memory.c:3841
RSP: 0018:88006a02f7d0 EFLAGS: 0246 ORIG_RAX: ff10
RAX: 0005dab0 RBX: 11000d405f00 RCX: 11000d28cd3f
RDX: dc00 RSI: 8025 RDI: 8800694669f8
RBP: 88006a02f9a8 R08: 014280ca R09: 11000d405eb0
R10: 88006a02f660 R11: dc00 R12: 88006a02f8e0
R13: 88006a02f980 R14: 8800694669b0 R15: 880069466a00
 
 handle_mm_fault+0x1aa/0x450 mm/memory.c:3878
 faultin_page mm/gup.c:408
 __get_user_pages+0x606/0x14a0 mm/gup.c:607
 populate_vma_page_range+0xd9/0x100 mm/gup.c:1062
 __mm_populate+0x278/0x540 mm/gup.c:1112
 mm_populate ./include/linux/mm.h:2132
 vm_mmap_pgoff+0x258/0x280 mm/util.c:314
 SYSC_mmap_pgoff mm/mmap.c:1503
 SyS_mmap_pgoff+0x22c/0x5e0 mm/mmap.c:1461
 SYSC_mmap arch/x86/kernel/sys_x86_64.c:96
 SyS_mmap+0x16/0x20 arch/x86/kernel/sys_x86_64.c:87
 entry_SYSCALL_64_fastpath+0x1a/0xa9 arch/x86/entry/entry_64.S:204
RIP: 0033:0x4458e9
RSP: 002b:7f3a15ce4b58 EFLAGS: 0282 ORIG_RAX: 0009
RAX: ffda RBX: 7f3a15ce5700 RCX: 004458e9
RDX: 0002 RSI: 009b8000 RDI: 2000
RBP:  R08:  R09: 8000
R10: 8032 R11: 0282 R12: 
R13:  R14: 7f3a15ce59c0 R15: 7f3a15ce5700
---[ end trace e36457085170396e ]---

>
> Thanx, Paul
>
>> kernel/rcu/tree.c:3269 at addr 88003b842280
>> Write of size 8 by task kworker/u10:1/180
>> CPU: 2 PID: 180 Comm: kworker/u10:1 Not tainted 4.11.0-rc8+ #270
>> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
>> Workqueue: events_unbound call_usermodehelper_exec_work
>> Call Trace:
>>  __dump_stack lib/dump_stack.c:16 [inline]
>>  dump_stack+0x192/0x22d lib/dump_stack.c:52
>>  kasan_object_err+0x1c/0x70 mm/kasan/report.c:164
>>  print_address_description mm/kasan/report.c:202 [inline]
>>  kasan_report_error mm/kasan/report.c:291 [inline]
>>  kasan_report+0x252/0x510 mm/kasan/report.c:347
>>  __asan_report_store8_noabort+0x17/0x20 mm/kasan/report.c:373
>>  __call_rcu.constprop.77+0x13be/0x1640 kernel/rcu/tree.c:3269
>>  call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3288
>>  free_pid+0x446/0x5d0 kernel/pid.c:293
>>  __change_pid+0x2a1/0x3d0 kernel/pid.c:411
>>  detach_pid+0x1f/0x30 kernel/pid.c:416
>>  __unhash_process kernel/exit.c:74 [inline]
>>  __exit_signal kernel/exit.c:155 [inline]
>>  release_task+0xbb0/0x1d90 kernel/exit.c:199
>>  wait_task_zombie kernel/exit.c:1230 [inline]
>>  wait_consider_task+0x11fe/0x3410 kernel/exit.c:1458
>>  do_wait_thread kernel/exit.c:1521 [inline]
>>  do_wait+0x3ea/0x8e0 kernel/exit.c:1592
>>  SYSC_wait4 kernel/exit.c:1720 [inline]
>>  SyS_wait4+0x208/0x340 kernel/exit.c:1689
>>  call_usermodehelper_exec_sync kernel/kmod.c:292 [inline]
>>  call_usermodehelper_exec_work+0x1a7/0x2b0 kernel/kmod.c:329
>>  process_one_work+0x9f7/0x1580 kernel/workqueue.c:2097
>>  worker_thread+0x1df/0x14b0 kernel/workqueue.c:2231
>>  kthread+0x31f/0x3f0 

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[Paul E. McKenney]

On Wed, Apr 26, 2017 at 02:34:15PM +0200, Andrey Konovalov wrote:
> Hi,
> 
> I've got the following error report while fuzzing the kernel with syzkaller.
> 
> On commit 5a7ad1146caa895ad718a534399e38bd2ba721b7 (4.11-rc8).
> 
> Unfortunately it's not reproducible.
> 
> I'm not sure whether is is an issue with rcu or ipv6.
> 
> ==
> BUG: KASAN: use-after-free in __call_rcu.constprop.77+0x13be/0x1640

Does building with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y show any splats?
(Yeah, kind of a stupid question if it is not reproducible, but had
to ask!)

Thanx, Paul

> kernel/rcu/tree.c:3269 at addr 88003b842280
> Write of size 8 by task kworker/u10:1/180
> CPU: 2 PID: 180 Comm: kworker/u10:1 Not tainted 4.11.0-rc8+ #270
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Workqueue: events_unbound call_usermodehelper_exec_work
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x192/0x22d lib/dump_stack.c:52
>  kasan_object_err+0x1c/0x70 mm/kasan/report.c:164
>  print_address_description mm/kasan/report.c:202 [inline]
>  kasan_report_error mm/kasan/report.c:291 [inline]
>  kasan_report+0x252/0x510 mm/kasan/report.c:347
>  __asan_report_store8_noabort+0x17/0x20 mm/kasan/report.c:373
>  __call_rcu.constprop.77+0x13be/0x1640 kernel/rcu/tree.c:3269
>  call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3288
>  free_pid+0x446/0x5d0 kernel/pid.c:293
>  __change_pid+0x2a1/0x3d0 kernel/pid.c:411
>  detach_pid+0x1f/0x30 kernel/pid.c:416
>  __unhash_process kernel/exit.c:74 [inline]
>  __exit_signal kernel/exit.c:155 [inline]
>  release_task+0xbb0/0x1d90 kernel/exit.c:199
>  wait_task_zombie kernel/exit.c:1230 [inline]
>  wait_consider_task+0x11fe/0x3410 kernel/exit.c:1458
>  do_wait_thread kernel/exit.c:1521 [inline]
>  do_wait+0x3ea/0x8e0 kernel/exit.c:1592
>  SYSC_wait4 kernel/exit.c:1720 [inline]
>  SyS_wait4+0x208/0x340 kernel/exit.c:1689
>  call_usermodehelper_exec_sync kernel/kmod.c:292 [inline]
>  call_usermodehelper_exec_work+0x1a7/0x2b0 kernel/kmod.c:329
>  process_one_work+0x9f7/0x1580 kernel/workqueue.c:2097
>  worker_thread+0x1df/0x14b0 kernel/workqueue.c:2231
>  kthread+0x31f/0x3f0 kernel/kthread.c:231
>  ret_from_fork+0x2c/0x40 arch/x86/entry/entry_64.S:430
> Object at 88003b842018, in cache kmalloc-1024 size: 1024
> Allocated:
> PID = 1
>  save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
>  save_stack+0x43/0xd0 mm/kasan/kasan.c:513
>  set_track mm/kasan/kasan.c:525 [inline]
>  kasan_kmalloc+0xad/0xe0 mm/kasan/kasan.c:616
>  kmem_cache_alloc_trace+0x61/0x170 mm/slub.c:2745
>  kmalloc include/linux/slab.h:490 [inline]
>  kzalloc include/linux/slab.h:663 [inline]
>  ipv6_add_dev+0x199/0x1380 net/ipv6/addrconf.c:380
>  addrconf_init+0xd0/0x29a net/ipv6/addrconf.c:6405
>  inet6_init+0x2f6/0x584 net/ipv6/af_inet6.c:962
>  do_one_initcall+0xf3/0x380 init/main.c:792
>  do_initcall_level init/main.c:858 [inline]
>  do_initcalls init/main.c:866 [inline]
>  do_basic_setup init/main.c:884 [inline]
>  kernel_init_freeable+0x54d/0x622 init/main.c:1035
>  kernel_init+0x13/0x180 init/main.c:959
>  ret_from_fork+0x2c/0x40 arch/x86/entry/entry_64.S:430
> Freed:
> PID = 6479
>  save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
>  save_stack+0x43/0xd0 mm/kasan/kasan.c:513
>  set_track mm/kasan/kasan.c:525 [inline]
>  kasan_slab_free+0x73/0xc0 mm/kasan/kasan.c:589
>  slab_free_hook mm/slub.c:1357 [inline]
>  slab_free_freelist_hook mm/slub.c:1379 [inline]
>  slab_free mm/slub.c:2961 [inline]
>  kfree+0x91/0x190 mm/slub.c:3882
>  in6_dev_finish_destroy_rcu+0x97/0xc0 net/ipv6/addrconf_core.c:150
>  __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
>  rcu_do_batch.isra.65+0x6de/0xbd0 kernel/rcu/tree.c:2879
>  invoke_rcu_callbacks kernel/rcu/tree.c:3142 [inline]
>  __rcu_process_callbacks kernel/rcu/tree.c:3109 [inline]
>  rcu_process_callbacks+0x23f/0x810 kernel/rcu/tree.c:3126
>  __do_softirq+0x253/0x78b kernel/softirq.c:284
> Memory state around the buggy address:
>  88003b842180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>  88003b842200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> >88003b842280: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>^
>  88003b842300: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>  88003b842380: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ==
> 

Re: net/ipv6: use-after-free in __call_rcu/in6_dev_finish_destroy_rcu

[Paul E. McKenney]

On Wed, Apr 26, 2017 at 02:34:15PM +0200, Andrey Konovalov wrote:
> Hi,
> 
> I've got the following error report while fuzzing the kernel with syzkaller.
> 
> On commit 5a7ad1146caa895ad718a534399e38bd2ba721b7 (4.11-rc8).
> 
> Unfortunately it's not reproducible.
> 
> I'm not sure whether is is an issue with rcu or ipv6.
> 
> ==
> BUG: KASAN: use-after-free in __call_rcu.constprop.77+0x13be/0x1640

Does building with CONFIG_DEBUG_OBJECTS_RCU_HEAD=y show any splats?
(Yeah, kind of a stupid question if it is not reproducible, but had
to ask!)

Thanx, Paul

> kernel/rcu/tree.c:3269 at addr 88003b842280
> Write of size 8 by task kworker/u10:1/180
> CPU: 2 PID: 180 Comm: kworker/u10:1 Not tainted 4.11.0-rc8+ #270
> Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS Bochs 01/01/2011
> Workqueue: events_unbound call_usermodehelper_exec_work
> Call Trace:
>  __dump_stack lib/dump_stack.c:16 [inline]
>  dump_stack+0x192/0x22d lib/dump_stack.c:52
>  kasan_object_err+0x1c/0x70 mm/kasan/report.c:164
>  print_address_description mm/kasan/report.c:202 [inline]
>  kasan_report_error mm/kasan/report.c:291 [inline]
>  kasan_report+0x252/0x510 mm/kasan/report.c:347
>  __asan_report_store8_noabort+0x17/0x20 mm/kasan/report.c:373
>  __call_rcu.constprop.77+0x13be/0x1640 kernel/rcu/tree.c:3269
>  call_rcu_sched+0x12/0x20 kernel/rcu/tree.c:3288
>  free_pid+0x446/0x5d0 kernel/pid.c:293
>  __change_pid+0x2a1/0x3d0 kernel/pid.c:411
>  detach_pid+0x1f/0x30 kernel/pid.c:416
>  __unhash_process kernel/exit.c:74 [inline]
>  __exit_signal kernel/exit.c:155 [inline]
>  release_task+0xbb0/0x1d90 kernel/exit.c:199
>  wait_task_zombie kernel/exit.c:1230 [inline]
>  wait_consider_task+0x11fe/0x3410 kernel/exit.c:1458
>  do_wait_thread kernel/exit.c:1521 [inline]
>  do_wait+0x3ea/0x8e0 kernel/exit.c:1592
>  SYSC_wait4 kernel/exit.c:1720 [inline]
>  SyS_wait4+0x208/0x340 kernel/exit.c:1689
>  call_usermodehelper_exec_sync kernel/kmod.c:292 [inline]
>  call_usermodehelper_exec_work+0x1a7/0x2b0 kernel/kmod.c:329
>  process_one_work+0x9f7/0x1580 kernel/workqueue.c:2097
>  worker_thread+0x1df/0x14b0 kernel/workqueue.c:2231
>  kthread+0x31f/0x3f0 kernel/kthread.c:231
>  ret_from_fork+0x2c/0x40 arch/x86/entry/entry_64.S:430
> Object at 88003b842018, in cache kmalloc-1024 size: 1024
> Allocated:
> PID = 1
>  save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
>  save_stack+0x43/0xd0 mm/kasan/kasan.c:513
>  set_track mm/kasan/kasan.c:525 [inline]
>  kasan_kmalloc+0xad/0xe0 mm/kasan/kasan.c:616
>  kmem_cache_alloc_trace+0x61/0x170 mm/slub.c:2745
>  kmalloc include/linux/slab.h:490 [inline]
>  kzalloc include/linux/slab.h:663 [inline]
>  ipv6_add_dev+0x199/0x1380 net/ipv6/addrconf.c:380
>  addrconf_init+0xd0/0x29a net/ipv6/addrconf.c:6405
>  inet6_init+0x2f6/0x584 net/ipv6/af_inet6.c:962
>  do_one_initcall+0xf3/0x380 init/main.c:792
>  do_initcall_level init/main.c:858 [inline]
>  do_initcalls init/main.c:866 [inline]
>  do_basic_setup init/main.c:884 [inline]
>  kernel_init_freeable+0x54d/0x622 init/main.c:1035
>  kernel_init+0x13/0x180 init/main.c:959
>  ret_from_fork+0x2c/0x40 arch/x86/entry/entry_64.S:430
> Freed:
> PID = 6479
>  save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
>  save_stack+0x43/0xd0 mm/kasan/kasan.c:513
>  set_track mm/kasan/kasan.c:525 [inline]
>  kasan_slab_free+0x73/0xc0 mm/kasan/kasan.c:589
>  slab_free_hook mm/slub.c:1357 [inline]
>  slab_free_freelist_hook mm/slub.c:1379 [inline]
>  slab_free mm/slub.c:2961 [inline]
>  kfree+0x91/0x190 mm/slub.c:3882
>  in6_dev_finish_destroy_rcu+0x97/0xc0 net/ipv6/addrconf_core.c:150
>  __rcu_reclaim kernel/rcu/rcu.h:118 [inline]
>  rcu_do_batch.isra.65+0x6de/0xbd0 kernel/rcu/tree.c:2879
>  invoke_rcu_callbacks kernel/rcu/tree.c:3142 [inline]
>  __rcu_process_callbacks kernel/rcu/tree.c:3109 [inline]
>  rcu_process_callbacks+0x23f/0x810 kernel/rcu/tree.c:3126
>  __do_softirq+0x253/0x78b kernel/softirq.c:284
> Memory state around the buggy address:
>  88003b842180: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>  88003b842200: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> >88003b842280: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>^
>  88003b842300: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
>  88003b842380: fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb fb
> ==
>