from:"DaeRyong Jeong"

Re: KASAN: use-after-free Read in vhost_chr_write_iter

2018-05-22 Thread DaeRyong Jeong

On Mon, May 21, 2018 at 10:38:10AM +0800, Jason Wang wrote:
> 
> 
> On 2018年05月18日 17:24, Jason Wang wrote:
> > 
> > 
> > On 2018年05月17日 21:45, DaeRyong Jeong wrote:
> > > We report the crash: KASAN: use-after-free Read in vhost_chr_write_iter
> > > 
> > > This crash has been found in v4.17-rc1 using RaceFuzzer (a modified
> > > version of Syzkaller), which we describe more at the end of this
> > > report. Our analysis shows that the race occurs when invoking two
> > > syscalls concurrently, write$vnet and ioctl$VHOST_RESET_OWNER.
> > > 
> > > 
> > > Analysis:
> > > We think the concurrent execution of vhost_process_iotlb_msg() and
> > > vhost_dev_cleanup() causes the crash.
> > > Both of functions can run concurrently (please see call sequence below),
> > > and possibly, there is a race on dev->iotlb.
> > > If the switch occurs right after vhost_dev_cleanup() frees
> > > dev->iotlb, vhost_process_iotlb_msg() still sees the non-null value
> > > and it
> > > keep executing without returning -EFAULT. Consequently, use-after-free
> > > occures
> > > 
> > > 
> > > Thread interleaving:
> > > CPU0 (vhost_process_iotlb_msg)    CPU1 (vhost_dev_cleanup)
> > > (In the case of both VHOST_IOTLB_UPDATE and
> > > VHOST_IOTLB_INVALIDATE)
> > > =    =
> > >     vhost_umem_clean(dev->iotlb);
> > > if (!dev->iotlb) {
> > >     ret = -EFAULT;
> > >     break;
> > > }
> > >     dev->iotlb = NULL;
> > > 
> > > 
> > > Call Sequence:
> > > CPU0
> > > =
> > > vhost_net_chr_write_iter
> > > vhost_chr_write_iter
> > >     vhost_process_iotlb_msg
> > > 
> > > CPU1
> > > =
> > > vhost_net_ioctl
> > > vhost_net_reset_owner
> > >     vhost_dev_reset_owner
> > >     vhost_dev_cleanup
> > 
> > Thanks a lot for the analysis.
> > 
> > This could be addressed by simply protect it with dev mutex.
> > 
> > Will post a patch.
> > 
> 
> Could you please help to test the attached patch? I've done some smoking
> test.
> 
> Thanks

Sorry to say this, but we don't have a reproducer for this bug since our
reproducer is being implemented.

This crash had occrued a few times in our fuzzer, so I inspected the code
manually.

It seems the patch is good for me, but we can't test the patch for now.
Sorry.

> From 88328386f3f652e684ee33dc4cf63dcaed871aea Mon Sep 17 00:00:00 2001
> From: Jason Wang <jasow...@redhat.com>
> Date: Fri, 18 May 2018 17:33:27 +0800
> Subject: [PATCH] vhost: synchronize IOTLB message with dev cleanup
> 
> DaeRyong Jeong reports a race between vhost_dev_cleanup() and
> vhost_process_iotlb_msg():
> 
> Thread interleaving:
> CPU0 (vhost_process_iotlb_msg)        CPU1 (vhost_dev_cleanup)
> (In the case of both VHOST_IOTLB_UPDATE and
> VHOST_IOTLB_INVALIDATE)
> = =
>   vhost_umem_clean(dev->iotlb);
> if (!dev->iotlb) {
>   ret = -EFAULT;
>   break;
> }
>   dev->iotlb = NULL;
> 
> The reason is we don't synchronize between them, fixing by protecting
> vhost_process_iotlb_msg() with dev mutex.
> 
> Reported-by: DaeRyong Jeong <threeear...@gmail.com>
> Fixes: 6b1e6cc7855b0 ("vhost: new device IOTLB API")
> Reported-by: DaeRyong Jeong <threeear...@gmail.com>
> ---
>  drivers/vhost/vhost.c | 3 +++
>  1 file changed, 3 insertions(+)
> 
> diff --git a/drivers/vhost/vhost.c b/drivers/vhost/vhost.c
> index f3bd8e9..f0be5f3 100644
> --- a/drivers/vhost/vhost.c
> +++ b/drivers/vhost/vhost.c
> @@ -981,6 +981,7 @@ static int vhost_process_iotlb_msg(struct vhost_dev *dev,
>  {
>   int ret = 0;
>  
> + mutex_lock(>mutex);
>   vhost_dev_lock_vqs(dev);
>   switch (msg->type) {
>   case VHOST_IOTLB_UPDATE:
> @@ -1016,6 +1017,8 @@ static int vhost_process_iotlb_msg(struct vhost_dev 
> *dev,
>   }
>  
>   vhost_dev_unlock_vqs(dev);
> + mutex_unlock(>mutex);
> +
>   return ret;
>  }
>  ssize_t vhost_chr_write_iter(struct vhost_dev *dev,
> -- 
> 2.7.4
>

WARNING in ip_recv_error

2018-05-18 Thread DaeRyong Jeong

We report the crash: WARNING in ip_recv_error
(I resend the email since I mistakenly missed the subject in my previous
email. I'm sorry.)


This crash has been found in v4.17-rc1 using RaceFuzzer (a modified
version of Syzkaller), which we describe more at the end of this
report. Our analysis shows that the race occurs when invoking two
syscalls concurrently, setsockopt$inet6_IPV6_ADDRFORM and recvmsg.


Diagnosis:
We think the concurrent execution of do_ipv6_setsockopt() with optname
IPV6_ADDRFORM and inet_recvmsg() causes the crash. do_ipv6_setsockopt()
can update sk->prot to _prot and sk->sk_family to PF_INET. But
inet_recvmsg() can execute sk->sk_prot->recvmsg() right after that
sk->prot is updated and sk->sk_family is not updated by
do_ipv6_setsockopt(). This will lead WARN_ON in ip_recv_error().


Thread interleaving:
CPU0 (do_ipv6_setsockopt)   CPU1 (inet_recvmsg)
=   =
struct proto *prot = _prot;
...
sk->sk_prot = prot;
sk->sk_socket->ops = _dgram_ops;
err = sk->sk_prot->recvmsg(sk, 
msg, size, flags & MSG_DONTWAIT,
flags & 
~MSG_DONTWAIT, _len);

(in udp_recvmsg)
if (flags & MSG_ERRQUEUE)
return 
ip_recv_error(sk, msg, len, addr_len);

(in ip_recv_error)
WARN_ON_ONCE(sk->sk_family == 
AF_INET6);
sk->sk_family = PF_INET;


Call Sequence:
CPU0
=
udpv6_setsockopt
ipv6_setsockopt
do_ipv6_setsockopt

CPU1
=
sock_recvmsg
sock_recvmsg_nosec
inet_recvmsg
udp_recvmsg


==
WARNING: CPU: 1 PID: 32600 at 
/home/daeryong/workspace/new-race-fuzzer/kernels_repo/kernel_v4.17-rc1/net/ipv4/ip_sockglue.c:508
 ip_recv_error+0x6f2/0x720 net/ipv4/ip_sockglue.c:508
Kernel panic - not syncing: panic_on_warn set ...

CPU: 1 PID: 32600 Comm: syz-executor0 Not tainted 4.17.0-rc1 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x166/0x21c lib/dump_stack.c:113
 panic+0x1a0/0x3a7 kernel/panic.c:184
 __warn+0x191/0x1a0 kernel/panic.c:536
 report_bug+0x132/0x1b0 lib/bug.c:186
 fixup_bug.part.11+0x28/0x50 arch/x86/kernel/traps.c:178
 fixup_bug arch/x86/kernel/traps.c:247 [inline]
 do_error_trap+0x28b/0x2d0 arch/x86/kernel/traps.c:296
 do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:315
 invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:992
RIP: 0010:ip_recv_error+0x6f2/0x720 net/ipv4/ip_sockglue.c:508
RSP: 0018:8801dadff630 EFLAGS: 00010212
RAX: 0004 RBX: 2002 RCX: 8327de12
RDX: 008a RSI: c90001a0c000 RDI: 8801be615010
RBP: 8801dadff720 R08: 2002 R09: 8801dadff918
R10: 8801dadff738 R11: 8801dadffaff R12: 8801be615000
R13: 8801dadffd50 R14: 11003b5bfece R15: 8801dadffb90
 udp_recvmsg+0x834/0xa10 net/ipv4/udp.c:1571
 inet_recvmsg+0x121/0x420 net/ipv4/af_inet.c:830
 sock_recvmsg_nosec net/socket.c:802 [inline]
 sock_recvmsg+0x7f/0xa0 net/socket.c:809
 ___sys_recvmsg+0x1f0/0x430 net/socket.c:2279
 __sys_recvmsg+0xfc/0x1c0 net/socket.c:2328
 __do_sys_recvmsg net/socket.c:2338 [inline]
 __se_sys_recvmsg net/socket.c:2335 [inline]
 __x64_sys_recvmsg+0x48/0x50 net/socket.c:2335
 do_syscall_64+0x15f/0x4a0 arch/x86/entry/common.c:287
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4563f9
RSP: 002b:7f24f6927b28 EFLAGS: 0246 ORIG_RAX: 002f
RAX: ffda RBX: 0072bfa0 RCX: 004563f9
RDX: 2002 RSI: 2240 RDI: 0016
RBP: 04e4 R08:  R09: 
R10:  R11: 0246 R12: 7f24f69286d4
R13:  R14: 006fc600 R15: 
Dumping ftrace buffer:
   (ftrace buffer empty)
Kernel Offset: disabled
Rebooting in 86400 seconds..
==


= About RaceFuzzer

RaceFuzzer is a customized version of Syzkaller, specifically tailored
to find race condition bugs in the Linux kernel. While we leverage
many different technique, the notable feature of RaceFuzzer is in
leveraging a custom hypervisor (QEMU/KVM) to interleave the
scheduling. In particular, we modified the hypervisor to intentionally
stall a per-core execution, which is similar to supporting per-core
breakpoint functionality. This allows RaceFuzzer to force the kernel
to deterministically trigger racy condition (which may rarely happen
in practice due to randomness

[no subject]

2018-05-18 Thread DaeRyong Jeong

Bcc: 
Subject: WARNING in ip_recv_error
Reply-To: 

We report the crash: WARNING in ip_recv_error


This crash has been found in v4.17-rc1 using RaceFuzzer (a modified
version of Syzkaller), which we describe more at the end of this
report. Our analysis shows that the race occurs when invoking two
syscalls concurrently, do_ipv6_setsockopt and inet_recvmsg.


Diagnosis:
We think the concurrent execution of do_ipv6_setsockopt() with optname
IPV6_ADDRFORM and inet_recvmsg() causes the crash. do_ipv6_setsockopt()
can update sk->prot to _prot and sk->sk_family to PF_INET. But
inet_recvmsg() can execute sk->sk_prot->recvmsg() right after that
sk->prot is updated and sk->sk_family is not updated by
do_ipv6_setsockopt(). This will lead WARN_ON in ip_recv_error().


Thread interleaving:
CPU0 (do_ipv6_setsockopt)   CPU1 (inet_recvmsg)
=   =
struct proto *prot = _prot;
...
sk->sk_prot = prot;
sk->sk_socket->ops = _dgram_ops;
err = sk->sk_prot->recvmsg(sk, 
msg, size, flags & MSG_DONTWAIT,
flags & 
~MSG_DONTWAIT, _len);

(in udp_recvmsg)
if (flags & MSG_ERRQUEUE)
return 
ip_recv_error(sk, msg, len, addr_len);

(in ip_recv_error)
WARN_ON_ONCE(sk->sk_family == 
AF_INET6);
sk->sk_family = PF_INET;


Call Sequence:
CPU0
=
udpv6_setsockopt
ipv6_setsockopt
do_ipv6_setsockopt

CPU1
=
sock_recvmsg
sock_recvmsg_nosec
inet_recvmsg
udp_recvmsg


==
WARNING: CPU: 1 PID: 32600 at 
/home/daeryong/workspace/new-race-fuzzer/kernels_repo/kernel_v4.17-rc1/net/ipv4/ip_sockglue.c:508
 ip_recv_error+0x6f2/0x720 net/ipv4/ip_sockglue.c:508
Kernel panic - not syncing: panic_on_warn set ...

CPU: 1 PID: 32600 Comm: syz-executor0 Not tainted 4.17.0-rc1 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x166/0x21c lib/dump_stack.c:113
 panic+0x1a0/0x3a7 kernel/panic.c:184
 __warn+0x191/0x1a0 kernel/panic.c:536
 report_bug+0x132/0x1b0 lib/bug.c:186
 fixup_bug.part.11+0x28/0x50 arch/x86/kernel/traps.c:178
 fixup_bug arch/x86/kernel/traps.c:247 [inline]
 do_error_trap+0x28b/0x2d0 arch/x86/kernel/traps.c:296
 do_invalid_op+0x1b/0x20 arch/x86/kernel/traps.c:315
 invalid_op+0x14/0x20 arch/x86/entry/entry_64.S:992
RIP: 0010:ip_recv_error+0x6f2/0x720 net/ipv4/ip_sockglue.c:508
RSP: 0018:8801dadff630 EFLAGS: 00010212
RAX: 0004 RBX: 2002 RCX: 8327de12
RDX: 008a RSI: c90001a0c000 RDI: 8801be615010
RBP: 8801dadff720 R08: 2002 R09: 8801dadff918
R10: 8801dadff738 R11: 8801dadffaff R12: 8801be615000
R13: 8801dadffd50 R14: 11003b5bfece R15: 8801dadffb90
 udp_recvmsg+0x834/0xa10 net/ipv4/udp.c:1571
 inet_recvmsg+0x121/0x420 net/ipv4/af_inet.c:830
 sock_recvmsg_nosec net/socket.c:802 [inline]
 sock_recvmsg+0x7f/0xa0 net/socket.c:809
 ___sys_recvmsg+0x1f0/0x430 net/socket.c:2279
 __sys_recvmsg+0xfc/0x1c0 net/socket.c:2328
 __do_sys_recvmsg net/socket.c:2338 [inline]
 __se_sys_recvmsg net/socket.c:2335 [inline]
 __x64_sys_recvmsg+0x48/0x50 net/socket.c:2335
 do_syscall_64+0x15f/0x4a0 arch/x86/entry/common.c:287
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4563f9
RSP: 002b:7f24f6927b28 EFLAGS: 0246 ORIG_RAX: 002f
RAX: ffda RBX: 0072bfa0 RCX: 004563f9
RDX: 2002 RSI: 2240 RDI: 0016
RBP: 04e4 R08:  R09: 
R10:  R11: 0246 R12: 7f24f69286d4
R13:  R14: 006fc600 R15: 
Dumping ftrace buffer:
   (ftrace buffer empty)
Kernel Offset: disabled
Rebooting in 86400 seconds..
==


= About RaceFuzzer

RaceFuzzer is a customized version of Syzkaller, specifically tailored
to find race condition bugs in the Linux kernel. While we leverage
many different technique, the notable feature of RaceFuzzer is in
leveraging a custom hypervisor (QEMU/KVM) to interleave the
scheduling. In particular, we modified the hypervisor to intentionally
stall a per-core execution, which is similar to supporting per-core
breakpoint functionality. This allows RaceFuzzer to force the kernel
to deterministically trigger racy condition (which may rarely happen
in practice due to randomness in scheduling).

RaceFuzzer's C repro always

WARNING in __static_key_slow_dec

2018-05-18 Thread DaeRyong Jeong

We report the crash: WARNING in __static_key_slow_dec

This crash has been found in v4.8 using RaceFuzzer (a modified
version of Syzkaller), which we describe more at the end of this
report.
Even though v4.8 is the relatively old version, we did manual verification
and we think the bug still exists.
Our analysis shows that the race occurs when invoking two syscalls
concurrently, setsockopt() with optname SO_TIMESTAMPING and ioctl() with
cmd SIOCGSTAMPNS.


Diagnosis:
We think if timestamp was previously enabled with
SOCK_TIMESTAMPING_RX_SOFTWARE flag, the concurrent execution of
sock_disable_timestamp() and sock_enable_timestamp() causes the crash.


Thread interleaving:
(Assume sk->flag has the SOCK_TIMESTAMPING_RX_SOFTWARE flag by the
previous setsockopt() call with SO_TIMESTAMPING)

CPU0 (sock_disable_timestamp()) CPU1 (sock_enable_timestamp())
=   =
(flag == 1UL << SOCK_TIMESTAMPING_RX_SOFTWARE)  (flag == SOCK_TIMESTAMP)

if (!sock_flag(sk, flag)) {
unsigned long 
previous_flags = sk->sk_flags;

if (sk->sk_flags & flags) {
sk->sk_flags &= ~flags;
if (sock_needs_netstamp(sk) &&
!(sk->sk_flags & SK_FLAGS_TIMESTAMP))
net_disable_timestamp();
sock_set_flag(sk, flag);

if 
(sock_needs_netstamp(sk) &&
!(previous_flags & 
SK_FLAGS_TIMESTAMP))

net_enable_timestamp();
/* Here, 
net_enable_timestamp() is not called because
 * previous_flags has 
the SOCK_TIMESTAMPING_RX_SOFTWARE
 * flag
 */
/* After the race, sk->sk has the flag SOCK_TIMESTAMP, but
 * net_enable_timestamp() is not called one more time.
 * Consequently, when the socket is closed, __sk_destruct()
 * calls net_disable_timestamp() that leads WARNING.
 */


Call Sequence:
CPU0
=
sock_setsockopt (SO_TIMESTAMPING)
sock_disable_timestamp

CPU1
=
inet_ioctl (SIOCGSTAMPNS)
sock_get_timestampns
sock_enable_timestamp


==
[ cut here ]
WARNING: CPU: 1 PID: 14465 at 
/home/blee/project/race-fuzzer/kernels/kernel_v4.8/kernel/jump_label.c:151 
__static_key_slow_dec+0xc7/0xd0 kernel/jump_label.c:150
jump label: negative count!
Kernel panic - not syncing: panic_on_warn set ...

CPU: 1 PID: 14465 Comm: syz-executor0 Not tainted 4.8.0 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
  8801f2f5f608 81694cc3 82445d20
 8801f2f5f6e0 82498700 8801f2f5f6e0 8801f2f5f6d0
 812534a0 41b58ab3 829cae04 81253336
Call Trace:
 [] __dump_stack lib/dump_stack.c:15 [inline]
 [] dump_stack+0xb3/0x110 lib/dump_stack.c:51
 [] panic+0x16a/0x336 kernel/panic.c:153
 [] __warn+0x164/0x170 kernel/panic.c:509
 [] warn_slowpath_fmt+0xac/0xd0 kernel/panic.c:532
 [] __static_key_slow_dec+0xc7/0xd0 kernel/jump_label.c:150
 [] static_key_slow_dec+0x33/0x60 kernel/jump_label.c:174
 [] net_disable_timestamp+0x3b/0x50 net/core/dev.c:1729
 [] sock_disable_timestamp+0x6e/0x80 net/core/sock.c:403
 [] __sk_destruct+0xd4/0x260 net/core/sock.c:1453
 [] sk_destruct+0x32/0x60 net/core/sock.c:1472
 [] __sk_free+0x72/0x140 net/core/sock.c:1480
 [] sk_free+0x23/0x30 net/core/sock.c:1491
 [] sock_put include/net/sock.h:1582 [inline]
 [] sk_common_release+0x1ac/0x250 net/core/sock.c:2766
 [] raw_close+0x21/0x30 net/ipv4/raw.c:687
 [] inet_release+0x80/0xd0 net/ipv4/af_inet.c:420
 [] sock_release+0x4e/0xf0 net/socket.c:573
 [] sock_close+0x16/0x20 net/socket.c:1020
 [] __fput+0x182/0x380 fs/file_table.c:208
 [] fput+0x15/0x20 fs/file_table.c:244
 [] task_work_run+0xcb/0x100 kernel/task_work.c:116
 [] exit_task_work include/linux/task_work.h:21 [inline]
 [] do_exit+0x5a1/0x17f0 kernel/exit.c:828
 [] do_group_exit+0xa7/0x190 kernel/exit.c:953
 [] get_signal+0x465/0xdf0 kernel/signal.c:2307
 [] do_signal+0x93/0xad0 arch/x86/kernel/signal.c:805
 [] exit_to_usermode_loop+0xe0/0x110 
arch/x86/entry/common.c:163
 [] prepare_exit_to_usermode arch/x86/entry/common.c:198 
[inline]
 [] syscall_return_slowpath+0x146/0x150 
arch/x86/entry/common.c:267
 [] entry_SYSCALL_64_fastpath+0xbb/0xbd
Dumping ftrace buffer:
   (ftrace buffer empty)
Kernel Offset: disabled
==


= About RaceFuzzer

RaceFuzzer is a customized version

KASAN: use-after-free Read in vhost_chr_write_iter

2018-05-17 Thread DaeRyong Jeong

We report the crash: KASAN: use-after-free Read in vhost_chr_write_iter

This crash has been found in v4.17-rc1 using RaceFuzzer (a modified
version of Syzkaller), which we describe more at the end of this
report. Our analysis shows that the race occurs when invoking two
syscalls concurrently, write$vnet and ioctl$VHOST_RESET_OWNER.


Analysis:
We think the concurrent execution of vhost_process_iotlb_msg() and
vhost_dev_cleanup() causes the crash.
Both of functions can run concurrently (please see call sequence below),
and possibly, there is a race on dev->iotlb.
If the switch occurs right after vhost_dev_cleanup() frees
dev->iotlb, vhost_process_iotlb_msg() still sees the non-null value and it
keep executing without returning -EFAULT. Consequently, use-after-free
occures


Thread interleaving:
CPU0 (vhost_process_iotlb_msg)  CPU1 (vhost_dev_cleanup)
(In the case of both VHOST_IOTLB_UPDATE and
VHOST_IOTLB_INVALIDATE)
=   =

vhost_umem_clean(dev->iotlb);
if (!dev->iotlb) {
ret = -EFAULT;
break;
}
dev->iotlb = NULL;


Call Sequence:
CPU0
=
vhost_net_chr_write_iter
vhost_chr_write_iter
vhost_process_iotlb_msg

CPU1
=
vhost_net_ioctl
vhost_net_reset_owner
vhost_dev_reset_owner
vhost_dev_cleanup


==
BUG: KASAN: use-after-free in vhost_umem_interval_tree_iter_first 
drivers/vhost/vhost.c:52 [inline]
BUG: KASAN: use-after-free in vhost_del_umem_range drivers/vhost/vhost.c:936 
[inline]
BUG: KASAN: use-after-free in vhost_process_iotlb_msg 
drivers/vhost/vhost.c:1010 [inline]
BUG: KASAN: use-after-free in vhost_chr_write_iter+0x44e/0xcd0 
drivers/vhost/vhost.c:1037
Read of size 8 at addr 8801d9d7bc00 by task syz-executor0/4997

CPU: 0 PID: 4997 Comm: syz-executor0 Not tainted 4.17.0-rc1 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x166/0x21c lib/dump_stack.c:113
 print_address_description+0x73/0x250 mm/kasan/report.c:256
 kasan_report_error mm/kasan/report.c:354 [inline]
 kasan_report+0x23f/0x360 mm/kasan/report.c:412
 check_memory_region_inline mm/kasan/kasan.c:260 [inline]
 __asan_load8+0x54/0x90 mm/kasan/kasan.c:699
 vhost_umem_interval_tree_iter_first drivers/vhost/vhost.c:52 [inline]
 vhost_del_umem_range drivers/vhost/vhost.c:936 [inline]
 vhost_process_iotlb_msg drivers/vhost/vhost.c:1010 [inline]
 vhost_chr_write_iter+0x44e/0xcd0 drivers/vhost/vhost.c:1037
 vhost_net_chr_write_iter+0x38/0x40 drivers/vhost/net.c:1380
 call_write_iter include/linux/fs.h:1784 [inline]
 new_sync_write fs/read_write.c:474 [inline]
 __vfs_write+0x355/0x480 fs/read_write.c:487
 vfs_write+0x12d/0x2d0 fs/read_write.c:549
 ksys_write+0xca/0x190 fs/read_write.c:598
 __do_sys_write fs/read_write.c:610 [inline]
 __se_sys_write fs/read_write.c:607 [inline]
 __x64_sys_write+0x43/0x50 fs/read_write.c:607
 do_syscall_64+0x15f/0x4a0 arch/x86/entry/common.c:287
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4563f9
RSP: 002b:7f4da7ce9b28 EFLAGS: 0246 ORIG_RAX: 0001
RAX: ffda RBX: 0072bee0 RCX: 004563f9
RDX: 0068 RSI: 26c0 RDI: 0015
RBP: 0729 R08:  R09: 
R10:  R11: 0246 R12: 7f4da7cea6d4
R13:  R14: 006ffc78 R15: 

Allocated by task 4997:
 save_stack+0x43/0xd0 mm/kasan/kasan.c:448
 set_track mm/kasan/kasan.c:460 [inline]
 kasan_kmalloc+0xae/0xe0 mm/kasan/kasan.c:553
 __do_kmalloc_node mm/slab.c:3682 [inline]
 __kmalloc_node+0x47/0x70 mm/slab.c:3689
 kmalloc_node include/linux/slab.h:554 [inline]
 kvmalloc_node+0x99/0xd0 mm/util.c:421
 kvmalloc include/linux/mm.h:550 [inline]
 kvzalloc include/linux/mm.h:558 [inline]
 vhost_umem_alloc+0x72/0x120 drivers/vhost/vhost.c:1260
 vhost_init_device_iotlb+0x1e/0x160 drivers/vhost/vhost.c:1548
 vhost_net_set_features drivers/vhost/net.c:1273 [inline]
 vhost_net_ioctl+0x849/0x1040 drivers/vhost/net.c:1338
 vfs_ioctl fs/ioctl.c:46 [inline]
 do_vfs_ioctl+0x145/0xd00 fs/ioctl.c:686
 ksys_ioctl+0x94/0xb0 fs/ioctl.c:701
 __do_sys_ioctl fs/ioctl.c:708 [inline]
 __se_sys_ioctl fs/ioctl.c:706 [inline]
 __x64_sys_ioctl+0x43/0x50 fs/ioctl.c:706
 do_syscall_64+0x15f/0x4a0 arch/x86/entry/common.c:287
 entry_SYSCALL_64_after_hwframe+0x49/0xbe

Freed by task 5000:
 save_stack+0x43/0xd0 mm/kasan/kasan.c:448
 set_track mm/kasan/kasan.c:460 [inline]
 __kasan_slab_free+0x11a/0x170 mm/kasan/kasan.c:521
 kasan_slab_free+0xe/0x10 mm/kasan/kasan.c:528
 __cache_free mm/slab.c:3498 [inline]
 kfree+0xd9/0x260

KASAN: null-ptr-deref Read in rds_ib_get_mr

2018-05-10 Thread DaeRyong Jeong

We report the crash: KASAN: null-ptr-deref Read in rds_ib_get_mr

Note that this bug is previously reported by syzkaller.
https://syzkaller.appspot.com/bug?id=0bb56a5a48b000b52aa2b0d8dd20b1f545214d91
Nonetheless, this bug has not fixed yet, and we hope that this report and our
analysis, which gets help by the RaceFuzzer's feature, will helpful to fix the
crash.

This crash has been found in v4.17-rc1 using RaceFuzzer (a modified
version of Syzkaller), which we describe more at the end of this
report. Our analysis shows that the race occurs when invoking two
syscalls concurrently, bind$rds and setsockopt$RDS_GET_MR.


Analysis:
We think the concurrent execution of __rds_rdma_map() and rds_bind()
causes the problem. __rds_rdma_map() checks whether rs->rs_bound_addr is 0
or not. But the concurrent execution with rds_bind() can by-pass this
check. Therefore, __rds_rdmap_map() calls rs->rs_transport->get_mr() and
rds_ib_get_mr() causes the null deref at ib_rdma.c:544 in v4.17-rc1, when
dereferencing rs_conn.


Thread interleaving:
CPU0 (__rds_rdma_map)   CPU1 (rds_bind)
// rds_add_bound() sets 
rs->bound_addr as none 0
ret = rds_add_bound(rs, 
sin->sin_addr.s_addr, >sin_port);
if (rs->rs_bound_addr == 0 || !rs->rs_transport) {
ret = -ENOTCONN; /* XXX not a great errno */
goto out;
}
if (rs->rs_transport) { 
/* previously bound */
trans = 
rs->rs_transport;
if 
(trans->laddr_check(sock_net(sock->sk),

   sin->sin_addr.s_addr) != 0) {
ret = 
-ENOPROTOOPT;
// 
rds_remove_bound() sets rs->bound_addr as 0

rds_remove_bound(rs);
...
trans_private = rs->rs_transport->get_mr(sg, nents, rs,
 >r_key);
(in rds_ib_get_mr())
struct rds_ib_connection *ic = rs->rs_conn->c_transport_data;


Call sequence (v4.17-rc1):
CPU0
rds_setsockopt
rds_get_mr
__rds_rdma_map
rds_ib_get_mr


CPU1
rds_bind
rds_add_bound
...
rds_remove_bound


Crash log:
==
BUG: KASAN: null-ptr-deref in rds_ib_get_mr+0x3a/0x150 net/rds/ib_rdma.c:544
Read of size 8 at addr 0068 by task syz-executor0/32067

CPU: 0 PID: 32067 Comm: syz-executor0 Not tainted 4.17.0-rc1 #1
Hardware name: QEMU Standard PC (i440FX + PIIX, 1996), BIOS 
rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
Call Trace:
 __dump_stack lib/dump_stack.c:77 [inline]
 dump_stack+0x166/0x21c lib/dump_stack.c:113
 kasan_report_error mm/kasan/report.c:352 [inline]
 kasan_report+0x140/0x360 mm/kasan/report.c:412
 check_memory_region_inline mm/kasan/kasan.c:260 [inline]
 __asan_load8+0x54/0x90 mm/kasan/kasan.c:699
 rds_ib_get_mr+0x3a/0x150 net/rds/ib_rdma.c:544
 __rds_rdma_map+0x521/0x9d0 net/rds/rdma.c:271
 rds_get_mr+0xad/0xf0 net/rds/rdma.c:333
 rds_setsockopt+0x57f/0x720 net/rds/af_rds.c:347
 __sys_setsockopt+0x147/0x230 net/socket.c:1903
 __do_sys_setsockopt net/socket.c:1914 [inline]
 __se_sys_setsockopt net/socket.c:1911 [inline]
 __x64_sys_setsockopt+0x67/0x80 net/socket.c:1911
 do_syscall_64+0x15f/0x4a0 arch/x86/entry/common.c:287
 entry_SYSCALL_64_after_hwframe+0x49/0xbe
RIP: 0033:0x4563f9
RSP: 002b:7f6a2b3c2b28 EFLAGS: 0246 ORIG_RAX: 0036
RAX: ffda RBX: 0072bee0 RCX: 004563f9
RDX: 0002 RSI: 0114 RDI: 0015
RBP: 0575 R08: 0020 R09: 
R10: 2140 R11: 0246 R12: 7f6a2b3c36d4
R13:  R14: 006fd398 R15: 
==


= About RaceFuzzer

RaceFuzzer is a customized version of Syzkaller, specifically tailored
to find race condition bugs in the Linux kernel. While we leverage
many different technique, the notable feature of RaceFuzzer is in
leveraging a custom hypervisor (QEMU/KVM) to interleave the
scheduling. In particular, we modified the hypervisor to intentionally
stall a per-core execution, which is similar to supporting per-core
breakpoint functionality. This allows RaceFuzzer to force the kernel
to deterministically trigger racy condition (which may rarely happen
in practice due to randomness in scheduling).

RaceFuzzer's C repro always pinpoints two racy syscalls. Since C
repro's scheduling synchronization should be performed at the

Re: kernel BUG at /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:LINE!

2018-04-19 Thread DaeRyong Jeong

Hello.
We have analyzed the cause of the crash, kernel BUG at
net/packet/af_packet.c:LINE!,
which is found by RaceFuzzer (a modified version of Syzkaller) in v4.16-rc7.

Since struct packet_sock's member variables, running, has_vnet_hdr, origdev
and auxdata are declared as bitfields, accessing these variables can race if
there is no synchronization mechanism.

We think racing between following lines in af_packet.c causes the crash.
In function __unregister_prot_hook,
po->running = 0;
In function packet_setsockopt,
po->has_vnet_hdr = !!val;

Analysis:
CPU0
pakcet_setsockopt
po->has_vnet_hdr = !!val;

CPU1
packet_do_bind
__unregister_prot_hook
po->running = 0;

In the CPU1, the value of po->running should become 0, but because of racing,
it is possible that po->running can keep the value 1.
Consequently, after returning from __unregister_prot_hook, BUG_ON at
net/packet/af_packet.c:3107 can be triggered.

Possible interleaving between racy C source lines is as follows (built with
gcc-7.1.0).
CPU0 (po->has_vnet_hdr = !!val)CPU1 (po->running = 0)
movzbl 0x6e0(%r15),%eax
   andb
$0xfe,0x6e0(%r13)
shl$0x3,%r12d
and$0xfff7,%eax
or %r12d,%eax
mov%al,0x6e0(%r15)

Please, check out the following reproducer.
C repro code : 
https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-repro.c
kernel config v4.16-rc3 :
https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc3.config
kernel config v4.16-rc7 :
https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc7.config
kernel config v4.15.14 :
https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.15.14.config

= About RaceFuzzer

RaceFuzzer is a customized version of Syzkaller, specifically tailored
to find race condition bugs in the Linux kernel. While we leverage
many different technique, the notable feature of RaceFuzzer is in
leveraging a custom hypervisor (QEMU/KVM) to interleave the
scheduling. In particular, we modified the hypervisor to intentionally
stall a per-core execution, which is similar to supporting per-core
breakpoint functionality. This allows RaceFuzzer to force the kernel
to deterministically trigger racy condition (which may rarely happen
in practice due to randomness in scheduling).

RaceFuzzer's C repro always pinpoints two racy syscalls. Since C
repro's scheduling synchronization should be performed at the user
space, its reproducibility is limited (reproduction may take from 1
second to 10 minutes (or even more), depending on a bug). This is
because, while RaceFuzzer precisely interleaves the scheduling at the
kernel's instruction level when finding this bug, C repro cannot fully
utilize such a feature. Please disregard all code related to
"should_hypercall" in the C repro, as this is only for our debugging
purposes using our own hypervisor.

On Sat, Mar 31, 2018 at 1:33 AM, DaeRyong Jeong <threeear...@gmail.com> wrote:
> We report the crash: kernel BUG at
> /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:LINE!
>
> This crash has been found in v4.16-rc3 using RaceFuzzer (a modified
> version of Syzkaller), which we describe more at the end of this
> report. Our analysis shows that the race occurs when invoking two
> syscalls concurrently, (setsockopt$packet_int) and (bind$packet).
> We have confirmed that the kernel v4.16-rc3, v4.16-rc7, and v4.15.14
> built with gcc 7.1.0 are crashing by running the provided C repro
> program within a few minutes (5 minutes).
> Note that this crash can be triggered from the user space.
>
> C repro code : 
> https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-repro.c
> kernel config v4.16-rc3 :
> https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc3.config
> kernel config v4.16-rc7 :
> https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc7.config
> kernel config v4.15.14 :
> https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.15.14.config
>
> [  881.047513] [ cut here ]
> [  881.048416] kernel BUG at
> /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:3107!
> [  881.050014] invalid opcode:  [#1] SMP KASAN
> [  881.050698] Dumping ftrace buffer:
> [  881.051244](ftrace buffer empty)
> [  881.051768] Modules linked in:
> [  881.052236] CPU: 1 PID: 18247 Comm: syz-executor0 Not tainted 4.16.0-rc3 #1
> [  881.053247] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
> BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
> [  881.054880] RIP: 0010:packet_do_bind+0x88d/0x950
> [  881.03] RSP: 0018:8802231d7b08 EFLAGS: 00010212
> [  881.056310] RAX: 0001 RBX: 88

Re: WARNING in refcount_dec

2018-04-19 Thread DaeRyong Jeong

Hello.
We have analyzed the cause of the crash in v4.16-rc3, WARNING in refcount_dec,
which is found by RaceFuzzer (a modified version of Syzkaller).

Since struct packet_sock's member variables, running, has_vnet_hdr, origdev
and auxdata are declared as bitfields, accessing these variables can race if
there is no synchronization mechanism.

We think racing between following lines in af_packet.c causes the crash.
In function __unregister_prot_hook,
po->running = 0;
In function packet_setsockopt,
po->has_vnet_hdr = !!val;

Analysis:
CPU0
pakcet_setsockopt
po->has_vnet_hdr = !!val;

CPU1
packet_setsockop
packet_set_ring
__unregister_prot_hook
po->running = 0;

In the CPU1, the value of po->running should become 0, but because of racing,
it is possible that po->running can keep the value 1.
Consequently, the followings can happen.
- When packet_set_ring calls register_prot_hook, register_prot_hook
return immediately without calling sock_hold(sk).
- When packet_release is called, __unregister_prot_hook will be called
because po->running == 1 and sk->sk_refcnt hits zero.

Possible interleaving between racy C source lines is as follows (built
with gcc-7.1.0).
CPU0 (po->has_vnet_hdr = !!val)  CPU1 (po->running = 0)
movzbl 0x6e0(%r15),%eax
 andb
 $0xfe,0x6e0(%r13)
shl$0x3,%r12d
and$0xfff7,%eax
or %r12d,%eax
mov%al,0x6e0(%r15)

Please, check out the following reproducer.
C repro code : 
https://kiwi.cs.purdue.edu/static/race-fuzzer/repro-refcount_dec.c
kernel config: 
https://kiwi.cs.purdue.edu/static/race-fuzzer/kernel-config-v4.16-rc3

Since there is a small room to race, it may take a long time to
reproduce the crash.

= About RaceFuzzer

RaceFuzzer is a customized version of Syzkaller, specifically tailored
to find race condition bugs in the Linux kernel. While we leverage
many different technique, the notable feature of RaceFuzzer is in
leveraging a custom hypervisor (QEMU/KVM) to interleave the
scheduling. In particular, we modified the hypervisor to intentionally
stall a per-core execution, which is similar to supporting per-core
breakpoint functionality. This allows RaceFuzzer to force the kernel
to deterministically trigger racy condition (which may rarely happen
in practice due to randomness in scheduling).

RaceFuzzer's C repro always pinpoints two racy syscalls. Since C
repro's scheduling synchronization should be performed at the user
space, its reproducibility is limited (reproduction may take from 1
second to 10 minutes (or even more), depending on a bug). This is
because, while RaceFuzzer precisely interleaves the scheduling at the
kernel's instruction level when finding this bug, C repro cannot fully
utilize such a feature. Please disregard all code related to
"should_hypercall" in the C repro, as this is only for our debugging
purposes using our own hypervisor.

On Tue, Apr 3, 2018 at 1:12 PM, DaeRyong Jeong <threeear...@gmail.com> wrote:
> No. Only the first crash (WARNING in refcount_dec) is reproduced by
> the attached reproducer.
>
> The second crash (kernel bug at af_packet.c:3107) is reproduced by
> another reproducer.
> We reported it here.
> http://lkml.iu.edu/hypermail/linux/kernel/1803.3/05324.html
>
> On Sun, Apr 1, 2018 at 4:38 PM, Willem de Bruijn
> <willemdebruijn.ker...@gmail.com> wrote:
>> On Thu, Mar 29, 2018 at 1:16 AM, Cong Wang <xiyou.wangc...@gmail.com> wrote:
>>> (Cc'ing netdev and Willem)
>>>
>>> On Wed, Mar 28, 2018 at 12:03 PM, Byoungyoung Lee
>>> <byoungyo...@purdue.edu> wrote:
>>>> Another crash patterns observed: race between (setsockopt$packet_int)
>>>> and (bind$packet).
>>>>
>>>> --
>>>> [  357.731597] kernel BUG at
>>>> /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:3107!
>>>> [  357.733382] invalid opcode:  [#1] SMP KASAN
>>>> [  357.734017] Modules linked in:
>>>> [  357.734662] CPU: 1 PID: 3871 Comm: repro.exe Not tainted 4.16.0-rc3 #1
>>>> [  357.735791] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>>> BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
>>>> [  357.737434] RIP: 0010:packet_do_bind+0x88d/0x950
>>>> [  357.738121] RSP: 0018:8800b2787b08 EFLAGS: 00010293
>>>> [  357.738906] RAX: 8800b2fdc780 RBX: 880234358cc0 RCX: 
>>>> 838b244c
>>>> [  357.739905] RDX:  RSI: 838b257d RDI: 
>>>> 0001
>>>> [  357.741315] RBP: 8800b2787c10 R08: 8800b2fdc780 R09: 
>>>> 
>>>> [  357.743055] R10:

Re: WARNING in refcount_dec

2018-04-02 Thread DaeRyong Jeong

No. Only the first crash (WARNING in refcount_dec) is reproduced by
the attached reproducer.

The second crash (kernel bug at af_packet.c:3107) is reproduced by
another reproducer.
We reported it here.
http://lkml.iu.edu/hypermail/linux/kernel/1803.3/05324.html

On Sun, Apr 1, 2018 at 4:38 PM, Willem de Bruijn
 wrote:
> On Thu, Mar 29, 2018 at 1:16 AM, Cong Wang  wrote:
>> (Cc'ing netdev and Willem)
>>
>> On Wed, Mar 28, 2018 at 12:03 PM, Byoungyoung Lee
>>  wrote:
>>> Another crash patterns observed: race between (setsockopt$packet_int)
>>> and (bind$packet).
>>>
>>> --
>>> [  357.731597] kernel BUG at
>>> /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:3107!
>>> [  357.733382] invalid opcode:  [#1] SMP KASAN
>>> [  357.734017] Modules linked in:
>>> [  357.734662] CPU: 1 PID: 3871 Comm: repro.exe Not tainted 4.16.0-rc3 #1
>>> [  357.735791] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
>>> BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
>>> [  357.737434] RIP: 0010:packet_do_bind+0x88d/0x950
>>> [  357.738121] RSP: 0018:8800b2787b08 EFLAGS: 00010293
>>> [  357.738906] RAX: 8800b2fdc780 RBX: 880234358cc0 RCX: 
>>> 838b244c
>>> [  357.739905] RDX:  RSI: 838b257d RDI: 
>>> 0001
>>> [  357.741315] RBP: 8800b2787c10 R08: 8800b2fdc780 R09: 
>>> 
>>> [  357.743055] R10: 0001 R11:  R12: 
>>> 88023352ecc0
>>> [  357.744744] R13:  R14: 0001 R15: 
>>> 1d00
>>> [  357.746377] FS:  7f4b43733700() GS:8800b8b0()
>>> knlGS:
>>> [  357.749599] CS:  0010 DS:  ES:  CR0: 80050033
>>> [  357.752096] CR2: 20058000 CR3: 0002334b8000 CR4: 
>>> 06e0
>>> [  357.755045] Call Trace:
>>> [  357.755822]  ? compat_packet_setsockopt+0x100/0x100
>>> [  357.757324]  ? __sanitizer_cov_trace_const_cmp8+0x18/0x20
>>> [  357.758810]  packet_bind+0xa2/0xe0
>>> [  357.759640]  SYSC_bind+0x279/0x2f0
>>> [  357.760364]  ? move_addr_to_kernel.part.19+0xc0/0xc0
>>> [  357.761491]  ? __handle_mm_fault+0x25d0/0x25d0
>>> [  357.762449]  ? __sanitizer_cov_trace_const_cmp4+0x16/0x20
>>> [  357.763663]  ? __do_page_fault+0x417/0xba0
>>> [  357.764569]  ? vmalloc_fault+0x910/0x910
>>> [  357.765405]  ? __sanitizer_cov_trace_const_cmp4+0x16/0x20
>>> [  357.766525]  ? mark_held_locks+0x25/0xb0
>>> [  357.767336]  ? SyS_socketpair+0x4a0/0x4a0
>>> [  357.768182]  SyS_bind+0x24/0x30
>>> [  357.768851]  do_syscall_64+0x209/0x5d0
>>> [  357.769650]  ? syscall_return_slowpath+0x3e0/0x3e0
>>> [  357.770665]  ? __sanitizer_cov_trace_const_cmp4+0x16/0x20
>>> [  357.771779]  ? syscall_return_slowpath+0x260/0x3e0
>>> [  357.772748]  ? mark_held_locks+0x25/0xb0
>>> [  357.773581]  ? __sanitizer_cov_trace_const_cmp4+0x16/0x20
>>> [  357.774720]  ? retint_user+0x18/0x18
>>> [  357.775493]  ? trace_hardirqs_off_caller+0xb5/0x120
>>> [  357.776567]  ? trace_hardirqs_off_thunk+0x1a/0x1c
>>> [  357.777512]  entry_SYSCALL_64_after_hwframe+0x42/0xb7
>>> [  357.778508] RIP: 0033:0x4503a9
>>> [  357.779156] RSP: 002b:7f4b43732ce8 EFLAGS: 0246 ORIG_RAX:
>>> 0031
>>> [  357.780737] RAX: ffda RBX:  RCX: 
>>> 004503a9
>>> [  357.782169] RDX: 0014 RSI: 20058000 RDI: 
>>> 0003
>>> [  357.783710] RBP: 7f4b43732d10 R08:  R09: 
>>> 
>>> [  357.785202] R10:  R11: 0246 R12: 
>>> 
>>> [  357.786664] R13:  R14: 7f4b437339c0 R15: 
>>> 7f4b43733700
>>> [  357.788210] Code: c0 fd 48 c7 c2 00 c8 d9 84 be ab 02 00 00 48 c7
>>> c7 60 c8 d9 84 c6 05 e7 a2 48 02 01 e8 3f 17 af fd e9 60 fb ff ff e8
>>> 43 b3 c0 fd <0f> 0b e8 3c b3 c0 fd 48 8b bd 20 ff ff ff e8 60 1e e7 fd
>>> 4c 89
>>> [  357.792260] RIP: packet_do_bind+0x88d/0x950 RSP: 8800b2787b08
>>> [  357.793698] ---[ end trace 0c5a2539f0247369 ]---
>>> [  357.794696] Kernel panic - not syncing: Fatal exception
>>> [  357.795918] Kernel Offset: disabled
>>> [  357.796614] Rebooting in 86400 seconds..
>>>
>>> On Wed, Mar 28, 2018 at 1:19 AM, Byoungyoung Lee  
>>> wrote:
 We report the crash: WARNING in refcount_dec

 This crash has been found in v4.16-rc3 using RaceFuzzer (a modified
 version of Syzkaller), which we describe more at the end of this
 report. Our analysis shows that the race occurs when invoking two
 syscalls concurrently, (setsockopt$packet_int) and
 (setsockopt$packet_rx_ring).

 C repro code : 
 https://kiwi.cs.purdue.edu/static/race-fuzzer/repro-refcount_dec.c
 kernel config: 
 https://kiwi.cs.purdue.edu/static/race-fuzzer/kernel-config-v4.16-rc3
>>
>>
>> I tried your

kernel BUG at /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:LINE!

2018-03-30 Thread DaeRyong Jeong

We report the crash: kernel BUG at
/home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:LINE!

This crash has been found in v4.16-rc3 using RaceFuzzer (a modified
version of Syzkaller), which we describe more at the end of this
report. Our analysis shows that the race occurs when invoking two
syscalls concurrently, (setsockopt$packet_int) and (bind$packet).
We have confirmed that the kernel v4.16-rc3, v4.16-rc7, and v4.15.14
built with gcc 7.1.0 are crashing by running the provided C repro
program within a few minutes (5 minutes).
Note that this crash can be triggered from the user space.

C repro code : 
https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-repro.c
kernel config v4.16-rc3 :
https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc3.config
kernel config v4.16-rc7 :
https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.16-rc7.config
kernel config v4.15.14 :
https://kiwi.cs.purdue.edu/static/race-fuzzer/afpacket-setsockopt-bind-v4.15.14.config

[  881.047513] [ cut here ]
[  881.048416] kernel BUG at
/home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:3107!
[  881.050014] invalid opcode:  [#1] SMP KASAN
[  881.050698] Dumping ftrace buffer:
[  881.051244](ftrace buffer empty)
[  881.051768] Modules linked in:
[  881.052236] CPU: 1 PID: 18247 Comm: syz-executor0 Not tainted 4.16.0-rc3 #1
[  881.053247] Hardware name: QEMU Standard PC (i440FX + PIIX, 1996),
BIOS rel-1.8.2-0-g33fbe13 by qemu-project.org 04/01/2014
[  881.054880] RIP: 0010:packet_do_bind+0x88d/0x950
[  881.03] RSP: 0018:8802231d7b08 EFLAGS: 00010212
[  881.056310] RAX: 0001 RBX: 8800af831740 RCX: c900025ce000
[  881.057318] RDX: 00a5 RSI: 838b257d RDI: 0001
[  881.058301] RBP: 8802231d7c10 R08: 8802342f2480 R09: 
[  881.059298] R10: 0001 R11:  R12: 8802309f8f00
[  881.060314] R13:  R14: 0001 R15: 1000
[  881.061320] FS:  7f7fab50d700() GS:88023fc0()
knlGS:
[  881.062467] CS:  0010 DS:  ES:  CR0: 80050033
[  881.063285] CR2: 20038000 CR3: b11c9000 CR4: 06e0
[  881.064317] Call Trace:
[  881.064686]  ? compat_packet_setsockopt+0x100/0x100
[  881.065430]  ? __sanitizer_cov_trace_const_cmp8+0x18/0x20
[  881.066188]  packet_bind+0xa2/0xe0
[  881.066690]  SYSC_bind+0x279/0x2f0
[  881.067180]  ? move_addr_to_kernel.part.19+0xc0/0xc0
[  881.067896]  ? do_futex+0x1e90/0x1e90
[  881.068435]  ? SyS_sched_getaffinity+0xe3/0x100
[  881.069112]  ? mark_held_locks+0x25/0xb0
[  881.069677]  ? SyS_socketpair+0x4a0/0x4a0
[  881.070265]  SyS_bind+0x24/0x30
[  881.070732]  do_syscall_64+0x209/0x5d0
[  881.071270]  ? syscall_return_slowpath+0x3e0/0x3e0
[  881.071929]  ? __sanitizer_cov_trace_const_cmp4+0x16/0x20
[  881.072675]  ? syscall_return_slowpath+0x260/0x3e0
[  881.073365]  ? mark_held_locks+0x25/0xb0
[  881.073950]  ? entry_SYSCALL_64_after_hwframe+0x52/0xb7
[  881.074693]  ? trace_hardirqs_off_caller+0xb5/0x120
[  881.075390]  ? trace_hardirqs_off_thunk+0x1a/0x1c
[  881.076079]  entry_SYSCALL_64_after_hwframe+0x42/0xb7
[  881.076797] RIP: 0033:0x453909
[  881.077238] RSP: 002b:7f7fab50caf8 EFLAGS: 0212 ORIG_RAX:
0031
[  881.078268] RAX: ffda RBX: 007080d8 RCX: 00453909
[  881.079239] RDX: 0014 RSI: 2001f000 RDI: 0015
[  881.080268] RBP: 0250 R08:  R09: 
[  881.081256] R10:  R11: 0212 R12: 004a82d3
[  881.082272] R13:  R14: 0015 R15: 2001f000
[  881.083251] Code: c0 fd 48 c7 c2 00 c8 d9 84 be ab 02 00 00 48 c7
c7 60 c8 d9 84 c6 05 e7 a2 48 02 01 e8 3f 17 af fd e9 60 fb ff ff e8
43 b3 c0 fd <0f> 0b e8 3c b3 c0 fd 48 8b bd 20 ff ff ff e8 60 1e e7 fd
4c 89
[  881.085828] RIP: packet_do_bind+0x88d/0x950 RSP: 8802231d7b08
[  881.086619] ---[ end trace 9c461502752b4f3e ]---
[  881.087181] Kernel panic - not syncing: Fatal exception
[  881.088352] Dumping ftrace buffer:
[  881.088877](ftrace buffer empty)
[  881.089414] Kernel Offset: disabled
[  881.089950] Rebooting in 86400 seconds..

= About RaceFuzzer

RaceFuzzer is a customized version of Syzkaller, specifically tailored
to find race condition bugs in the Linux kernel. While we leverage
many different technique, the notable feature of RaceFuzzer is in
leveraging a custom hypervisor (QEMU/KVM) to interleave the
scheduling. In particular, we modified the hypervisor to intentionally
stall a per-core execution, which is similar to supporting per-core
breakpoint functionality. This allows RaceFuzzer to force the kernel
to deterministically trigger racy condition (which may rarely happen
in practice due to randomness in

Re: KASAN: use-after-free Read in vhost_chr_write_iter

WARNING in ip_recv_error

[no subject]

WARNING in __static_key_slow_dec

KASAN: use-after-free Read in vhost_chr_write_iter

KASAN: null-ptr-deref Read in rds_ib_get_mr

Re: kernel BUG at /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:LINE!

Re: WARNING in refcount_dec

Re: WARNING in refcount_dec

kernel BUG at /home/blee/project/race-fuzzer/kernels/kernel_v4.16-rc3/net/packet/af_packet.c:LINE!

10 matches

Site Navigation

Mail list logo

Footer information