Re: KASAN: use-after-free Read in sctp_association_free (2)
On Mon, Mar 12, 2018 at 04:16:27PM +0800, Xin Long wrote: > On Sun, Mar 11, 2018 at 3:04 AM, Neil Horman wrote: > > On Sun, Mar 11, 2018 at 12:22:32AM +0800, Xin Long wrote: > >> On Sat, Mar 10, 2018 at 9:13 PM, Neil Horman wrote: > >> > On Sat, Mar 10, 2018 at 03:58:04PM +0800, Xin Long wrote: > >> >> On Sat, Mar 10, 2018 at 6:08 AM, Neil Horman > >> >> wrote: > >> >> > On Fri, Mar 09, 2018 at 12:59:06PM -0800, syzbot wrote: > >> >> >> Hello, > >> >> >> > >> >> >> syzbot hit the following crash on net-next commit > >> >> >> fd372a7a9e5e9d8011a0222d10edd3523abcd3b1 (Thu Mar 8 19:43:48 2018 > >> >> >> +) > >> >> >> Merge tag 'mlx5-updates-2018-02-28-2' of > >> >> >> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux > >> >> >> > >> >> >> So far this crash happened 2 times on net-next. > >> >> >> C reproducer is attached. > >> >> >> syzkaller reproducer is attached. > >> >> >> Raw console output is attached. > >> >> >> compiler: gcc (GCC) 7.1.1 20170620 > >> >> >> .config is attached. > >> >> >> > >> >> >> IMPORTANT: if you fix the bug, please add the following tag to the > >> >> >> commit: > >> >> >> Reported-by: syzbot+a4e4112c3aff00c8c...@syzkaller.appspotmail.com > >> >> >> It will help syzbot understand when the bug is fixed. See footer for > >> >> >> details. > >> >> >> If you forward the report, please keep this part and the footer. > >> >> >> > >> >> >> IPVS: ftp: loaded support on port[0] = 21 > >> >> >> IPVS: ftp: loaded support on port[0] = 21 > >> >> >> IPVS: ftp: loaded support on port[0] = 21 > >> >> >> IPVS: ftp: loaded support on port[0] = 21 > >> >> >> == > >> >> >> BUG: KASAN: use-after-free in sctp_association_free+0x7b7/0x930 > >> >> >> net/sctp/associola.c:332 > >> >> >> Read of size 8 at addr 8801d8006ae0 by task syzkaller914861/4202 > >> >> >> > >> >> >> CPU: 1 PID: 4202 Comm: syzkaller914861 Not tainted 4.16.0-rc4+ #258 > >> >> >> Hardware name: Google Google Compute Engine/Google Compute Engine, > >> >> >> BIOS > >> >> >> Google 01/01/2011 > >> >> >> Call Trace: > >> >> >> __dump_stack lib/dump_stack.c:17 [inline] > >> >> >> dump_stack+0x194/0x24d lib/dump_stack.c:53 > >> >> >> print_address_description+0x73/0x250 mm/kasan/report.c:256 > >> >> >> kasan_report_error mm/kasan/report.c:354 [inline] > >> >> >> kasan_report+0x23c/0x360 mm/kasan/report.c:412 > >> >> >> __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:433 > >> >> >> sctp_association_free+0x7b7/0x930 net/sctp/associola.c:332 > >> >> >> sctp_sendmsg+0xc67/0x1a80 net/sctp/socket.c:2075 > >> >> >> inet_sendmsg+0x11f/0x5e0 net/ipv4/af_inet.c:763 > >> >> >> sock_sendmsg_nosec net/socket.c:629 [inline] > >> >> >> sock_sendmsg+0xca/0x110 net/socket.c:639 > >> >> >> SYSC_sendto+0x361/0x5c0 net/socket.c:1748 > >> >> >> SyS_sendto+0x40/0x50 net/socket.c:1716 > >> >> >> do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 > >> >> >> entry_SYSCALL_64_after_hwframe+0x42/0xb7 > >> >> >> RIP: 0033:0x446d09 > >> >> >> RSP: 002b:7f5dbac21da8 EFLAGS: 0216 ORIG_RAX: > >> >> >> 002c > >> >> >> RAX: ffda RBX: 006e29fc RCX: 00446d09 > >> >> >> RDX: 0001 RSI: 2340 RDI: 0003 > >> >> >> RBP: 006e29f8 R08: 204d9000 R09: 001c > >> >> >> R10: R11: 0216 R12: > >> >> >> R13: 7fff7b26fb1f R14: 7f5dbac229c0 R15: 006e2b60 > >> >> >> > >> >> > I think we have a corner case with > >> >> > a0ff660058b88d12625a783ce9e5c1371c87951f > >> >> > here. If a peeloff event happens during a wait for sendbuf space, > >> >> > EPIPE will be > >> >> > returned, and the code path appears to call sctp_association_put > >> >> > twice, leading > >> >> > to the use after free situation. I'll write a patch this weekend > >> >> Hi, Neil, you're right. > >> >> > >> >> I didn't expect peeloff can be done on a NEW asoc, as peeloff needs > >> >> assoc_id, which can only be set when connecting has started. > >> >> > >> >> But I realized that: > >> >> f84af33 sctp: factor out sctp_sendmsg_to_asoc from sctp_sendmsg > >> >> > >> >> moved sctp_primitive_ASSOCIATE(connecting) before sctp_wait_for_sndbuf > >> >> (snd buffer waiting). It means peeloff can be done on a NEW asoc. > >> >> So you may want to move it back. > >> >> > >> > I agree with the root cause, but I'm not sure I agree with just moving > >> > the > >> > wait_for_sndbuf call back above the call to associate. I'm not sure I > >> > like > >> > relying on placing a call in a spcific order solely to avoid an error > >> > condition > >> > that might legitimately occur. I think would rather check the return > >> > code at > >> > the call site for the complete set of conditions for which we should not > >> > free > >> > the association. Something like this: > >> > > >> > diff --git a/net/sctp/socket.c b/net/sctp/socket.c > >> > index 7d347
Re: KASAN: use-after-free Read in sctp_association_free (2)
On Sun, Mar 11, 2018 at 3:04 AM, Neil Horman wrote: > On Sun, Mar 11, 2018 at 12:22:32AM +0800, Xin Long wrote: >> On Sat, Mar 10, 2018 at 9:13 PM, Neil Horman wrote: >> > On Sat, Mar 10, 2018 at 03:58:04PM +0800, Xin Long wrote: >> >> On Sat, Mar 10, 2018 at 6:08 AM, Neil Horman >> >> wrote: >> >> > On Fri, Mar 09, 2018 at 12:59:06PM -0800, syzbot wrote: >> >> >> Hello, >> >> >> >> >> >> syzbot hit the following crash on net-next commit >> >> >> fd372a7a9e5e9d8011a0222d10edd3523abcd3b1 (Thu Mar 8 19:43:48 2018 >> >> >> +) >> >> >> Merge tag 'mlx5-updates-2018-02-28-2' of >> >> >> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux >> >> >> >> >> >> So far this crash happened 2 times on net-next. >> >> >> C reproducer is attached. >> >> >> syzkaller reproducer is attached. >> >> >> Raw console output is attached. >> >> >> compiler: gcc (GCC) 7.1.1 20170620 >> >> >> .config is attached. >> >> >> >> >> >> IMPORTANT: if you fix the bug, please add the following tag to the >> >> >> commit: >> >> >> Reported-by: syzbot+a4e4112c3aff00c8c...@syzkaller.appspotmail.com >> >> >> It will help syzbot understand when the bug is fixed. See footer for >> >> >> details. >> >> >> If you forward the report, please keep this part and the footer. >> >> >> >> >> >> IPVS: ftp: loaded support on port[0] = 21 >> >> >> IPVS: ftp: loaded support on port[0] = 21 >> >> >> IPVS: ftp: loaded support on port[0] = 21 >> >> >> IPVS: ftp: loaded support on port[0] = 21 >> >> >> == >> >> >> BUG: KASAN: use-after-free in sctp_association_free+0x7b7/0x930 >> >> >> net/sctp/associola.c:332 >> >> >> Read of size 8 at addr 8801d8006ae0 by task syzkaller914861/4202 >> >> >> >> >> >> CPU: 1 PID: 4202 Comm: syzkaller914861 Not tainted 4.16.0-rc4+ #258 >> >> >> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS >> >> >> Google 01/01/2011 >> >> >> Call Trace: >> >> >> __dump_stack lib/dump_stack.c:17 [inline] >> >> >> dump_stack+0x194/0x24d lib/dump_stack.c:53 >> >> >> print_address_description+0x73/0x250 mm/kasan/report.c:256 >> >> >> kasan_report_error mm/kasan/report.c:354 [inline] >> >> >> kasan_report+0x23c/0x360 mm/kasan/report.c:412 >> >> >> __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:433 >> >> >> sctp_association_free+0x7b7/0x930 net/sctp/associola.c:332 >> >> >> sctp_sendmsg+0xc67/0x1a80 net/sctp/socket.c:2075 >> >> >> inet_sendmsg+0x11f/0x5e0 net/ipv4/af_inet.c:763 >> >> >> sock_sendmsg_nosec net/socket.c:629 [inline] >> >> >> sock_sendmsg+0xca/0x110 net/socket.c:639 >> >> >> SYSC_sendto+0x361/0x5c0 net/socket.c:1748 >> >> >> SyS_sendto+0x40/0x50 net/socket.c:1716 >> >> >> do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 >> >> >> entry_SYSCALL_64_after_hwframe+0x42/0xb7 >> >> >> RIP: 0033:0x446d09 >> >> >> RSP: 002b:7f5dbac21da8 EFLAGS: 0216 ORIG_RAX: 002c >> >> >> RAX: ffda RBX: 006e29fc RCX: 00446d09 >> >> >> RDX: 0001 RSI: 2340 RDI: 0003 >> >> >> RBP: 006e29f8 R08: 204d9000 R09: 001c >> >> >> R10: R11: 0216 R12: >> >> >> R13: 7fff7b26fb1f R14: 7f5dbac229c0 R15: 006e2b60 >> >> >> >> >> > I think we have a corner case with >> >> > a0ff660058b88d12625a783ce9e5c1371c87951f >> >> > here. If a peeloff event happens during a wait for sendbuf space, >> >> > EPIPE will be >> >> > returned, and the code path appears to call sctp_association_put twice, >> >> > leading >> >> > to the use after free situation. I'll write a patch this weekend >> >> Hi, Neil, you're right. >> >> >> >> I didn't expect peeloff can be done on a NEW asoc, as peeloff needs >> >> assoc_id, which can only be set when connecting has started. >> >> >> >> But I realized that: >> >> f84af33 sctp: factor out sctp_sendmsg_to_asoc from sctp_sendmsg >> >> >> >> moved sctp_primitive_ASSOCIATE(connecting) before sctp_wait_for_sndbuf >> >> (snd buffer waiting). It means peeloff can be done on a NEW asoc. >> >> So you may want to move it back. >> >> >> > I agree with the root cause, but I'm not sure I agree with just moving the >> > wait_for_sndbuf call back above the call to associate. I'm not sure I like >> > relying on placing a call in a spcific order solely to avoid an error >> > condition >> > that might legitimately occur. I think would rather check the return code >> > at >> > the call site for the complete set of conditions for which we should not >> > free >> > the association. Something like this: >> > >> > diff --git a/net/sctp/socket.c b/net/sctp/socket.c >> > index 7d3476a4860d..a68846d2b0ef 100644 >> > --- a/net/sctp/socket.c >> > +++ b/net/sctp/socket.c >> > @@ -2071,8 +2071,9 @@ static int sctp_sendmsg(struct sock *sk, struct >> > msghdr *msg, size_t msg_len) >> > >> > /* Send msg to the asoc */ >> > err = sctp_sendmsg_to_asoc(asoc, ms
Re: KASAN: use-after-free Read in sctp_association_free (2)
On Sun, Mar 11, 2018 at 12:22:32AM +0800, Xin Long wrote: > On Sat, Mar 10, 2018 at 9:13 PM, Neil Horman wrote: > > On Sat, Mar 10, 2018 at 03:58:04PM +0800, Xin Long wrote: > >> On Sat, Mar 10, 2018 at 6:08 AM, Neil Horman wrote: > >> > On Fri, Mar 09, 2018 at 12:59:06PM -0800, syzbot wrote: > >> >> Hello, > >> >> > >> >> syzbot hit the following crash on net-next commit > >> >> fd372a7a9e5e9d8011a0222d10edd3523abcd3b1 (Thu Mar 8 19:43:48 2018 +) > >> >> Merge tag 'mlx5-updates-2018-02-28-2' of > >> >> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux > >> >> > >> >> So far this crash happened 2 times on net-next. > >> >> C reproducer is attached. > >> >> syzkaller reproducer is attached. > >> >> Raw console output is attached. > >> >> compiler: gcc (GCC) 7.1.1 20170620 > >> >> .config is attached. > >> >> > >> >> IMPORTANT: if you fix the bug, please add the following tag to the > >> >> commit: > >> >> Reported-by: syzbot+a4e4112c3aff00c8c...@syzkaller.appspotmail.com > >> >> It will help syzbot understand when the bug is fixed. See footer for > >> >> details. > >> >> If you forward the report, please keep this part and the footer. > >> >> > >> >> IPVS: ftp: loaded support on port[0] = 21 > >> >> IPVS: ftp: loaded support on port[0] = 21 > >> >> IPVS: ftp: loaded support on port[0] = 21 > >> >> IPVS: ftp: loaded support on port[0] = 21 > >> >> == > >> >> BUG: KASAN: use-after-free in sctp_association_free+0x7b7/0x930 > >> >> net/sctp/associola.c:332 > >> >> Read of size 8 at addr 8801d8006ae0 by task syzkaller914861/4202 > >> >> > >> >> CPU: 1 PID: 4202 Comm: syzkaller914861 Not tainted 4.16.0-rc4+ #258 > >> >> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS > >> >> Google 01/01/2011 > >> >> Call Trace: > >> >> __dump_stack lib/dump_stack.c:17 [inline] > >> >> dump_stack+0x194/0x24d lib/dump_stack.c:53 > >> >> print_address_description+0x73/0x250 mm/kasan/report.c:256 > >> >> kasan_report_error mm/kasan/report.c:354 [inline] > >> >> kasan_report+0x23c/0x360 mm/kasan/report.c:412 > >> >> __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:433 > >> >> sctp_association_free+0x7b7/0x930 net/sctp/associola.c:332 > >> >> sctp_sendmsg+0xc67/0x1a80 net/sctp/socket.c:2075 > >> >> inet_sendmsg+0x11f/0x5e0 net/ipv4/af_inet.c:763 > >> >> sock_sendmsg_nosec net/socket.c:629 [inline] > >> >> sock_sendmsg+0xca/0x110 net/socket.c:639 > >> >> SYSC_sendto+0x361/0x5c0 net/socket.c:1748 > >> >> SyS_sendto+0x40/0x50 net/socket.c:1716 > >> >> do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 > >> >> entry_SYSCALL_64_after_hwframe+0x42/0xb7 > >> >> RIP: 0033:0x446d09 > >> >> RSP: 002b:7f5dbac21da8 EFLAGS: 0216 ORIG_RAX: 002c > >> >> RAX: ffda RBX: 006e29fc RCX: 00446d09 > >> >> RDX: 0001 RSI: 2340 RDI: 0003 > >> >> RBP: 006e29f8 R08: 204d9000 R09: 001c > >> >> R10: R11: 0216 R12: > >> >> R13: 7fff7b26fb1f R14: 7f5dbac229c0 R15: 006e2b60 > >> >> > >> > I think we have a corner case with > >> > a0ff660058b88d12625a783ce9e5c1371c87951f > >> > here. If a peeloff event happens during a wait for sendbuf space, EPIPE > >> > will be > >> > returned, and the code path appears to call sctp_association_put twice, > >> > leading > >> > to the use after free situation. I'll write a patch this weekend > >> Hi, Neil, you're right. > >> > >> I didn't expect peeloff can be done on a NEW asoc, as peeloff needs > >> assoc_id, which can only be set when connecting has started. > >> > >> But I realized that: > >> f84af33 sctp: factor out sctp_sendmsg_to_asoc from sctp_sendmsg > >> > >> moved sctp_primitive_ASSOCIATE(connecting) before sctp_wait_for_sndbuf > >> (snd buffer waiting). It means peeloff can be done on a NEW asoc. > >> So you may want to move it back. > >> > > I agree with the root cause, but I'm not sure I agree with just moving the > > wait_for_sndbuf call back above the call to associate. I'm not sure I like > > relying on placing a call in a spcific order solely to avoid an error > > condition > > that might legitimately occur. I think would rather check the return code > > at > > the call site for the complete set of conditions for which we should not > > free > > the association. Something like this: > > > > diff --git a/net/sctp/socket.c b/net/sctp/socket.c > > index 7d3476a4860d..a68846d2b0ef 100644 > > --- a/net/sctp/socket.c > > +++ b/net/sctp/socket.c > > @@ -2071,8 +2071,9 @@ static int sctp_sendmsg(struct sock *sk, struct > > msghdr *msg, size_t msg_len) > > > > /* Send msg to the asoc */ > > err = sctp_sendmsg_to_asoc(asoc, msg, msg_len, transport, sinfo); > > - if (err < 0 && err != -ESRCH && new) > > - sctp_association_free(asoc); > > + if ((err != -ESRCH) && (err !=
Re: KASAN: use-after-free Read in sctp_association_free (2)
On Sat, Mar 10, 2018 at 9:13 PM, Neil Horman wrote: > On Sat, Mar 10, 2018 at 03:58:04PM +0800, Xin Long wrote: >> On Sat, Mar 10, 2018 at 6:08 AM, Neil Horman wrote: >> > On Fri, Mar 09, 2018 at 12:59:06PM -0800, syzbot wrote: >> >> Hello, >> >> >> >> syzbot hit the following crash on net-next commit >> >> fd372a7a9e5e9d8011a0222d10edd3523abcd3b1 (Thu Mar 8 19:43:48 2018 +) >> >> Merge tag 'mlx5-updates-2018-02-28-2' of >> >> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux >> >> >> >> So far this crash happened 2 times on net-next. >> >> C reproducer is attached. >> >> syzkaller reproducer is attached. >> >> Raw console output is attached. >> >> compiler: gcc (GCC) 7.1.1 20170620 >> >> .config is attached. >> >> >> >> IMPORTANT: if you fix the bug, please add the following tag to the commit: >> >> Reported-by: syzbot+a4e4112c3aff00c8c...@syzkaller.appspotmail.com >> >> It will help syzbot understand when the bug is fixed. See footer for >> >> details. >> >> If you forward the report, please keep this part and the footer. >> >> >> >> IPVS: ftp: loaded support on port[0] = 21 >> >> IPVS: ftp: loaded support on port[0] = 21 >> >> IPVS: ftp: loaded support on port[0] = 21 >> >> IPVS: ftp: loaded support on port[0] = 21 >> >> == >> >> BUG: KASAN: use-after-free in sctp_association_free+0x7b7/0x930 >> >> net/sctp/associola.c:332 >> >> Read of size 8 at addr 8801d8006ae0 by task syzkaller914861/4202 >> >> >> >> CPU: 1 PID: 4202 Comm: syzkaller914861 Not tainted 4.16.0-rc4+ #258 >> >> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS >> >> Google 01/01/2011 >> >> Call Trace: >> >> __dump_stack lib/dump_stack.c:17 [inline] >> >> dump_stack+0x194/0x24d lib/dump_stack.c:53 >> >> print_address_description+0x73/0x250 mm/kasan/report.c:256 >> >> kasan_report_error mm/kasan/report.c:354 [inline] >> >> kasan_report+0x23c/0x360 mm/kasan/report.c:412 >> >> __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:433 >> >> sctp_association_free+0x7b7/0x930 net/sctp/associola.c:332 >> >> sctp_sendmsg+0xc67/0x1a80 net/sctp/socket.c:2075 >> >> inet_sendmsg+0x11f/0x5e0 net/ipv4/af_inet.c:763 >> >> sock_sendmsg_nosec net/socket.c:629 [inline] >> >> sock_sendmsg+0xca/0x110 net/socket.c:639 >> >> SYSC_sendto+0x361/0x5c0 net/socket.c:1748 >> >> SyS_sendto+0x40/0x50 net/socket.c:1716 >> >> do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 >> >> entry_SYSCALL_64_after_hwframe+0x42/0xb7 >> >> RIP: 0033:0x446d09 >> >> RSP: 002b:7f5dbac21da8 EFLAGS: 0216 ORIG_RAX: 002c >> >> RAX: ffda RBX: 006e29fc RCX: 00446d09 >> >> RDX: 0001 RSI: 2340 RDI: 0003 >> >> RBP: 006e29f8 R08: 204d9000 R09: 001c >> >> R10: R11: 0216 R12: >> >> R13: 7fff7b26fb1f R14: 7f5dbac229c0 R15: 006e2b60 >> >> >> > I think we have a corner case with a0ff660058b88d12625a783ce9e5c1371c87951f >> > here. If a peeloff event happens during a wait for sendbuf space, EPIPE >> > will be >> > returned, and the code path appears to call sctp_association_put twice, >> > leading >> > to the use after free situation. I'll write a patch this weekend >> Hi, Neil, you're right. >> >> I didn't expect peeloff can be done on a NEW asoc, as peeloff needs >> assoc_id, which can only be set when connecting has started. >> >> But I realized that: >> f84af33 sctp: factor out sctp_sendmsg_to_asoc from sctp_sendmsg >> >> moved sctp_primitive_ASSOCIATE(connecting) before sctp_wait_for_sndbuf >> (snd buffer waiting). It means peeloff can be done on a NEW asoc. >> So you may want to move it back. >> > I agree with the root cause, but I'm not sure I agree with just moving the > wait_for_sndbuf call back above the call to associate. I'm not sure I like > relying on placing a call in a spcific order solely to avoid an error > condition > that might legitimately occur. I think would rather check the return code at > the call site for the complete set of conditions for which we should not free > the association. Something like this: > > diff --git a/net/sctp/socket.c b/net/sctp/socket.c > index 7d3476a4860d..a68846d2b0ef 100644 > --- a/net/sctp/socket.c > +++ b/net/sctp/socket.c > @@ -2071,8 +2071,9 @@ static int sctp_sendmsg(struct sock *sk, struct msghdr > *msg, size_t msg_len) > > /* Send msg to the asoc */ > err = sctp_sendmsg_to_asoc(asoc, msg, msg_len, transport, sinfo); > - if (err < 0 && err != -ESRCH && new) > - sctp_association_free(asoc); > + if ((err != -ESRCH) && (err != -EPIPE)) > + if (err < 0 && new) > + sctp_association_free(asoc); > > out_unlock: > release_sock(sk); > > Which I think also avoids the noted conflict. > > Thoughts? If sctp_association_free is called for general asoc, yes, I agree wit
Re: KASAN: use-after-free Read in sctp_association_free (2)
On Sat, Mar 10, 2018 at 03:58:04PM +0800, Xin Long wrote: > On Sat, Mar 10, 2018 at 6:08 AM, Neil Horman wrote: > > On Fri, Mar 09, 2018 at 12:59:06PM -0800, syzbot wrote: > >> Hello, > >> > >> syzbot hit the following crash on net-next commit > >> fd372a7a9e5e9d8011a0222d10edd3523abcd3b1 (Thu Mar 8 19:43:48 2018 +) > >> Merge tag 'mlx5-updates-2018-02-28-2' of > >> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux > >> > >> So far this crash happened 2 times on net-next. > >> C reproducer is attached. > >> syzkaller reproducer is attached. > >> Raw console output is attached. > >> compiler: gcc (GCC) 7.1.1 20170620 > >> .config is attached. > >> > >> IMPORTANT: if you fix the bug, please add the following tag to the commit: > >> Reported-by: syzbot+a4e4112c3aff00c8c...@syzkaller.appspotmail.com > >> It will help syzbot understand when the bug is fixed. See footer for > >> details. > >> If you forward the report, please keep this part and the footer. > >> > >> IPVS: ftp: loaded support on port[0] = 21 > >> IPVS: ftp: loaded support on port[0] = 21 > >> IPVS: ftp: loaded support on port[0] = 21 > >> IPVS: ftp: loaded support on port[0] = 21 > >> == > >> BUG: KASAN: use-after-free in sctp_association_free+0x7b7/0x930 > >> net/sctp/associola.c:332 > >> Read of size 8 at addr 8801d8006ae0 by task syzkaller914861/4202 > >> > >> CPU: 1 PID: 4202 Comm: syzkaller914861 Not tainted 4.16.0-rc4+ #258 > >> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS > >> Google 01/01/2011 > >> Call Trace: > >> __dump_stack lib/dump_stack.c:17 [inline] > >> dump_stack+0x194/0x24d lib/dump_stack.c:53 > >> print_address_description+0x73/0x250 mm/kasan/report.c:256 > >> kasan_report_error mm/kasan/report.c:354 [inline] > >> kasan_report+0x23c/0x360 mm/kasan/report.c:412 > >> __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:433 > >> sctp_association_free+0x7b7/0x930 net/sctp/associola.c:332 > >> sctp_sendmsg+0xc67/0x1a80 net/sctp/socket.c:2075 > >> inet_sendmsg+0x11f/0x5e0 net/ipv4/af_inet.c:763 > >> sock_sendmsg_nosec net/socket.c:629 [inline] > >> sock_sendmsg+0xca/0x110 net/socket.c:639 > >> SYSC_sendto+0x361/0x5c0 net/socket.c:1748 > >> SyS_sendto+0x40/0x50 net/socket.c:1716 > >> do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 > >> entry_SYSCALL_64_after_hwframe+0x42/0xb7 > >> RIP: 0033:0x446d09 > >> RSP: 002b:7f5dbac21da8 EFLAGS: 0216 ORIG_RAX: 002c > >> RAX: ffda RBX: 006e29fc RCX: 00446d09 > >> RDX: 0001 RSI: 2340 RDI: 0003 > >> RBP: 006e29f8 R08: 204d9000 R09: 001c > >> R10: R11: 0216 R12: > >> R13: 7fff7b26fb1f R14: 7f5dbac229c0 R15: 006e2b60 > >> > > I think we have a corner case with a0ff660058b88d12625a783ce9e5c1371c87951f > > here. If a peeloff event happens during a wait for sendbuf space, EPIPE > > will be > > returned, and the code path appears to call sctp_association_put twice, > > leading > > to the use after free situation. I'll write a patch this weekend > Hi, Neil, you're right. > > I didn't expect peeloff can be done on a NEW asoc, as peeloff needs > assoc_id, which can only be set when connecting has started. > > But I realized that: > f84af33 sctp: factor out sctp_sendmsg_to_asoc from sctp_sendmsg > > moved sctp_primitive_ASSOCIATE(connecting) before sctp_wait_for_sndbuf > (snd buffer waiting). It means peeloff can be done on a NEW asoc. > So you may want to move it back. > I agree with the root cause, but I'm not sure I agree with just moving the wait_for_sndbuf call back above the call to associate. I'm not sure I like relying on placing a call in a spcific order solely to avoid an error condition that might legitimately occur. I think would rather check the return code at the call site for the complete set of conditions for which we should not free the association. Something like this: diff --git a/net/sctp/socket.c b/net/sctp/socket.c index 7d3476a4860d..a68846d2b0ef 100644 --- a/net/sctp/socket.c +++ b/net/sctp/socket.c @@ -2071,8 +2071,9 @@ static int sctp_sendmsg(struct sock *sk, struct msghdr *msg, size_t msg_len) /* Send msg to the asoc */ err = sctp_sendmsg_to_asoc(asoc, msg, msg_len, transport, sinfo); - if (err < 0 && err != -ESRCH && new) - sctp_association_free(asoc); + if ((err != -ESRCH) && (err != -EPIPE)) + if (err < 0 && new) + sctp_association_free(asoc); out_unlock: release_sock(sk); Which I think also avoids the noted conflict. Thoughts? Neil > One good thing is the fix shouldn't touch the conflict on > https://lkml.org/lkml/2018/3/7/1175 > We can fix it right now, I think. But pls double check it before > submitting the patch. We just can't grow up that fixup for linus > tree's
Re: KASAN: use-after-free Read in sctp_association_free (2)
On Sat, Mar 10, 2018 at 6:08 AM, Neil Horman wrote: > On Fri, Mar 09, 2018 at 12:59:06PM -0800, syzbot wrote: >> Hello, >> >> syzbot hit the following crash on net-next commit >> fd372a7a9e5e9d8011a0222d10edd3523abcd3b1 (Thu Mar 8 19:43:48 2018 +) >> Merge tag 'mlx5-updates-2018-02-28-2' of >> git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux >> >> So far this crash happened 2 times on net-next. >> C reproducer is attached. >> syzkaller reproducer is attached. >> Raw console output is attached. >> compiler: gcc (GCC) 7.1.1 20170620 >> .config is attached. >> >> IMPORTANT: if you fix the bug, please add the following tag to the commit: >> Reported-by: syzbot+a4e4112c3aff00c8c...@syzkaller.appspotmail.com >> It will help syzbot understand when the bug is fixed. See footer for >> details. >> If you forward the report, please keep this part and the footer. >> >> IPVS: ftp: loaded support on port[0] = 21 >> IPVS: ftp: loaded support on port[0] = 21 >> IPVS: ftp: loaded support on port[0] = 21 >> IPVS: ftp: loaded support on port[0] = 21 >> == >> BUG: KASAN: use-after-free in sctp_association_free+0x7b7/0x930 >> net/sctp/associola.c:332 >> Read of size 8 at addr 8801d8006ae0 by task syzkaller914861/4202 >> >> CPU: 1 PID: 4202 Comm: syzkaller914861 Not tainted 4.16.0-rc4+ #258 >> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS >> Google 01/01/2011 >> Call Trace: >> __dump_stack lib/dump_stack.c:17 [inline] >> dump_stack+0x194/0x24d lib/dump_stack.c:53 >> print_address_description+0x73/0x250 mm/kasan/report.c:256 >> kasan_report_error mm/kasan/report.c:354 [inline] >> kasan_report+0x23c/0x360 mm/kasan/report.c:412 >> __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:433 >> sctp_association_free+0x7b7/0x930 net/sctp/associola.c:332 >> sctp_sendmsg+0xc67/0x1a80 net/sctp/socket.c:2075 >> inet_sendmsg+0x11f/0x5e0 net/ipv4/af_inet.c:763 >> sock_sendmsg_nosec net/socket.c:629 [inline] >> sock_sendmsg+0xca/0x110 net/socket.c:639 >> SYSC_sendto+0x361/0x5c0 net/socket.c:1748 >> SyS_sendto+0x40/0x50 net/socket.c:1716 >> do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 >> entry_SYSCALL_64_after_hwframe+0x42/0xb7 >> RIP: 0033:0x446d09 >> RSP: 002b:7f5dbac21da8 EFLAGS: 0216 ORIG_RAX: 002c >> RAX: ffda RBX: 006e29fc RCX: 00446d09 >> RDX: 0001 RSI: 2340 RDI: 0003 >> RBP: 006e29f8 R08: 204d9000 R09: 001c >> R10: R11: 0216 R12: >> R13: 7fff7b26fb1f R14: 7f5dbac229c0 R15: 006e2b60 >> > I think we have a corner case with a0ff660058b88d12625a783ce9e5c1371c87951f > here. If a peeloff event happens during a wait for sendbuf space, EPIPE will > be > returned, and the code path appears to call sctp_association_put twice, > leading > to the use after free situation. I'll write a patch this weekend Hi, Neil, you're right. I didn't expect peeloff can be done on a NEW asoc, as peeloff needs assoc_id, which can only be set when connecting has started. But I realized that: f84af33 sctp: factor out sctp_sendmsg_to_asoc from sctp_sendmsg moved sctp_primitive_ASSOCIATE(connecting) before sctp_wait_for_sndbuf (snd buffer waiting). It means peeloff can be done on a NEW asoc. So you may want to move it back. One good thing is the fix shouldn't touch the conflict on https://lkml.org/lkml/2018/3/7/1175 We can fix it right now, I think. But pls double check it before submitting the patch. We just can't grow up that fixup for linus tree's merge. Thanks.
Re: KASAN: use-after-free Read in sctp_association_free (2)
On Fri, Mar 09, 2018 at 12:59:06PM -0800, syzbot wrote: > Hello, > > syzbot hit the following crash on net-next commit > fd372a7a9e5e9d8011a0222d10edd3523abcd3b1 (Thu Mar 8 19:43:48 2018 +) > Merge tag 'mlx5-updates-2018-02-28-2' of > git://git.kernel.org/pub/scm/linux/kernel/git/mellanox/linux > > So far this crash happened 2 times on net-next. > C reproducer is attached. > syzkaller reproducer is attached. > Raw console output is attached. > compiler: gcc (GCC) 7.1.1 20170620 > .config is attached. > > IMPORTANT: if you fix the bug, please add the following tag to the commit: > Reported-by: syzbot+a4e4112c3aff00c8c...@syzkaller.appspotmail.com > It will help syzbot understand when the bug is fixed. See footer for > details. > If you forward the report, please keep this part and the footer. > > IPVS: ftp: loaded support on port[0] = 21 > IPVS: ftp: loaded support on port[0] = 21 > IPVS: ftp: loaded support on port[0] = 21 > IPVS: ftp: loaded support on port[0] = 21 > == > BUG: KASAN: use-after-free in sctp_association_free+0x7b7/0x930 > net/sctp/associola.c:332 > Read of size 8 at addr 8801d8006ae0 by task syzkaller914861/4202 > > CPU: 1 PID: 4202 Comm: syzkaller914861 Not tainted 4.16.0-rc4+ #258 > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS > Google 01/01/2011 > Call Trace: > __dump_stack lib/dump_stack.c:17 [inline] > dump_stack+0x194/0x24d lib/dump_stack.c:53 > print_address_description+0x73/0x250 mm/kasan/report.c:256 > kasan_report_error mm/kasan/report.c:354 [inline] > kasan_report+0x23c/0x360 mm/kasan/report.c:412 > __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:433 > sctp_association_free+0x7b7/0x930 net/sctp/associola.c:332 > sctp_sendmsg+0xc67/0x1a80 net/sctp/socket.c:2075 > inet_sendmsg+0x11f/0x5e0 net/ipv4/af_inet.c:763 > sock_sendmsg_nosec net/socket.c:629 [inline] > sock_sendmsg+0xca/0x110 net/socket.c:639 > SYSC_sendto+0x361/0x5c0 net/socket.c:1748 > SyS_sendto+0x40/0x50 net/socket.c:1716 > do_syscall_64+0x281/0x940 arch/x86/entry/common.c:287 > entry_SYSCALL_64_after_hwframe+0x42/0xb7 > RIP: 0033:0x446d09 > RSP: 002b:7f5dbac21da8 EFLAGS: 0216 ORIG_RAX: 002c > RAX: ffda RBX: 006e29fc RCX: 00446d09 > RDX: 0001 RSI: 2340 RDI: 0003 > RBP: 006e29f8 R08: 204d9000 R09: 001c > R10: R11: 0216 R12: > R13: 7fff7b26fb1f R14: 7f5dbac229c0 R15: 006e2b60 > I think we have a corner case with a0ff660058b88d12625a783ce9e5c1371c87951f here. If a peeloff event happens during a wait for sendbuf space, EPIPE will be returned, and the code path appears to call sctp_association_put twice, leading to the use after free situation. I'll write a patch this weekend Neil
Re: KASAN: use-after-free Read in sctp_association_free
On Thu, Nov 02, 2017 at 08:07:27PM +0800, Xin Long wrote:
> On Thu, Nov 2, 2017 at 1:55 AM, syzbot
>
> wrote:
> > Hello,
> >
> > syzkaller hit the following crash on
> > 25a5d23b47994cdb451dcd2bc8ac310a1492f71b
> > git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/master
> > compiler: gcc (GCC) 7.1.1 20170620
> > .config is attached
> > Raw console output is attached.
> > C reproducer is attached
> > syzkaller reproducer is attached. See https://goo.gl/kgGztJ
> > for information about syzkaller reproducers
> >
> >
> > ==
> > BUG: KASAN: use-after-free in sctp_association_free+0x7b7/0x930
> > net/sctp/associola.c:333
> > Read of size 8 at addr 8801d21d4720 by task syzkaller504854/3007
> >
> > CPU: 0 PID: 3007 Comm: syzkaller504854 Not tainted 4.14.0-rc6+ #62
> > Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
> > Google 01/01/2011
> > Call Trace:
> > __dump_stack lib/dump_stack.c:16 [inline]
> > dump_stack+0x194/0x257 lib/dump_stack.c:52
> > print_address_description+0x73/0x250 mm/kasan/report.c:252
> > kasan_report_error mm/kasan/report.c:351 [inline]
> > kasan_report+0x25b/0x340 mm/kasan/report.c:409
> > __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:430
> > sctp_association_free+0x7b7/0x930 net/sctp/associola.c:333
> asoc could have been freed by sctp_stop_t1_and_abort or elsewhere
> when waiting for buf without holding sk lock.
>
> One possible fix:
>
> diff --git a/net/sctp/socket.c b/net/sctp/socket.c
> index c75acdf..e2ea12a 100644
> --- a/net/sctp/socket.c
> +++ b/net/sctp/socket.c
> @@ -2015,7 +2015,7 @@ static int sctp_sendmsg(struct sock *sk, struct
> msghdr *msg, size_t msg_len)
> goto out_unlock;
>
> out_free:
> - if (new_asoc)
> + if (new_asoc && err != -ESRCH)
> sctp_association_free(asoc);
> out_unlock:
> release_sock(sk);
> @@ -7976,10 +7976,11 @@ static int sctp_wait_for_sndbuf(struct
> sctp_association *asoc, long *timeo_p,
> for (;;) {
> prepare_to_wait_exclusive(&asoc->wait, &wait,
> TASK_INTERRUPTIBLE);
> + if (asoc->base.dead)
> + goto do_dead;
> if (!*timeo_p)
> goto do_nonblock;
> - if (sk->sk_err || asoc->state >= SCTP_STATE_SHUTDOWN_PENDING
> ||
> - asoc->base.dead)
> + if (sk->sk_err || asoc->state >= SCTP_STATE_SHUTDOWN_PENDING)
> goto do_error;
> if (signal_pending(current))
> goto do_interrupted;
> @@ -8004,6 +8005,10 @@ static int sctp_wait_for_sndbuf(struct
> sctp_association *asoc, long *timeo_p,
>
> return err;
>
> +do_dead:
> + err = -ESRCH;
> + goto out;
> +
> do_error:
> err = -EPIPE;
> goto out;
>
> will check for sure before posting. thanks.
>
> > sctp_sendmsg+0x2311/0x31f0 net/sctp/socket.c:2011
> > inet_sendmsg+0x11f/0x5e0 net/ipv4/af_inet.c:762
> > sock_sendmsg_nosec net/socket.c:633 [inline]
> > sock_sendmsg+0xca/0x110 net/socket.c:643
> > SYSC_sendto+0x352/0x5a0 net/socket.c:1750
> > SyS_sendto+0x40/0x50 net/socket.c:1718
> > do_syscall_32_irqs_on arch/x86/entry/common.c:329 [inline]
> > do_fast_syscall_32+0x3f2/0xf05 arch/x86/entry/common.c:391
> > entry_SYSENTER_compat+0x51/0x60 arch/x86/entry/entry_64_compat.S:124
> > RIP: 0023:0xf7fd2c79
> > RSP: 002b:f5fca1ec EFLAGS: 0292 ORIG_RAX: 0171
> > RAX: ffda RBX: 0003 RCX: 20925000
> > RDX: 0002 RSI: RDI: 209e1000
> > RBP: 001c R08: R09:
> > R10: R11: R12:
> > R13: R14: R15:
> >
This crash has stopped occurring. I assume it was fixed by commit ca3af4dd28cff
(thanks Xin!), so let's tell syzbot so that it can continue to report crashes in
the same place:
#syz fix: sctp: do not free asoc when it is already dead in sctp_sendmsg
- Eric
Re: KASAN: use-after-free Read in sctp_association_free
On Thu, Nov 2, 2017 at 1:55 AM, syzbot
wrote:
> Hello,
>
> syzkaller hit the following crash on
> 25a5d23b47994cdb451dcd2bc8ac310a1492f71b
> git://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/master
> compiler: gcc (GCC) 7.1.1 20170620
> .config is attached
> Raw console output is attached.
> C reproducer is attached
> syzkaller reproducer is attached. See https://goo.gl/kgGztJ
> for information about syzkaller reproducers
>
>
> ==
> BUG: KASAN: use-after-free in sctp_association_free+0x7b7/0x930
> net/sctp/associola.c:333
> Read of size 8 at addr 8801d21d4720 by task syzkaller504854/3007
>
> CPU: 0 PID: 3007 Comm: syzkaller504854 Not tainted 4.14.0-rc6+ #62
> Hardware name: Google Google Compute Engine/Google Compute Engine, BIOS
> Google 01/01/2011
> Call Trace:
> __dump_stack lib/dump_stack.c:16 [inline]
> dump_stack+0x194/0x257 lib/dump_stack.c:52
> print_address_description+0x73/0x250 mm/kasan/report.c:252
> kasan_report_error mm/kasan/report.c:351 [inline]
> kasan_report+0x25b/0x340 mm/kasan/report.c:409
> __asan_report_load8_noabort+0x14/0x20 mm/kasan/report.c:430
> sctp_association_free+0x7b7/0x930 net/sctp/associola.c:333
asoc could have been freed by sctp_stop_t1_and_abort or elsewhere
when waiting for buf without holding sk lock.
One possible fix:
diff --git a/net/sctp/socket.c b/net/sctp/socket.c
index c75acdf..e2ea12a 100644
--- a/net/sctp/socket.c
+++ b/net/sctp/socket.c
@@ -2015,7 +2015,7 @@ static int sctp_sendmsg(struct sock *sk, struct
msghdr *msg, size_t msg_len)
goto out_unlock;
out_free:
- if (new_asoc)
+ if (new_asoc && err != -ESRCH)
sctp_association_free(asoc);
out_unlock:
release_sock(sk);
@@ -7976,10 +7976,11 @@ static int sctp_wait_for_sndbuf(struct
sctp_association *asoc, long *timeo_p,
for (;;) {
prepare_to_wait_exclusive(&asoc->wait, &wait,
TASK_INTERRUPTIBLE);
+ if (asoc->base.dead)
+ goto do_dead;
if (!*timeo_p)
goto do_nonblock;
- if (sk->sk_err || asoc->state >= SCTP_STATE_SHUTDOWN_PENDING ||
- asoc->base.dead)
+ if (sk->sk_err || asoc->state >= SCTP_STATE_SHUTDOWN_PENDING)
goto do_error;
if (signal_pending(current))
goto do_interrupted;
@@ -8004,6 +8005,10 @@ static int sctp_wait_for_sndbuf(struct
sctp_association *asoc, long *timeo_p,
return err;
+do_dead:
+ err = -ESRCH;
+ goto out;
+
do_error:
err = -EPIPE;
goto out;
will check for sure before posting. thanks.
> sctp_sendmsg+0x2311/0x31f0 net/sctp/socket.c:2011
> inet_sendmsg+0x11f/0x5e0 net/ipv4/af_inet.c:762
> sock_sendmsg_nosec net/socket.c:633 [inline]
> sock_sendmsg+0xca/0x110 net/socket.c:643
> SYSC_sendto+0x352/0x5a0 net/socket.c:1750
> SyS_sendto+0x40/0x50 net/socket.c:1718
> do_syscall_32_irqs_on arch/x86/entry/common.c:329 [inline]
> do_fast_syscall_32+0x3f2/0xf05 arch/x86/entry/common.c:391
> entry_SYSENTER_compat+0x51/0x60 arch/x86/entry/entry_64_compat.S:124
> RIP: 0023:0xf7fd2c79
> RSP: 002b:f5fca1ec EFLAGS: 0292 ORIG_RAX: 0171
> RAX: ffda RBX: 0003 RCX: 20925000
> RDX: 0002 RSI: RDI: 209e1000
> RBP: 001c R08: R09:
> R10: R11: R12:
> R13: R14: R15:
>
> Allocated by task 3007:
> save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
> save_stack+0x43/0xd0 mm/kasan/kasan.c:447
> set_track mm/kasan/kasan.c:459 [inline]
> kasan_kmalloc+0xad/0xe0 mm/kasan/kasan.c:551
> kmem_cache_alloc_trace+0x136/0x750 mm/slab.c:3627
> kmalloc include/linux/slab.h:493 [inline]
> kzalloc include/linux/slab.h:666 [inline]
> sctp_association_new+0x114/0x21e0 net/sctp/associola.c:309
> sctp_sendmsg+0x128c/0x31f0 net/sctp/socket.c:1838
> inet_sendmsg+0x11f/0x5e0 net/ipv4/af_inet.c:762
> sock_sendmsg_nosec net/socket.c:633 [inline]
> sock_sendmsg+0xca/0x110 net/socket.c:643
> SYSC_sendto+0x352/0x5a0 net/socket.c:1750
> SyS_sendto+0x40/0x50 net/socket.c:1718
> do_syscall_32_irqs_on arch/x86/entry/common.c:329 [inline]
> do_fast_syscall_32+0x3f2/0xf05 arch/x86/entry/common.c:391
> entry_SYSENTER_compat+0x51/0x60 arch/x86/entry/entry_64_compat.S:124
>
> Freed by task 3007:
> save_stack_trace+0x16/0x20 arch/x86/kernel/stacktrace.c:59
> save_stack+0x43/0xd0 mm/kasan/kasan.c:447
> set_track mm/kasan/kasan.c:459 [inline]
> kasan_slab_free+0x71/0xc0 mm/kasan/kasan.c:524
> __cache_free mm/slab.c:3503 [inline]
> kfree+0xca/0x250 mm/slab.c:3820
> sctp_association_destroy net/sctp/associola.c:435 [inline]
> sctp_association_put+0x21c/0x2f0 net/s
