Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-02-04 Thread Tommi Rantala 

On 02.02.2018 14:34, Neil Horman wrote:

Patch looks good, but if you could please submit it with the proper title in a
separate thread so it gets davem's attention properly, I'd appreciate it.
Additional points if you update it to include the ipv6 fixes :)


Thanks, I'll send this patch properly.

I think I will let Alexey post the ipv6 fixes separately, as he's also 
planning another fix to sctp_v6_get_dst.


Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-02-02 Thread Neil Horman 
On Thu, Feb 01, 2018 at 10:41:27PM +0200, Tommi Rantala wrote:
> 2018-02-01 21:23 GMT+02:00 Neil Horman :
> > No, I can't say I saw the patch on the list.  Can you resend it?
> 
> Here's the patch again, sending from gmail this time.
> 
> The previous mail is now also at spinics, dunno what happened.
> https://www.spinics.net/lists/linux-sctp/msg07005.html
> 
> 
> From b94c037d27e36a3053e6862b7e7da5f07f2f5597 Mon Sep 17 00:00:00 2001
> From: Tommi Rantala 
> Date: Wed, 31 Jan 2018 11:24:31 +
> Subject: [PATCH] sctp: fix dst leak in sctp_v4_get_dst
> 
> Fix dst reference leak in sctp_v4_get_dst() introduced in commit
> 410f03831 ("sctp: add routing output fallback"):
> 
> When walking the address_list, successive ip_route_output_key() calls
> may return the same rt->dst with the reference incremented on each call.
> 
> The code would not decrement the dst refcount when the dst pointer was
> identical from the previous iteration, causing the dst leak.
> 
> Testcase:
>   ip netns add TEST
>   ip netns exec TEST ip link set lo up
>   ip link add dummy0 type dummy
>   ip link add dummy1 type dummy
>   ip link add dummy2 type dummy
>   ip link set dev dummy0 netns TEST
>   ip link set dev dummy1 netns TEST
>   ip link set dev dummy2 netns TEST
>   ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
>   ip netns exec TEST ip link set dummy0 up
>   ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
>   ip netns exec TEST ip link set dummy1 up
>   ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
>   ip netns exec TEST ip link set dummy2 up
>   ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1
> -p 2 -s -B 192.168.1.3
>   ip netns del TEST
> 
Patch looks good, but if you could please submit it with the proper title in a
separate thread so it gets davem's attention properly, I'd appreciate it.
Additional points if you update it to include the ipv6 fixes :)
Neil

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-02-02 Thread Alexey Kodanev 
On 02.02.2018 11:27, Tommi Rantala wrote:
> 2018-02-02 1:57 GMT+02:00 Alexey Kodanev :
>> For ipv6 part, shouldn't we release 'bdst' there if the previous address
>> match is better and we continue to the next iteration?
> 
> Good catch!
> 

On the second thought, I think, we should also check 'bdst' ptr for
the error earlier, i.e. right after 'bdst = ip6_dst_lookup_flow(...)'.
I'll prepare the patch.

Thanks,
Alexey


> Didn't see that one.
> 
> Tommi
> 
>> diff --git a/net/sctp/ipv6.c b/net/sctp/ipv6.c
>> index 5d4c15b..a044096 100644
>> --- a/net/sctp/ipv6.c
>> +++ b/net/sctp/ipv6.c
>> @@ -336,8 +336,11 @@ static void sctp_v6_get_dst(struct sctp_transport *t, 
>> union sctp_addr *saddr,
>> }
>>
>> bmatchlen = sctp_v6_addr_match_len(daddr, >a);
>> -   if (matchlen > bmatchlen)
>> +   if (matchlen > bmatchlen) {
>> +   if (!IS_ERR(bdst))
>> +   dst_release(bdst);
>> continue;
>> +   }
>>
>> if (!IS_ERR_OR_NULL(dst))
>> dst_release(dst);
>>
>> Thanks,
>> Alexey

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-02-02 Thread Tommi Rantala 
2018-02-02 1:57 GMT+02:00 Alexey Kodanev :
> For ipv6 part, shouldn't we release 'bdst' there if the previous address
> match is better and we continue to the next iteration?

Good catch!

Didn't see that one.

Tommi

> diff --git a/net/sctp/ipv6.c b/net/sctp/ipv6.c
> index 5d4c15b..a044096 100644
> --- a/net/sctp/ipv6.c
> +++ b/net/sctp/ipv6.c
> @@ -336,8 +336,11 @@ static void sctp_v6_get_dst(struct sctp_transport *t, 
> union sctp_addr *saddr,
> }
>
> bmatchlen = sctp_v6_addr_match_len(daddr, >a);
> -   if (matchlen > bmatchlen)
> +   if (matchlen > bmatchlen) {
> +   if (!IS_ERR(bdst))
> +   dst_release(bdst);
> continue;
> +   }
>
> if (!IS_ERR_OR_NULL(dst))
> dst_release(dst);
>
> Thanks,
> Alexey

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-02-01 Thread Alexey Kodanev 
On 01.02.2018 21:02, Tommi Rantala wrote:
> 2018-01-31 19:51 GMT+02:00 Tommi Rantala :
>> On 31.01.2018 14:31, Neil Horman wrote:
>>>
>>> On Wed, Jan 31, 2018 at 11:42:24AM +0200, Tommi Rantala wrote:

 I think there's a problem in the dst refcounting in sctp_v4_get_dst()

 There's a dst_entry struct that has >0 refcnt after running the testcase,
 which makes it impossible to delete the loopback device, as that dst is
 never freed.

 I'll try to make a patch.

>>> Are you looking at the second for loop there, which uses
>>> ip_route_output_key,
>>> but discards the result if dst is already set?  That does look a bit
>>> wonky, and
>>> the same problem may exist in the ipv6 path.  Let me know what the result
>>> is.
>>
>>
>> Yes, that was it. Did you receive the email I sent with the patch?
>> (I'm not seeing that message e.g. at spinics.net linux-sctp archive, so just
>> wondering if that email got lost somehow...)
>>
>> I'll check the ipv6 case, did not try it yet.
>>
>> Tommi
> 
> As far as I can tell, the ipv6 code does not suffer from this.
> The dst handling there looks good to me.
> 

For ipv6 part, shouldn't we release 'bdst' there if the previous address
match is better and we continue to the next iteration?

diff --git a/net/sctp/ipv6.c b/net/sctp/ipv6.c
index 5d4c15b..a044096 100644
--- a/net/sctp/ipv6.c
+++ b/net/sctp/ipv6.c
@@ -336,8 +336,11 @@ static void sctp_v6_get_dst(struct sctp_transport *t, 
union sctp_addr *saddr,
}
 
bmatchlen = sctp_v6_addr_match_len(daddr, >a);
-   if (matchlen > bmatchlen)
+   if (matchlen > bmatchlen) {
+   if (!IS_ERR(bdst))
+   dst_release(bdst);
continue;
+   }
 
if (!IS_ERR_OR_NULL(dst))
dst_release(dst);

Thanks,
Alexey

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-02-01 Thread Tommi Rantala 
2018-02-01 21:23 GMT+02:00 Neil Horman <nhor...@tuxdriver.com>:
> No, I can't say I saw the patch on the list.  Can you resend it?

Here's the patch again, sending from gmail this time.

The previous mail is now also at spinics, dunno what happened.
https://www.spinics.net/lists/linux-sctp/msg07005.html


>From b94c037d27e36a3053e6862b7e7da5f07f2f5597 Mon Sep 17 00:00:00 2001
From: Tommi Rantala <tommi.t.rant...@nokia.com>
Date: Wed, 31 Jan 2018 11:24:31 +
Subject: [PATCH] sctp: fix dst leak in sctp_v4_get_dst

Fix dst reference leak in sctp_v4_get_dst() introduced in commit
410f03831 ("sctp: add routing output fallback"):

When walking the address_list, successive ip_route_output_key() calls
may return the same rt->dst with the reference incremented on each call.

The code would not decrement the dst refcount when the dst pointer was
identical from the previous iteration, causing the dst leak.

Testcase:
  ip netns add TEST
  ip netns exec TEST ip link set lo up
  ip link add dummy0 type dummy
  ip link add dummy1 type dummy
  ip link add dummy2 type dummy
  ip link set dev dummy0 netns TEST
  ip link set dev dummy1 netns TEST
  ip link set dev dummy2 netns TEST
  ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
  ip netns exec TEST ip link set dummy0 up
  ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
  ip netns exec TEST ip link set dummy1 up
  ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
  ip netns exec TEST ip link set dummy2 up
  ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1
-p 2 -s -B 192.168.1.3
  ip netns del TEST

In 4.4 and 4.9 kernels this results to:
  [  354.179591] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  [  364.419674] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  [  374.663664] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  [  384.903717] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  [  395.143724] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  [  405.383645] unregister_netdevice: waiting for lo to become free.
Usage count = 1
  ...

Fixes: 410f03831 ("sctp: add routing output fallback")
Signed-off-by: Tommi Rantala <tommi.t.rant...@nokia.com>
---
 net/sctp/protocol.c | 9 -
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/net/sctp/protocol.c b/net/sctp/protocol.c
index 8b4ff315695e..aadb9e7f60e2 100644
--- a/net/sctp/protocol.c
+++ b/net/sctp/protocol.c
@@ -508,21 +508,20 @@ static void sctp_v4_get_dst(struct
sctp_transport *t, union sctp_addr *saddr,
  if (IS_ERR(rt))
  continue;

- if (!dst)
- dst = >dst;
-
  /* Ensure the src address belongs to the output
  * interface.
  */
  odev = __ip_dev_find(sock_net(sk), laddr->a.v4.sin_addr.s_addr,
   false);
  if (!odev || odev->ifindex != fl4->flowi4_oif) {
- if (>dst != dst)
+ if (!dst)
+ dst = >dst;
+ else
  dst_release(>dst);
  continue;
  }

- if (dst != >dst)
+ if (dst && dst != >dst)
  dst_release(dst);
  dst = >dst;
  break;
-- 
2.15.1

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-02-01 Thread Neil Horman 
On Thu, Feb 01, 2018 at 08:02:07PM +0200, Tommi Rantala wrote:
> 2018-01-31 19:51 GMT+02:00 Tommi Rantala :
> > On 31.01.2018 14:31, Neil Horman wrote:
> >>
> >> On Wed, Jan 31, 2018 at 11:42:24AM +0200, Tommi Rantala wrote:
> >>>
> >>> I think there's a problem in the dst refcounting in sctp_v4_get_dst()
> >>>
> >>> There's a dst_entry struct that has >0 refcnt after running the testcase,
> >>> which makes it impossible to delete the loopback device, as that dst is
> >>> never freed.
> >>>
> >>> I'll try to make a patch.
> >>>
> >> Are you looking at the second for loop there, which uses
> >> ip_route_output_key,
> >> but discards the result if dst is already set?  That does look a bit
> >> wonky, and
> >> the same problem may exist in the ipv6 path.  Let me know what the result
> >> is.
> >
> >
> > Yes, that was it. Did you receive the email I sent with the patch?
> > (I'm not seeing that message e.g. at spinics.net linux-sctp archive, so just
> > wondering if that email got lost somehow...)
> >
No, I can't say I saw the patch on the list.  Can you resend it?

Neil

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-02-01 Thread Tommi Rantala 
2018-01-31 19:51 GMT+02:00 Tommi Rantala :
> On 31.01.2018 14:31, Neil Horman wrote:
>>
>> On Wed, Jan 31, 2018 at 11:42:24AM +0200, Tommi Rantala wrote:
>>>
>>> I think there's a problem in the dst refcounting in sctp_v4_get_dst()
>>>
>>> There's a dst_entry struct that has >0 refcnt after running the testcase,
>>> which makes it impossible to delete the loopback device, as that dst is
>>> never freed.
>>>
>>> I'll try to make a patch.
>>>
>> Are you looking at the second for loop there, which uses
>> ip_route_output_key,
>> but discards the result if dst is already set?  That does look a bit
>> wonky, and
>> the same problem may exist in the ipv6 path.  Let me know what the result
>> is.
>
>
> Yes, that was it. Did you receive the email I sent with the patch?
> (I'm not seeing that message e.g. at spinics.net linux-sctp archive, so just
> wondering if that email got lost somehow...)
>
> I'll check the ipv6 case, did not try it yet.
>
> Tommi

As far as I can tell, the ipv6 code does not suffer from this.
The dst handling there looks good to me.

Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-01-31 Thread Tommi Rantala 

On 31.01.2018 14:31, Neil Horman wrote:

On Wed, Jan 31, 2018 at 11:42:24AM +0200, Tommi Rantala wrote:

I think there's a problem in the dst refcounting in sctp_v4_get_dst()

There's a dst_entry struct that has >0 refcnt after running the testcase,
which makes it impossible to delete the loopback device, as that dst is
never freed.

I'll try to make a patch.


Are you looking at the second for loop there, which uses ip_route_output_key,
but discards the result if dst is already set?  That does look a bit wonky, and
the same problem may exist in the ipv6 path.  Let me know what the result is.


Yes, that was it. Did you receive the email I sent with the patch?
(I'm not seeing that message e.g. at spinics.net linux-sctp archive, so 
just wondering if that email got lost somehow...)


I'll check the ipv6 case, did not try it yet.

Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-01-31 Thread Neil Horman 
On Wed, Jan 31, 2018 at 11:42:24AM +0200, Tommi Rantala wrote:
> On 30.01.2018 23:03, Neil Horman wrote:
> > On Tue, Jan 30, 2018 at 09:24:17PM +0200, Tommi Rantala wrote:
> > > On 30.01.2018 17:59, Neil Horman wrote:
> > > > On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
> > > > > 
> > > > > ip netns add TEST
> > > > > ip netns exec TEST ip link set lo up
> > > > > ip link add dummy0 type dummy
> > > > > ip link add dummy1 type dummy
> > > > > ip link add dummy2 type dummy
> > > > > ip link set dev dummy0 netns TEST
> > > > > ip link set dev dummy1 netns TEST
> > > > > ip link set dev dummy2 netns TEST
> > > > > ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
> > > > > ip netns exec TEST ip link set dummy0 up
> > > > > ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
> > > > > ip netns exec TEST ip link set dummy1 up
> > > > > ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
> > > > > ip netns exec TEST ip link set dummy2 up
> > > > > ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 
> > > > > -p 2
> > > > > -s -B 192.168.1.3
> > > > > ip netns del TEST
> > > > > 
> > > > Does the problem occur if you don't set lo up?
> > > 
> > > Still happens after dropping "ip netns exec TEST ip link set lo up".
> > > 
> > > Omitting "-B 192.168.1.3" from the sctp_test args helps.
> > > 
> > Thatswierd.  I'll look at the sctp_test code in the AM
> 
> I think there's a problem in the dst refcounting in sctp_v4_get_dst()
> 
> There's a dst_entry struct that has >0 refcnt after running the testcase,
> which makes it impossible to delete the loopback device, as that dst is
> never freed.
> 
> I'll try to make a patch.
> 
Are you looking at the second for loop there, which uses ip_route_output_key,
but discards the result if dst is already set?  That does look a bit wonky, and
the same problem may exist in the ipv6 path.  Let me know what the result is.
Neil

> Tommi
> 
>

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-01-31 Thread Tommi Rantala 

On 31.01.2018 11:42, Tommi Rantala wrote:

I think there's a problem in the dst refcounting in sctp_v4_get_dst()

There's a dst_entry struct that has >0 refcnt after running the 
testcase, which makes it impossible to delete the loopback device, as 
that dst is never freed.


I'll try to make a patch.

Tommi



Hi,

This fixes the problem for me, does it look good?

Tested it briefly on 4.4.113 and 4.15-rc9.


From b94c037d27e36a3053e6862b7e7da5f07f2f5597 Mon Sep 17 00:00:00 2001
From: Tommi Rantala <tommi.t.rant...@nokia.com>
Date: Wed, 31 Jan 2018 11:24:31 +
Subject: [PATCH] sctp: fix dst leak in sctp_v4_get_dst

Fix dst reference leak in sctp_v4_get_dst() introduced in commit
410f03831 ("sctp: add routing output fallback"):

When walking the address_list, successive ip_route_output_key() calls
may return the same rt->dst with the reference incremented on each call.

The code would not decrement the dst refcount when the dst pointer was
identical from the previous iteration, causing the dst leak.

Testcase:
  ip netns add TEST
  ip netns exec TEST ip link set lo up
  ip link add dummy0 type dummy
  ip link add dummy1 type dummy
  ip link add dummy2 type dummy
  ip link set dev dummy0 netns TEST
  ip link set dev dummy1 netns TEST
  ip link set dev dummy2 netns TEST
  ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
  ip netns exec TEST ip link set dummy0 up
  ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
  ip netns exec TEST ip link set dummy1 up
  ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
  ip netns exec TEST ip link set dummy2 up
  ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 
-p 2 -s -B 192.168.1.3

  ip netns del TEST

In 4.4 and 4.9 kernels this results to:
  [  354.179591] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
  [  364.419674] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
  [  374.663664] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
  [  384.903717] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
  [  395.143724] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
  [  405.383645] unregister_netdevice: waiting for lo to become free. 
Usage count = 1

  ...

Fixes: 410f03831 ("sctp: add routing output fallback")
Signed-off-by: Tommi Rantala <tommi.t.rant...@nokia.com>
---
 net/sctp/protocol.c | 9 -
 1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/net/sctp/protocol.c b/net/sctp/protocol.c
index 8b4ff315695e..aadb9e7f60e2 100644
--- a/net/sctp/protocol.c
+++ b/net/sctp/protocol.c
@@ -508,21 +508,20 @@ static void sctp_v4_get_dst(struct sctp_transport 
*t, union sctp_addr *saddr,

if (IS_ERR(rt))
continue;

-   if (!dst)
-   dst = >dst;
-
/* Ensure the src address belongs to the output
 * interface.
 */
odev = __ip_dev_find(sock_net(sk), laddr->a.v4.sin_addr.s_addr,
 false);
if (!odev || odev->ifindex != fl4->flowi4_oif) {
-   if (>dst != dst)
+   if (!dst)
+   dst = >dst;
+   else
dst_release(>dst);
continue;
}

-   if (dst != >dst)
+   if (dst && dst != >dst)
dst_release(dst);
dst = >dst;
break;
--
2.15.1

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-01-31 Thread Tommi Rantala 

On 30.01.2018 23:03, Neil Horman wrote:

On Tue, Jan 30, 2018 at 09:24:17PM +0200, Tommi Rantala wrote:

On 30.01.2018 17:59, Neil Horman wrote:

On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:


ip netns add TEST
ip netns exec TEST ip link set lo up
ip link add dummy0 type dummy
ip link add dummy1 type dummy
ip link add dummy2 type dummy
ip link set dev dummy0 netns TEST
ip link set dev dummy1 netns TEST
ip link set dev dummy2 netns TEST
ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
ip netns exec TEST ip link set dummy0 up
ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
ip netns exec TEST ip link set dummy1 up
ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
ip netns exec TEST ip link set dummy2 up
ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 2
-s -B 192.168.1.3
ip netns del TEST


Does the problem occur if you don't set lo up?


Still happens after dropping "ip netns exec TEST ip link set lo up".

Omitting "-B 192.168.1.3" from the sctp_test args helps.


Thatswierd.  I'll look at the sctp_test code in the AM


I think there's a problem in the dst refcounting in sctp_v4_get_dst()

There's a dst_entry struct that has >0 refcnt after running the 
testcase, which makes it impossible to delete the loopback device, as 
that dst is never freed.


I'll try to make a patch.

Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-01-30 Thread Tommi Rantala 

On 31.01.2018 00:52, Marcelo Ricardo Leitner wrote:

On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:

Hi,

When running sctp_test from lksctp-tools in netns in 4.4 and 4.9 with
suitable arguments, the local loopback device in the netns is not getting
destroyed after deleting the netns.


...


Based on a quick test, 4.14 and 4.15 does not suffer from this, but its
reproducible e.g. in 4.4.113 and 4.9.75

Any ideas?


By the versions you mentioned and report, maybe f186ce61bb82 ("Fix an
intermittent pr_emerg warning about lo becoming free.") fixed it.


Thanks, found that commit too, but it does not help. It has been 
included 4.4.76 already.


Also the warning is not intermittent, but permanent...

It makes it impossible to add another netns, in 4.4 "ip netns add" gets 
stuck in uninterruptible sleep:


$ grep State /proc/$(pidof ip)/status
State:  D (disk sleep)

$ cat /proc/$(pidof ip)/stack
[] copy_net_ns+0x72/0x100
[] create_new_namespaces+0xf9/0x1b0
[] unshare_nsproxy_namespaces+0x61/0xa0
[] SyS_unshare+0x1ab/0x2e0
[] entry_SYSCALL_64_fastpath+0x12/0x6d
[] 0x


Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-01-30 Thread Marcelo Ricardo Leitner 
On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
> Hi,
> 
> When running sctp_test from lksctp-tools in netns in 4.4 and 4.9 with
> suitable arguments, the local loopback device in the netns is not getting
> destroyed after deleting the netns.
> 
...
> 
> Based on a quick test, 4.14 and 4.15 does not suffer from this, but its
> reproducible e.g. in 4.4.113 and 4.9.75
> 
> Any ideas?

By the versions you mentioned and report, maybe f186ce61bb82 ("Fix an
intermittent pr_emerg warning about lo becoming free.") fixed it.

  Marcelo

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-01-30 Thread Neil Horman 
On Tue, Jan 30, 2018 at 09:24:17PM +0200, Tommi Rantala wrote:
> On 30.01.2018 17:59, Neil Horman wrote:
> > On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
> > > 
> > > ip netns add TEST
> > > ip netns exec TEST ip link set lo up
> > > ip link add dummy0 type dummy
> > > ip link add dummy1 type dummy
> > > ip link add dummy2 type dummy
> > > ip link set dev dummy0 netns TEST
> > > ip link set dev dummy1 netns TEST
> > > ip link set dev dummy2 netns TEST
> > > ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
> > > ip netns exec TEST ip link set dummy0 up
> > > ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
> > > ip netns exec TEST ip link set dummy1 up
> > > ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
> > > ip netns exec TEST ip link set dummy2 up
> > > ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 
> > > 2
> > > -s -B 192.168.1.3
> > > ip netns del TEST
> > > 
> > Does the problem occur if you don't set lo up?
> 
> Still happens after dropping "ip netns exec TEST ip link set lo up".
> 
> Omitting "-B 192.168.1.3" from the sctp_test args helps.
> 
Thatswierd.  I'll look at the sctp_test code in the AM
Neil

> Tommi
>

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-01-30 Thread Tommi Rantala 

On 30.01.2018 17:59, Neil Horman wrote:

On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:


ip netns add TEST
ip netns exec TEST ip link set lo up
ip link add dummy0 type dummy
ip link add dummy1 type dummy
ip link add dummy2 type dummy
ip link set dev dummy0 netns TEST
ip link set dev dummy1 netns TEST
ip link set dev dummy2 netns TEST
ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
ip netns exec TEST ip link set dummy0 up
ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
ip netns exec TEST ip link set dummy1 up
ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
ip netns exec TEST ip link set dummy2 up
ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 2
-s -B 192.168.1.3
ip netns del TEST


Does the problem occur if you don't set lo up?


Still happens after dropping "ip netns exec TEST ip link set lo up".

Omitting "-B 192.168.1.3" from the sctp_test args helps.

Tommi

Re: sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-01-30 Thread Neil Horman 
On Mon, Jan 29, 2018 at 05:55:45PM +0200, Tommi Rantala wrote:
> Hi,
> 
> When running sctp_test from lksctp-tools in netns in 4.4 and 4.9 with
> suitable arguments, the local loopback device in the netns is not getting
> destroyed after deleting the netns.
> 
> For example:
> 
> ip netns add TEST
> ip netns exec TEST ip link set lo up
> ip link add dummy0 type dummy
> ip link add dummy1 type dummy
> ip link add dummy2 type dummy
> ip link set dev dummy0 netns TEST
> ip link set dev dummy1 netns TEST
> ip link set dev dummy2 netns TEST
> ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
> ip netns exec TEST ip link set dummy0 up
> ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
> ip netns exec TEST ip link set dummy1 up
> ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
> ip netns exec TEST ip link set dummy2 up
> ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 2
> -s -B 192.168.1.3
> ip netns del TEST
> 
> Results to:
> 
> [  354.179591] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> [  364.419674] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> [  374.663664] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> [  384.903717] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> [  395.143724] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> [  405.383645] unregister_netdevice: waiting for lo to become free. Usage
> count = 1
> ...
> 
> Based on a quick test, 4.14 and 4.15 does not suffer from this, but its
> reproducible e.g. in 4.4.113 and 4.9.75
> 
> Any ideas?
> 
> Tommi
> 
Does the problem occur if you don't set lo up?

Neil

sctp netns "unregister_netdevice: waiting for lo to become free. Usage count = 1"

2018-01-29 Thread Tommi Rantala 

Hi,

When running sctp_test from lksctp-tools in netns in 4.4 and 4.9 with 
suitable arguments, the local loopback device in the netns is not 
getting destroyed after deleting the netns.


For example:

ip netns add TEST
ip netns exec TEST ip link set lo up
ip link add dummy0 type dummy
ip link add dummy1 type dummy
ip link add dummy2 type dummy
ip link set dev dummy0 netns TEST
ip link set dev dummy1 netns TEST
ip link set dev dummy2 netns TEST
ip netns exec TEST ip addr add 192.168.1.1/24 dev dummy0
ip netns exec TEST ip link set dummy0 up
ip netns exec TEST ip addr add 192.168.1.2/24 dev dummy1
ip netns exec TEST ip link set dummy1 up
ip netns exec TEST ip addr add 192.168.1.3/24 dev dummy2
ip netns exec TEST ip link set dummy2 up
ip netns exec TEST sctp_test -H 192.168.1.2 -P 20002 -h 192.168.1.1 -p 
2 -s -B 192.168.1.3

ip netns del TEST

Results to:

[  354.179591] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
[  364.419674] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
[  374.663664] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
[  384.903717] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
[  395.143724] unregister_netdevice: waiting for lo to become free. 
Usage count = 1
[  405.383645] unregister_netdevice: waiting for lo to become free. 
Usage count = 1

...

Based on a quick test, 4.14 and 4.15 does not suffer from this, but its 
reproducible e.g. in 4.4.113 and 4.9.75


Any ideas?

Tommi

unregister_netdevice: waiting for lo to become free. Usage count = 1

2017-11-26 Thread Cengiz Can 
Hello!

In case anyone wondering (like I did) if this is still an issue, it's not.

It was fixed in 4.12.

Re: unregister_netdevice: waiting for lo to become free. Usage count = 1

2017-06-23 Thread Andrei Vagin 
On Fri, Jun 23, 2017 at 02:49:58PM -0700, Andrei Vagin wrote:
> Hello Everyone,
> 
> Today I've met a problem, when any attempts to create a new network
> namespace hang up.
> I see that one of previous namespaces can't be destroyed, because a
> usage count for one
> of its devices isn't zero. To reproduce the problem, you need to
> executed an attached program
> in a separate network namespace:
> 
> [root@fc24 net]# cat run.sh
> ip link set up dev lo
> ./tcp-bug
> [root@fc24 net]# unshare -n sh run.sh

If this program is executed in a new user, pid, mnt, and net namespaces,
the kernel reports a bug:

[root@fc24 net]# unshare -Urmfpn sh run.sh
[root@fc24 net]# unshare -Urmfpn sh run.sh
^C

[   30.592295] unregister_netdevice: waiting for lo to become free. Usage count 
= 1
[   37.958180] 
=
[   37.963675] BUG kmalloc-1024 (Not tainted): Poison overwritten
[   37.966160] 
-

[   37.969497] Disabling lock debugging due to kernel taint
[   37.971181] INFO: 0x93eef8bc3760-0x93eef8bc3760. First byte 0x38 
instead of 0x6b
[   37.973066] INFO: Slab 0xe69fc1e2f000 objects=15 used=15 fp=0x  
(null) flags=0x1fffc08100
[   37.974902] INFO: Object 0x93eef8bc35a8 @offset=13736 
fp=0x93eef8bc1088

[   37.976618] Redzone 93eef8bc35a0: bb bb bb bb bb bb bb bb
  
[   37.978355] Object 93eef8bc35a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.980239] Object 93eef8bc35b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.982209] Object 93eef8bc35c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.984072] Object 93eef8bc35d8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.985729] Object 93eef8bc35e8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.987273] Object 93eef8bc35f8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.988930] Object 93eef8bc3608: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.990582] Object 93eef8bc3618: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.991921] Object 93eef8bc3628: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.993541] Object 93eef8bc3638: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.994922] Object 93eef8bc3648: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.996745] Object 93eef8bc3658: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.998135] Object 93eef8bc3668: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   37.999350] Object 93eef8bc3678: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.000656] Object 93eef8bc3688: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.001920] Object 93eef8bc3698: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.003142] Object 93eef8bc36a8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.004549] Object 93eef8bc36b8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.005925] Object 93eef8bc36c8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.007296] Object 93eef8bc36d8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.008852] Object 93eef8bc36e8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.010375] Object 93eef8bc36f8: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.011686] Object 93eef8bc3708: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.012967] Object 93eef8bc3718: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.014244] Object 93eef8bc3728: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.015513] Object 93eef8bc3738: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.016796] Object 93eef8bc3748: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.018101] Object 93eef8bc3758: 6b 6b 6b 6b 6b 6b 6b 6b 38 6b 6b 6b 6b 
6b 6b 6b  8kkk
[   38.019151] Object 93eef8bc3768: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.020349] Object 93eef8bc3778: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.021622] Object 93eef8bc3788: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.022931] Object 93eef8bc3798: 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 6b 
6b 6b 6b  
[   38.024223] Object 93eef8bc37a8: 6b

unregister_netdevice: waiting for lo to become free. Usage count = 1

2017-06-23 Thread Andrei Vagin 
Hello Everyone,

Today I've met a problem, when any attempts to create a new network
namespace hang up.
I see that one of previous namespaces can't be destroyed, because a
usage count for one
of its devices isn't zero. To reproduce the problem, you need to
executed an attached program
in a separate network namespace:

[root@fc24 net]# cat run.sh
ip link set up dev lo
./tcp-bug
[root@fc24 net]# unshare -n sh run.sh
[root@fc24 net]# echo $?
0
[root@fc24 ~]# cat /proc/40/stack
[] msleep+0x3e/0x50
[] netdev_run_todo+0x12a/0x320
[] rtnl_unlock+0xe/0x10
[] default_device_exit_batch+0x14a/0x170
[] ops_exit_list.isra.6+0x52/0x60
[] cleanup_net+0x1ee/0x2f0
[] process_one_work+0x205/0x620
[] worker_thread+0x4e/0x3b0
[] kthread+0x114/0x150
[] ret_from_fork+0x2a/0x40
[] 0x
[root@fc24 ~]# dmesg | tail
[   97.071533] unregister_netdevice: waiting for lo to become free.
Usage count = 1
[   97.079561] systemd-journald[180]: Sent WATCHDOG=1 notification.
[  107.319260] unregister_netdevice: waiting for lo to become free.
Usage count = 1
[  117.567180] unregister_netdevice: waiting for lo to become free.
Usage count = 1
[  127.807401] unregister_netdevice: waiting for lo to become free.
Usage count = 1
[  138.055324] unregister_netdevice: waiting for lo to become free.
Usage count = 1
[  148.303308] unregister_netdevice: waiting for lo to become free.
Usage count = 1
[  158.559118] unregister_netdevice: waiting for lo to become free.
Usage count = 1
[  168.807423] unregister_netdevice: waiting for lo to become free.
Usage count = 1
[  179.055590] unregister_netdevice: waiting for lo to become free.
Usage count = 1

This program creates a server tcp socket, then it creates a pair of
connected tcp sockets
and then it does actions which trigger this problem. It calls
connect() with AF_UNSPEC
for one of connected sockets and then call connect() with the address
of the server socket.


Thanks,
Andrei
#include 
#include 
#include   /* for sockaddr_in and inet_ntoa() */

#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 


#define pr_err(fmt, ...)   \
printf("Error: " fmt, ##__VA_ARGS__)

#define pr_perror(fmt, ...) \
pr_err(fmt ": %s\n", ##__VA_ARGS__, strerror(errno))

#define fail(fmt, ...) \
pr_err(fmt ": %s\n", ##__VA_ARGS__, strerror(errno))

union sockaddr_inet {
	struct sockaddr addr;
	struct sockaddr_in v4;
	struct sockaddr_in6 v6;
};

int tcp_init_server(int family, int *port)
{
	union sockaddr_inet addr;
	int sock;
	int yes = 1, ret;

	memset(,0,sizeof(addr));
	if (family == AF_INET) {
		addr.v4.sin_family = family;
		inet_pton(family, "0.0.0.0", &(addr.v4.sin_addr));
	} else if (family == AF_INET6){
		addr.v6.sin6_family = family;
		inet_pton(family, "::0", &(addr.v6.sin6_addr));
	} else
		return -1;

	sock = socket(family, SOCK_STREAM, IPPROTO_TCP);
	if (sock == -1) {
		pr_perror("socket() failed");
		return -1;
	}

	if (setsockopt(sock, SOL_SOCKET, SO_REUSEADDR, , sizeof(int)) == -1 ) {
		pr_perror("setsockopt() error");
		return -1;
	}

	while (1) {
		if (family == AF_INET)
			addr.v4.sin_port = htons(*port);
		else if (family == AF_INET6)
			addr.v6.sin6_port = htons(*port);

		ret = bind(sock, (struct sockaddr *) , sizeof(addr));

		/* criu doesn't restore sock opts, so we need this hack */
		if (ret == -1 && errno == EADDRINUSE) {
			(*port)++;
			continue;
		}
		break;
	}

	if (ret == -1) {
		pr_perror("bind() failed");
		return -1;
	}

	if (listen(sock, 1) == -1) {
		pr_perror("listen() failed");
		return -1;
	}
	return sock;
}

int tcp_accept_server(int sock)
{
	struct sockaddr_in maddr;
	int sock2;
	socklen_t addrlen;
#ifdef DEBUG
	test_msg ("Waiting for connection..\n");
#endif
	addrlen = sizeof(maddr);
	sock2 = accept(sock,(struct sockaddr *) , );

	if (sock2 == -1) {
		pr_perror("accept() failed");
		return -1;
	}

#ifdef DEBUG
	test_msg ("Connection!!\n");
#endif
	return sock2;
}

int __tcp_init_client(int sock, int family, char *servIP, unsigned short servPort)
{
	union sockaddr_inet servAddr;

	/* Construct the server address structure */
	memset(, 0, sizeof(servAddr));
	if (family == AF_INET) {
		servAddr.v4.sin_family  = AF_INET;
		servAddr.v4.sin_port= htons(servPort);
		inet_pton(AF_INET, servIP, _addr);
	} else {
		servAddr.v6.sin6_family  = AF_INET6;
		servAddr.v6.sin6_port= htons(servPort);
		inet_pton(AF_INET6, servIP, _addr);
	}
	if (connect(sock, (struct sockaddr *) , sizeof(servAddr)) < 0) {
		pr_perror("can't connect to server");
		return -1;
	}
	return sock;
}

int tcp_init_client(int family, char *servIP, unsigned short servPort)
{
	int sock;

	if ((sock = socket(family, SOCK_STREAM, IPPROTO_TCP)) < 0)

Re: linux-next: unregister_netdevice: waiting for lo to become free. Usage count = 1

2015-08-19 Thread Andrey Wagin 
2015-08-18 18:27 GMT+03:00 David Ahern d...@cumulusnetworks.com:
 On 8/18/15 9:24 AM, Andrey Wagin wrote:

 Hello David,

 CRIU tests detetect that references on net devices leak on
 4.2.0-rc6-next-20150817. Looks like it started with
 v4.2-rc6-882-g3bfd847.


 1e3136789975f03e461798149309034e5213c1b4 should have fixed it.

Yes, it works now. Thanks!


 David
--
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

linux-next: unregister_netdevice: waiting for lo to become free. Usage count = 1

2015-08-18 Thread Andrey Wagin 
Hello David,

CRIU tests detetect that references on net devices leak on
4.2.0-rc6-next-20150817. Looks like it started with
v4.2-rc6-882-g3bfd847.

David, could you check your changes?

The problem can be reproduced by the following script:
ip netns add test
ip netns exec test ip link set lo up
ip netns exec test ip addr add 1.2.3.4 dev lo
ip netns exec test ip route add 1.2.3.5 dev lo
ip netns exec test ip route add 1.2.3.6 via 1.2.3.5
ip netns del test

You need to execute it twice.

Thanks,
Andrey
--
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: linux-next: unregister_netdevice: waiting for lo to become free. Usage count = 1

2015-08-18 Thread David Ahern 

On 8/18/15 9:24 AM, Andrey Wagin wrote:

Hello David,

CRIU tests detetect that references on net devices leak on
4.2.0-rc6-next-20150817. Looks like it started with
v4.2-rc6-882-g3bfd847.


1e3136789975f03e461798149309034e5213c1b4 should have fixed it.

David
--
To unsubscribe from this list: send the line unsubscribe netdev in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html