Re: [PATCH] net: take care of bonding in build_skb_flow_key (v3)

2016-01-20 Thread Wengang Wang



在 2016年01月21日 12:05, Jay Vosburgh 写道:

Wengang Wang  wrote:
[...]

For ipip,  yes seems update_pmtu is called in line for each call of
queue_xmit.  Do you know if it's a good configuration for ipip + bonding?

Yes, it is.


Other's comment and suggestion?

I agree with Sabrina Dubroca 's suggestions
from yesterday.


Thank you! I will follow.

thanks,
wengang

-J

---
-Jay Vosburgh, jay.vosbu...@canonical.com




Re: [PATCH] net: take care of bonding in build_skb_flow_key (v3)

2016-01-20 Thread Wengang Wang



在 2016年01月20日 23:18, Sabrina Dubroca 写道:

2016-01-20, 13:32:13 +0800, Wengang Wang wrote:

In a bonding setting, we determines fragment size according to MTU and
PMTU associated to the bonding master. If the slave finds the fragment
size is too big, it drops the fragment and calls ip_rt_update_pmtu(),
passing _skb_ and _pmtu_, trying to update the path MTU.
Problem is that the target device that function ip_rt_update_pmtu actually
tries to update is the slave (skb->dev), not the master. Thus since no
PMTU change happens on master, the fragment size for later packets doesn't
change so all later fragments/packets are dropped too.

The fix is letting build_skb_flow_key() take care of the transition of
device index from bonding slave to the master. That makes the master become
the target device that ip_rt_update_pmtu tries to update PMTU to.

Signed-off-by: Wengang Wang 
---
  net/ipv4/route.c | 13 -
  1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 85f184e..c59fb0d 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -523,10 +523,21 @@ static void build_skb_flow_key(struct flowi4 *fl4, const 
struct sk_buff *skb,
   const struct sock *sk)
  {
const struct iphdr *iph = ip_hdr(skb);
-   int oif = skb->dev->ifindex;
+   struct net_device *master = NULL;
u8 tos = RT_TOS(iph->tos);
u8 prot = iph->protocol;
u32 mark = skb->mark;
+   int oif;
+
+   if (skb->dev->flags & IFF_SLAVE) {

Maybe use netif_is_bond_slave here instead, since you have this
problem with bonding slaves?



+   rtnl_lock();
+   master = netdev_master_upper_dev_get(skb->dev);
+   rtnl_unlock();
+   }

As zhuyj said, this is called from dev_queue_xmit, so you cannot take
rtnl_lock here.


+   if (master)
+   oif = master->ifindex;

You cannot dereference master after you release the rtnl lock.

So it would probably be best to use netdev_master_upper_dev_get_rcu,
as zhuyj suggested earlier, and make sure that you only use the result
between rcu_read_lock()/rcu_read_unlock():

 rcu_read_lock();
 master = netdev_master_upper_dev_get_rcu(skb->dev);
 if (master)
 oif = master->ifindex;
 rcu_read_unlock();


OK, thanks for advising.

thanks,
wengang


Thanks,





Re: [PATCH] net: take care of bonding in build_skb_flow_key (v3)

2016-01-20 Thread Wengang Wang



在 2016年01月20日 17:56, zhuyj 写道:

On 01/20/2016 05:47 PM, Wengang Wang wrote:



在 2016年01月20日 15:54, zhuyj 写道:

On 01/20/2016 03:38 PM, Wengang Wang wrote:



在 2016年01月20日 14:24, zhuyj 写道:

On 01/20/2016 01:32 PM, Wengang Wang wrote:
In a bonding setting, we determines fragment size according to 
MTU and
PMTU associated to the bonding master. If the slave finds the 
fragment
size is too big, it drops the fragment and calls 
ip_rt_update_pmtu(),

passing _skb_ and _pmtu_, trying to update the path MTU.
Problem is that the target device that function ip_rt_update_pmtu 
actually
tries to update is the slave (skb->dev), not the master. Thus 
since no
PMTU change happens on master, the fragment size for later 
packets doesn't

change so all later fragments/packets are dropped too.

The fix is letting build_skb_flow_key() take care of the 
transition of
device index from bonding slave to the master. That makes the 
master become

the target device that ip_rt_update_pmtu tries to update PMTU to.

Signed-off-by: Wengang Wang 
---
  net/ipv4/route.c | 13 -
  1 file changed, 12 insertions(+), 1 deletion(-)

diff --git a/net/ipv4/route.c b/net/ipv4/route.c
index 85f184e..c59fb0d 100644
--- a/net/ipv4/route.c
+++ b/net/ipv4/route.c
@@ -523,10 +523,21 @@ static void build_skb_flow_key(struct 
flowi4 *fl4, const struct sk_buff *skb,

 const struct sock *sk)
  {
  const struct iphdr *iph = ip_hdr(skb);
-int oif = skb->dev->ifindex;
+struct net_device *master = NULL;
  u8 tos = RT_TOS(iph->tos);
  u8 prot = iph->protocol;
  u32 mark = skb->mark;
+int oif;
+
+if (skb->dev->flags & IFF_SLAVE) {
+rtnl_lock();
+master = netdev_master_upper_dev_get(skb->dev);
+rtnl_unlock();
update_pmtu is called very frequently. Is it appropriate to use 
rtnl_lock here?
By "very frequently", how frequently it is expected? And what 
situation can cause that?

For my case, the update_pmtu is called only once.

ip_tunnel_xmit


Can you please explain with more details?


dev_queue_xmit->ipip_tunnel_xmit->ip_tunnel_xmit->tnl_update_pmtu-> 
skb_dst(skb)->ops->update_pmtu
For ipip,  yes seems update_pmtu is called in line for each call of 
queue_xmit.  Do you know if it's a good configuration for ipip + bonding?

Other's comment and suggestion?

thanks,
wengang


Re: [PATCH] net: take care of bonding in build_skb_flow_key (v3)

2016-01-20 Thread Jay Vosburgh
Wengang Wang  wrote:
[...]
>For ipip,  yes seems update_pmtu is called in line for each call of
>queue_xmit.  Do you know if it's a good configuration for ipip + bonding?

Yes, it is.

>Other's comment and suggestion?

I agree with Sabrina Dubroca 's suggestions
from yesterday.

-J

---
-Jay Vosburgh, jay.vosbu...@canonical.com


Re: [PATCH] net: take care of bonding in build_skb_flow_key (v3)

2016-01-20 Thread Sabrina Dubroca
2016-01-20, 13:32:13 +0800, Wengang Wang wrote:
> In a bonding setting, we determines fragment size according to MTU and
> PMTU associated to the bonding master. If the slave finds the fragment
> size is too big, it drops the fragment and calls ip_rt_update_pmtu(),
> passing _skb_ and _pmtu_, trying to update the path MTU.
> Problem is that the target device that function ip_rt_update_pmtu actually
> tries to update is the slave (skb->dev), not the master. Thus since no
> PMTU change happens on master, the fragment size for later packets doesn't
> change so all later fragments/packets are dropped too.
> 
> The fix is letting build_skb_flow_key() take care of the transition of
> device index from bonding slave to the master. That makes the master become
> the target device that ip_rt_update_pmtu tries to update PMTU to.
> 
> Signed-off-by: Wengang Wang 
> ---
>  net/ipv4/route.c | 13 -
>  1 file changed, 12 insertions(+), 1 deletion(-)
> 
> diff --git a/net/ipv4/route.c b/net/ipv4/route.c
> index 85f184e..c59fb0d 100644
> --- a/net/ipv4/route.c
> +++ b/net/ipv4/route.c
> @@ -523,10 +523,21 @@ static void build_skb_flow_key(struct flowi4 *fl4, 
> const struct sk_buff *skb,
>  const struct sock *sk)
>  {
>   const struct iphdr *iph = ip_hdr(skb);
> - int oif = skb->dev->ifindex;
> + struct net_device *master = NULL;
>   u8 tos = RT_TOS(iph->tos);
>   u8 prot = iph->protocol;
>   u32 mark = skb->mark;
> + int oif;
> +
> + if (skb->dev->flags & IFF_SLAVE) {

Maybe use netif_is_bond_slave here instead, since you have this
problem with bonding slaves?


> + rtnl_lock();
> + master = netdev_master_upper_dev_get(skb->dev);
> + rtnl_unlock();
> + }

As zhuyj said, this is called from dev_queue_xmit, so you cannot take
rtnl_lock here.

> + if (master)
> + oif = master->ifindex;

You cannot dereference master after you release the rtnl lock.

So it would probably be best to use netdev_master_upper_dev_get_rcu,
as zhuyj suggested earlier, and make sure that you only use the result
between rcu_read_lock()/rcu_read_unlock():

rcu_read_lock();
master = netdev_master_upper_dev_get_rcu(skb->dev);
if (master)
oif = master->ifindex;
rcu_read_unlock();


Thanks,

-- 
Sabrina