Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-28 Thread Ian Wells
Path MTU discovery works on a path - something with an L3 router in the way
- where the outbound interface has a smaller MTU than the inbound one.
You're transmitting across an L2 network - no L3 routers present.  You send
a 1500 byte packet, the network fabric (which is not L3, has no address,
and therefore has no means to answer you) does all that it can do with that
packet - it drops it.  The sender retransmits, assuming congestion, but the
same thing happens.  Eventually the sender decides there's a network
problem and times out.

This is a common problem with Openstack deployments, although various
features of the virtual networking let you get away with it, with some
configs and not others.  OVS used to fake a PMTU exceeded message from the
destination if you tried to pass an overlarge packet - not in spec, but it
hid the problem nicely.  I have a suspicion that some implementations will
fragment the containing UDP packet, which is also not in spec and also
solves the problem (albeit with poor performance).

The right answer for you is to set the MTU in your machines to the same MTU
you've given the network, that is, 1450 bytes.  You can do this by setting
a DHCP option for MTU, providing your VMs support that option (search the
web for the solution, I don't have it offhand) or lower the MTU by hand or
by script when you start your VM.

The right answer for everyone is to properly determine and advertise the
network MTU to VMs (which, with provider networks, is not even consistent
from one network to the next) and that's the spec Kyle is referring to.
We'll be fixing this in Kilo.
-- 
Ian.


On 27 October 2014 20:14, Li Tianqing jaze...@163.com wrote:






 --
 Best
 Li Tianqing


 At 2014-10-27 17:42:41, Ihar Hrachyshka ihrac...@redhat.com wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA512
 
 On 27/10/14 02:18, Li Tianqing wrote:
  Hello, Right now, we test neutron under havana release. We
  configured network_device_mtu=1450 in neutron.conf, After create
  vm, we found the vm interface's mtu is 1500, the ping, ssh, is ok.
  But if we scp large file between vms then scp display 'stalled'.
  And iperf is also can not completed. If we configured vm's mtu to
  1450, then iperf, scp all is ok. If we iperf with -M 1300, then the
  iperf is ok too. The vms path mtu discovery is set by default. I do
  not know why the vm whose mtu is 1500 can not send large file.
 
 There is a neutron spec currently in discussion for Kilo to finally
 fix MTU issues due to tunneling, that also tries to propagate MTU
 inside instances: https://review.openstack.org/#/c/105989/

 The problem is i do not know why the vm with 1500 mtu can not send large file?
 I found the packet send out all with DF, and is it because the DF set default 
 by linux cause the packet
 be dropped? And the application do not handle the return back icmp packet 
 with the smaller mtu?

  
 /Ihar
 -BEGIN PGP SIGNATURE-
 Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
 
 iQEcBAEBCgAGBQJUThORAAoJEC5aWaUY1u571u4H/3EqEVPL1Q9KgymrudLpAdRh
 fwNarwPWT8Ed+0x7WIXAr7OFXX1P90cKRAZKTlAEEI94vOrdr0s608ZX8awMuLeu
 +LB6IA7nMpgJammfDb8zNmYLHuTQGGatXblOinvtm3XXIcNbkNu8840MTV3y/Jdq
 Mndtz69TrjTrjn7r9REJ4bnRIlL4DGo+gufXPD49+yax1y/woefqwZPU13kO6j6R
 Q0+MAy13ptg2NwX26OI+Sb801W0kpDXby6WZjfekXqxqv62fY1/lPQ3oqqJBd95K
 EFe5NuogLV7UGH5vydQJa0eO2jw5lh8qLuHSShGcDEp/N6oQWiDzXYYYoEQdUic=
 =jRQ/
 -END PGP SIGNATURE-
 
 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev




 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-28 Thread Li Tianqing
The problem is that it is not at the begining to transmit large file. It is 
after some packets trasmited, then the connection is choked. 
After the connection choked, from the bridge in compute host we can see the 
sender send packets, and the receiver can not get the packets.
If it is the pmtud, then at the very begining, the packet can not transmit from 
the begining.


At 2014-10-28 14:10:09, Ian Wells ijw.ubu...@cack.org.uk wrote:

Path MTU discovery works on a path - something with an L3 router in the way - 
where the outbound interface has a smaller MTU than the inbound one.  You're 
transmitting across an L2 network - no L3 routers present.  You send a 1500 
byte packet, the network fabric (which is not L3, has no address, and therefore 
has no means to answer you) does all that it can do with that packet - it drops 
it.  The sender retransmits, assuming congestion, but the same thing happens.  
Eventually the sender decides there's a network problem and times out.

This is a common problem with Openstack deployments, although various features 
of the virtual networking let you get away with it, with some configs and not 
others.  OVS used to fake a PMTU exceeded message from the destination if you 
tried to pass an overlarge packet - not in spec, but it hid the problem nicely. 
 I have a suspicion that some implementations will fragment the containing UDP 
packet, which is also not in spec and also solves the problem (albeit with poor 
performance).

The right answer for you is to set the MTU in your machines to the same MTU 
you've given the network, that is, 1450 bytes.  You can do this by setting a 
DHCP option for MTU, providing your VMs support that option (search the web for 
the solution, I don't have it offhand) or lower the MTU by hand or by script 
when you start your VM.


The right answer for everyone is to properly determine and advertise the 
network MTU to VMs (which, with provider networks, is not even consistent from 
one network to the next) and that's the spec Kyle is referring to.  We'll be 
fixing this in Kilo.
--

Ian.




On 27 October 2014 20:14, Li Tianqing jaze...@163.com wrote:







--

Best
Li Tianqing



At 2014-10-27 17:42:41, Ihar Hrachyshka ihrac...@redhat.com wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 27/10/14 02:18, Li Tianqing wrote:
 Hello, Right now, we test neutron under havana release. We
 configured network_device_mtu=1450 in neutron.conf, After create
 vm, we found the vm interface's mtu is 1500, the ping, ssh, is ok.
 But if we scp large file between vms then scp display 'stalled'.
 And iperf is also can not completed. If we configured vm's mtu to
 1450, then iperf, scp all is ok. If we iperf with -M 1300, then the
 iperf is ok too. The vms path mtu discovery is set by default. I do
 not know why the vm whose mtu is 1500 can not send large file.

There is a neutron spec currently in discussion for Kilo to finally
fix MTU issues due to tunneling, that also tries to propagate MTU

inside instances: https://review.openstack.org/#/c/105989/


The problem is i do not know why the vm with 1500 mtu can not send large file? 
I found the packet send out all with DF, and is it because the DF set default 
by linux cause the packet
be dropped? And the application do not handle the return back icmp packet with 
the smaller mtu?



/Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJUThORAAoJEC5aWaUY1u571u4H/3EqEVPL1Q9KgymrudLpAdRh
fwNarwPWT8Ed+0x7WIXAr7OFXX1P90cKRAZKTlAEEI94vOrdr0s608ZX8awMuLeu
+LB6IA7nMpgJammfDb8zNmYLHuTQGGatXblOinvtm3XXIcNbkNu8840MTV3y/Jdq
Mndtz69TrjTrjn7r9REJ4bnRIlL4DGo+gufXPD49+yax1y/woefqwZPU13kO6j6R
Q0+MAy13ptg2NwX26OI+Sb801W0kpDXby6WZjfekXqxqv62fY1/lPQ3oqqJBd95K
EFe5NuogLV7UGH5vydQJa0eO2jw5lh8qLuHSShGcDEp/N6oQWiDzXYYYoEQdUic=
=jRQ/
-END PGP SIGNATURE-

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-28 Thread Li Tianqing
lan, you are right, the receiver only receive packet that small than 1450. 
Because the sender does not send large packets at the begining, so
tcpdump can catch some small packets. 


Another question about the mtu, what if we clear the  DF in the ip packets?  
Then l2 can split packets into smaller mtu size?

At 2014-10-28 15:15:51, Li Tianqing jaze...@163.com wrote:

The problem is that it is not at the begining to transmit large file. It is 
after some packets trasmited, then the connection is choked. 
After the connection choked, from the bridge in compute host we can see the 
sender send packets, and the receiver can not get the packets.
If it is the pmtud, then at the very begining, the packet can not transmit from 
the begining.


At 2014-10-28 14:10:09, Ian Wells ijw.ubu...@cack.org.uk wrote:

Path MTU discovery works on a path - something with an L3 router in the way - 
where the outbound interface has a smaller MTU than the inbound one.  You're 
transmitting across an L2 network - no L3 routers present.  You send a 1500 
byte packet, the network fabric (which is not L3, has no address, and therefore 
has no means to answer you) does all that it can do with that packet - it drops 
it.  The sender retransmits, assuming congestion, but the same thing happens.  
Eventually the sender decides there's a network problem and times out.

This is a common problem with Openstack deployments, although various features 
of the virtual networking let you get away with it, with some configs and not 
others.  OVS used to fake a PMTU exceeded message from the destination if you 
tried to pass an overlarge packet - not in spec, but it hid the problem nicely. 
 I have a suspicion that some implementations will fragment the containing UDP 
packet, which is also not in spec and also solves the problem (albeit with poor 
performance).

The right answer for you is to set the MTU in your machines to the same MTU 
you've given the network, that is, 1450 bytes.  You can do this by setting a 
DHCP option for MTU, providing your VMs support that option (search the web for 
the solution, I don't have it offhand) or lower the MTU by hand or by script 
when you start your VM.


The right answer for everyone is to properly determine and advertise the 
network MTU to VMs (which, with provider networks, is not even consistent from 
one network to the next) and that's the spec Kyle is referring to.  We'll be 
fixing this in Kilo.
--

Ian.




On 27 October 2014 20:14, Li Tianqing jaze...@163.com wrote:







--

Best
Li Tianqing



At 2014-10-27 17:42:41, Ihar Hrachyshka ihrac...@redhat.com wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 27/10/14 02:18, Li Tianqing wrote:
 Hello, Right now, we test neutron under havana release. We
 configured network_device_mtu=1450 in neutron.conf, After create
 vm, we found the vm interface's mtu is 1500, the ping, ssh, is ok.
 But if we scp large file between vms then scp display 'stalled'.
 And iperf is also can not completed. If we configured vm's mtu to
 1450, then iperf, scp all is ok. If we iperf with -M 1300, then the
 iperf is ok too. The vms path mtu discovery is set by default. I do
 not know why the vm whose mtu is 1500 can not send large file.

There is a neutron spec currently in discussion for Kilo to finally
fix MTU issues due to tunneling, that also tries to propagate MTU

inside instances: https://review.openstack.org/#/c/105989/


The problem is i do not know why the vm with 1500 mtu can not send large file? 
I found the packet send out all with DF, and is it because the DF set default 
by linux cause the packet
be dropped? And the application do not handle the return back icmp packet with 
the smaller mtu?



/Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJUThORAAoJEC5aWaUY1u571u4H/3EqEVPL1Q9KgymrudLpAdRh
fwNarwPWT8Ed+0x7WIXAr7OFXX1P90cKRAZKTlAEEI94vOrdr0s608ZX8awMuLeu
+LB6IA7nMpgJammfDb8zNmYLHuTQGGatXblOinvtm3XXIcNbkNu8840MTV3y/Jdq
Mndtz69TrjTrjn7r9REJ4bnRIlL4DGo+gufXPD49+yax1y/woefqwZPU13kO6j6R
Q0+MAy13ptg2NwX26OI+Sb801W0kpDXby6WZjfekXqxqv62fY1/lPQ3oqqJBd95K
EFe5NuogLV7UGH5vydQJa0eO2jw5lh8qLuHSShGcDEp/N6oQWiDzXYYYoEQdUic=
=jRQ/
-END PGP SIGNATURE-

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev






___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-28 Thread Ian Wells
On 28 October 2014 00:18, A, Keshava keshav...@hp.com wrote:

  Hi,



 Currently OpenStack have any framework to notify the Tennant/Service-VM
 for such kind of notification based on VM’s interest ?


It's possible to use DHCP or RA to notify a VM of the MTU but there are
limitations (RAs don't let you increase the MTU, only decrease it, and
obviously VMs must support the MTU element of DHCP) and Openstack doesn't
currently use it.  You can statically configure the DHCP MTU number that
DHCP transmits; this is useful to work around problems but not really the
right answer to the problem.


  VM may be very much interested for such kind of notification like

 1.   Path MTU.

This will be correctly discovered from the ICMP PMTU exceeded message, and
Neutron routers should certainly be expected to send that.  (In fact the
namespace implementation of routers would do this if the router ever had
different MTUs on its ports; it's in the kernel network stack.)  There's no
requirement for a special notification, and indeed you couldn't do it that
way anyway.

  2.   Based on specific incoming Tennant traffic, block/Allow
  particular traffic flow at infrastructure level itself, instead of at VM.

I don't see the relevance; and you appear to be describing security groups.

  This may require OpenStack infrastructure notification support to
 Tenant/Service VM.

Not particularly, as MTU doesn't generally change, and I think we would
forbid changing the MTU of a network after creation.  It's only an initial
configuration thing, therefore.  It might involve better cloud-init support
for network configuration, something that gets discussed periodically.

-- 
Ian.



 …

 Thanks  regards,

 Keshava



 *From:* Ian Wells [mailto:ijw.ubu...@cack.org.uk]
 *Sent:* Tuesday, October 28, 2014 11:40 AM
 *To:* OpenStack Development Mailing List (not for usage questions)
 *Subject:* Re: [openstack-dev] [neutron] vm can not transport large file
 under neutron ml2 + linux bridge + vxlan



 Path MTU discovery works on a path - something with an L3 router in the
 way - where the outbound interface has a smaller MTU than the inbound one.
 You're transmitting across an L2 network - no L3 routers present.  You send
 a 1500 byte packet, the network fabric (which is not L3, has no address,
 and therefore has no means to answer you) does all that it can do with that
 packet - it drops it.  The sender retransmits, assuming congestion, but the
 same thing happens.  Eventually the sender decides there's a network
 problem and times out.

 This is a common problem with Openstack deployments, although various
 features of the virtual networking let you get away with it, with some
 configs and not others.  OVS used to fake a PMTU exceeded message from the
 destination if you tried to pass an overlarge packet - not in spec, but it
 hid the problem nicely.  I have a suspicion that some implementations will
 fragment the containing UDP packet, which is also not in spec and also
 solves the problem (albeit with poor performance).

 The right answer for you is to set the MTU in your machines to the same
 MTU you've given the network, that is, 1450 bytes.  You can do this by
 setting a DHCP option for MTU, providing your VMs support that option
 (search the web for the solution, I don't have it offhand) or lower the MTU
 by hand or by script when you start your VM.

 The right answer for everyone is to properly determine and advertise the
 network MTU to VMs (which, with provider networks, is not even consistent
 from one network to the next) and that's the spec Kyle is referring to.
 We'll be fixing this in Kilo.
 --

 Ian.



 On 27 October 2014 20:14, Li Tianqing jaze...@163.com wrote:





  --

 Best

 Li Tianqing




 At 2014-10-27 17:42:41, Ihar Hrachyshka ihrac...@redhat.com wrote:

 -BEGIN PGP SIGNED MESSAGE-

 Hash: SHA512

 

 On 27/10/14 02:18, Li Tianqing wrote:

  Hello, Right now, we test neutron under havana release. We

  configured network_device_mtu=1450 in neutron.conf, After create

  vm, we found the vm interface's mtu is 1500, the ping, ssh, is ok.

  But if we scp large file between vms then scp display 'stalled'.

  And iperf is also can not completed. If we configured vm's mtu to

  1450, then iperf, scp all is ok. If we iperf with -M 1300, then the

  iperf is ok too. The vms path mtu discovery is set by default. I do

  not know why the vm whose mtu is 1500 can not send large file.

 

 There is a neutron spec currently in discussion for Kilo to finally

 fix MTU issues due to tunneling, that also tries to propagate MTU

  inside instances: https://review.openstack.org/#/c/105989/



  The problem is i do not know why the vm with 1500 mtu can not send large 
 file?

  I found the packet send out all with DF, and is it because the DF set 
 default by linux cause the packet

  be dropped? And the application do not handle the return back icmp packet 
 with the smaller mtu?





 /Ihar

Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-28 Thread Li Tianqing
One more question.
Vxlan use udp, how can vxlan promise reliability.


在 2014-10-28 15:51:02,Ian Wells ijw.ubu...@cack.org.uk 写道:

On 28 October 2014 00:30, Li Tianqing jaze...@163.com wrote:

lan, you are right, the receiver only receive packet that small than 1450. 
Because the sender does not send large packets at the begining, so
tcpdump can catch some small packets. 


Another question about the mtu, what if we clear the  DF in the ip packets?  
Then l2 can split packets into smaller mtu size?


Routers can split packets.  L2 networks don't understand IP headers and 
therefore can't fragment packets.  DF doesn't change that.


(DF is there to make PMTU discovery work, incidentially; it's what prompts 
routers to return PMTU exceeded messages.)
--

Ian.



At 2014-10-28 15:15:51, Li Tianqing jaze...@163.com wrote:

The problem is that it is not at the begining to transmit large file. It is 
after some packets trasmited, then the connection is choked. 
After the connection choked, from the bridge in compute host we can see the 
sender send packets, and the receiver can not get the packets.
If it is the pmtud, then at the very begining, the packet can not transmit from 
the begining.


At 2014-10-28 14:10:09, Ian Wells ijw.ubu...@cack.org.uk wrote:

Path MTU discovery works on a path - something with an L3 router in the way - 
where the outbound interface has a smaller MTU than the inbound one.  You're 
transmitting across an L2 network - no L3 routers present.  You send a 1500 
byte packet, the network fabric (which is not L3, has no address, and therefore 
has no means to answer you) does all that it can do with that packet - it drops 
it.  The sender retransmits, assuming congestion, but the same thing happens.  
Eventually the sender decides there's a network problem and times out.

This is a common problem with Openstack deployments, although various features 
of the virtual networking let you get away with it, with some configs and not 
others.  OVS used to fake a PMTU exceeded message from the destination if you 
tried to pass an overlarge packet - not in spec, but it hid the problem nicely. 
 I have a suspicion that some implementations will fragment the containing UDP 
packet, which is also not in spec and also solves the problem (albeit with poor 
performance).

The right answer for you is to set the MTU in your machines to the same MTU 
you've given the network, that is, 1450 bytes.  You can do this by setting a 
DHCP option for MTU, providing your VMs support that option (search the web for 
the solution, I don't have it offhand) or lower the MTU by hand or by script 
when you start your VM.


The right answer for everyone is to properly determine and advertise the 
network MTU to VMs (which, with provider networks, is not even consistent from 
one network to the next) and that's the spec Kyle is referring to.  We'll be 
fixing this in Kilo.
--

Ian.




On 27 October 2014 20:14, Li Tianqing jaze...@163.com wrote:







--

Best
Li Tianqing



At 2014-10-27 17:42:41, Ihar Hrachyshka ihrac...@redhat.com wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 27/10/14 02:18, Li Tianqing wrote:
 Hello, Right now, we test neutron under havana release. We
 configured network_device_mtu=1450 in neutron.conf, After create
 vm, we found the vm interface's mtu is 1500, the ping, ssh, is ok.
 But if we scp large file between vms then scp display 'stalled'.
 And iperf is also can not completed. If we configured vm's mtu to
 1450, then iperf, scp all is ok. If we iperf with -M 1300, then the
 iperf is ok too. The vms path mtu discovery is set by default. I do
 not know why the vm whose mtu is 1500 can not send large file.

There is a neutron spec currently in discussion for Kilo to finally
fix MTU issues due to tunneling, that also tries to propagate MTU

inside instances: https://review.openstack.org/#/c/105989/


The problem is i do not know why the vm with 1500 mtu can not send large file? 
I found the packet send out all with DF, and is it because the DF set default 
by linux cause the packet
be dropped? And the application do not handle the return back icmp packet with 
the smaller mtu?



/Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJUThORAAoJEC5aWaUY1u571u4H/3EqEVPL1Q9KgymrudLpAdRh
fwNarwPWT8Ed+0x7WIXAr7OFXX1P90cKRAZKTlAEEI94vOrdr0s608ZX8awMuLeu
+LB6IA7nMpgJammfDb8zNmYLHuTQGGatXblOinvtm3XXIcNbkNu8840MTV3y/Jdq
Mndtz69TrjTrjn7r9REJ4bnRIlL4DGo+gufXPD49+yax1y/woefqwZPU13kO6j6R
Q0+MAy13ptg2NwX26OI+Sb801W0kpDXby6WZjfekXqxqv62fY1/lPQ3oqqJBd95K
EFe5NuogLV7UGH5vydQJa0eO2jw5lh8qLuHSShGcDEp/N6oQWiDzXYYYoEQdUic=
=jRQ/
-END PGP SIGNATURE-

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org

Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-28 Thread A, Keshava
Hi,
Pl find my reply .

Regards,
keshava

From: Ian Wells [mailto:ijw.ubu...@cack.org.uk]
Sent: Tuesday, October 28, 2014 1:19 PM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] vm can not transport large file under 
neutron ml2 + linux bridge + vxlan

On 28 October 2014 00:18, A, Keshava 
keshav...@hp.commailto:keshav...@hp.com wrote:
Hi,

Currently OpenStack have any framework to notify the Tennant/Service-VM for 
such kind of notification based on VM’s interest ?

It's possible to use DHCP or RA to notify a VM of the MTU but there are 
limitations (RAs don't let you increase the MTU, only decrease it, and 
obviously VMs must support the MTU element of DHCP) and Openstack doesn't 
currently use it.  You can statically configure the DHCP MTU number that DHCP 
transmits; this is useful to work around problems but not really the right 
answer to the problem.

VM may be very much interested for such kind of notification like

1.   Path MTU.
This will be correctly discovered from the ICMP PMTU exceeded message, and 
Neutron routers should certainly be expected to send that.  (In fact the 
namespace implementation of routers would do this if the router ever had 
different MTUs on its ports; it's in the kernel network stack.)  There's no 
requirement for a special notification, and indeed you couldn't do it that way 
anyway.

In the network interface/router going down is  common scenario. In that case  
the packet will take different path which may have different MTU.
In that case the PATH MTU calculated at the source may be different and should 
be notified dynamically to VM. So that  VM can originate the packet with 
requirement MTU size .
If there is no notification mechanism ( as per this reply):
If there is no such dynamic PATH MTU change notification to VM,  how VM  can 
change the  packet size  ?
Or
Do we expect ICMP too big message reaches all the way  to VM ?
Or
VM itself the run the PATH MTU ?


2.   Based on specific incoming Tennant traffic, block/Allow  particular 
traffic flow at infrastructure level itself, instead of at VM.
I don't see the relevance; and you appear to be describing security groups.

This may require OpenStack infrastructure notification support to 
Tenant/Service VM.
Not particularly, as MTU doesn't generally change, and I think we would forbid 
changing the MTU of a network after creation.  It's only an initial 
configuration thing, therefore.  It might involve better cloud-init support for 
network configuration, something that gets discussed periodically.

--
Ian.

…
Thanks  regards,
Keshava

From: Ian Wells [mailto:ijw.ubu...@cack.org.ukmailto:ijw.ubu...@cack.org.uk]
Sent: Tuesday, October 28, 2014 11:40 AM
To: OpenStack Development Mailing List (not for usage questions)
Subject: Re: [openstack-dev] [neutron] vm can not transport large file under 
neutron ml2 + linux bridge + vxlan

Path MTU discovery works on a path - something with an L3 router in the way - 
where the outbound interface has a smaller MTU than the inbound one.  You're 
transmitting across an L2 network - no L3 routers present.  You send a 1500 
byte packet, the network fabric (which is not L3, has no address, and therefore 
has no means to answer you) does all that it can do with that packet - it drops 
it.  The sender retransmits, assuming congestion, but the same thing happens.  
Eventually the sender decides there's a network problem and times out.

This is a common problem with Openstack deployments, although various features 
of the virtual networking let you get away with it, with some configs and not 
others.  OVS used to fake a PMTU exceeded message from the destination if you 
tried to pass an overlarge packet - not in spec, but it hid the problem nicely. 
 I have a suspicion that some implementations will fragment the containing UDP 
packet, which is also not in spec and also solves the problem (albeit with poor 
performance).

The right answer for you is to set the MTU in your machines to the same MTU 
you've given the network, that is, 1450 bytes.  You can do this by setting a 
DHCP option for MTU, providing your VMs support that option (search the web for 
the solution, I don't have it offhand) or lower the MTU by hand or by script 
when you start your VM.
The right answer for everyone is to properly determine and advertise the 
network MTU to VMs (which, with provider networks, is not even consistent from 
one network to the next) and that's the spec Kyle is referring to.  We'll be 
fixing this in Kilo.
--
Ian.

On 27 October 2014 20:14, Li Tianqing jaze...@163.commailto:jaze...@163.com 
wrote:



--
Best
Li Tianqing


At 2014-10-27 17:42:41, Ihar Hrachyshka 
ihrac...@redhat.commailto:ihrac...@redhat.com wrote:

-BEGIN PGP SIGNED MESSAGE-

Hash: SHA512



On 27/10/14 02:18, Li Tianqing wrote:

 Hello, Right now, we test neutron under havana release. We

 configured network_device_mtu=1450

Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-28 Thread Cory Benfield
On Tue, Oct 28, 2014 at 08:32:11, Li Tianqing wrote:
 One more question.
 Vxlan use udp, how can vxlan promise reliability.
 

It can't, but that doesn't matter.

VXLAN emulates a single layer 2 broadcast domain: conceptually, a series of 
machines all plugged into the same Ethernet switch. This kind of network 
*isn't* reliable: you can lose Ethernet frames. There's no reason to require 
reliability from VXLAN, and it would increase VXLAN's overhead to add it in. If 
you need reliability, the encapsulated transport will provide it for you 
exactly as it does on a non-encapsulated network.

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-27 Thread Ihar Hrachyshka
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 27/10/14 02:18, Li Tianqing wrote:
 Hello, Right now, we test neutron under havana release. We
 configured network_device_mtu=1450 in neutron.conf, After create
 vm, we found the vm interface's mtu is 1500, the ping, ssh, is ok.
 But if we scp large file between vms then scp display 'stalled'.
 And iperf is also can not completed. If we configured vm's mtu to
 1450, then iperf, scp all is ok. If we iperf with -M 1300, then the
 iperf is ok too. The vms path mtu discovery is set by default. I do
 not know why the vm whose mtu is 1500 can not send large file.

There is a neutron spec currently in discussion for Kilo to finally
fix MTU issues due to tunneling, that also tries to propagate MTU
inside instances: https://review.openstack.org/#/c/105989/

/Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJUThORAAoJEC5aWaUY1u571u4H/3EqEVPL1Q9KgymrudLpAdRh
fwNarwPWT8Ed+0x7WIXAr7OFXX1P90cKRAZKTlAEEI94vOrdr0s608ZX8awMuLeu
+LB6IA7nMpgJammfDb8zNmYLHuTQGGatXblOinvtm3XXIcNbkNu8840MTV3y/Jdq
Mndtz69TrjTrjn7r9REJ4bnRIlL4DGo+gufXPD49+yax1y/woefqwZPU13kO6j6R
Q0+MAy13ptg2NwX26OI+Sb801W0kpDXby6WZjfekXqxqv62fY1/lPQ3oqqJBd95K
EFe5NuogLV7UGH5vydQJa0eO2jw5lh8qLuHSShGcDEp/N6oQWiDzXYYYoEQdUic=
=jRQ/
-END PGP SIGNATURE-

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-27 Thread Kyle Mestery
On Mon, Oct 27, 2014 at 4:42 AM, Ihar Hrachyshka ihrac...@redhat.com wrote:
 -BEGIN PGP SIGNED MESSAGE-
 Hash: SHA512

 On 27/10/14 02:18, Li Tianqing wrote:
 Hello, Right now, we test neutron under havana release. We
 configured network_device_mtu=1450 in neutron.conf, After create
 vm, we found the vm interface's mtu is 1500, the ping, ssh, is ok.
 But if we scp large file between vms then scp display 'stalled'.
 And iperf is also can not completed. If we configured vm's mtu to
 1450, then iperf, scp all is ok. If we iperf with -M 1300, then the
 iperf is ok too. The vms path mtu discovery is set by default. I do
 not know why the vm whose mtu is 1500 can not send large file.

 There is a neutron spec currently in discussion for Kilo to finally
 fix MTU issues due to tunneling, that also tries to propagate MTU
 inside instances: https://review.openstack.org/#/c/105989/

This is something which would be quite fantastic to land in Kilo, as
multiple people have been bit by this particular issue.

 /Ihar
 -BEGIN PGP SIGNATURE-
 Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

 iQEcBAEBCgAGBQJUThORAAoJEC5aWaUY1u571u4H/3EqEVPL1Q9KgymrudLpAdRh
 fwNarwPWT8Ed+0x7WIXAr7OFXX1P90cKRAZKTlAEEI94vOrdr0s608ZX8awMuLeu
 +LB6IA7nMpgJammfDb8zNmYLHuTQGGatXblOinvtm3XXIcNbkNu8840MTV3y/Jdq
 Mndtz69TrjTrjn7r9REJ4bnRIlL4DGo+gufXPD49+yax1y/woefqwZPU13kO6j6R
 Q0+MAy13ptg2NwX26OI+Sb801W0kpDXby6WZjfekXqxqv62fY1/lPQ3oqqJBd95K
 EFe5NuogLV7UGH5vydQJa0eO2jw5lh8qLuHSShGcDEp/N6oQWiDzXYYYoEQdUic=
 =jRQ/
 -END PGP SIGNATURE-

 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-27 Thread Li Tianqing






--

Best
Li Tianqing



At 2014-10-27 17:42:41, Ihar Hrachyshka ihrac...@redhat.com wrote:
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA512

On 27/10/14 02:18, Li Tianqing wrote:
 Hello, Right now, we test neutron under havana release. We
 configured network_device_mtu=1450 in neutron.conf, After create
 vm, we found the vm interface's mtu is 1500, the ping, ssh, is ok.
 But if we scp large file between vms then scp display 'stalled'.
 And iperf is also can not completed. If we configured vm's mtu to
 1450, then iperf, scp all is ok. If we iperf with -M 1300, then the
 iperf is ok too. The vms path mtu discovery is set by default. I do
 not know why the vm whose mtu is 1500 can not send large file.

There is a neutron spec currently in discussion for Kilo to finally
fix MTU issues due to tunneling, that also tries to propagate MTU

inside instances: https://review.openstack.org/#/c/105989/


The problem is i do not know why the vm with 1500 mtu can not send large file? 
I found the packet send out all with DF, and is it because the DF set default 
by linux cause the packet
be dropped? And the application do not handle the return back icmp packet with 
the smaller mtu?



/Ihar
-BEGIN PGP SIGNATURE-
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)

iQEcBAEBCgAGBQJUThORAAoJEC5aWaUY1u571u4H/3EqEVPL1Q9KgymrudLpAdRh
fwNarwPWT8Ed+0x7WIXAr7OFXX1P90cKRAZKTlAEEI94vOrdr0s608ZX8awMuLeu
+LB6IA7nMpgJammfDb8zNmYLHuTQGGatXblOinvtm3XXIcNbkNu8840MTV3y/Jdq
Mndtz69TrjTrjn7r9REJ4bnRIlL4DGo+gufXPD49+yax1y/woefqwZPU13kO6j6R
Q0+MAy13ptg2NwX26OI+Sb801W0kpDXby6WZjfekXqxqv62fY1/lPQ3oqqJBd95K
EFe5NuogLV7UGH5vydQJa0eO2jw5lh8qLuHSShGcDEp/N6oQWiDzXYYYoEQdUic=
=jRQ/
-END PGP SIGNATURE-

___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-26 Thread Damon Wang
Hi Tangqing,

This is a well-known problem, it is because GRE/VxLan 's packet header use
some bytes, length of data packet should reduced.
You can get refer to bellow pages:
http://www.chriscowley.me.uk/blog/2014/03/31/openstack-neutron-performance-problems/
https://ask.openstack.org/en/question/25947/openstack-neutron-stability-problems-with-openvswitch/
https://openstack.redhat.com/forum/discussion/302/quantumneutron-and-ip-packet-mtus-questions-answers-and-suggestions/p1

Hope it helps,
Damon

2014-10-27 9:18 GMT+08:00 Li Tianqing jaze...@163.com:

 Hello,
 Right now, we test neutron under havana release. We configured
 network_device_mtu=1450 in neutron.conf,
After create vm, we found the vm interface's mtu is 1500, the ping,
 ssh, is ok. But if we scp large file between vms
then scp display 'stalled'. And iperf is also can not completed.
If we configured vm's mtu to 1450, then iperf, scp all is ok.
If we iperf with -M 1300, then the iperf is ok too.
The vms path mtu discovery is set by default. I do not know why the vm
 whose mtu is 1500 can not send large file.


 Thanks.




 --
 Best
 Li Tianqing



 ___
 OpenStack-dev mailing list
 OpenStack-dev@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev


Re: [openstack-dev] [neutron] vm can not transport large file under neutron ml2 + linux bridge + vxlan

2014-10-26 Thread Li Tianqing
Thanks, it helps a lot. But i want to know, why the mtu discovery can not find 
the appropriate mtu to send?


In 
http://www.chriscowley.me.uk/blog/2014/03/31/openstack-neutron-performance-problems/,
 it says that
 Everything matches, however I was using GRE tunnels, which add a header to 
each frame. This was pushing them over 1500 on entry to br-tun causing massive 
network fragmentation, which basically destroyed Openvswitch every time I 
performed a large transfer. It showed up when deploying an image, because that 
is hitting the Glance API over http.


Dose it means that the pmtu is not work?





At 2014-10-27 09:51:31, Damon Wang damon.dev...@gmail.com wrote:

Hi Tangqing,


This is a well-known problem, it is because GRE/VxLan 's packet header use some 
bytes, length of data packet should reduced.

You can get refer to bellow pages:
http://www.chriscowley.me.uk/blog/2014/03/31/openstack-neutron-performance-problems/
https://ask.openstack.org/en/question/25947/openstack-neutron-stability-problems-with-openvswitch/
https://openstack.redhat.com/forum/discussion/302/quantumneutron-and-ip-packet-mtus-questions-answers-and-suggestions/p1


Hope it helps,

Damon


2014-10-27 9:18 GMT+08:00 Li Tianqing jaze...@163.com:

Hello,
Right now, we test neutron under havana release. We configured 
network_device_mtu=1450 in neutron.conf, 
   After create vm, we found the vm interface's mtu is 1500, the ping, ssh, is 
ok. But if we scp large file between vms
   then scp display 'stalled'. And iperf is also can not completed.
   If we configured vm's mtu to 1450, then iperf, scp all is ok.
   If we iperf with -M 1300, then the iperf is ok too.
   The vms path mtu discovery is set by default. I do not know why the vm whose 
mtu is 1500 can not send large file.




Thanks.





--

Best
Li Tianqing



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev



___
OpenStack-dev mailing list
OpenStack-dev@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev