Re: [WISPA] frame size and fps - Mikrotik large packets
I guess technically, it doesn't need to be encrypted, as security was not the goal, it was interoperabilty and management of traffic. But why not add the benefit of security/encryptions? GRE is one way to do it (without encryption), but we chose against doing it that way, but I forget why. It may be possible to do it with EoIP (IP-IP?), but the problem with these other methods or similar methods is that they all have greater/significant negative performance impacts or overhead. The idea is reduce header overhead and increase data packet size. The reason we liked CIPE, is that we could not identify any trade off in using it. So why not use it, if it was the best choice? I've yet to be given a reason why CIPE is not the best choice. I will add that we had difficulty/reliability issues with the authentication components of CIPE, where it would not re-connect properly, so we disable that step of the protocol most of the time. But why do we need authentication? Once the tunnel gets established it generally stays that way for a really long time (until a link failure or reboot). And once established the data is encrypted. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Jeff Broadwick [EMAIL PROTECTED] To: 'WISPA General List' wireless@wispa.org Sent: Thursday, August 03, 2006 1:24 PM Subject: RE: [WISPA] frame size and fps - Mikrotik large packets I guess the first question is--does it need to be encrypted? If not, GRE would be a better choice (or IP-IP, if interoperability isn't as important). If it does need to be encrypted, what are the encryption requirements? Jeff Broadwick ImageStream 800-813-5123 x106 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Wednesday, August 02, 2006 12:17 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - Mikrotik large packets Jeff, know that, under Linux, they do that, but it doesn't make it correct. :-) It just makes it confusing. It's one of the reasons that I think so many people confuse MTU and MSS. I agree. Just for the record, ImageStream supports but does not recommend using CIPE as our first choice. We highly encourage customers to use OpenVPN. It is more secure, far more extensible and feature-rich and has considerable market force behind it (unlike CIPE). That statement definately needs further clarification, as we need to define which applications you are referring to in that comment. We use and support OPEN VPN on our routers as well. We use it frequently from customer's central office to customer's remote offices. We also were considering on using it as the VPN method between our Central Management / Monitoring PCs and our remote distributed Routers in our network redesign. For any of those Purposes, specific to a customer or our own use, CIPE would not have been our choice. I look at CIPE as a good alternative to IPSEC. I do not see how Open VPN could be a replacement for CIPE in the applications where we feel CIPE is required. We use CIPE in one broad application. We need to make a connection between Cell Site A and Cell Site B. And between those two cell sites, the interconnecting backhaul (usually third party ethernet fiber provider ) does not support passing packets with an MTU higher than 1500 bytes. In most cases, the Fiber provider uses layer3 private IP addresses to route our data across their network. To maintain and property distribute our IP space, we need to cross these connections in some method of tunneling. We can NOT chose a method of tunneling that increases the MTU or the Fiber carrier will not pass our data across the circuit without forcing packets to be permanently fragmented. (some ISPs will drop fragmented packets as well as not Customer VPN friendly). We can not chose a tunneling method that shrinks the MTU, as then customer's data would not pass our network. The reason for us to tunnel is to maintain IP, not to provide optimal security. We need to select a VPN method that will work with any other VPN that the customer might use for their own need for their corporate network. We need to support Any/All VPN within our VPN compatibility. To the best of my knowledge, the only tunneling protocol to be able to handle this in an amicable way is CIPE. Because CIPE takes care of spliting the packet to fit into 1500 MTU, and reassemble packet on other end, only when needed for optimal performance. To the best of my knowledge, OPEN VPN does not support this application as listed above. If you feel Open VPN can accomplish this, please clarify how. (On a side note, OPEN VPN is also a technology that supports unique reasons to choose Linux routers over name brand applications like Cisco.) Actually I'm interested in learning/comparing ANY tunneling protocol that can accomplsih the above requirements. But performance
RE: [WISPA] frame size and fps - Mikrotik large packets
I guess the first question is--does it need to be encrypted? If not, GRE would be a better choice (or IP-IP, if interoperability isn't as important). If it does need to be encrypted, what are the encryption requirements? Jeff Broadwick ImageStream 800-813-5123 x106 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Wednesday, August 02, 2006 12:17 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - Mikrotik large packets Jeff, know that, under Linux, they do that, but it doesn't make it correct. :-) It just makes it confusing. It's one of the reasons that I think so many people confuse MTU and MSS. I agree. Just for the record, ImageStream supports but does not recommend using CIPE as our first choice. We highly encourage customers to use OpenVPN. It is more secure, far more extensible and feature-rich and has considerable market force behind it (unlike CIPE). That statement definately needs further clarification, as we need to define which applications you are referring to in that comment. We use and support OPEN VPN on our routers as well. We use it frequently from customer's central office to customer's remote offices. We also were considering on using it as the VPN method between our Central Management / Monitoring PCs and our remote distributed Routers in our network redesign. For any of those Purposes, specific to a customer or our own use, CIPE would not have been our choice. I look at CIPE as a good alternative to IPSEC. I do not see how Open VPN could be a replacement for CIPE in the applications where we feel CIPE is required. We use CIPE in one broad application. We need to make a connection between Cell Site A and Cell Site B. And between those two cell sites, the interconnecting backhaul (usually third party ethernet fiber provider ) does not support passing packets with an MTU higher than 1500 bytes. In most cases, the Fiber provider uses layer3 private IP addresses to route our data across their network. To maintain and property distribute our IP space, we need to cross these connections in some method of tunneling. We can NOT chose a method of tunneling that increases the MTU or the Fiber carrier will not pass our data across the circuit without forcing packets to be permanently fragmented. (some ISPs will drop fragmented packets as well as not Customer VPN friendly). We can not chose a tunneling method that shrinks the MTU, as then customer's data would not pass our network. The reason for us to tunnel is to maintain IP, not to provide optimal security. We need to select a VPN method that will work with any other VPN that the customer might use for their own need for their corporate network. We need to support Any/All VPN within our VPN compatibility. To the best of my knowledge, the only tunneling protocol to be able to handle this in an amicable way is CIPE. Because CIPE takes care of spliting the packet to fit into 1500 MTU, and reassemble packet on other end, only when needed for optimal performance. To the best of my knowledge, OPEN VPN does not support this application as listed above. If you feel Open VPN can accomplish this, please clarify how. (On a side note, OPEN VPN is also a technology that supports unique reasons to choose Linux routers over name brand applications like Cisco.) Actually I'm interested in learning/comparing ANY tunneling protocol that can accomplsih the above requirements. But performance and compatibilty is a key factor. When we selected CIPE, we did it because it was able to solve the problem in the most efficient manner with the least amount of overhead to maintain performance. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
RE: [WISPA] frame size and fps - Mikrotik large packets
Back atcha Tom: The only comment I disagree with is The proper VLAN aware drivers show 1500 MTU for both the underlying interface and the VLAN interface but it treats VLAN packets with caution, so as not to truncate or drop them because of their longer size. If that's true, then it isn't a proper VLAN aware driver. The MTU should be set correctly and not just show 1500 and use something else. My statement was just bringing up that the MTU is for the size of the packet before VLAN tags were added, and VLAN header bits are added on top of the existing full size packets. The reason for this is that we guarantee to deliver 1500 MTU to the consumer, and we want the setting to show what the customer expects to get. If we tag it with VLAN for our own use, we must be able to pass the higher size packet, and strip the VLAN off before delivering down to the client on the other end. The consumer's local Ethernet interface has an MTU of 1500. Nothing you do on the WAN side with VLANs, VPNs, etc. will affect that. They should see 1500 even if you use a StarOS box that clamps the WAN MTU down. I don't think that it is a good thing for interfaces to show an MTU of 1500 on both the primary Ethernet and VLAN devices. I know that, under Linux, they do that, but it doesn't make it correct. :-) It just makes it confusing. It's one of the reasons that I think so many people confuse MTU and MSS. Opinions on this depend on what the provider is trying to do with the router. The needs as a customer premise router is different than the needs of a ISP transport provider router. The way StarOS does it, to shring the MTU, is appropriate for Customer premise routers, and it would make sense under that circumstance for the MTU setting to reflect the new MTU size that the customer would see. I guess that would depend on your customer. :-) I can't imagine creating that problem for customers. receive 1500 MTU capability. The problem is not all ethernet devices pass IPSEC higher MTU. Its why it ended up not being a preferred method for tunneling across our network or other's network, without compromising delivery of 1500 MTU to subscribers. Thus the reason we switched to CIPE tunneling as our standard tunneling method. And major reason for selecting Linux based routers. I'd like to add, that I believe ImageStream supports CIPE tunneling. One of the disadvantages of Mikrotik and StarOS, is that they do NOT support CIPE tunneling. It was one of the reasons we built our own routers 5 years ago. (Before MPLS switches allowing larger packets were mainstream and affordable). Just for the record, ImageStream supports but does not recommend using CIPE as our first choice. We highly encourage customers to use OpenVPN. It is more secure, far more extensible and feature-rich and has considerable market force behind it (unlike CIPE). Regards, Jeff Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] frame size and fps - Mikrotik large packets
is there will be one of two things that happen the next two years. 100mbps and 1GB wireless products will become affordable to solve the problem with raw speed, or major feature adds will happen on Routers made for wireless provider's cell sites. This QOS problem is not something solveable from the Radio (bridge) manufacturers themselves, it has to be solved at the router software level. I call the needed technique adaptive bandwidth management. Because bandwidth management logic will need to dynamically change based on the conditions of the links and network and type of data. If you put your mind to it, its chaotic, so many possible random changing variable to effect the logic. Part of the solution is routing protocols designed to spread bandwdith equally across multiple link paths, so you can increase capacity by adding radios not just by increasing capacity of them, which may not be possible in noisy long range links. The current load balancing type protocols are not adequate, as its not likely the paths will be going to the same destinations, to completely solve the problem. And there needs to be a way to tell in real time which path data tends to take, so troubleshooting of network performance problems is managable. The logic behind MPLS is, label each packet with a priority, and then pass them based on priority. And the amount of banwdith you have is the amount of bandwdith you have period. To do better than that, the option is to increase capacity/speed when your average capacity starts to get saturated. So MPLS is a good structure for making the best of what you got. But MPLS does not help you gain more capacity. Smarter routing is needed to take advantage of the full capacity of all your links. The idea is not to have links sitting idle when one link is getting maxxed out and slowed down. Even if the path is longer, it may be faster if uncongested. The advantage Fiber carriers have is that the capacity is unlimited more or less, and sending the data From Dc to NewYork and back to DC again, is no sweat if its what the routing says to do. With Wireless, thats not the case. Every additional wireless hop is a change for packet loss or runnning into a degraded interfered with Link. LOS issues may prevent direct paths between perferred sites. Distance and capacity is limited. So again, the routing needs to be smarter than what was adequate for fiber networks. Just my 2 cents on the topic. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Gino A. Villarini [EMAIL PROTECTED] To: 'WISPA General List' wireless@wispa.org Sent: Tuesday, August 01, 2006 9:38 PM Subject: RE: [WISPA] frame size and fps - Mikrotik large packets (Before MPLS switches allowing larger packets were mainstream and affordable) Where ? care to share ? Gino A. Villarini [EMAIL PROTECTED] Aeronet Wireless Broadband Corp. tel 787.273.4143 fax 787.273.4145 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Tuesday, August 01, 2006 4:23 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - Mikrotik large packets Jeff, Thanks for the clarifications from your engineer. The only comment I disagree with is The proper VLAN aware drivers show 1500 MTU for both the underlying interface and the VLAN interface but it treats VLAN packets with caution, so as not to truncate or drop them because of their longer size. If that's true, then it isn't a proper VLAN aware driver. The MTU should be set correctly and not just show 1500 and use something else. My statement was just bringing up that the MTU is for the size of the packet before VLAN tags were added, and VLAN header bits are added on top of the existing full size packets. The reason for this is that we guarantee to deliver 1500 MTU to the consumer, and we want the setting to show what the customer expects to get. If we tag it with VLAN for our own use, we must be able to pass the higher size packet, and strip the VLAN off before delivering down to the client on the other end. Opinions on this depend on what the provider is trying to do with the router. The needs as a customer premise router is different than the needs of a ISP transport provider router. The way StarOS does it, to shring the MTU, is appropriate for Customer premise routers, and it would make sense under that circumstance for the MTU setting to reflect the new MTU size that the customer would see. But in our application as a transport provider, our cell site routers want to appear transparent, to any application the consumer may want to use. Its standard that Ethernet is limited to 1500 MTU, and the Consumer should have not problem working around that, and when they want to lower their own MTU. However, as a provider, we never have a need to lower an MTU, as lowering an MTU would compromise the service
Re: [WISPA] frame size and fps - Mikrotik large packets
Jeff, know that, under Linux, they do that, but it doesn't make it correct. :-) It just makes it confusing. It's one of the reasons that I think so many people confuse MTU and MSS. I agree. Just for the record, ImageStream supports but does not recommend using CIPE as our first choice. We highly encourage customers to use OpenVPN. It is more secure, far more extensible and feature-rich and has considerable market force behind it (unlike CIPE). That statement definately needs further clarification, as we need to define which applications you are referring to in that comment. We use and support OPEN VPN on our routers as well. We use it frequently from customer's central office to customer's remote offices. We also were considering on using it as the VPN method between our Central Management / Monitoring PCs and our remote distributed Routers in our network redesign. For any of those Purposes, specific to a customer or our own use, CIPE would not have been our choice. I look at CIPE as a good alternative to IPSEC. I do not see how Open VPN could be a replacement for CIPE in the applications where we feel CIPE is required. We use CIPE in one broad application. We need to make a connection between Cell Site A and Cell Site B. And between those two cell sites, the interconnecting backhaul (usually third party ethernet fiber provider ) does not support passing packets with an MTU higher than 1500 bytes. In most cases, the Fiber provider uses layer3 private IP addresses to route our data across their network. To maintain and property distribute our IP space, we need to cross these connections in some method of tunneling. We can NOT chose a method of tunneling that increases the MTU or the Fiber carrier will not pass our data across the circuit without forcing packets to be permanently fragmented. (some ISPs will drop fragmented packets as well as not Customer VPN friendly). We can not chose a tunneling method that shrinks the MTU, as then customer's data would not pass our network. The reason for us to tunnel is to maintain IP, not to provide optimal security. We need to select a VPN method that will work with any other VPN that the customer might use for their own need for their corporate network. We need to support Any/All VPN within our VPN compatibility. To the best of my knowledge, the only tunneling protocol to be able to handle this in an amicable way is CIPE. Because CIPE takes care of spliting the packet to fit into 1500 MTU, and reassemble packet on other end, only when needed for optimal performance. To the best of my knowledge, OPEN VPN does not support this application as listed above. If you feel Open VPN can accomplish this, please clarify how. (On a side note, OPEN VPN is also a technology that supports unique reasons to choose Linux routers over name brand applications like Cisco.) Actually I'm interested in learning/comparing ANY tunneling protocol that can accomplsih the above requirements. But performance and compatibilty is a key factor. When we selected CIPE, we did it because it was able to solve the problem in the most efficient manner with the least amount of overhead to maintain performance. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
RE: [WISPA] frame size and fps - Mikrotik large packets
Hi Tom, Ran this past engineering: Vlan is level two information. A VLAN packet has a different type in the Ethernet header, which is read by the card driver. So a VLAN aware driver will allow a packet which physical size is 1518 bytes long (1500 bytes of payload + 2*6 of ethernet address + 2 bytes of type/len + 4 bytes VLAN extra info) instead of the normal 1514. Yep. Typically, drivers will allow even larger packets than this, too, since they'll allow frames with bridge headers, too. On the other hand, IPSec (more precisely ESP and AH) are IP protocols. I.e. the ethernet drivers knows nothing about it. And an IPSec packet can be transported in an ethernet packet, a vlan packet or over a ppp connection. It is IP. Plus, the overhead of IPSec is a lot more than 4 bytes, more 40 bytes or so, but I don't remember the exact value. It's about 32-36, depending on the packet and without compression, for an ESP packet. So my recollection is as followed: - the unpatched drivers on our Linux box were dumb and would simply drop packets that where too big. That used to be the case, but has been resolved by the community. Of course, commercial vendors that run Linux solved this long ago, too. A Linux system will display and use the proper MTU. But this has no bearing on IPSec. This is a different ball games. And that's why I was asking the question: what is it for? To create tunnels for you and they need to have 1500 MTU? Or to create tunnels for the customers and it is then a non-issue: they'll have to deal with the lower MTU size of the IPSec tunnel and most of the time it just works (thanks to path MTU discovery). As an expansion on that point, PMTU is just as important--if you have a bottleneck somewhere in the middle of the network that only accepts a smaller packet, you'll encounter problems. MTU path discovery can help, but it is unreliable and not always available. To clarify. The MTU is only the size of the payload. It doesn't take into account the Ethernet header. Of course, the IP header, TCP/UDP header, etc. are considered payload for ethernet and indeed counted in the ethernet payload. This is incorrect. The MTU is the size of the packet less non-TCP headers, as you mentioned above. It considers the entire packet with all headers attached. The MSS is the value that you are defining here--the size of the allowable payload. The MSS is negotiated during the SYN and SYN/ACK phases of TCP. There are two MTU to consider. The MTU of the underlying ethernet interface and the MTU of the VLAN interface itself. The second MTU is the effective MTU, the one seen by application, networks, using this interface. The first MTU is the one of the hardware interface. I think that calling the second value an MTU is a misnomer. The IPSec interface has an MTU that is an actual MTU (not an effective one), and it will be lower than either the VLAN or Ethernet interface upon which that VPN rides. The trick used by StarOS is to reduced the effective MTU. I think the term you are searching to find is MSS. Either way, the result is the same: you get less payload so that the packet (headers+payload) fits within a normal MTU. Therefore, gaining 4 bytes off the payload to expand the header into it, without the underlying interface having to be aware of it. If it was possible, leaving the effective MTU at the same value and increasing the underlying interface MTU by 4 bytes would have the same effect. Exactly, though just to be clearer, you're talking about dropping the MSS, which would lower the MTU as well (all other things being equal). The proper VLAN aware drivers show 1500 MTU for both the underlying interface and the VLAN interface but it treats VLAN packets with caution, so as not to truncate or drop them because of their longer size. If that's true, then it isn't a proper VLAN aware driver. The MTU should be set correctly and not just show 1500 and use something else. I know the gigabit ports would, but not the Mikrotik 100mbps ports? Actually, not all GigE ports will have jumbo frames enabled. It's not a safe assumption that your packets won't get fragmented on a GigE port. So I'm not even sure how to test :-) You have to prevent or detect fragmentation to know what's going on. With ping, the option '-M do' will set the DF flag (don't fragment). The test is to see that without fragmentation, you can ping with '-s 1468' and not with '-s 1472'. This would indicate a VLAN MTU issue. Sniffing with tcpdump, where appropriate, is also very informative. In particular look at the flags: [DF] means that the don't fragment flag is set, [+] means that the more fragment to come flag is set (i.e. the message is fragmented). Examples: # sudo tcpdump -i eth4 -l -n -v icmp tcpdump: listening on eth4, link-type EN10MB (Ethernet), capture size 68 bytes 19:05:27.714176 IP (tos 0x0, ttl 64, id 12940, offset 0, flags [DF], length: 1500) 10.0.162.1 10.0.162.3: icmp 1480: echo
Re: [WISPA] frame size and fps - Mikrotik large packets
Jeff, Thanks for the clarifications from your engineer. The only comment I disagree with is The proper VLAN aware drivers show 1500 MTU for both the underlying interface and the VLAN interface but it treats VLAN packets with caution, so as not to truncate or drop them because of their longer size. If that's true, then it isn't a proper VLAN aware driver. The MTU should be set correctly and not just show 1500 and use something else. My statement was just bringing up that the MTU is for the size of the packet before VLAN tags were added, and VLAN header bits are added on top of the existing full size packets. The reason for this is that we guarantee to deliver 1500 MTU to the consumer, and we want the setting to show what the customer expects to get. If we tag it with VLAN for our own use, we must be able to pass the higher size packet, and strip the VLAN off before delivering down to the client on the other end. Opinions on this depend on what the provider is trying to do with the router. The needs as a customer premise router is different than the needs of a ISP transport provider router. The way StarOS does it, to shring the MTU, is appropriate for Customer premise routers, and it would make sense under that circumstance for the MTU setting to reflect the new MTU size that the customer would see. But in our application as a transport provider, our cell site routers want to appear transparent, to any application the consumer may want to use. Its standard that Ethernet is limited to 1500 MTU, and the Consumer should have not problem working around that, and when they want to lower their own MTU. However, as a provider, we never have a need to lower an MTU, as lowering an MTU would compromise the service delivered to the consumerthat would expect to receive 1500 MTU capabilty. The problem is not all ethernet devices pass IPSEC higher MTU. Its why it ended up not being a preferred method for tunneling across our network or other's network, without compromising delivery of 1500 MTU to subscribers. Thus the reason we switched to CIPE tunneling as our standard tunneling method. And major reason for selecting Linux based routers. I'd like to add, that I believe ImageStream supports CIPE tunneling. One of the disadvantages of Mikrotik and StarOS, is that they do NOT support CIPE tunneling. It was one of the reasons we built our own routers 5 years ago. (Before MPLS switches allowing larger packets were mainstream and affordable). Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
RE: [WISPA] frame size and fps - Mikrotik large packets
(Before MPLS switches allowing larger packets were mainstream and affordable) Where ? care to share ? Gino A. Villarini [EMAIL PROTECTED] Aeronet Wireless Broadband Corp. tel 787.273.4143 fax 787.273.4145 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Tuesday, August 01, 2006 4:23 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - Mikrotik large packets Jeff, Thanks for the clarifications from your engineer. The only comment I disagree with is The proper VLAN aware drivers show 1500 MTU for both the underlying interface and the VLAN interface but it treats VLAN packets with caution, so as not to truncate or drop them because of their longer size. If that's true, then it isn't a proper VLAN aware driver. The MTU should be set correctly and not just show 1500 and use something else. My statement was just bringing up that the MTU is for the size of the packet before VLAN tags were added, and VLAN header bits are added on top of the existing full size packets. The reason for this is that we guarantee to deliver 1500 MTU to the consumer, and we want the setting to show what the customer expects to get. If we tag it with VLAN for our own use, we must be able to pass the higher size packet, and strip the VLAN off before delivering down to the client on the other end. Opinions on this depend on what the provider is trying to do with the router. The needs as a customer premise router is different than the needs of a ISP transport provider router. The way StarOS does it, to shring the MTU, is appropriate for Customer premise routers, and it would make sense under that circumstance for the MTU setting to reflect the new MTU size that the customer would see. But in our application as a transport provider, our cell site routers want to appear transparent, to any application the consumer may want to use. Its standard that Ethernet is limited to 1500 MTU, and the Consumer should have not problem working around that, and when they want to lower their own MTU. However, as a provider, we never have a need to lower an MTU, as lowering an MTU would compromise the service delivered to the consumerthat would expect to receive 1500 MTU capabilty. The problem is not all ethernet devices pass IPSEC higher MTU. Its why it ended up not being a preferred method for tunneling across our network or other's network, without compromising delivery of 1500 MTU to subscribers. Thus the reason we switched to CIPE tunneling as our standard tunneling method. And major reason for selecting Linux based routers. I'd like to add, that I believe ImageStream supports CIPE tunneling. One of the disadvantages of Mikrotik and StarOS, is that they do NOT support CIPE tunneling. It was one of the reasons we built our own routers 5 years ago. (Before MPLS switches allowing larger packets were mainstream and affordable). Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/ -- WISPA Wireless List: wireless@wispa.org Subscribe/Unsubscribe: http://lists.wispa.org/mailman/listinfo/wireless Archives: http://lists.wispa.org/pipermail/wireless/
Re: [WISPA] frame size and fps - Mikrotik large packets
Charles, I asked one of our Engineers to clarify on Large Packet Support On Linux Routers and VLAN vs IPSEC. Here is his response. Vlan is level two information. A VLAN packet has a different type in the Ethernet header, which is read by the card driver. So a VLAN aware driver will allow a packet which physical size is 1518 bytes long (1500 bytes of payload + 2*6 of ethernet address + 2 bytes of type/len + 4 bytes VLAN extra info) instead of the normal 1514. On the other hand, IPSec (more precisely ESP and AH) are IP protocols. I.e. the ethernet drivers knows nothing about it. And an IPSec packet can be transported in an ethernet packet, a vlan packet or over a ppp connection. It is IP. Plus, the overhead of IPSec is a lot more than 4 bytes, more 40 bytes or so, but I don't remember the exact value. So my recollection is as followed: - the unpatched drivers on our Linux box were dumb and would simply drop packets that where too big. - the starOS has unpatched drivers but drops to 1496 the MTU of the VLAN interface so that no extra large packet would be generated at the ethernet interface. It works and is correct, but not the behavior we were looking after, only if all devices agree to this behavior and it doesn't mimic the capability of VLAN switches. - the patched drivers (RapidDSL Router Code) and Mikrotik drivers, although they display a MTU of 1500, will accept larger packets to accommodate VLANs. The fact that you cannot change the MTU in Mikrotik doesn't mean that I doesn't pass properly the VLAN packets. Many VLANs are setup all over the place over our Mikrotik routers and RapidDSL routers, and to the best of my knowledge, it works properly. But this has no bearing on IPSec. This is a different ball games. And that's why I was asking the question: what is it for? To create tunnels for you and they need to have 1500 MTU? Or to create tunnels for the customers and it is then a non-issue: they'll have to deal with the lower MTU size of the IPSec tunnel and most of the time it just works (thanks to path MTU discovery). To clarify. The MTU is only the size of the payload. It doesn't take into account the Ethernet header. Of course, the IP header, TCP/UDP header, etc. are considered payload for ethernet and indeed counted in the ethernet payload. There are two MTU to consider. The MTU of the underlying ethernet interface and the MTU of the VLAN interface itself. The second MTU is the effective MTU, the one seen by application, networks, using this interface. The first MTU is the one of the hardware interface. The trick used by StarOS is to reduced the effective MTU. Therefore, gaining 4 bytes off the payload to expand the header into it, without the underlying interface having to be aware of it. If it was possible, leaving the effective MTU at the same value and increasing the underlying interface MTU by 4 bytes would have the same effect. The proper VLAN aware drivers show 1500 MTU for both the underlying interface and the VLAN interface, but it treats VLAN packets with caution, so as not to truncate or drop them because of their longer size. On some places on our network (Reston) I could only ping -s 1470 IPaddres, because any higher ping wouldn;t work. Actually, if 1470 works, so should 1471 and 1472. The size in the ping command is the payload of the ping packet. So the actual size of the IP packet is this size plus the icmp header (8 bytes) and the IP header (20 bytes). 1472 + 20 + 8 = 1500. If you have a VLAN issue, it will cut off at 1468, 1472 - 4 extra bytes of the VLAN tagging. But at Dulles, I tried ping -s 9600 IPaddres and the pings returned. There is no way the Ethernet devices would pass 9600 byte packets would they? ICMP traffic is IP traffic, and as such can go through IP fragmentation (http://www.geocities.com/siliconvalley/vista/8672/network/ ipfrag.html). But some dumb device with a limited IP stack implementation will not support IP reassembling, especially on ICMP traffic. For example, at Dulles, pinging su-peter-knob with -s 1753 fails but pinging ap-wifi-peter-knob works. Note that the ap is Linux based, and hence has a full network stack, and the su (Teletronics) is not. I know the gigabit ports would, but not the Mikrotik 100mbps ports? So I'm not even sure how to test :-) You have to prevent or detect fragmentation to know what's going on. With ping, the option '-M do' will set the DF flag (don't fragment). The test is to see that without fragmentation, you can ping with '-s 1468' and not with '-s 1472'. This would indicate a VLAN MTU issue. Sniffing with tcpdump, where appropriate, is also very informative. In particular look at the flags: [DF] means that the don't fragment flag is set, [+] means that the more fragment to come flag is set (i.e. the message is fragmented). Examples: # sudo tcpdump -i eth4 -l -n -v icmp tcpdump: listening on eth4, link-type EN10MB (Ethernet), capture size 68 bytes 19:05:27.714176 IP (tos 0x0, ttl 64, id
Re: [WISPA] frame size and fps - Mikrotik large packets
Disclaimer- My techs did all the testing and hands on configurations, so I'm responding third hand. To bridge correctly (based on 802.11 client design limitations) WDS is required. So we only tested Large Packets across WDS. As we only needed large packets when we were bridging to pass VLAN and IPSEC data that initiated at our cell site router and terminated at the building router. The topology was: Cell router - wreless - Mikrotik AP - wireless - Microtik building router. The physical Mikrotik hardware was tested by us to pass the large packets in our lab as well as the software. I am not sure that Mikrotik allows the increase of MTU on ports or configurations that don't allow it by specification such as wireless client mode ports. Maybe it can maybe it can't? However our model was not necessarilly to pass large packet in any configuration. Our goal was to deliver the Client a Full 1500 MTU packet size, so that if we (our transport network) layered on VLAN tagging or VPN tunneling to transport it across our network, it would be added on top (up to 1542 packet size) without compromising the customers 1500 MTU guaranteed to them. In our application, I'm not even sure if there was a need to increase the MTU above 1500 on the Mikrotik interface configuration, as the VLAN data just got inserted above the 1500 mtu, and the VLAN configured port on the other end allowed it in. I'd have to double check. But I'll ask tech tommorrow. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Charles Wu [EMAIL PROTECTED] To: 'WISPA General List' wireless@wispa.org Sent: Wednesday, July 26, 2006 11:10 AM Subject: RE: [WISPA] frame size and fps - Mikrotik large packets The other day, I was trying to configure the mtu setting on a Mikrotik, and even though the manual said it supports up to 1600-byte -- the interface configuration won't let me set anything above 1500 Anyone? Tricks? Thoughts? Suggestions? (Tom -- you mentioned in the post that you tested Mikrotik w/ large packets)? -Charles --- WiNOG Wireless Roadshows Coming to a City Near You http://www.winog.com -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Tuesday, June 20, 2006 6:44 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K I only mentioned Mikrotik as its abilty to pass large packets has been tested. I believe we couldn't do that with StarOS as a limitation of Wifi clients (although not positive, as I did not investigate WDS options on StarOS which allows the large packets and full passing bridge features.) With that asside, I guess it would be fair to consider StarOS, Ikarus, and Mikrotik as the same class product. I actually wanted to classify it by hardware class such as OEM Atheros products. But technically thatdefinition would include Alvarion. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Charles Wu [EMAIL PROTECTED] To: 'WISPA General List' wireless@wispa.org Sent: Tuesday, June 20, 2006 3:15 PM Subject: RE: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K Hi Tom, Not to add another chink to your debate -- but it is worth noting that Mikrotik is more of a jack of all trades solution (they do routing, hotspot, etc) than a wireless solution While they do an ok job w/ wireless, IMO, their strength is more the convenience coming from the integration of multiple packages and its flexibility rather than the performance of any single feature If you're looking at purely a wireless solution (in this do-it-yourself genre) -- you need to include Star-OS / Ikarus in your evaluation (but then, documentation gets a bit sparse there...) -Charles --- CWLab Technology Architects http://www.cwlab.com -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Monday, June 19, 2006 5:37 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K Paul, Although many have reported very high speeds with Mikrotik. Our live tests in noisy environments (wether accepted as accurate or not) showed we were not able to get the peak speeds out of Mikrotik where we could get them from Alvarion. Our comparative tests were done with the Alvarion ver 3 firmware (not 4 yet). The Alvarion speeds that we got were right on the numbers with the speeds test Alvarion tech support sent us. Actually our tested speeds were a bit higher in some some cases. (Take note we only got accurate speeds when we hard set modulation to optimal (picked the best one for the situation) modulation for testing). I do not mean this as a negative comment on Mikrotik. Our competition to Alvarion is NOT Trango, Trango does not yet have a 20
Re: [WISPA] frame size and fps - Mikrotik large packets
Jeff, Although what you said is true, by standard. MikrotikOS and various Manufactuers (Trango) have hacked the drivers to allow larger size packets. For example, MPLS 100 mbps routers require Higher MTU than 1500. Service providers need packet size larger than 1500, so that they can deliver the standard 1500 MTU size to the end user. This is becomming a requirement for complex configurations such as VLAN and virtual circuit based Ethernet networks. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Jeff Broadwick [EMAIL PROTECTED] To: 'WISPA General List' wireless@wispa.org Sent: Wednesday, July 26, 2006 4:54 PM Subject: RE: [WISPA] frame size and fps - Mikrotik large packets The Ethernet standard maximum packet size is 1500 bytes (not counting the header). You can use jumbo frames in GigE applications, but not in 10/100, and not all switches support them, because there is no standard. Jeff Broadwick ImageStream 800-813-5123 x106 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Charles Wu Sent: Wednesday, July 26, 2006 11:11 AM To: 'WISPA General List' Subject: RE: [WISPA] frame size and fps - Mikrotik large packets The other day, I was trying to configure the mtu setting on a Mikrotik, and even though the manual said it supports up to 1600-byte -- the interface configuration won't let me set anything above 1500 Anyone? Tricks? Thoughts? Suggestions? (Tom -- you mentioned in the post that you tested Mikrotik w/ large packets)? -Charles --- WiNOG Wireless Roadshows Coming to a City Near You http://www.winog.com -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Tuesday, June 20, 2006 6:44 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K I only mentioned Mikrotik as its abilty to pass large packets has been tested. I believe we couldn't do that with StarOS as a limitation of Wifi clients (although not positive, as I did not investigate WDS options on StarOS which allows the large packets and full passing bridge features.) With that asside, I guess it would be fair to consider StarOS, Ikarus, and Mikrotik as the same class product. I actually wanted to classify it by hardware class such as OEM Atheros products. But technically thatdefinition would include Alvarion. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Charles Wu [EMAIL PROTECTED] To: 'WISPA General List' wireless@wispa.org Sent: Tuesday, June 20, 2006 3:15 PM Subject: RE: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K Hi Tom, Not to add another chink to your debate -- but it is worth noting that Mikrotik is more of a jack of all trades solution (they do routing, hotspot, etc) than a wireless solution While they do an ok job w/ wireless, IMO, their strength is more the convenience coming from the integration of multiple packages and its flexibility rather than the performance of any single feature If you're looking at purely a wireless solution (in this do-it-yourself genre) -- you need to include Star-OS / Ikarus in your evaluation (but then, documentation gets a bit sparse there...) -Charles --- CWLab Technology Architects http://www.cwlab.com -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Monday, June 19, 2006 5:37 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K Paul, Although many have reported very high speeds with Mikrotik. Our live tests in noisy environments (wether accepted as accurate or not) showed we were not able to get the peak speeds out of Mikrotik where we could get them from Alvarion. Our comparative tests were done with the Alvarion ver 3 firmware (not 4 yet). The Alvarion speeds that we got were right on the numbers with the speeds test Alvarion tech support sent us. Actually our tested speeds were a bit higher in some some cases. (Take note we only got accurate speeds when we hard set modulation to optimal (picked the best one for the situation) modulation for testing). I do not mean this as a negative comment on Mikrotik. Our competition to Alvarion is NOT Trango, Trango does not yet have a 20 mbps product for PtMP. We look at our Trango as the best choice to tackle the worse noisy environments (for us almost everywhere :-) Our competition for Alvarion is actually Mikrotik. Mikrotik probably has the single highest value from a feature cost perspective. Why pay Alvarion price, when Mikrotik can do almost the same thing at a fraction of the cost. Mikrotik has changed this market and forced competing vendors to look at how to be more competitive
RE: [WISPA] frame size and fps - Mikrotik large packets
The other day, I was trying to configure the mtu setting on a Mikrotik, and even though the manual said it supports up to 1600-byte -- the interface configuration won't let me set anything above 1500 Anyone? Tricks? Thoughts? Suggestions? (Tom -- you mentioned in the post that you tested Mikrotik w/ large packets)? -Charles --- WiNOG Wireless Roadshows Coming to a City Near You http://www.winog.com -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Tuesday, June 20, 2006 6:44 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K I only mentioned Mikrotik as its abilty to pass large packets has been tested. I believe we couldn't do that with StarOS as a limitation of Wifi clients (although not positive, as I did not investigate WDS options on StarOS which allows the large packets and full passing bridge features.) With that asside, I guess it would be fair to consider StarOS, Ikarus, and Mikrotik as the same class product. I actually wanted to classify it by hardware class such as OEM Atheros products. But technically thatdefinition would include Alvarion. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Charles Wu [EMAIL PROTECTED] To: 'WISPA General List' wireless@wispa.org Sent: Tuesday, June 20, 2006 3:15 PM Subject: RE: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K Hi Tom, Not to add another chink to your debate -- but it is worth noting that Mikrotik is more of a jack of all trades solution (they do routing, hotspot, etc) than a wireless solution While they do an ok job w/ wireless, IMO, their strength is more the convenience coming from the integration of multiple packages and its flexibility rather than the performance of any single feature If you're looking at purely a wireless solution (in this do-it-yourself genre) -- you need to include Star-OS / Ikarus in your evaluation (but then, documentation gets a bit sparse there...) -Charles --- CWLab Technology Architects http://www.cwlab.com -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Monday, June 19, 2006 5:37 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K Paul, Although many have reported very high speeds with Mikrotik. Our live tests in noisy environments (wether accepted as accurate or not) showed we were not able to get the peak speeds out of Mikrotik where we could get them from Alvarion. Our comparative tests were done with the Alvarion ver 3 firmware (not 4 yet). The Alvarion speeds that we got were right on the numbers with the speeds test Alvarion tech support sent us. Actually our tested speeds were a bit higher in some some cases. (Take note we only got accurate speeds when we hard set modulation to optimal (picked the best one for the situation) modulation for testing). I do not mean this as a negative comment on Mikrotik. Our competition to Alvarion is NOT Trango, Trango does not yet have a 20 mbps product for PtMP. We look at our Trango as the best choice to tackle the worse noisy environments (for us almost everywhere :-) Our competition for Alvarion is actually Mikrotik. Mikrotik probably has the single highest value from a feature cost perspective. Why pay Alvarion price, when Mikrotik can do almost the same thing at a fraction of the cost. Mikrotik has changed this market and forced competing vendors to look at how to be more competitive. Mikrotik is doing what Trango did 4 years ago to drive the price down. (I'd argue that Trango is still doing it also). It will be real interesting to see how Alvarion performs side by side to Mikrotik. The initial look show to me that Alvarion adds significant features that make it the premium choice, possibly the leader in OFDM today, if price not part of the consideration. However, Mikrotik's flexibilty and price clearly will keep them a major player for many WISPs. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Paul Hendry [EMAIL PROTECTED] To: 'WISPA General List' wireless@wispa.org Sent: Friday, June 16, 2006 3:45 PM Subject: RE: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K Are these figures in the lab? I have seen similar with a Mikrotik/N-Streme solution. -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Patrick Leary Sent: 16 June 2006 19:57 To: WISPA General List Subject: RE: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K So I have more data for you Matt I just received about what firmware 4.0 delivers in terms of frame sizes and what it can mean to the business case. Remember, this is multipoint, not PtP. All
RE: [WISPA] frame size and fps - Mikrotik large packets
The Ethernet standard maximum packet size is 1500 bytes (not counting the header). You can use jumbo frames in GigE applications, but not in 10/100, and not all switches support them, because there is no standard. Jeff Broadwick ImageStream 800-813-5123 x106 -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Charles Wu Sent: Wednesday, July 26, 2006 11:11 AM To: 'WISPA General List' Subject: RE: [WISPA] frame size and fps - Mikrotik large packets The other day, I was trying to configure the mtu setting on a Mikrotik, and even though the manual said it supports up to 1600-byte -- the interface configuration won't let me set anything above 1500 Anyone? Tricks? Thoughts? Suggestions? (Tom -- you mentioned in the post that you tested Mikrotik w/ large packets)? -Charles --- WiNOG Wireless Roadshows Coming to a City Near You http://www.winog.com -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Tuesday, June 20, 2006 6:44 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K I only mentioned Mikrotik as its abilty to pass large packets has been tested. I believe we couldn't do that with StarOS as a limitation of Wifi clients (although not positive, as I did not investigate WDS options on StarOS which allows the large packets and full passing bridge features.) With that asside, I guess it would be fair to consider StarOS, Ikarus, and Mikrotik as the same class product. I actually wanted to classify it by hardware class such as OEM Atheros products. But technically thatdefinition would include Alvarion. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Charles Wu [EMAIL PROTECTED] To: 'WISPA General List' wireless@wispa.org Sent: Tuesday, June 20, 2006 3:15 PM Subject: RE: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K Hi Tom, Not to add another chink to your debate -- but it is worth noting that Mikrotik is more of a jack of all trades solution (they do routing, hotspot, etc) than a wireless solution While they do an ok job w/ wireless, IMO, their strength is more the convenience coming from the integration of multiple packages and its flexibility rather than the performance of any single feature If you're looking at purely a wireless solution (in this do-it-yourself genre) -- you need to include Star-OS / Ikarus in your evaluation (but then, documentation gets a bit sparse there...) -Charles --- CWLab Technology Architects http://www.cwlab.com -Original Message- From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of Tom DeReggi Sent: Monday, June 19, 2006 5:37 PM To: WISPA General List Subject: Re: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K Paul, Although many have reported very high speeds with Mikrotik. Our live tests in noisy environments (wether accepted as accurate or not) showed we were not able to get the peak speeds out of Mikrotik where we could get them from Alvarion. Our comparative tests were done with the Alvarion ver 3 firmware (not 4 yet). The Alvarion speeds that we got were right on the numbers with the speeds test Alvarion tech support sent us. Actually our tested speeds were a bit higher in some some cases. (Take note we only got accurate speeds when we hard set modulation to optimal (picked the best one for the situation) modulation for testing). I do not mean this as a negative comment on Mikrotik. Our competition to Alvarion is NOT Trango, Trango does not yet have a 20 mbps product for PtMP. We look at our Trango as the best choice to tackle the worse noisy environments (for us almost everywhere :-) Our competition for Alvarion is actually Mikrotik. Mikrotik probably has the single highest value from a feature cost perspective. Why pay Alvarion price, when Mikrotik can do almost the same thing at a fraction of the cost. Mikrotik has changed this market and forced competing vendors to look at how to be more competitive. Mikrotik is doing what Trango did 4 years ago to drive the price down. (I'd argue that Trango is still doing it also). It will be real interesting to see how Alvarion performs side by side to Mikrotik. The initial look show to me that Alvarion adds significant features that make it the premium choice, possibly the leader in OFDM today, if price not part of the consideration. However, Mikrotik's flexibilty and price clearly will keep them a major player for many WISPs. Tom DeReggi RapidDSL Wireless, Inc IntAirNet- Fixed Wireless Broadband - Original Message - From: Paul Hendry [EMAIL PROTECTED] To: 'WISPA General List' wireless@wispa.org Sent: Friday, June 16, 2006 3:45 PM Subject: RE: [WISPA] frame size and fps - was OT: about 70Mbps for under $ 6K Are these figures in the lab? I