Re: [Nanog-futures] Draft Policy re individual sites
On Mon, May 11, 2009 at 7:05 PM, Jo Rhett jrh...@netconsonance.com wrote: On May 1, 2009, at 1:34 PM, Martin Hannigan wrote: Loudness != majority I think most of us are broad minded and appreciate common sense topics related to network operations. Most know what that is. No need to make rules to assault the few, IMHO. While I agree with your points in theory, Martin, I would ask that you do an actual analysis of useful content on NANOG. I did one some months ago based on a week's backlog of Nanog in my mail folder, and found (quoted from an e-mail I sent to someone at the time) Jo, My analysis would come out dramatically different than yours. That's would be the point. Best Regards, Martin -- Martin Hannigan mar...@theicelandguy.com p: +16178216079 Power, Network, and Costs Consulting for Iceland Datacenters and Occupants ___ Nanog-futures mailing list Nanog-futures@nanog.org http://mailman.nanog.org/mailman/listinfo/nanog-futures
Re: Anomalies with AS13214 ?
On 11/5/09 16:30, Jay Hennigan wrote: We're getting cyclops[1] alerts that AS13214 is advertising itself as origin for all of our prefixes. Their anomaly report shows thousands of prefixes originating there. Anyone else seeing evidence of this or being affected? [1] http://cyclops.cs.ucla.edu/ I'm seeing alerts for AS13214 advertising our prefixes from cyclops also. However a quick look at a few looking glasses and route servers doesnt seem to show any rogue advertisments, and we havent see any drop in traffic as yet. Vince -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Re: PPP multilink help
Gents, On Mon, May 11, 2009 at 10:54 AM, Dan White dwh...@olp.net wrote: Andrey Gordon wrote: [snip] When I transfer a large file over FTP (or CIFS, or anything else), I'd expect it to max out either one or both T1, but instead utilization on the T1s is hoovering at 70% on both and sometimes MLPPP link utilization even drops below 50%. What am I'm not gettting here? Sounds like the TCP window is either set 'small' or TCP window scaling either isn't enabled or isn't scaling to your bandwidth/delay product (for the hosts in question). Since FTP is a 'stream' based transport of file data (like http), you should see this scale to nearly all of or most of your links (assuming TCP isn't your issue). Additionally, when using CIFS, SMB, TFTP, NFS, and other command-acknowledgment style protocols over wide-area links (which aren't stream-based operations, but rather iterative operations on blocks or parts of a file), you likely will never observe a single transfer filling up the links. -Tk
Re: Anomalies with AS13214 ?
Same here. Cyclops reporting an origin change but we are seeing no change in traffic levels. Still investigating at the moment... 2009/5/11 Jay Hennigan j...@west.net We're getting cyclops[1] alerts that AS13214 is advertising itself as origin for all of our prefixes. Their anomaly report shows thousands of prefixes originating there. Anyone else seeing evidence of this or being affected? [1] http://cyclops.cs.ucla.edu/ -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Re: PPP multilink help
On Mon, May 11, 2009 at 10:37:25AM -0400, Andrey Gordon wrote: Hey folks, I'm sure to you it's peanuts, but I'm a bit puzzled (most likely because of the lack of knowledge, I bet). I'm buying an IP backbone from VNZ (presumably MPLS). I get a MLPPP hand off on all sites, so I don't do the actual labeling and switching, so I guess for practical purposes what I'm trying to say is that I have no physical control over the other side of my MLPPP links. When I transfer a large file over FTP (or CIFS, or anything else), I'd expect it to max out either one or both T1, Most MLPPP implementations don't has the flows at the IP layer to an individual MLPPP member link. The bundle is a virtual L3 interface and the packets themselves are distributed over the member links. Some people reference it as a load balancing scenario vs. load sharing as the traffic is given to the link that isn't currently busy. but instead utilization on the T1s is hoovering at 70% on both and sometimes MLPPP link utilization even drops below 50%. What am I'm not gettting here? If you have Multilink fragmentation disabled it sends a packet down each path. It could be a reordering delay causing just enough variance in the packet stream that the application thorttles back. If you have a bunch of individual streams going you would probably see a higher throughput. Remember there is additional overhead for the MLPPP. Rodney Tx, Andrey Below is a snip of my config. controller T1 0/0/0 cablelength long 0db channel-group 1 timeslots 1-24 ! controller T1 0/0/1 cablelength long 0db channel-group 1 timeslots 1-24 ! ip nbar custom rdesktop tcp 3389 ip cef ! class-map match-any VoIP match dscp ef class-map match-any interactive match protocol rdesktop match protocol telnet match protocol ssh ! policy-map QWAS class VoIP priority 100 class interactive bandwidth 500 class class-default fair-queue 4096 ! interface Multilink1 description Verizon Business MPLS Circuit ip address x.x.x.150 255.255.255.252 ip flow ingress ip nat inside ip virtual-reassembly load-interval 30 no peer neighbor-route ppp chap hostname R1 ppp multilink ppp multilink links minimum 1 ppp multilink group 1 ppp multilink fragment disable service-policy output QWAS ! interface Serial0/0/0:1 no ip address ip flow ingress encapsulation ppp load-interval 30 fair-queue 4096 256 0 ppp chap hostname R1 ppp multilink ppp multilink group 1 ! interface Serial0/0/1:1 no ip address ip flow ingress encapsulation ppp load-interval 30 fair-queue 4096 256 0 ppp chap hostname R1 ppp multilink ppp multilink group 1 - Andrey Gordon [andrey.gor...@gmail.com]
Re: Anomalies with AS13214 ?
On Mon, 11 May 2009, Russell Heilling wrote: Same here. Cyclops reporting an origin change but we are seeing no change in traffic levels. Still investigating at the moment... Somewhere, something is confused. I'm seeing cyclops report some of my prefixes with origins of 6364 (correct), 13214 6364 (no), and 6364 13214 (not right either). I'm also not seeing any unusual reduction of input traffic. -- Jon Lewis | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
Re: Anomalies with AS13214 ?
Seeing the same issues with AS13214 and no corresponding drop in traffic, route views doesn't show any rogue adverts for out prefixes either. -James On May 11, 2009, at 9:01 AM, Vincent Hoffman wrote: On 11/5/09 16:30, Jay Hennigan wrote: We're getting cyclops[1] alerts that AS13214 is advertising itself as origin for all of our prefixes. Their anomaly report shows thousands of prefixes originating there. Anyone else seeing evidence of this or being affected? [1] http://cyclops.cs.ucla.edu/ I'm seeing alerts for AS13214 advertising our prefixes from cyclops also. However a quick look at a few looking glasses and route servers doesnt seem to show any rogue advertisments, and we havent see any drop in traffic as yet. Vince -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV Ravioli is the square root of pasta. - Max K., Age 11
RE: PPP multilink help
I would also think the problem is with flow control not allowing the maximum bandwidth. Trying multiple ftp streams and seeing if that would max it out would help. I would think you would want to add a WRED to the class-default entry to prevent global tcp synchronization ... class class-default fair-queue 4096 random-detect dscp-based Matthew Huff | One Manhattanville Rd OTA Management LLC | Purchase, NY 10577 http://www.ox.com | Phone: 914-460-4039 aim: matthewbhuff | Fax: 914-460-4139 -Original Message- From: Rodney Dunn [mailto:rod...@cisco.com] Sent: Monday, May 11, 2009 12:06 PM To: Andrey Gordon Cc: nanog@nanog.org Subject: Re: PPP multilink help On Mon, May 11, 2009 at 10:37:25AM -0400, Andrey Gordon wrote: Hey folks, I'm sure to you it's peanuts, but I'm a bit puzzled (most likely because of the lack of knowledge, I bet). I'm buying an IP backbone from VNZ (presumably MPLS). I get a MLPPP hand off on all sites, so I don't do the actual labeling and switching, so I guess for practical purposes what I'm trying to say is that I have no physical control over the other side of my MLPPP links. When I transfer a large file over FTP (or CIFS, or anything else), I'd expect it to max out either one or both T1, Most MLPPP implementations don't has the flows at the IP layer to an individual MLPPP member link. The bundle is a virtual L3 interface and the packets themselves are distributed over the member links. Some people reference it as a load balancing scenario vs. load sharing as the traffic is given to the link that isn't currently busy. but instead utilization on the T1s is hoovering at 70% on both and sometimes MLPPP link utilization even drops below 50%. What am I'm not gettting here? If you have Multilink fragmentation disabled it sends a packet down each path. It could be a reordering delay causing just enough variance in the packet stream that the application thorttles back. If you have a bunch of individual streams going you would probably see a higher throughput. Remember there is additional overhead for the MLPPP. Rodney Tx, Andrey Below is a snip of my config. controller T1 0/0/0 cablelength long 0db channel-group 1 timeslots 1-24 ! controller T1 0/0/1 cablelength long 0db channel-group 1 timeslots 1-24 ! ip nbar custom rdesktop tcp 3389 ip cef ! class-map match-any VoIP match dscp ef class-map match-any interactive match protocol rdesktop match protocol telnet match protocol ssh ! policy-map QWAS class VoIP priority 100 class interactive bandwidth 500 class class-default fair-queue 4096 ! interface Multilink1 description Verizon Business MPLS Circuit ip address x.x.x.150 255.255.255.252 ip flow ingress ip nat inside ip virtual-reassembly load-interval 30 no peer neighbor-route ppp chap hostname R1 ppp multilink ppp multilink links minimum 1 ppp multilink group 1 ppp multilink fragment disable service-policy output QWAS ! interface Serial0/0/0:1 no ip address ip flow ingress encapsulation ppp load-interval 30 fair-queue 4096 256 0 ppp chap hostname R1 ppp multilink ppp multilink group 1 ! interface Serial0/0/1:1 no ip address ip flow ingress encapsulation ppp load-interval 30 fair-queue 4096 256 0 ppp chap hostname R1 ppp multilink ppp multilink group 1 - Andrey Gordon [andrey.gor...@gmail.com]
Re: Anomalies with AS13214 ?
Randy doing testing again? Jay Hennigan wrote: We're getting cyclops[1] alerts that AS13214 is advertising itself as origin for all of our prefixes. Their anomaly report shows thousands of prefixes originating there. Anyone else seeing evidence of this or being affected? [1] http://cyclops.cs.ucla.edu/ -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Re: Anomalies with AS13214 ?
Robert D. Scott wrote: It looks like Cyclops is seeing these from AS 48285, but I see no indication they are being advertised to any production upstream provider. Our /16 is being alerted in Cyclops, but I can not find any advert on any looking glass. That's what I'm seeing as well. It's possible that 13214 is broken but not causing an issue except to their customers. Or 48285 is broken or just giving bad data to Cyclops. Cyclops has hundreds of monitors and this is the only one showing the issue. I suspect that if there's a real problem it isn't affecting anyone other than 48285 and maybe 13214. -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Re: PPP multilink help
To address the concerns about the overhead (FTP is still transferring that file: core.bvzn#sh proc cpu hist core.bvzn 12:44:07 PM Monday May 11 2009 EST 43322322 100 90 80 70 60 50 40 30 20 10 0511223344556 05050505050 CPU% per second (last 60 seconds) 4 4 3434435334445545545545566444555145454544 100 90 80 70 60 50 40 * * 30 * * 20 * * 10 * ** ** ** # ***# * * * 0511223344556 05050505050 CPU% per minute (last 60 minutes) * = maximum CPU% # = average CPU% 1 433433344343443344334343344334301332 497289281236443538550242449336950644007664423513486377362431706922208088 100* 90* 80* 70* 60* 50* * *** *** ******* 40 ** ** * ** * ** * * * 30 *** 20 ***# 10 ***# 051122334455667.. 0505050505050 CPU% per hour (last 72 hours) * = maximum CPU% # = average CPU% core.bvzn#sh inv NAME: 2821 chassis, DESCR: 2821 chassis snip Serial0/0/0:1 is up, line protocol is up Hardware is GT96K Serial Description: MTU 1500 bytes, BW 1536 Kbit/sec, DLY 2 usec, reliability 255/255, txload 149/255, rxload 15/255 Encapsulation PPP, LCP Open, multilink Open Link is a member of Multilink bundle Multilink1, loopback not set Keepalive set (10 sec) Last input 00:00:00, output 00:00:00, output hang never Last clearing of show interface counters 14w0d Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair [suspended, using FIFO] FIFO output queue 0/40, 0 drops 30 second input rate 93000 bits/sec, 86 packets/sec 30 second output rate 899000 bits/sec, 122 packets/sec 105433994 packets input, 3520749026 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 155813204 packets output, 1174780375 bytes, 0 underruns 0 output errors, 0 collisions, 1 interface resets 0 unknown protocol drops 0 output buffer failures, 0 output buffers swapped out 1 carrier transitions Timeslot(s) Used:1-24, SCC: 0, Transmitter delay is 0 flags Serial0/0/1:1 is up, line protocol is up Hardware is GT96K Serial Description: MTU 1500 bytes, BW 1536 Kbit/sec, DLY 2 usec, reliability 255/255, txload 149/255, rxload 15/255 Encapsulation PPP, LCP Open, multilink Open Link is a member of Multilink bundle Multilink1, loopback not set Keepalive set (10 sec) Last input 00:00:00, output 00:00:00, output hang never Last clearing of show interface counters 14w0d Input queue: 0/75/0/0 (size/max/drops/flushes); Total output drops: 0 Queueing strategy: weighted fair [suspended, using FIFO] FIFO output queue 0/40, 0 drops 30 second input rate 94000 bits/sec, 86 packets/sec 30 second output rate 898000 bits/sec, 122 packets/sec 105441924 packets input, 3518841511 bytes, 0 no buffer Received 0 broadcasts, 0 runts, 0 giants, 0 throttles 0 input errors, 0 CRC, 0 frame, 0 overrun, 0 ignored, 0 abort 155734625 packets output, 1156759105 bytes, 0 underruns 0 output errors, 0 collisions, 1 interface resets 0 unknown protocol drops 0 output buffer failures, 0 output buffers swapped out 1 carrier transitions Timeslot(s) Used:1-24, SCC: 1, Transmitter delay is 0 flags Multilink1 is up, line protocol is up Hardware is multilink group interface Description: Verizon Business MPLS Circuit Internet address is x.x.x.150/30 MTU 1500 bytes, BW 3072 Kbit/sec, DLY 10 usec, reliability 255/255, txload 148/255, rxload 14/255
Re: Anomalies with AS13214 ?
Yeah, interesting contact name on this: person: Fredrik Neij address:DCPNetworks address:Box 161 address:SE-11479 Stockholm address:Sweden mnt-by: MNT-DCP phone: +46 707 323819 nic-hdl:FN2233-RIPE source: RIPE # Filtered Christopher Morrow wrote: On Mon, May 11, 2009 at 12:41 PM, David Freedman david.freed...@uk.clara.net wrote: Randy doing testing again? 13214 != 3130
Re: Anomalies with AS13214 ?
On Mon, May 11, 2009 at 2:29 PM, Andree Toonk andree+na...@toonk.nl wrote: .-- My secret spy satellite informs me that at Mon, 11 May 2009, Jay Hennigan wrote: We're getting cyclops[1] alerts that AS13214 is advertising itself as origin for all of our prefixes. Their anomaly report shows thousands of prefixes originating there. Anyone else seeing evidence of this or being affected? It seems it was picked up by route-views4. Non of the RIS peers seem to have seen this. Looking at the raw bgp data from route-views4: AS13214 leaked a full table (~266294 prefixes) with 13214 as OriginAS to AS48285 which is a routeviews4 peer. Routeviews4 saw these announcements as: ASpath 48285 13214. Since 48285 == robtex, is it possible TPB was just setting up a monitoring/route-feed session to robtex and either missed their outbound policy or sent them the wrong form of outbound policy (full routes not customer only routes)?? -chris
Re: Anomalies with AS13214 ?
Hello, Jay Hennigan wrote: We're getting cyclops[1] alerts that AS13214 is advertising itself as origin for all of our prefixes. Their anomaly report shows thousands of prefixes originating there. Anyone else seeing evidence of this or being affected? I have also seen this today for our prefixes where Cyclops reported the as path 48285 13214. After sending an e-mail to both ASN I got the following answer from AS48285: Our transit 13214 had interesting router problems affecting bgp origins for the entire bgp table. The next-hop and thus routing was still working fine though. Since we collect bgp data from several transits and announce it to multiple route servers and for our own publicly available bgp-tools, it looked worse than it was, but as far as we can tell it was actually never propagated by them to the Internet except to downstreams, where traffic still worked, although via an unusually short path. Regards, Christian Seitz Network Operations
Re: Anomalies with AS13214 ?
Hi all, First, thanks for using Cyclops, and thanks for all the Cyclops users that drop me a message about this. It seems some router in AS13214 decided to originate all the prefixes and send them to AS48285 in the Caymans, all the ASPATHs are 48285 13214. The first announcement was on 2009-05-11 11:03:11 UTC and last on 2009-05-11 12:16:32 UTC, there were 266,289 prefixes leaked (they were withdrawn afterwards) As indicated in the Cyclops alerts, only a single monitor(AS48285) in route-views4 detected this leak. I checked on other neighbors of AS13214 and they seem fine, so it seems it was only a single router issue. This incident shows the advantage of having a wide set of peers for detection, it seems Cyclops was the only tool to detect this incident. Given the amount of banks and financial institutions in the Caymans, i would otherwise have raised a red flag, but it seems this case was an unintentional misconfig by AS13214. Would appreciate any further comment on the tool, and happy cyclopying! --Ricardo the Cyclops guy http://cyclops.cs.ucla.edu On May 11, 2009, at 8:30 AM, Jay Hennigan wrote: We're getting cyclops[1] alerts that AS13214 is advertising itself as origin for all of our prefixes. Their anomaly report shows thousands of prefixes originating there. Anyone else seeing evidence of this or being affected? [1] http://cyclops.cs.ucla.edu/ -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Speaking of weird ASPATHs
I started seeing these on May 8th.
* 95.87.192.0/18 3257 9070 43561 {196738}
* 8928 9070 43561 {196738}
* 8928 9070 43561 {196738}
* 1273 9050 8866 43561 {196738}
* 6762 8400 8866 43561 {196738}
Re: Anomalies with AS13214 ?
On Mon, 11 May 2009, bmann...@vacation.karoshi.com wrote: I certainly do. This time it is a config error, next time it will be researcher X doing some testing for a NANOG paper, and the time after that it will be some RBN test to see if anyone cares anymore to look deeply into what they are trying to pull off. Our level of sensitivity will eventually be nullified and we will all be the worse for it. -Hank anyone but me find it unusual that we accept behaviours by some that we would find unacceptable by others... its stuff like that which provides my strongest motivation for things like SIDR... --bill On Mon, May 11, 2009 at 05:41:36PM +0100, David Freedman wrote: Randy doing testing again? Jay Hennigan wrote: We're getting cyclops[1] alerts that AS13214 is advertising itself as origin for all of our prefixes. Their anomaly report shows thousands of prefixes originating there. Anyone else seeing evidence of this or being affected? [1] http://cyclops.cs.ucla.edu/ -- Jay Hennigan - CCIE #7880 - Network Engineering - j...@impulse.net Impulse Internet Service - http://www.impulse.net/ Your local telephone and internet company - 805 884-6323 - WB6RDV
Re: Speaking of weird ASPATHs
Jason Lewis wrote:
I started seeing these on May 8th.
* 95.87.192.0/18 3257 9070 43561 {196738}
* 8928 9070 43561 {196738}
* 8928 9070 43561 {196738}
* 1273 9050 8866 43561 {196738}
* 6762 8400 8866 43561 {196738}
Google(32bit ASN)
Greets,
Jeroen
signature.asc
Description: OpenPGP digital signature
two interfaces one subnet
Hi, This is a pretty moronic question, but I've been searching RFC's on- and-off for a couple of weeks and can't find an answer. So I'm hoping someone here will know it offhand. I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. The OS in this case is Linux. I know it can be done with clever routing and prioritization and such, but this has to do with vanilla config, just setting up two interfaces in one network. I would be grateful for a pointer to such an RFC statement, assuming it exists. Thanks! Chris
Re: two interfaces one subnet
On May 11, 2009, at 4:29 PM, Chris Meidinger wrote: This is a pretty moronic question, but I've been searching RFC's on- and-off for a couple of weeks and can't find an answer. So I'm hoping someone here will know it offhand. I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. The OS in this case is Linux. I know it can be done with clever routing and prioritization and such, but this has to do with vanilla config, just setting up two interfaces in one network. I would be grateful for a pointer to such an RFC statement, assuming it exists. Why would an RFC prohibit this? Most _implementations_ do, but as far as network rules in general it is a valid configuration. -- TTFN, patrick
Re: two interfaces one subnet
On Mon, 11 May 2009, Chris Meidinger wrote: I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. I don't know if it still works, but it did in Linux little over 10 years back. Proxy-arp:ed all the IPs in the /27 in the /24 and everything was fine (legacy reasons plus radiolink which I didn't want to run a lot of broadcasts over). There are legitimate cases where you might want to do this. -- Mikael Abrahamssonemail: swm...@swm.pp.se
Re: two interfaces one subnet
On 11.05.2009, at 22:34, Patrick W. Gilmore wrote: On May 11, 2009, at 4:29 PM, Chris Meidinger wrote: I would be grateful for a pointer to such an RFC statement, assuming it exists. Why would an RFC prohibit this? Most _implementations_ do, but as far as network rules in general it is a valid configuration. That was essentially my conclusion as well: logically it can't work, but I wasn't certain where it might be forbidden. Thusly did I come to NANOG with the question, thinking smarter people than I might know. If it's completely down to implementation, or really to the interaction between TCP and underlying IP, then so be it. I was hoping that I might just not have thought of the right place to look. On 11.05.2009, at 22:39, Mikael Abrahamsson wrote: On Mon, 11 May 2009, Chris Meidinger wrote: I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. I don't know if it still works, but it did in Linux little over 10 years back. Proxy-arp:ed all the IPs in the /27 in the /24 and everything was fine (legacy reasons plus radiolink which I didn't want to run a lot of broadcasts over). There are legitimate cases where you might want to do this. Yes, I've gotten it to work as well as little as 10 days ago, but it's not something that $random_customer should be doing as a matter of practice. Thus, again, my hope that I just wasn't thinking of the right place to look to find an IETF recommendation against doing so. Thanks for the input! Chris
Re: two interfaces one subnet
On Mon, May 11, 2009 at 3:39 PM, Mikael Abrahamsson swm...@swm.pp.se wrote: ... There are legitimate cases where you might want to do this. NetApp filers consider this to be a legitimate configuration, even a supported and recommended one. If I understand the documentation correctly, NetApp will (somehow) remember the physical interface a request arrived on, and make sure to its send replies out same. It sounds like a slight breakage of the expected behavior in order to gain load-sharing for their multiple NICs attached to the same subnet. IIRC, you can turn the feature off if it makes an issue. --D
Re: two interfaces one subnet
What does two interfaces in one subnet mean? Two NICs? Or virtual interfaces? Mikael Abrahamsson wrote: On Mon, 11 May 2009, Chris Meidinger wrote: I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. I don't know if it still works, but it did in Linux little over 10 years back. Proxy-arp:ed all the IPs in the /27 in the /24 and everything was fine (legacy reasons plus radiolink which I didn't want to run a lot of broadcasts over). There are legitimate cases where you might want to do this.
Re: two interfaces one subnet
On 11.05.2009, at 23:00, Charles Wyble wrote: What does two interfaces in one subnet mean? Two NICs? Or virtual interfaces? Two NICs, as in physical interfaces.
Re: two interfaces one subnet
Unless you configure Layer 2 for two interfaces, it's not going to work. It is invalid from networking principle. If you have to send the traffic for host in same subnet you configured, which interface it should send out ? Basically it may create broadcast storm loop by putting two ip addresses in same subnet in different interface. It may be allowed from host-level, but from router equipment, I don't think it was allowed at all. Alex Chris Meidinger wrote: Hi, This is a pretty moronic question, but I've been searching RFC's on-and-off for a couple of weeks and can't find an answer. So I'm hoping someone here will know it offhand. I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. The OS in this case is Linux. I know it can be done with clever routing and prioritization and such, but this has to do with vanilla config, just setting up two interfaces in one network. I would be grateful for a pointer to such an RFC statement, assuming it exists. Thanks! Chris
Re: two interfaces one subnet
Chris, I work with iHDTV http://ihdtv.org, a project that sends uncompressed high definition television (1.5 Gbps) as UDP over two 1 Gbps interfaces. If both interfaces are on the same subnet, the OS sees the same router (gateway) address on both interfaces, and the results are sub-optimal ... around 50% packet loss. Dave On Mon, May 11, 2009 at 3:29 PM, Chris Meidinger cmeidin...@sendmail.comwrote: Hi, This is a pretty moronic question, but I've been searching RFC's on-and-off for a couple of weeks and can't find an answer. So I'm hoping someone here will know it offhand. I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. The OS in this case is Linux. I know it can be done with clever routing and prioritization and such, but this has to do with vanilla config, just setting up two interfaces in one network. I would be grateful for a pointer to such an RFC statement, assuming it exists. Thanks! Chris
Re: two interfaces one subnet
On 11.05.2009, at 23:19, Alex H. Ryu wrote: Unless you configure Layer 2 for two interfaces, it's not going to work. It is invalid from networking principle. If you have to send the traffic for host in same subnet you configured, which interface it should send out ? Basically it may create broadcast storm loop by putting two ip addresses in same subnet in different interface. It may be allowed from host-level, but from router equipment, I don't think it was allowed at all. Alex, I _personally_ know that it's a problem. I was hoping for an RFC- reference, or similar standards document, to show to customers to convince them to stop trying to hack things to make it work. Chris
Re: two interfaces one subnet
On Mon, May 11, 2009 at 2:22 PM, David Devereaux-Weber ddevereauxwe...@gmail.com wrote: Chris, I work with iHDTV http://ihdtv.org, a project that sends uncompressed high definition television (1.5 Gbps) as UDP over two 1 Gbps interfaces. If both interfaces are on the same subnet, the OS sees the same router (gateway) address on both interfaces, and the results are sub-optimal ... around 50% packet loss. packet loss is probably due to the network switch having to re-learn the location of the MAC address constantly as it sees packets on two or more ports with the same MAC address (think STP loops). If your network stack and network device (switch) supports LACP, then you can have multiple connections between a host and a network device. That is a very easy way to increase capacity and add redundancy. That is how all of our VMWare ESX 3.5i servers are connected. Hector
Re: two interfaces one subnet
Chris Meidinger wrote: Hi, This is a pretty moronic question, but I've been searching RFC's on-and-off for a couple of weeks and can't find an answer. So I'm hoping someone here will know it offhand. I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. The OS in this case is Linux. I know it can be done with clever routing and prioritization and such, but this has to do with vanilla config, just setting up two interfaces in one network. I would be grateful for a pointer to such an RFC statement, assuming it exists. If your goal is to achieve redundancy or to increase bandwidth, you can bond the interfaces together - assuming that you have a switch / switch stack that supports 802.3ad. Then you could assign multiple IPs to the bonded interface without any layer 3 messyness. - Dan
Re: two interfaces one subnet
On 11.05.2009, at 23:31, Dan White wrote: Chris Meidinger wrote: Hi, This is a pretty moronic question, but I've been searching RFC's on- and-off for a couple of weeks and can't find an answer. So I'm hoping someone here will know it offhand. I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. The OS in this case is Linux. I know it can be done with clever routing and prioritization and such, but this has to do with vanilla config, just setting up two interfaces in one network. I would be grateful for a pointer to such an RFC statement, assuming it exists. If your goal is to achieve redundancy or to increase bandwidth, you can bond the interfaces together - assuming that you have a switch / switch stack that supports 802.3ad. Then you could assign multiple IPs to the bonded interface without any layer 3 messyness. I should have been clearer. The case in point is having two physical interfaces, each with a unique IP, in the same subnet. For example, eth0 is 10.0.0.1/24 and eth1 is 10.0.0.2/24, nothing like bonding going on. The customers usually have the idea of running one interface for administration and another for production (which is a _good_ idea) but they want to do it in the same subnet (not such a good idea...) Chris
RE: two interfaces one subnet
Hi Chris, I remember this. I remember it in an early IP RFC, but couldn't find it in 10 minutes of searching. It had to do with intefaces cannot have overlapping address space. One of the IETF greybeards ought to know. It's been a while since I was writing code with marked up rfc's in front of me... Chris
Re: two interfaces one subnet
On May 11, 2009, at 5:19 PM, Alex H. Ryu wrote: Unless you configure Layer 2 for two interfaces, it's not going to work. It can work. Of course it _may_ not, depending upon your implementation, but then some implementations can't get a single interface to work properly per subnet. It is invalid from networking principle. You are confused, there is nothing invalid about the configuration. If you have to send the traffic for host in same subnet you configured, which interface it should send out ? Pick an interface and send the packet. It's not rocket science. I can come up with half a dozen algorithms off the top of my head while typing the last sentence. Basically it may create broadcast storm loop by putting two ip addresses in same subnet in different interface. That is an interesting statement. Could you explain how this can happen without crafting an idiotic implementation spec (e.g. every packet goes out both interfaces)? It may be allowed from host-level, but from router equipment, I don't think it was allowed at all. Ever used HSRP / VRRP? Two interfaces in the same subnet. Works fine. In fact, most people think it works _better_ than one interface in the same subnet. -- TTFN, patrick
Re: two interfaces one subnet
On Mon, May 11, 2009 at 5:38 PM, Chris Meidinger cmeidin...@sendmail.com wrote: For example, eth0 is 10.0.0.1/24 and eth1 is 10.0.0.2/24, nothing like bonding going on. The customers usually have the idea of running one interface for administration and another for production (which is a _good_ idea) but they want to do it in the same subnet (not such a good idea...) I just posted on this, but I didn't really address your original question, so: I'm not aware of anything in the RFCs or other standards which prohibits this. But then, I haven't gone looking, because... It *can* be made to work in practice, for certain scenarios. For example, if you're talking a web server, and you bind the production site to 10.0.0.2 and the administration site to 10.0.0.1, and configure policy routing (you said Linux, right?) to route appropriately, it should work. It works because Apache can bind sites to individual interfaces. -- Ben
Re: two interfaces one subnet
:Hi, : :This is a pretty moronic question, but I've been searching RFC's on- :and-off for a couple of weeks and can't find an answer. So I'm hoping :someone here will know it offhand. : :I've been looking through RFC's trying to find a clear statement that :having two interfaces in the same subnet does not work, but can't find :it that statement anywhere. : :The OS in this case is Linux. I know it can be done with clever :routing and prioritization and such, but this has to do with vanilla :config, just setting up two interfaces in one network. : :I would be grateful for a pointer to such an RFC statement, assuming :it exists. RFC1122, Section 3.3.4.1 explicitly says this IS a legal config from an IP perspective: 3.3.4 Local Multihoming 3.3.4.1 Introduction A multihomed host has multiple IP addresses, which we may think of as logical interfaces. These logical interfaces may be associated with one or more physical interfaces, and these physical interfaces may be connected to the same or different networks. There are other considerations here -- OS, link-layer, etc. Obviously, you want to do such things with care. But simply from a standards perspective, it's ok. There are a lot of hosts that historically didn't have enough RFC1122 compliance to make such configurations problematic (e.g. section 3.3.1.2 and multiple default route support vs. old BSD IP stacks) but that doesn't invalidate the standards. -- Michael J. O'Connor m...@dojo.mi.org =--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--==--= Pain has an element of blank. -Emily Dickinson
Re: two interfaces one subnet
On 11.05.2009, at 23:42, Kevin Oberman wrote: Date: Mon, 11 May 2009 16:19:56 -0500 From: Alex H. Ryu r.hyuns...@ieee.org Unless you configure Layer 2 for two interfaces, it's not going to work. It is invalid from networking principle. If you have to send the traffic for host in same subnet you configured, which interface it should send out ? Basically it may create broadcast storm loop by putting two ip addresses in same subnet in different interface. It may be allowed from host-level, but from router equipment, I don't think it was allowed at all. Alex I am a bit baffled as to why people think: 1. It won't work 2. It is a bad thing to do if it would work Neither is true. If it is two separate interfaces with two MAC addresses, it will work fine IF one of the interfaces is configured with a netmask of 255.255.255.255 (/32). Of course, you will have to add routes for the second interface if you expect to source traffic from it, but it really in not rare. This is, of course, how I've done it at times in the past. Routing management can, however, become quite a pain over time. The customer expectation is, naturally, that any traffic related to a connection that comes in to the first interface should go back out that interface, and anything related to a connection that came into the second interface should go back out there. (All this without any specific routing etc.) I think we both know that that's not going to happen automagically. Chris
Re: two interfaces one subnet
From: Chris Meidinger cmeidin...@sendmail.com Date: Mon, 11 May 2009 23:38:30 +0200 On 11.05.2009, at 23:31, Dan White wrote: Chris Meidinger wrote: Hi, This is a pretty moronic question, but I've been searching RFC's on- and-off for a couple of weeks and can't find an answer. So I'm hoping someone here will know it offhand. I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. The OS in this case is Linux. I know it can be done with clever routing and prioritization and such, but this has to do with vanilla config, just setting up two interfaces in one network. I would be grateful for a pointer to such an RFC statement, assuming it exists. If your goal is to achieve redundancy or to increase bandwidth, you can bond the interfaces together - assuming that you have a switch / switch stack that supports 802.3ad. Then you could assign multiple IPs to the bonded interface without any layer 3 messyness. I should have been clearer. The case in point is having two physical interfaces, each with a unique IP, in the same subnet. For example, eth0 is 10.0.0.1/24 and eth1 is 10.0.0.2/24, nothing like bonding going on. The customers usually have the idea of running one interface for administration and another for production (which is a _good_ idea) but they want to do it in the same subnet (not such a good idea...) This will not work right. One interface can be 10.0.0.1/24, but any added interfaces would need to be /32 (10.0.0.2/32). What your customer wants can probably be done, but it is a really bad idea. Put them in different subnets. If you need to, break off a /30 from the /24. (That is a bit messy as you meed to break the /24 into a /25, a /26, a /27..., but it should work fine. Since the main interface has to talk to ALL of the subnets, you will need to use one address from each and that is pretty wasteful, but it should work.) Just really UGLY! If only a part of the address space need be used, it gets easier and less ugly. If a /25 will work, it's pretty much normal configuration on both interfaces. -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: ober...@es.net Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751
RE: two interfaces one subnet
If it were me and had the requirement of having both NICs in the same L2 segment, but unique IP addresses, I'd assign a secondary IP address to the Layer3 SVI on the upstream device, and give the 2nd NIC on the server an IP on that secondary IP block. Ken Matlock Network Analyst Exempla Healthcare (303) 467-4671 matlo...@exempla.org -Original Message- From: Chris Meidinger [mailto:cmeidin...@sendmail.com] Sent: Monday, May 11, 2009 3:39 PM To: Dan White Cc: nanog@nanog.org Subject: Re: two interfaces one subnet On 11.05.2009, at 23:31, Dan White wrote: Chris Meidinger wrote: Hi, This is a pretty moronic question, but I've been searching RFC's on- and-off for a couple of weeks and can't find an answer. So I'm hoping someone here will know it offhand. I've been looking through RFC's trying to find a clear statement that having two interfaces in the same subnet does not work, but can't find it that statement anywhere. The OS in this case is Linux. I know it can be done with clever routing and prioritization and such, but this has to do with vanilla config, just setting up two interfaces in one network. I would be grateful for a pointer to such an RFC statement, assuming it exists. If your goal is to achieve redundancy or to increase bandwidth, you can bond the interfaces together - assuming that you have a switch / switch stack that supports 802.3ad. Then you could assign multiple IPs to the bonded interface without any layer 3 messyness. I should have been clearer. The case in point is having two physical interfaces, each with a unique IP, in the same subnet. For example, eth0 is 10.0.0.1/24 and eth1 is 10.0.0.2/24, nothing like bonding going on. The customers usually have the idea of running one interface for administration and another for production (which is a _good_ idea) but they want to do it in the same subnet (not such a good idea...) Chris
Re: two interfaces one subnet
On May 11, 2009, at 4:45 PM, Chris Meidinger wrote: On 11.05.2009, at 22:34, Patrick W. Gilmore wrote: On May 11, 2009, at 4:29 PM, Chris Meidinger wrote: I would be grateful for a pointer to such an RFC statement, assuming it exists. Why would an RFC prohibit this? Most _implementations_ do, but as far as network rules in general it is a valid configuration. That was essentially my conclusion as well: logically it can't work, but I wasn't certain where it might be forbidden. How did you read what I wrote and say what you said? I've read it several times and I can't get from point A to point B. Did you do a major typo or something? I said it is valid. After saying you came to the same conclusion, you then said logically it can't work. Your statement not only contradicts what I said, but contradicts the fact is has and does work. Let me be clear: IT IS NOT FORBIDDEN. IT WORKS FINE. -- TTFN, patrick
Re: two interfaces one subnet
On 5/11/09 3:23 PM, Chris Meidinger wrote: On 11.05.2009, at 23:19, Alex H. Ryu wrote: Unless you configure Layer 2 for two interfaces, it's not going to work. It is invalid from networking principle. If you have to send the traffic for host in same subnet you configured, which interface it should send out ? Basically it may create broadcast storm loop by putting two ip addresses in same subnet in different interface. It may be allowed from host-level, but from router equipment, I don't think it was allowed at all. Alex, I _personally_ know that it's a problem. I was hoping for an RFC-reference, or similar standards document, to show to customers to convince them to stop trying to hack things to make it work. Chris In Linux, I ran into the exact situation talked about in the link: http://lwn.net/Articles/45373/ Basically, recent versions of Linux will respond to arp requests for IPs on another interface on the receiving interface. Basically, you end up with traffic going in/out of unexpected interfaces. I discovered my iptables rules weren't quite working right and I couldn't get into one of my boxen because the allow was set to eth0, and the packets were coming in/out of eth1 even though the IP was bound to eth0. One of the more interesting gotchas that had me stumped for hours before I found out what was really going on. -- Brielle Bruns The Summit Open Source Development Group http://www.sosdg.org/ http://www.ahbl.org
Re: two interfaces one subnet
On 11.05.2009 23:47 Patrick W. Gilmore wrote On May 11, 2009, at 5:19 PM, Alex H. Ryu wrote: It may be allowed from host-level, but from router equipment, I don't think it was allowed at all. Ever used HSRP / VRRP? Two interfaces in the same subnet. Works fine. In fact, most people think it works _better_ than one interface in the same subnet. I guess you are mixing interfaces with IPs now. Don't you? Arnold -- Arnold Nipper / nIPper consulting, Sandhausen, Germany email: arn...@nipper.de phone: +49 6224 9259 299 mobile: +49 172 2650958 fax: +49 6224 9259 333 signature.asc Description: OpenPGP digital signature
