Re: Muni fiber: L1 or L2?
Hi Jay, Le 29/01/2013 18:54, Jay Ashworth a écrit : Hmmm. I tend to be a Layer-2-available guy, cause I think it lets smaller players play. Please let me present the french regulatory rules about that. It has been an ongoing debate for a few years and is now almost stable. French regulation has divided the territory in thow zones : dense and non-dense areas, dense beeing city centers wuth multi-tenant buildings. In both case, it is mandatory to install at least two point to point fibers between a residence and a patch-panel. In dense areas, building owners or home owner associations are to choose a building operator to install the fiber strands in the private areas and the patch panel made available to other service providers. This building operator then informs service provider of the location of the patch panel and provide a public offer to ISPs to either buy a strand or rent one, and get some space for their own patch chords in the panel. In non-dense areas, zone operators have to build concentration points (kind of MMRs) for at least 300 residences (when chaining MMRs) or 1000 residences (for a single MMR per zone). Theses MMRs often take the form of street cabinets or shelters and have to be equiped with power and cooling units to enable any ISP yo install active equipments (either OLT or ethernet switch). Building and zone operators can be public (muni-owned) infrastructure operators or public-owned corporations. We've also seen NFP associations applying for such roles. It is mandatory for them to provide a L1 point to point service to ISPs. Infrastructure operators can also provide a L2 service but are still required to offer L1 service to any willing ISP. In such case, collocation space in street cabinets (or the ability to install their own side by side with passive cabinets) is required. This model has been choosed because it lets both network types be deployed : either point to multipoint (GePON) or point to point is possible on any of these fiber networks, thanks to the local-loop (between residences and MMRs) beeing point to point only. Smaller ISPs usually go for L2 services, provided by the infrastructure operator or another ISP already present on site. But some tends to stick to L1 service and deply their own eqipments for many reasons. What comes to mind is the usual incompetence of infrastructure operators regarding to multicast services or maintenance-windows beeing too loose for most SLAs. Some ISPs also stick to P2P topologies because it's simplier to manage and brings less features in the network equipment. They strongly believe that a robust network is a stupid network (and I tend to agree with them, seeing many interoperability and scalability issues in P2MP network equipments). Now, about individual rights, civil liberties and constitutional vantage point, infrastructure operators can't operate a network without an L1 offer, and most also propose an L2 offer. Still, ISPs are the only enitites capable of identifying a user because the infrastructure operator don't have a contract with the end-user in any case. Therefore court orders are sent to ISPs and infrastructure operators ain't concerned. I hope it clarifies what's beeing done on actual fiber networks and how can this issue be regulated (either by common sense or law). Best regards, -- Jérôme Nicolle +33 6 19 31 27 14
Re: Muni network ownership and the Fourth
Jay Ashworth j...@baylink.com writes: Still, the power budget improvements by not going with a single strand active ethernet solution (which were another suggested technology and has actually been deployed by some muni PON folks like Clarkesville, TN) are huge. Imagine a 24 port switch that draws 100 watts. OK, that's 4w per customer. 30k customers from a served location, that's 120kw ($13k power bill if you had 100% efficient UPSes and 0 cost cooling, neither of which is true) just for the edge, not counting any aggregation devices or northbound switch gear. Hmm. the optics don't have auto power control? Auto power control would apply to launch levels for the light; assuming a launch level of -3 dBm and lasers that were only 1 percent efficient (combination of spec max launch power for LX optics and unrealistically crummy efficiency lasers) your total power budget for the laser is only 50 milliwatts out of that 4 watts - wrong place to look for power savings. The rest is taken up by stuff like the ethernet chip and supporting logic in the switch, inefficiencies in the power supply, etc. etc. Back at NN, we discounted this as a technology almost immediately based on energy efficiency alone. Anyway, in summary, for PON deployments the part that matters *is* a greenfield deployment and if the fiber plant is planned and scaled accordingly the cost differential is noise. I assume you mean the cost diff between GPON plant and home-run plant; that's the answer I was hoping for. Close; I meant the cost difference between a home run fiber architecture with centralized splitters for *PON and distributed splitters in the field is minimal, and one gains it back in future-proofing and avoiding forklift upgrades down the road. The question of where one puts the splitters (if any) is coupled to the PON vs. active ethernet question only insofar as AE doesn't need splitters - but assuming: * $10k/month cost differential for power in the scenario above * unity cost for head end equipment (almost certainly wrong) * a 16 way split ratio (worst case; you might get 24 or 32) * $100 apiece splitters (24 or 32 would be marginally more) * today's stupid-low cost of capital break-even point on the decision to go with a PON type of technology is still less than two years. If you have a customer who needs the whole pipe to himself (or next generation optics for 10g or 100g to the couch), with centralized splitters the solution is easy. You re-patch him with an attenuator instead of a splitter (or hook him to the new kit), re-range, and go to town. Of course you lose the power advantages of a PON architecture but those customers are the exception not the rule. -r
Re: Metro Ethernet, VPLS clarifications
Metro-Ethernet is generally the term used to describe Ethernet used as a WAN connection or as a point to point connection. There was at one time the concept of a MAN (Metro Area Network) but metro ethernet is now available in more scenarios than that described. The connectivity can be over fiber or copper and the speed delivered can be as low as a few mbps but commercially available offerings normally start at 5-10 mbps. On the high end its possible to get gigabit and faster connections in certain areas. http://en.wikipedia.org/wiki/Metro_E VPLS stands for Virtual Private Lan Services. This an umbrella technology that allows for the bridging of layer 2 traffic across various layer 2 3 networks. This is generally used as a replacement for a point to point metro ethernet (or other) connection. http://en.wikipedia.org/wiki/VPLS On Mon, Feb 4, 2013 at 11:06 PM, Abzal Sembay serian@gmail.com wrote: Hi experts, I need some clarifications on these terms. Could somebody give explanations or share some links? When and how are these technologies used? Thanks in advance. -- Regards, Abzal -- Scott Helms Vice President of Technology ZCorum (678) 507-5000 http://twitter.com/kscotthelms
L3 East cost maint / fiber 05FEB2012 maintenance
I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. -- *Josh Reynolds* ess...@gmail.com - (270) 302-3552
Re: L3 East cost maint / fiber 05FEB2012 maintenance
We also noticed outage due to L3 Maintenance that went into the outage. We were not even notified about the Maintenance itself. We also noticed black hauling in their network. -Thanks, Viral On 5 February 2013 21:09, Josh Reynolds ess...@gmail.com wrote: I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. -- *Josh Reynolds* ess...@gmail.com - (270) 302-3552
RE: L3 East cost maint / fiber 05FEB2012 maintenance
We saw the same thing out of their Tampa location; there was a brief drop around 2am EST and a more severe one around 4:05 AM which lasted about 10 minutes for us. Unfortunately whatever they did, they did it in a way that our BGP sessions stayed up so we couldn't react until bgpmon altered me about some route withdrawals but by that time things were back to normal and remained stable. -Original Message- From: Josh Reynolds [mailto:ess...@gmail.com] Sent: Tuesday, February 05, 2013 10:40 AM To: nanog@nanog.org Subject: L3 East cost maint / fiber 05FEB2012 maintenance I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. -- *Josh Reynolds* ess...@gmail.com - (270) 302-3552
Re: L3 East cost maint / fiber 05FEB2012 maintenance
On Tue, 5 Feb 2013, Josh Reynolds wrote: I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. We're a Level3 customer in Orlando. Our BGP sessions stayed up, but the number of routes received from Level3 fell to only a few tens of thousands at about 4:10am, and gradually returned to normal numbers by about 4:35am. -- Jon Lewis, MCP :) | I route Senior Network Engineer | therefore you are Atlantic Net| _ http://www.lewis.org/~jlewis/pgp for PGP public key_
RE: L3 East cost maint / fiber 05FEB2012 maintenance
We saw the same here, However our session did tear down. I was told they were doing scheduled emergency maintenance about 3:30PM EST Yesterday. We're hung off the orlando market. Nick Olsen Network Operations (855) FLSPEED x106 From: David Hubbard dhubb...@dino.hostasaurus.com Sent: Tuesday, February 05, 2013 10:53 AM To: nanog@nanog.org Subject: RE: L3 East cost maint / fiber 05FEB2012 maintenance We saw the same thing out of their Tampa location; there was a brief drop around 2am EST and a more severe one around 4:05 AM which lasted about 10 minutes for us. Unfortunately whatever they did, they did it in a way that our BGP sessions stayed up so we couldn't react until bgpmon altered me about some route withdrawals but by that time things were back to normal and remained stable. -Original Message- From: Josh Reynolds [mailto:ess...@gmail.com] Sent: Tuesday, February 05, 2013 10:40 AM To: nanog@nanog.org Subject: L3 East cost maint / fiber 05FEB2012 maintenance I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. -- *Josh Reynolds* ess...@gmail.com - (270) 302-3552
Re: L3 East cost maint / fiber 05FEB2012 maintenance
I got notification of their maintenance window, albeit with 24 hours notice. Notice came in at 11:00GMT-5 yesterday, maintenance was scheduled for 00:00GMT-5 this morning. That said, the notice said that the maintenance was in Phoenix but I got a notice about my IPT circuit at 60 Hudson which I found confusing. Based on my logs, our BGP session with them went down at 03:06GMT-5 and back up at 03:15GMT-5. Down again at 03:37GMT-5 until 04:20GMT-5. A third time at 06:41GMT-5 and back at 06:45GMT-5. Traffic graphs tell a bit of a different story. Just before 05:00GMT-5, our outbound traffic to Level 3 dropped substantially. About that time, I started getting reports about issues to Level 3 destinations. Traces seemed to indicate a black hole condition within Level 3's network in NYC, seemingly at, or just past csw3.NewYork1.Level3.net. Stuff seemed to correct itself by about 06:45GMT-5, but due to Level 3 sending only about 180k routes. About 20 minutes later, the table was back to ~431K and all's been fine since. On 2013-02-05, at 10:39 AM, Josh Reynolds ess...@gmail.com wrote: I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. -- *Josh Reynolds* ess...@gmail.com - (270) 302-3552
RE: L3 East cost maint / fiber 05FEB2012 maintenance
I acknowledge sliding past the maintenance window, and we're seeing similar bumps, 09:42 - 09:46 CST is most recent. This are with our Wisconsin and Netherlands locations. They seem to be having a bad day all around. KG Hi Andrey!
How far must muni fiber operators protect ISP competition?
- Original Message - From: Owen DeLong o...@delong.com Actually, as I understood what was proposed, you would bring Cable Coop and/or other such vendors into the colo space adjacent to the MMR and let them sell directly to the other service providers and/or customers. I am of two minds at this point, on this topic. The goal of this project, lying just atop improving the city's position in the world, is to do so by making practical competition between service providers, to keep prices as low as possible. when I delve into the realm of things like this, some people could make a relatively defensible argument that I am disadvantaging ISPs who are smart enough to know about this sort of service on their own, by helping out those who are not. I'm not sure if that argument outweighs the opposing one, which is that I should be *trying* to advantage those smaller, less savvy operators, as they're the sort I want as providers. I think this particular point is one of opinion; I solicit such. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
Re: Muni fiber: L1 or L2?
- Original Message - From: Masataka Ohta mo...@necom830.hpcl.titech.ac.jp My point is that a conduit capable of storing additional 10 guage copper can, instead, store 10 guage fiber. Or, if you assume a conduit without any extra space, upgrading to PON is also impossible. Sure. My install will be greenfield, down to new conduit, so I may have different contstraints than other planners. I will, in fact, be over-sizing the conduit as well, and I'll offer space leasing to potential providers who want to go that far as well. But, since conduit space will be a much more limited quantity, it will cost quite a bit more to do it that way, even before you blow the fiber, than to lease my L1 or L2 services to the subs. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
Re: L3 East cost maint / fiber 05FEB2012 maintenance
We lost our peering with them in Baton Rouge (Houston) but not in Jackson MS (Atlanta). It was less than 10 minutes. No advanced notification. On Tue, Feb 5, 2013 at 10:06 AM, 2asx1y...@sneakemail.com wrote: I acknowledge sliding past the maintenance window, and we're seeing similar bumps, 09:42 - 09:46 CST is most recent. This are with our Wisconsin and Netherlands locations. They seem to be having a bad day all around. KG Hi Andrey!
Re: How far must muni fiber operators protect ISP competition?
On the video side or the total data project? Both? On Tue, Feb 5, 2013 at 11:08 AM, Jay Ashworth j...@baylink.com wrote: - Original Message - From: Owen DeLong o...@delong.com Actually, as I understood what was proposed, you would bring Cable Coop and/or other such vendors into the colo space adjacent to the MMR and let them sell directly to the other service providers and/or customers. I am of two minds at this point, on this topic. The goal of this project, lying just atop improving the city's position in the world, is to do so by making practical competition between service providers, to keep prices as low as possible. when I delve into the realm of things like this, some people could make a relatively defensible argument that I am disadvantaging ISPs who are smart enough to know about this sort of service on their own, by helping out those who are not. I'm not sure if that argument outweighs the opposing one, which is that I should be *trying* to advantage those smaller, less savvy operators, as they're the sort I want as providers. I think this particular point is one of opinion; I solicit such. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274 -- Scott Helms Vice President of Technology ZCorum (678) 507-5000 http://twitter.com/kscotthelms
Re: Muni fiber: L1 or L2?
- Original Message - From: Scott Helms khe...@zcorum.com Yes it does... It locks you into whatever is supported on the ring. I don't know how I can explain this more plainly, I can (more accurately have) taken a fiber build that was created as a ring spoke SONET system and with the same fiber plant overlaid that with GigE and ATM (further back in time) to backhaul for PON, DSL, VOIP, and direct Active Ethernet. Overlaid? Could you clarify that? Owen's assertion (and mine) is that a loop architecture *requires* active equipment, suited to the phy layer protocol, at each node. And while those loop fibers are running SONET, they can't be running anything else at the same time. There is nothing about a hub spoke architecture is this harmful or even suboptimal for doing Gig-E directly to end users today. You propose to run a ring *for each subscriber*? Or put active gear in the field to mux the subscriber AE loops into a SONET ring? Or some other approach I don't know it possible? This wasn't always true because we've only had 40G and 100G Ethernet for carrier networks for a few years. In the past we were limited by how big of an etherchannel network we could use for the ring. I'd also point out that the ring architecture is optimal for redundancy since you have fewer fiber bundles to get cut in the field and any cut to your ring gets routed around the ring by ERPS (http://en.wikipedia.org/wiki/ERPS) in less than 50 milliseconds. I infer from that continuation of your thought that you mean the second: active optical muxes out in the plant. I'm sure I've made clear why that design limits me in ways I don't want to be limited when building a fiber plant for a 50 year lifetime, but let's address your responses below. Lower the price per instance and you very likely find new demands. The vast majority of business don't WANT that kind of connectivity. The vast majority of businesses don't want it at the price they have to pay for it now -- or more to the point, the consultants who do their IT don't. You have no real way, I should think, to extrapolate whether that will continue as prices drop, especially if sharply. How many MPLS connections get purchased by SMBs? That's the same kind of connectivity at layer 3 and that's a market that is almost entirely used by large corportations. Sure; most small businesses don't need that. But there are some that do, and there are some that it doesn't matter *where they are at*. Fiber on your wall with no upfront engineering charge is a pretty strong call, in some markets, and I won't have to do most of the publicity myself; it'll make the news. But the vendors do and it makes a huge difference to the barrier to entry price for competing vendors offering different services. (I'm talking about more than just IP at this point). What vendors? ISPs don't. And your assertion here is based on what? How many places have ISPs had a *choice* as to whether to take a L1 optical or L2 aggregated handoff? What I'm proposing is a hub and spoke architecture. It's just a much larger hub with much longer spokes. That's called home running, but as I've said that's ok in some scenarios, its just that in most cases there is no benefit. Today. Neither you nor I know how that will change in 20, 30, or 50 years. But that's the horizon I'm planning not to block. You're assuming the current business model of incumbent-provider owned fiber. In a case where you have service providers not allowed to own fiber and a fiber provider not allowed to provide services, the incentives all work towards cooperation and the conflicts of interest between them are eliminated. I understand what you're saying about field technicians and their motivations, but, again those are based largely on the current business models and compensation schemes. In the proposed arena, there's no reason management at the service provider and management at the fiber provider cannot work together to address these issues. Further, the technician that blames the fiber plant for everything rather than cooperating to resolve said issues together will inherently have his installations take longer than the ones that cooperate, so he is actually already automatically incentivized in the correct direction. This is my goal. Admittedly, without some education, that may not be intuitively obvious to him, but I find that education is usually possible when attempted. You need to understand that I've built the exact network your describing several times and in all those case this was for a muni network in a relatively small town (25,000 residents). I also know who the installers are in that sized community (as a group, not personally) and even if you get the best ISP partners on the planet they're going to have normal installers doing much of the work. When you say
Re: Muni fiber: L1 or L2?
- Original Message - From: Jérôme Nicolle jer...@ceriz.fr Le 29/01/2013 18:54, Jay Ashworth a écrit : Hmmm. I tend to be a Layer-2-available guy, cause I think it lets smaller players play. Please let me present the french regulatory rules about that. It has been an ongoing debate for a few years and is now almost stable. [ ... ] Infrastructure operators can also provide a L2 service but are still required to offer L1 service to any willing ISP. In such case, collocation space in street cabinets (or the ability to install their own side by side with passive cabinets) is required. This model has been choosed because it lets both network types be deployed : either point to multipoint (GePON) or point to point is possible on any of these fiber networks, thanks to the local-loop (between residences and MMRs) beeing point to point only. Smaller ISPs usually go for L2 services, provided by the infrastructure operator or another ISP already present on site. But some tends to stick to L1 service and deply their own eqipments for many reasons. Hmmm. Sounds familiar, Jerome. :-) How is it working out in practice, since it's within about 10% of what I proposed to do? Are there any public numbers we can look at? Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
Re: Muni network ownership and the Fourth
- Original Message - From: Robert E. Seastrom r...@seastrom.com Hmm. the optics don't have auto power control? Auto power control would apply to launch levels for the light; assuming a launch level of -3 dBm and lasers that were only 1 percent efficient (combination of spec max launch power for LX optics and unrealistically crummy efficiency lasers) your total power budget for the laser is only 50 milliwatts out of that 4 watts - wrong place to look for power savings. The rest is taken up by stuff like the ethernet chip and supporting logic in the switch, inefficiencies in the power supply, etc. etc. Ah. Didn't realize that was the split. Anyway, in summary, for PON deployments the part that matters *is* a greenfield deployment and if the fiber plant is planned and scaled accordingly the cost differential is noise. I assume you mean the cost diff between GPON plant and home-run plant; that's the answer I was hoping for. Close; I meant the cost difference between a home run fiber architecture with centralized splitters for *PON and distributed splitters in the field is minimal, and one gains it back in future-proofing and avoiding forklift upgrades down the road. I believe that's the same assertion, yes. :-) The question of where one puts the splitters (if any) is coupled to the PON vs. active ethernet question only insofar as AE doesn't need splitters - but assuming: * $10k/month cost differential for power in the scenario above * unity cost for head end equipment (almost certainly wrong) * a 16 way split ratio (worst case; you might get 24 or 32) * $100 apiece splitters (24 or 32 would be marginally more) * today's stupid-low cost of capital break-even point on the decision to go with a PON type of technology is still less than two years. Well, some of it is how many access chassis you need to sink the ports; Calix, for example, can do 480 ports per 10U at AE, but ... well, they say 10k ports, but since each card is 8-GPON (x 16 subs), that's 128 * 20, which is 2560, so I have to assume they're quoting 64x GPON, which people are telling me isn't actually practical. Just the capital cost, though, of 20 chassis vs 1 or 2 is really notable, at the prices those things go for. If you have a customer who needs the whole pipe to himself (or next generation optics for 10g or 100g to the couch), with centralized splitters the solution is easy. You re-patch him with an attenuator instead of a splitter (or hook him to the new kit), re-range, and go to town. Of course you lose the power advantages of a PON architecture but those customers are the exception not the rule. Sure. Unless, as we've been discussing, an ISP comes to town who has all their kit pre-designed and trained, and wants to do one or the other. (My underlying assumptions are in the rollup posts I put out on Friday, if you missed it.) Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
RE: How far must muni fiber operators protect ISP competition?
IMHO: level of clue is a minor point, as that can be bought. The fundamental issues for a project like this are funding, and intent. Well-funded organizations that lack intent are just problem children that like to tie up the courts to keep others from making progress. The target for a project like you describe is the organization with intent, but lacks funding. Yes some of those will have an easier time by not having to acquire the appropriate level of clue, but they may not last long if they don't. Part of your calculation has to be level of churn you are willing to impose on the city as the low-price competitors come and go. Tony -Original Message- From: Jay Ashworth [mailto:j...@baylink.com] Sent: Tuesday, February 05, 2013 8:09 AM To: NANOG Subject: How far must muni fiber operators protect ISP competition? - Original Message - From: Owen DeLong o...@delong.com Actually, as I understood what was proposed, you would bring Cable Coop and/or other such vendors into the colo space adjacent to the MMR and let them sell directly to the other service providers and/or customers. I am of two minds at this point, on this topic. The goal of this project, lying just atop improving the city's position in the world, is to do so by making practical competition between service providers, to keep prices as low as possible. when I delve into the realm of things like this, some people could make a relatively defensible argument that I am disadvantaging ISPs who are smart enough to know about this sort of service on their own, by helping out those who are not. I'm not sure if that argument outweighs the opposing one, which is that I should be *trying* to advantage those smaller, less savvy operators, as they're the sort I want as providers. I think this particular point is one of opinion; I solicit such. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
Re: How far must muni fiber operators protect ISP competition?
- Original Message - From: Scott Helms khe...@zcorum.com On the video side or the total data project? Both? The point of open fiber is to level the competitive marketplace as much as possible for provider. Which approach better services that goal: telling them all about all the providers who might make their services more complete, or not doing so? Whether we provide shared space, treating such providers as other clients, and tying them all through an IX switch, is a subsidiary issue. Cheers -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
Re: How far must muni fiber operators protect ISP competition?
- Original Message - From: Tony Hain alh-i...@tndh.net IMHO: level of clue is a minor point, as that can be bought. The fundamental issues for a project like this are funding, and intent. Well-funded organizations that lack intent are just problem children that like to tie up the courts to keep others from making progress. The target for a project like you describe is the organization with intent, but lacks funding. Yes some of those will have an easier time by not having to acquire the appropriate level of clue, but they may not last long if they don't. Part of your calculation has to be level of churn you are willing to impose on the city as the low-price competitors come and go. So you're saying I *should* provide all comers with the research in question, and deal with shared IX access right up front, even if that means I have multiple providers offering the same good as separate retailers... in the service of avoiding provider churn? Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
Re: L3 East cost maint / fiber 05FEB2012 maintenance
My hunch is that this is fallout and repairs from Juniper PR839412. Only fix is an upgrade. Not sure why they're not able to do a hitless upgrade though; that's unfortunate. Specially-crafted TCP packets that can get past RE/loopback filters can crash the box. --j On Tue, Feb 5, 2013 at 7:39 AM, Josh Reynolds ess...@gmail.com wrote: I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. -- *Josh Reynolds* ess...@gmail.com - (270) 302-3552
Re: L3 East cost maint / fiber 05FEB2012 maintenance
Workaround is proper filtering and other techniques on the RE/Loopback to prevent the issue from happening. Should an upgrade be performed? Yes, but certainly doesn't have to have right away or without notice to customers. On Tue, Feb 5, 2013 at 11:23 AM, Jonathan Lassoff j...@thejof.com wrote: My hunch is that this is fallout and repairs from Juniper PR839412. Only fix is an upgrade. Not sure why they're not able to do a hitless upgrade though; that's unfortunate. Specially-crafted TCP packets that can get past RE/loopback filters can crash the box. --j On Tue, Feb 5, 2013 at 7:39 AM, Josh Reynolds ess...@gmail.com wrote: I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. -- *Josh Reynolds* ess...@gmail.com - (270) 302-3552 -- Jason
Re: Muni fiber: L1 or L2?
On Tue, Feb 5, 2013 at 11:30 AM, Jay Ashworth j...@baylink.com wrote: - Original Message - From: Scott Helms khe...@zcorum.com Yes it does... It locks you into whatever is supported on the ring. I don't know how I can explain this more plainly, I can (more accurately have) taken a fiber build that was created as a ring spoke SONET system and with the same fiber plant overlaid that with GigE and ATM (further back in time) to backhaul for PON, DSL, VOIP, and direct Active Ethernet. Overlaid? Could you clarify that? Sure, ring, hub spoke, home run, star these are all descriptions of the physical architecture and many layer 2 technologies will happily use them all including Ethernet. To use a specific example an existing SONET ring (OC-3 to be precise) had be in service with an ILEC for more than a decade. This physical topology was a common one with a physical ring of fiber (32 strands, yes this was built back in the day) connected to Add/Drop Multiplexers (Fujitsu IIRC) along the ring as needed to deliver 25,000 or shorter copper loops either directly from the same cabinet that ADM was in or from a subtended Digital Loop Carrier off of a spur (collapsed ring) of the ring. Now, SONET connections work off a pair of fibers, one for transmit and one for receive. To run Ethernet (initially 100mbps but now 10G) we simply lit 2 of the remaining 30 strands to overlay an Ethernet ring on top of the SONET ring. We then placed switches in the same remote cabinets we had the ADMs and DLCs and started trenching the fiber drops. Owen's assertion (and mine) is that a loop architecture *requires* active equipment, suited to the phy layer protocol, at each node. And while those loop fibers are running SONET, they can't be running anything else at the same time. You're confounding the physical layer topology with the layer 2 protocol. You can't run SONET and Ethernet on the same physical fiber at the same time (unless you use WDM but that's confusing the discussion) but you'd never build a ring of fiber with only two strands. There is nothing about a hub spoke architecture is this harmful or even suboptimal for doing Gig-E directly to end users today. You propose to run a ring *for each subscriber*? Or put active gear in the field to mux the subscriber AE loops into a SONET ring? Or some other approach I don't know it possible? SONET is simply the legacy (and expensive) way that telco's used to build rings. I'd neither use it nor recommend it for much of anything today. Calix, Occam(also Calix now), Adtran, and all the other guys who play in this space will happily construct a Gig/10G/40G Ethernet ring in the same shelf you're going to be buying to put your GPON or AE line cards in. This wasn't always true because we've only had 40G and 100G Ethernet for carrier networks for a few years. In the past we were limited by how big of an etherchannel network we could use for the ring. I'd also point out that the ring architecture is optimal for redundancy since you have fewer fiber bundles to get cut in the field and any cut to your ring gets routed around the ring by ERPS (http://en.wikipedia.org/wiki/ERPS) in less than 50 milliseconds. I infer from that continuation of your thought that you mean the second: active optical muxes out in the plant. I'm sure I've made clear why that design limits me in ways I don't want to be limited when building a fiber plant for a 50 year lifetime, but let's address your responses below. The only limitation you have is a limited supply of total fibers (hint, this is a big reason why its cheaper to build and run). Lower the price per instance and you very likely find new demands. The vast majority of business don't WANT that kind of connectivity. The vast majority of businesses don't want it at the price they have to pay for it now -- or more to the point, the consultants who do their IT don't. You have no real way, I should think, to extrapolate whether that will continue as prices drop, especially if sharply. The vast majority of businesses don't know and don't care about HOW their connectivity is delivered and wouldn't know the difference between Layer 1 and Layer 2 if it punched them in the face. Almost all businesses want INTERNET connectivity at the highest quality speed at the lowest cost and that's it. There are a small percentage, mainly larger businesses, that do have special requirements, but those special requirements very seldom include a L1 anything. How many MPLS connections get purchased by SMBs? That's the same kind of connectivity at layer 3 and that's a market that is almost entirely used by large corportations. Sure; most small businesses don't need that. Nor medium businesses, and that's where knowing your (potential) customer base matters more than anything I can tell you. If you're
Re: L3 East cost maint / fiber 05FEB2012 maintenance
On Tue, Feb 5, 2013 at 9:33 AM, Jason Biel ja...@biel-tech.com wrote: Workaround is proper filtering and other techniques on the RE/Loopback to prevent the issue from happening. Agreed. However, if it only takes one packet, what if an attacker sources the traffic from your management address space? Guarding against this requires either a separate VRF/table for management traffic or transit traffic, RPF checking, or TTL security. If these weren't setup ahead of time, maybe it would be easier to upgrade than lab, test, and deploy a new configuration. This is all speculation about Level3 on my part; I don't know their network from an internal perspective. --j Should an upgrade be performed? Yes, but certainly doesn't have to have right away or without notice to customers. On Tue, Feb 5, 2013 at 11:23 AM, Jonathan Lassoff j...@thejof.com wrote: My hunch is that this is fallout and repairs from Juniper PR839412. Only fix is an upgrade. Not sure why they're not able to do a hitless upgrade though; that's unfortunate. Specially-crafted TCP packets that can get past RE/loopback filters can crash the box. --j On Tue, Feb 5, 2013 at 7:39 AM, Josh Reynolds ess...@gmail.com wrote: I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. -- *Josh Reynolds* ess...@gmail.com - (270) 302-3552 -- Jason
Re: How far must muni fiber operators protect ISP competition?
Jay, On the data side that's certainly possible, but the content guys won't play ball on a shared L2 network. This actually undermines my position on how to architect your system, but sharing anything from one of the big content guys isn't something I've seen them allow as of yet. Organizations like TVN(Avail now?) or NCTC also require direct agreements and I've never seen them do anything at an aggregation level. On Tue, Feb 5, 2013 at 11:48 AM, Jay Ashworth j...@baylink.com wrote: - Original Message - From: Scott Helms khe...@zcorum.com On the video side or the total data project? Both? The point of open fiber is to level the competitive marketplace as much as possible for provider. Which approach better services that goal: telling them all about all the providers who might make their services more complete, or not doing so? Whether we provide shared space, treating such providers as other clients, and tying them all through an IX switch, is a subsidiary issue. Cheers -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274 -- Scott Helms Vice President of Technology ZCorum (678) 507-5000 http://twitter.com/kscotthelms
Re: L3 East cost maint / fiber 05FEB2012 maintenance
Agree as well. Bad assumption on my part that Level3 would doing the items listed in the workaround already. On Tue, Feb 5, 2013 at 11:41 AM, Jonathan Lassoff j...@thejof.com wrote: On Tue, Feb 5, 2013 at 9:33 AM, Jason Biel ja...@biel-tech.com wrote: Workaround is proper filtering and other techniques on the RE/Loopback to prevent the issue from happening. Agreed. However, if it only takes one packet, what if an attacker sources the traffic from your management address space? Guarding against this requires either a separate VRF/table for management traffic or transit traffic, RPF checking, or TTL security. If these weren't setup ahead of time, maybe it would be easier to upgrade than lab, test, and deploy a new configuration. This is all speculation about Level3 on my part; I don't know their network from an internal perspective. --j Should an upgrade be performed? Yes, but certainly doesn't have to have right away or without notice to customers. On Tue, Feb 5, 2013 at 11:23 AM, Jonathan Lassoff j...@thejof.com wrote: My hunch is that this is fallout and repairs from Juniper PR839412. Only fix is an upgrade. Not sure why they're not able to do a hitless upgrade though; that's unfortunate. Specially-crafted TCP packets that can get past RE/loopback filters can crash the box. --j On Tue, Feb 5, 2013 at 7:39 AM, Josh Reynolds ess...@gmail.com wrote: I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. -- *Josh Reynolds* ess...@gmail.com - (270) 302-3552 -- Jason -- Jason
Re: Muni fiber: L1 or L2?
- Original Message - From: Scott Helms khe...@zcorum.com Overlaid? Could you clarify that? Sure, ring, hub spoke, home run, star these are all descriptions of the physical architecture and many layer 2 technologies will happily use them all including Ethernet. To use a specific example an existing SONET ring (OC-3 to be precise) had be in service with an ILEC for more than a decade. Yup; with you so far; I was an OC-12 tail circuit off of L3/telcove's Pinellas County ring at an earlier job. (And I had a fault on one side, because an... This physical topology was a common one with a physical ring of fiber (32 strands, yes this was built back in the day) connected to Add/Drop Multiplexers (Fujitsu IIRC) ADM at a site adjacent to me was in a business that had closed down, and L3 couldn't get it out of the loop, or hadn't, or what have you, so I was unprotected the entire 2.5 years I was there. Only went out once or twice, though. Mine was a Lucent DMXplore, delivering 6 DS1s and a 10BaseT. along the ring as needed to deliver 25,000 or shorter copper loops either directly from the same cabinet that ADM was in or from a subtended Digital Loop Carrier off of a spur (collapsed ring) of the ring. Now, SONET connections work off a pair of fibers, one for transmit and one for receive. To run Ethernet (initially 100mbps but now 10G) we simply lit 2 of the remaining 30 strands to overlay an Ethernet ring on top of the SONET ring. We then placed switches in the same remote cabinets we had the ADMs and DLCs and started trenching the fiber drops. Surely. You *put active equipment out in the physical plant*. I'm sure that there are some physical plant design criteria that permit that decision, but mine isn't one of them, for reasons I believe I've made fairly clear. You disagree with some of those as well, of course, but you understand *that* I have made them, and I would expect, therefore, also why this entire subthread isn't germane to the problem I'm trying to solve, right? Owen's assertion (and mine) is that a loop architecture *requires* active equipment, suited to the phy layer protocol, at each node. And while those loop fibers are running SONET, they can't be running anything else at the same time. You're confounding the physical layer topology with the layer 2 protocol. You can't run SONET and Ethernet on the same physical fiber at the same time (unless you use WDM but that's confusing the discussion) but you'd never build a ring of fiber with only two strands. Certainly not. But a ring a) requires *some kind* of active equipment between the MDF and the ONT, and b) does not support PtP at all. So, *for my stated purposes*, it's not an acceptable alternative. There is nothing about a hub spoke architecture is this harmful or even suboptimal for doing Gig-E directly to end users today. You propose to run a ring *for each subscriber*? Or put active gear in the field to mux the subscriber AE loops into a SONET ring? Or some other approach I don't know is possible? SONET is simply the legacy (and expensive) way that telco's used to build rings. I'd neither use it nor recommend it for much of anything today. Calix, Occam(also Calix now), Adtran, and all the other guys who play in this space will happily construct a Gig/10G/40G Ethernet ring in the same shelf you're going to be buying to put your GPON or AE line cards in. I'm sure, but it's still a ring. If I ever want to upgrade it, I have to do a lot more than rack new gear in my CO, and then move patch cords one at a time. I infer from that continuation of your thought that you mean the second: active optical muxes out in the plant. I'm sure I've made clear why that design limits me in ways I don't want to be limited when building a fiber plant for a 50 year lifetime, but let's address your responses below. The only limitation you have is a limited supply of total fibers (hint, this is a big reason why its cheaper to build and run). Nope, that is, in fact, not the only limitation; the others have been expressed or implied, but are left as an exercise for the student. Lower the price per instance and you very likely find new demands. The vast majority of business don't WANT that kind of connectivity. The vast majority of businesses don't want it at the price they have to pay for it now -- or more to the point, the consultants who do their IT don't. You have no real way, I should think, to extrapolate whether that will continue as prices drop, especially if sharply. The vast majority of businesses don't know and don't care about HOW their connectivity is delivered and wouldn't know the difference between Layer 1 and Layer 2 if it punched them in the face. No one in this conversation, Scott, has ever suggested that *subscribers* care how the ISP delivers the service, as long as it's fast -- though the
Re: How far must muni fiber operators protect ISP competition?
- Original Message - From: Scott Helms khe...@zcorum.com On the data side that's certainly possible, but the content guys won't play ball on a shared L2 network. This actually undermines my position on how to architect your system, but sharing anything from one of the big content guys isn't something I've seen them allow as of yet. Organizations like TVN(Avail now?) or NCTC also require direct agreements and I've never seen them do anything at an aggregation level. I'm aware of how pissy content providers/transport aggregators are likely to be; I'm been involved in the mythTV project for about 7 years. My point was that if any of them provide on-site equipment as, say, Akamai do (and yes, I realize we're discussing real-time now, not caching), if they have multiple clients in the same place, it's in *their* best interest not to provision multiple racks just because they have contracts with multiple providers; perhaps such racks would connect directly, and mentioning my IX was a red-herring; my apologies for confusing the matter. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://baylink.pitas.com 2000 Land Rover DII St Petersburg FL USA #natog +1 727 647 1274
2013.02.05 NANOG57 day2 morning session notes are up
I posted my notes from this morning's session at http://kestrel3.netflight.com/2013.02.05-NANOG57-day2-morning-session.txt Sorry about the gap in the notes about the telegeography talk; my player decided to wig out, and then die, and I lost a chunk while switching to the redundant computer. Awesome content this morning; definitely getting kudos in the survey feedback! Matt
Re: Muni fiber: L1 or L2?
You *put active equipment out in the physical plant*. I'm sure that there are some physical plant design criteria that permit that decision, but mine isn't one of them, for reasons I believe I've made fairly clear. You disagree with some of those as well, of course, but you understand *that* I have made them, and I would expect, therefore, also why this entire subthread isn't germane to the problem I'm trying to solve, right? I've tried to make clear that yes, in some scenarios (and your situation may well fit here) that it makes sense so I think we can drop this portion. Owen's assertion (and mine) is that a loop architecture *requires* active equipment, suited to the phy layer protocol, at each node. And while those loop fibers are running SONET, they can't be running anything else at the same time. You're confounding the physical layer topology with the layer 2 protocol. You can't run SONET and Ethernet on the same physical fiber at the same time (unless you use WDM but that's confusing the discussion) but you'd never build a ring of fiber with only two strands. Certainly not. But a ring a) requires *some kind* of active equipment between the MDF and the ONT, and b) does not support PtP at all. So, *for my stated purposes*, it's not an acceptable alternative. Right, I'm questioning the value of and required number of point to point connections. You certainly can do dozens of point to point connections with a hub and spoke by simply having a patch panel where your cabinets (which you'll probably have anyhow). There is nothing about a hub spoke architecture is this harmful or even suboptimal for doing Gig-E directly to end users today. You propose to run a ring *for each subscriber*? Or put active gear in the field to mux the subscriber AE loops into a SONET ring? Or some other approach I don't know is possible? SONET is simply the legacy (and expensive) way that telco's used to build rings. I'd neither use it nor recommend it for much of anything today. Calix, Occam(also Calix now), Adtran, and all the other guys who play in this space will happily construct a Gig/10G/40G Ethernet ring in the same shelf you're going to be buying to put your GPON or AE line cards in. I'm sure, but it's still a ring. If I ever want to upgrade it, I have to do a lot more than rack new gear in my CO, and then move patch cords one at a time. Not really, all that changes (and this does matter) is where you swap cards out. I infer from that continuation of your thought that you mean the second: active optical muxes out in the plant. I'm sure I've made clear why that design limits me in ways I don't want to be limited when building a fiber plant for a 50 year lifetime, but let's address your responses below. The only limitation you have is a limited supply of total fibers (hint, this is a big reason why its cheaper to build and run). Nope, that is, in fact, not the only limitation; the others have been expressed or implied, but are left as an exercise for the student. Then I'd have continue to say none, since I've done all of the things you're saying are limitations. If your position was something like, We did the economic study and it will cost us less to home run everything than to place remote cabinets with power. I'd have never questioned you at all. I know you've made a decision, but you _seem_ to have made it on faulty assumptions: 1) You will have demand for layer 1 connectivity sufficient to offset the higher costs of home running all the fiber both today and in 10 years. 2) Not home running creates limitations, mainly on assumption #1, that make it untenable. If #1 isn't true (and I strongly doubt it is) then #2 can't be either. That doesn't mean that home running is wrong for you, but if you did your math on those two assumptions then its certainly questionable. Almost all businesses want INTERNET connectivity at the highest quality speed at the lowest cost and that's it. There are a small percentage, mainly larger businesses, that do have special requirements, but those special requirements very seldom include a L1 anything. Yes, but now we're into Whorf's Hypothesis: your vocabulary limits the things you're *able* to think about; it hasn't been practical to *supply* MAN L1 fiber at reasonable prices until about now. I'm basing my views on talking to ISPs around North America and beyond and helping them plan their networks. You're basing your view on? I could certainly be wrong and it wouldn't be the first time nor will it be the last. Having said that, if you don't have some solid market research or some interested ISPs telling you what they want exactly what are you basing your opinion on? Sure, and I don't expect to sell a lot of it up front, unless my launch ISP wants to use their own L2 gear.
Re: L3 East cost maint / fiber 05FEB2012 maintenance
On 2/5/13 10:02 AM, Jason Biel wrote: Agree as well. Bad assumption on my part that Level3 would doing the items listed in the workaround already. On Tue, Feb 5, 2013 at 11:41 AM, Jonathan Lassoff j...@thejof.com wrote: On Tue, Feb 5, 2013 at 9:33 AM, Jason Biel ja...@biel-tech.com wrote: Workaround is proper filtering and other techniques on the RE/Loopback to prevent the issue from happening. Agreed. However, if it only takes one packet, what if an attacker sources the traffic from your management address space? Guarding against this requires either a separate VRF/table for management traffic or transit traffic, RPF checking, or TTL security. If these weren't setup ahead of time, maybe it would be easier to upgrade than lab, test, and deploy a new configuration. This is all speculation about Level3 on my part; I don't know their network from an internal perspective. Routers that show up on exchange fabrics are a particular problem... For this issue... For what it's worth we have several dzone circuits with them from 100mb/s office links to 10Gb/s paths and we have notifications for maintenances last night and tonight and touching locations in europe us east and us west coasts. I'm presuming that there is further internal work that is not directly impactful. I have evidence of various other providers as well as ourselves undertaking fixes to this issue. --j Should an upgrade be performed? Yes, but certainly doesn't have to have right away or without notice to customers. On Tue, Feb 5, 2013 at 11:23 AM, Jonathan Lassoff j...@thejof.com wrote: My hunch is that this is fallout and repairs from Juniper PR839412. Only fix is an upgrade. Not sure why they're not able to do a hitless upgrade though; that's unfortunate. Specially-crafted TCP packets that can get past RE/loopback filters can crash the box. --j On Tue, Feb 5, 2013 at 7:39 AM, Josh Reynolds ess...@gmail.com wrote: I know a lot of you are out of the office right now, but does anybody have any info on what happened with L3 this morning? They went into a 5 hour maintenance window with expected downtime of about 30 minutes while they upgraded something like *40* of their core routers (their words), but also did this during some fiber work and completely cut off several of their east coast peers for the entirety of the 5 hour window. If anybody has any more info on this, on a NOC contact for them on the East Coast for future issues, you can hit me off off-list if you don't feel comfortable replying with that info here. Thanks, and I hope hope you guys are enjoying Orlando. -- *Josh Reynolds* ess...@gmail.com - (270) 302-3552 -- Jason
REMINDER - Register Now for ARIN Public Policy Consultation @ NANOG 57
REMINDER - If you are remotely participating in the NANOG 57 meeting, and intend to participate in the ARIN Public Policy Consultation, you must register to participate in the jabber session and thus ask questions and be counted in any polls conducted. For those not already registered at this point, you may still do so quickly by going to arin.net/ppchttp://arin.net/ppc and clicking on the Register Now button... FYI (and Thanks!) /John Begin forwarded message: From: John Curran jcur...@arin.netmailto:jcur...@arin.net Subject: Register Now for ARIN Public Policy Consultation @ NANOG 57 Date: January 15, 2013 1:33:53 PM EST To: NANOG list nanog@nanog.orgmailto:nanog@nanog.org NANOGers - If you are going to be at NANOG 57 in Orlando, then please note that ARIN will be holding a Public Policy Consultation (PPC) there regarding several number resource policy proposals and you are very much encouraged to participate and make your views on these proposals known. Your NANOG 57 registration includes attending the ARIN Public Policy Consultation onsite if you so desire to do so. As ARIN's Public Policy Consultations are open to all, it is also possible to attend _just_ the PPC without charge, either in person or remotely. One needs to register separately to just participate in the public policy consultation, and this registration does not provide you entry to any other NANOG programming or social events. This is not likely to be relevant to many folks on this list (since I'll be seeing most of you onsite at NANOG 57!) but if you are going to be remotely watching NANOG 57, please take note and register for the ARIN PPC if you intend on participating in that session (and details are available in the attached announcement.) I'd like to take a moment to thank NANOG's Executive Director Betty Burke and the NANOG Planning Committee for making possible the ARIN Public Policy Consultation @ NANOG 57! Thanks! /John John Curran President and CEO ARIN Begin forwarded message: From: ARIN i...@arin.netmailto:i...@arin.netmailto:i...@arin.net Subject: [arin-ppml] Register Now for ARIN Public Policy Consultation @ NANOG 57 Date: January 15, 2013 5:17:30 AM HST To: arin-p...@arin.netmailto:arin-p...@arin.netmailto:arin-p...@arin.net Registration is now open for ARIN's first Public Policy Consultation (PPC), which will be held during NANOG 57 in Orlando, FL on 5 February 2013 at the Renaissance Orlando at Seaworld. The PPC is part of ARIN's new Policy Development Process, and it is an open public discussion of Internet number resource policy. Registered NANOG 57 attendees do not need to register to participate in this session. ARIN welcomes members of the NANOG community who will not be in Orlando to register as remote participants. If you plan to attend and are not registered for NANOG you must register for the PPC at the URL below. There is no registration fee for this 90-minute session, and it does not provide you entry to any other NANOG programming or social events. Learn more at https://www.arin.net/ppc_nanog57/index.html. Current policy proposals up for discussion at this meeting are: * ARIN-2012-2: IPv6 Subsequent Allocations Utilization Requirement - https://www.arin.net/policy/proposals/2012_2.html * ARIN-prop-182 Update Residential Customer Definition to not exclude wireless as Residential Service - http://lists.arin.net/pipermail/arin-ppml/2012-October/026116.html * ARIN-prop-183 Section 8.4 Transfer enhancement- http://lists.arin.net/pipermail/arin-ppml/2012-October/026203.html The PPC will also include a Policy Experience Report and Open Microphone. ARIN will offer a webcast, live transcript, and Jabber chat options for remote participants. Registered remote participants can submit comments and questions to the discussions during the meeting. Register to attend in person or remotely today! Visit https://www.arin.net/app/meeting/registration/. Regards, Communications and Member Services American Registry for Internet Numbers (ARIN)
ATT Uverse/DSL Network Engineer DNS question
Hi, Can a ATT Uverse/DSL Network Engineer answer a question about the DNS server IPs that are handed out to customers please? I am currently testing from a Florida IP. Can you please let me know if all Uverse and DSL customers across the United States only use these 2 IPs as their primary and secondary DNS servers? 68.94.156.1 68.94.157.1 We provide services based on IP GEO-location. Since the 2 recursive resolvers below are registered in Texas every DNS query for any of our records return results that are intended for IPs in that region. In other words, users on the east coast would actually resolve to a central part of the US or west coast IP. Thanks in advance,Tim
Re: ATT Uverse/DSL Network Engineer DNS question
These appear to be an anycasted service, as I reach different destinations based on my source address. Hopefully each deployment has unique origin IPs for their recursive queries. I would recommend against looking at RIR registration data to determine IP location. There's often little to no correlation, there. --j On Tue, Feb 5, 2013 at 1:01 PM, Tim Haak thaiti...@hotmail.com wrote: Hi, Can a ATT Uverse/DSL Network Engineer answer a question about the DNS server IPs that are handed out to customers please? I am currently testing from a Florida IP. Can you please let me know if all Uverse and DSL customers across the United States only use these 2 IPs as their primary and secondary DNS servers? 68.94.156.1 68.94.157.1 We provide services based on IP GEO-location. Since the 2 recursive resolvers below are registered in Texas every DNS query for any of our records return results that are intended for IPs in that region. In other words, users on the east coast would actually resolve to a central part of the US or west coast IP. Thanks in advance,Tim
Re: ATT Uverse/DSL Network Engineer DNS question
On Tue, Feb 5, 2013 at 1:10 PM, Jonathan Lassoff j...@thejof.com wrote: These appear to be an anycasted service, as I reach different destinations based on my source address. Hopefully each deployment has unique origin IPs for their recursive queries. Just confirmed this. As these resolvers traverse and query your servers, they'll have different source IPs, depending on the regional resolver. Return differentiated DNS responses, based on that. --j I would recommend against looking at RIR registration data to determine IP location. There's often little to no correlation, there. --j On Tue, Feb 5, 2013 at 1:01 PM, Tim Haak thaiti...@hotmail.com wrote: Hi, Can a ATT Uverse/DSL Network Engineer answer a question about the DNS server IPs that are handed out to customers please? I am currently testing from a Florida IP. Can you please let me know if all Uverse and DSL customers across the United States only use these 2 IPs as their primary and secondary DNS servers? 68.94.156.1 68.94.157.1 We provide services based on IP GEO-location. Since the 2 recursive resolvers below are registered in Texas every DNS query for any of our records return results that are intended for IPs in that region. In other words, users on the east coast would actually resolve to a central part of the US or west coast IP. Thanks in advance,Tim
Re: ATT Uverse/DSL Network Engineer DNS question
Here in Orange County, CA I've got a /28 with Uverse Residential with the same DNS servers as mentioned below. FYI On 2/5/13 1:10 PM, Jonathan Lassoff j...@thejof.com wrote: These appear to be an anycasted service, as I reach different destinations based on my source address. Hopefully each deployment has unique origin IPs for their recursive queries. I would recommend against looking at RIR registration data to determine IP location. There's often little to no correlation, there. --j On Tue, Feb 5, 2013 at 1:01 PM, Tim Haak thaiti...@hotmail.com wrote: Hi, Can a ATT Uverse/DSL Network Engineer answer a question about the DNS server IPs that are handed out to customers please? I am currently testing from a Florida IP. Can you please let me know if all Uverse and DSL customers across the United States only use these 2 IPs as their primary and secondary DNS servers? 68.94.156.1 68.94.157.1 We provide services based on IP GEO-location. Since the 2 recursive resolvers below are registered in Texas every DNS query for any of our records return results that are intended for IPs in that region. In other words, users on the east coast would actually resolve to a central part of the US or west coast IP. Thanks in advance,Tim
2013.02.05 NANOG57 day2 afternoon session
Notes, complete with typos are up at http://kestrel3.netflight.com/2013.02.05-NANOG57-day2-afternoon-session.txt definitely awesome content today; bummed i missed out, sounds like tonight should be an absolute blast at seaworld--have fun, and we'll see what tomorrow brings. :) Matt
Re: Muni fiber: L1 or L2?
On Feb 5, 2013, at 9:37 AM, Scott Helms khe...@zcorum.com wrote: On Tue, Feb 5, 2013 at 11:30 AM, Jay Ashworth j...@baylink.com wrote: - Original Message - From: Scott Helms khe...@zcorum.com Yes it does... It locks you into whatever is supported on the ring. I don't know how I can explain this more plainly, I can (more accurately have) taken a fiber build that was created as a ring spoke SONET system and with the same fiber plant overlaid that with GigE and ATM (further back in time) to backhaul for PON, DSL, VOIP, and direct Active Ethernet. Overlaid? Could you clarify that? Sure, ring, hub spoke, home run, star these are all descriptions of the physical architecture and many layer 2 technologies will happily use them all including Ethernet. To use a specific example an existing SONET ring (OC-3 to be precise) had be in service with an ILEC for more than a decade. This physical topology was a common one with a physical ring of fiber (32 strands, yes this was built back in the day) connected to Add/Drop Multiplexers (Fujitsu IIRC) along the ring as needed to deliver 25,000 or shorter copper loops either directly from the same cabinet that ADM was in or from a subtended Digital Loop Carrier off of a spur (collapsed ring) of the ring. Now, SONET connections work off a pair of fibers, one for transmit and one for receive. To run Ethernet (initially 100mbps but now 10G) we simply lit 2 of the remaining 30 strands to overlay an Ethernet ring on top of the SONET ring. We then placed switches in the same remote cabinets we had the ADMs and DLCs and started trenching the fiber drops. However, for any given ring, you are locked into a single technology and you have to put active electronics out in the field. You can't, given a ring architecture, provide dark fiber leases. I realize it is your argument that one doesn't need to do so, there's no market for it, etc. However, I don't agree with you. Owen's assertion (and mine) is that a loop architecture *requires* active equipment, suited to the phy layer protocol, at each node. And while those loop fibers are running SONET, they can't be running anything else at the same time. You're confounding the physical layer topology with the layer 2 protocol. You can't run SONET and Ethernet on the same physical fiber at the same time (unless you use WDM but that's confusing the discussion) but you'd never build a ring of fiber with only two strands. Sure, but, you're ring only works with things that do L2 aggregation in the field with active electronics in the field. This means that for any L2 technology a particular subscriber wants to use, you need to either already have that L2 technology deployed on a ring, or, you need to deploy another ring to support that technology. Lower the price per instance and you very likely find new demands. The vast majority of business don't WANT that kind of connectivity. The vast majority of businesses don't want it at the price they have to pay for it now -- or more to the point, the consultants who do their IT don't. You have no real way, I should think, to extrapolate whether that will continue as prices drop, especially if sharply. The vast majority of businesses don't know and don't care about HOW their connectivity is delivered and wouldn't know the difference between Layer 1 and Layer 2 if it punched them in the face. Almost all businesses want INTERNET connectivity at the highest quality speed at the lowest cost and that's it. There are a small percentage, mainly larger businesses, that do have special requirements, but those special requirements very seldom include a L1 anything. VPNs are popular today (whether MPLS, IPSEC, or otherwise) because L1 connections are expensive and VPNS are (relatively) cheap. If dark fiber can be provided for $30/month per termination (we've already agreed that the cost is $20 or less), that changes the equation quite a bit. If, as a business, I can provide corporate connectivity and internet access to my employees for $30/month/employee without having to use a VPN, but just 802.1q trunking and providing them a router (or switch) that has different ports for Corporate and Personal LANs in their house, that changes the equation quite a bit. Admittedly, this only works for the employees that live within range, but it's an example of the kinds of services that nobody even imagines today because we can't get good L1 services cheap yet. You're assuming the current business model of incumbent-provider owned fiber. In a case where you have service providers not allowed to own fiber and a fiber provider not allowed to provide services, the incentives all work towards cooperation and the conflicts of interest between them are eliminated. I understand what you're saying about field technicians and their motivations, but, again those are based largely on the current business models and
Re: Muni fiber: L1 or L2?
Scott Helms wrote: They are not soo different, as long as you try to recover initial cost not so quickly, which is why copper costs about $10/M or so. I know several dozen companies that do this kind of construction and they don't agree. That is, they are trying to recover initial cost quickly. And, you can see the slide contain POP Active Equipment Cost, which you thought most of the cost is in lighting the fiber, is already included. Google is making their own access gear. Their economy is very very different from all of us here. If you think google access gear is much less expensive than others, let google be the dominant supplier of the access gear for all of us. If you throw away optical MDF, there is no point to discuss L1 unbundling. OK, historically the main distribution frame was where all of the copper pairs came into a central office which means they have enough space to accommodate optical MDF. note that a phone company often had several central offices to cover their territory in the time before there were remotes (Digital Loop Carriers). Each CO has its own MDF, where competing ISPs must have their routers. No different from competing ISPs using DSL or PON. Today even when you home run all of your fiber connections you bring it to a central patch panel(s) which really doesn't look like a main distribution frame. If so, it is merely because they want to make L1 unbundling difficult. Surely, transition from copper to fiber is not trivial, but it helps a lot that fiber cables are thinner than copper cables. Really, so you think that the thickness of the cable has an impact on how much it should cost? So, tell you what I'll exchange some nice thick 10 gauge copper wire for 4 gauge platinum, since its much thinner that ought to be a good trade for you, right? ;) My point is that a conduit capable of storing additional 10 guage copper can, instead, store 10 guage fiber. Or, if you assume a conduit without any extra space, upgrading to PON is also impossible. OK, twisted pair cabling isn't run in conduit. Each fiber in an access cable, neither. You cannot remove the twisted pair in whole or part and then run fiber through that cabling. Are you saying you can remove a fiber from an access cable? No, you can't. Well., it is not impossible if you use quite fatty cable in which each fiber is stored in its own conduit. But, it costs a lot. Worse, if a cable is cut, you must repair all the conduit to be air tight again, which means it is practically impossible. You can of course use the same trench IF you have buried cable and there is room. There is room for another cable mostly always, because, without the room, you can not replace copper cables without much service interruption. To replace a damaged copper cable without much service interruption, you have to lay a new cable before removing the damaged cable. Masataka Ohta
RE: Muni fiber: L1 or L2?
In the past the ISP simply needed a nice big ATM pipe to the ILEC for DSL service. The ILEC provided a PVC from the customer endpoint to the ISP. As understand it this is no longer the case, but only because of non-technical issues. We currently use XO, Covad, etc to connect to the customer We get a fiber connection to them and the provide use L2 connectivity to the custom endpoint using an Ethernet VLAN, Frame Relay PVC, etc complete with QoS. I assume XO, etc use UNE access to the local loop. There is no reason a Muni can't do something similar. -Original Message- From: Masataka Ohta [mailto:mo...@necom830.hpcl.titech.ac.jp] Sent: Tuesday, February 05, 2013 7:17 PM To: Scott Helms Cc: NANOG Subject: Re: Muni fiber: L1 or L2? note that a phone company often had several central offices to cover their territory in the time before there were remotes (Digital Loop Carriers). Each CO has its own MDF, where competing ISPs must have their routers. No different from competing ISPs using DSL or PON.
Re: Muni fiber: L1 or L2?
Eric Wieling wrote: In the past the ISP simply needed a nice big ATM pipe to the ILEC for DSL service. The ILEC provided a PVC from the customer endpoint to the ISP. As understand it this is no longer the case, but only because of non-technical issues. The non-technical issue is *COST*! No one considered to use so expensive ATM as L2 for DSL unbundling, at least in Japan, which made DSL in Japan quite inexpensive. We currently use XO, Covad, etc to connect to the customer We get a fiber connection to them and the provide use L2 connectivity to the custom endpoint using an Ethernet VLAN, Frame Relay PVC, etc complete with QoS. I assume XO, etc use UNE access to the local loop. There is no reason a Muni can't do something similar. Muni can. However, there is no reason Muni can't offer L1 unbundling. Masataka Ohta
RE: Muni fiber: L1 or L2?
The ILECs basically got large portions of the 1996 telecom reform rules gutted via lawsuits. DSL unbundling was part of this. See http://quello.msu.edu/sites/default/files/pdf/wp-05-02.pdf The ILECs already need a DSLAM in each CO and already use ATM PVCs to provide L2 connectivity from the DSLAM to their IP network, I don't think it is that much more expensive to allow other ISPs an ATM PVC into their network. ATM may not be the best technology to do this, but the basic concept is not bad. Ethernet VLANs would be another option, as would Frame Relay, as would simply DAXing multiple 64k channels from the customer endpoint to the ISP if you want more L1 style connectivity. What *I* want as an ISP is to connect to customers, I don't care what the local loop is. It could be fiber, twisted pair, coax, or even licensed wireless and hand it off to me over a nice fat fiber link with a PVC or VLAN or whatever to the customer endpoint. What I don't want is to have to install equipment at each and every CO I want to provide service out of. This would be astoundingly expensive for us. -Original Message- From: Masataka Ohta [mailto:mo...@necom830.hpcl.titech.ac.jp] Sent: Tuesday, February 05, 2013 7:42 PM To: nanog@nanog.org Subject: Re: Muni fiber: L1 or L2? Eric Wieling wrote: In the past the ISP simply needed a nice big ATM pipe to the ILEC for DSL service. The ILEC provided a PVC from the customer endpoint to the ISP. As understand it this is no longer the case, but only because of non-technical issues. The non-technical issue is *COST*! No one considered to use so expensive ATM as L2 for DSL unbundling, at least in Japan, which made DSL in Japan quite inexpensive. We currently use XO, Covad, etc to connect to the customer We get a fiber connection to them and the provide use L2 connectivity to the custom endpoint using an Ethernet VLAN, Frame Relay PVC, etc complete with QoS. I assume XO, etc use UNE access to the local loop. There is no reason a Muni can't do something similar. Muni can. However, there is no reason Muni can't offer L1 unbundling. Masataka Ohta
Re: Metro Ethernet, VPLS clarifications
05.02.2013 19:58, Scott Helms ?: Metro-Ethernet is generally the term used to describe Ethernet used as a WAN connection or as a point to point connection. There was at one time the concept of a MAN (Metro Area Network) but metro ethernet is now available in more scenarios than that described. The connectivity can be over fiber or copper and the speed delivered can be as low as a few mbps but commercially available offerings normally start at 5-10 mbps. On the high end its possible to get gigabit and faster connections in certain areas. http://en.wikipedia.org/wiki/Metro_E VPLS stands for Virtual Private Lan Services. This an umbrella technology that allows for the bridging of layer 2 traffic across various layer 2 3 networks. This is generally used as a replacement for a point to point metro ethernet (or other) connection. http://en.wikipedia.org/wiki/VPLS On Mon, Feb 4, 2013 at 11:06 PM, Abzal Sembay serian@gmail.com mailto:serian@gmail.com wrote: Hi experts, I need some clarifications on these terms. Could somebody give explanations or share some links? When and how are these technologies used? Thanks in advance. -- Regards, Abzal -- Scott Helms Vice President of Technology ZCorum (678) 507-5000 http://twitter.com/kscotthelms Thank you, Scott and all of you for your answers and time. From my understanding M-Ethernet is a some kind of service. Standartized technology that allows to connect multiple different networks. And it is independent from physical and datalink layers. And nowadays which tecnology is the most used(VPLS or Metro)? What about MPLS? Sorry I'm a little confused. I really want to understand. -- Regards, Abzal
Re: Metro Ethernet, VPLS clarifications
The Metro Ethernet Forum (MEF) develops standards for Metro Ethernet, which are generally implemented by telcos/cablecos. See the following link: http://metroethernetforum.org/ The 2 biggest problems I have found with telco/cableco MEF services are: 1. In network configurations where all sites are relatively close together ( 500 miles), the telco/cableco SLAs are meaningless, bordering on being fraudulent. For instance SLAs of 50 ms round trip for bronze service, and 20 ms for gold service are enough network transit time to send packets 5000 miles and 2000 miles respectively. This is like buying homeowners' insurance on a $500K house with a $10 million deductible (50 ms SLA), and a more expensive policy has a $5 million deductible (20 ms SLA). 2. The MEF spec does not address directed multicast, as opposed to a native Ethernet switched network which updates the mac tables with each next hop for the multicast requestor (video for instance) tracking the Layer 3 multicast routing protocol shortest path. So in MEF implementations where users view a constant 10 Mbps (for example) multicast video stream between a requestor and a multicast source, this 10 Mbps gets broadcast out all switch ports in a users' MEF VLAN, rendering low speed MEF connections at all other users' locations useless. David On Tue, Feb 5, 2013 at 7:54 PM, Abzal Sembay serian@gmail.com wrote: 05.02.2013 19:58, Scott Helms ?: Metro-Ethernet is generally the term used to describe Ethernet used as a WAN connection or as a point to point connection. There was at one time the concept of a MAN (Metro Area Network) but metro ethernet is now available in more scenarios than that described. The connectivity can be over fiber or copper and the speed delivered can be as low as a few mbps but commercially available offerings normally start at 5-10 mbps. On the high end its possible to get gigabit and faster connections in certain areas. http://en.wikipedia.org/wiki/**Metro_Ehttp://en.wikipedia.org/wiki/Metro_E VPLS stands for Virtual Private Lan Services. This an umbrella technology that allows for the bridging of layer 2 traffic across various layer 2 3 networks. This is generally used as a replacement for a point to point metro ethernet (or other) connection. http://en.wikipedia.org/wiki/**VPLS http://en.wikipedia.org/wiki/VPLS On Mon, Feb 4, 2013 at 11:06 PM, Abzal Sembay serian@gmail.commailto: serian@gmail.com wrote: Hi experts, I need some clarifications on these terms. Could somebody give explanations or share some links? When and how are these technologies used? Thanks in advance. -- Regards, Abzal -- Scott Helms Vice President of Technology ZCorum (678) 507-5000 --**-- http://twitter.com/kscotthelms --**-- Thank you, Scott and all of you for your answers and time. From my understanding M-Ethernet is a some kind of service. Standartized technology that allows to connect multiple different networks. And it is independent from physical and datalink layers. And nowadays which tecnology is the most used(VPLS or Metro)? What about MPLS? Sorry I'm a little confused. I really want to understand. -- Regards, Abzal
Re: Muni fiber: L1 or L2?
Eric Wieling wrote: I don't think it is that much more expensive to allow other ISPs an ATM PVC into their network. Wrong, which is why ATM has disappeared. ATM may not be the best technology to do this, It is not. but the basic concept is not bad. It is not enough, even if you use inexpensive Ethernet. See the subject. What *I* want as an ISP is to connect to customers, You may. However, the customers care cost for you to do so, a lot. L1 unbundling allows the customers to choose an ISP with best (w.r.t. cost, performance, etc.) L2 and L3 technology, whereas L2 unbundling allows ILECs choose stupid L2 technologies such as ATM or PON, which is locally best for their short term revenue, which, in the long run, delays global deployment of broadband environment, because of high cost to the customers. Masataka Ohta
