Re: Cogent Abuse - Bogus Propagation of ASN 36471
On Thu Jul 20 Mike Hammet wrote: > If they (or anyone else) want to give me free service to use as I see fit > (well, legally), I'll gladly accept their offer. I once had free IP transit from Cogent for about a year after I told them to shove it. Not that it did me much good. - Jared
Re: Routed optical networks
On 5/5/23, Mark Tinka wrote: > Juxtapose that against 100Gbps pricing: > > * EUR473 @ 10km. > * EUR1,300 @ 25km. > * EUR1,500 @ 30km. > * EUR2,600 @ 40km. > * EUR3,925 @ 80km. You can get 100G optics for less than half those prices. For reference, here are publicly listed prices for optics from an European vendor I have in production: 100G 4WDM & CLR4 QSFP28 * 10 km * 225€ 100GBASE-LR4 & OTU4 & 128GFC QSFP28 * 20km *305€ 100GBASE-ER4 Lite & OTU4 & 128GFC QSFP28 * 40 km * 1030€ 100GBASE-ZR4 QSFP28 * 80 km * 1650€ 100GBASE-ZR4+ QSFP28 * 100 km * 2100€ These prices are for single units without discounts. - Jared
Re: Coherent 100G in QSFP28
On 2/28/23, Pascal Masha wrote: > How much will these cost? ADVA said 4ish grand each. Probably less than five. - Jared
Coherent 100G in QSFP28
Looks like coherent 100G in the QSFP28 form factor is finally on the horizon. From the datasheet: * 100G coherent DWDM in QSFP28 form factor * tunable, flexible grid * 300 km with amplification, 120 km without * industrial or commercial temperature * 5 watts https://www.adva.com/en/products/open-optical-transport/pluggables-and-subsystems/coherent-100zr - Jared
Re: V6 still not supported
Owen DeLong via NANOG wrote: >> I would expect the trend to become that ISP's refuse to accommodate 3rd >> party vendors shenanigans to the point where it hampers their operations or >> to the point where it cost them more to do so. > > $ISP_1 refuses to accommodate Sony’s shenanigans… > Three possible outcomes: The three possible outcomes assume status quo is maintained. However, if ISP A makes a business decision to not accommodate 3rd party shenanigans and modifies policies accordingly, then we have a new equilibrium. Outcome 1 is maintained: Customer churns off ISP A. Everybody wins. Outcome 2 is no longer a single outcome, but rather several: a. Customer is upsold to gaming package which includes a static IP. b. Customer returns Playstation and buys Xbox instead. c. Customer declines gaming package, but continues to bother customer service. Customer is directed to 3rd party customer support. Further customer contact is handled via self service portals and other low cost customer service channels. d. Customer terminates contract and goes offline. Outcome 3 is resolved by ISP A telling returning customers that service at that address is only available if ordered together with the gaming package. > All of this, of course, becomes an effective non-issue if both $ISP and Sony > deploy IPv6 and get rid of the stupid NAT tricks. Well yes... ... but why would Sony do that when they have so conveniently externalized all costs? - Jared
Re: V6 still not supported
Francis Booth wrote: > I think you’re jumping to conclusions that Sony is doing this purely from the > darkness in their hearts. I confess to being momentously surprised if this wasn't the driving reason :) > The same thing could be said about Netflix and Hulu blocking traffic from > addresses that appear as proxies/VPNs. This is not quite the same. Netflix and Hulu have contractual reasons for not allowing out of market access, as they do not have distribution rights to content in all markets. Then there is also the question of password sharing, which is a legitimate reason to restrict access. IIRC Netflix will still let you watch Netflix originals even if they think you are using a proxy or VPN. They will even occasionally fix misdesignated IP space. > Like it or not we had many years where the primary expectation of the > Internet was that you could map a single ISP customer back to an IP address > and MANY services still cling to this belief. Even the courts are coming around to the fact that an IP address does not equal a person. When even ultraprogressive instances like these are starting to get it, maybe it's time for all the other neanderthals to get with the times? - Jared
Re: V6 still not supported
JORDI PALET MARTINEZ via NANOG wrote: > If I'm a gamer, and one of my possible ISPs is using CGN, and from time to > time stops working, and another ISP is providing me a public and/or static > IPv4 address, always working, and there is not too much price difference, > what I will do? Changing providers only works in a competitive market, but even there a little bit of market segmentation isn't necessarily a bad thing. The main thing is that ISPs should not be so accommodating to these malfeasants, who via their practices make a bad situation worse. Sony et al. are externalizing costs and that shouldn't be accepted. - Jared
Re: V6 still not supported
My apologies for expressing myself poorly. What I meant to say is that this is primarily a problem caused by Sony and the Sonys of the world. Less so a problem inherent to IPv4. A root cause fix would address Sony's hostile behavior. - Jared Jordi Palet wrote: No, isn't only a Sony problem, becomes a problem for every ISP that has customers using Sony PSN and have CGN (NAT444), their IP blocks are black-listed when they are detected as used CGN. This blocking is "forever" (I'm not aware of anyone that has been able to convince PSN to unblock them). Then the ISP will rotate the addresses that are in the CGN (which means some work renumbering other parts of the network). You do this with all your IPv4 blocks, and at some point, you don't have any "not black-listed" block. Then you need to transfer more addresses. So realistically, in many cases, for residential ISPs it makes a lot of sense to analyze if you have a relevant number of customers using PSN and make your numbers about if it makes sense or not to buy CGN vs transfer IPv4 addresses vs the real long term solution, which is IPv6 even if you need to invest in replacing the customer CPEs. Regards, Jordi @jordipalet El 30/3/22, 21:02, "NANOG en nombre de Jared Brown" escribió: Not to necessarily disagree with you, but that is more of a Sony problem than an IPv4 problem. - Jared Jordi Palet wrote: It is not a fixed one-time cost ... because if your users are gamers behind PSP, Sony is blocking IPv4 ranges behind CGN. So, you keep rotating your addresses until all then are blocked, then you need to transfer more IPv4 addresses ... So under this perspective, in many cases it makes more sense to NOT invest in CGN, and use that money to transfer up-front more IPv4 addresses at once, you will get a better price than if you transfer them every few months. Regards, Jordi @jordipalet El 30/3/22, 18:38, "NANOG en nombre de Jared Brown" escribió: Randy Carpenter wrote: > >> >> Owen DeLong via NANOG wrote: > >> >> When your ISP starts charging $X/Month for legacy protocol support > >> > > >> > Out of interest, how would this come about? > >> > >> ISPs are facing ever growing costs to continue providing IPv4 services. > > Could you please be more specific about which costs you are referring to? > > > > It's not like IP transit providers care if they deliver IPv4 or IPv6 bits to > > you. > > Have you priced blocks of IPv4 addresses lately? IPv4 address blocks have a fixed one-time cost, not an ongoing $X/month cost. - Jared
Re: V6 still not supported
> Owen DeLong via NANOG wrote: > When your ISP starts charging $X/Month for legacy protocol support > >>> > >>> Out of interest, how would this come about? > >> > >> ISPs are facing ever growing costs to continue providing IPv4 services. > > Could you please be more specific about which costs you are referring to? > > Costs of address acquisition > Costs of CGNAT systems in lieu of address acquisition costs > Costs of increasing support calls due to IPv4 life support measures in other > networks. > etc. > > > It's not like IP transit providers care if they deliver IPv4 or IPv6 bits > > to you. > > True, but adding customers requires additional addresses at some point. IPv6 > addresses are cheap compared to IPv4 addresses. As an aside, all this demonstrates quite well one of the impediments to accelerated IPv6 adoption: None of these costs apply to parties not growing or ones that are only growing withing their existing IPv4 allocation. The status quo does not promote IPv6 adoption, which is obviously a problem since transitioning to IPv6-only requires all parties to be aboard. I'll even add that there is a perverse incentive for ISPs and others to delay IPv6 adoption in certain segments. As there is a scarcity of IPv4, ISPs can charge a premium for access to IPv4 addresses, something you cannot do with IPv6. Furthermore as IPv4 blocks are acting like an appreciating asset, there is both an incentive to acquire more, regardless of need, and to hoard what you have, even if you don't need it. For cloud providers your IPv4 blocks become your moat. - Jared
Re: V6 still not supported
Not to necessarily disagree with you, but that is more of a Sony problem than an IPv4 problem. - Jared Jordi Palet wrote: It is not a fixed one-time cost ... because if your users are gamers behind PSP, Sony is blocking IPv4 ranges behind CGN. So, you keep rotating your addresses until all then are blocked, then you need to transfer more IPv4 addresses ... So under this perspective, in many cases it makes more sense to NOT invest in CGN, and use that money to transfer up-front more IPv4 addresses at once, you will get a better price than if you transfer them every few months. Regards, Jordi @jordipalet El 30/3/22, 18:38, "NANOG en nombre de Jared Brown" escribió: Randy Carpenter wrote: > >> >> Owen DeLong via NANOG wrote: > >> >> When your ISP starts charging $X/Month for legacy protocol support > >> > > >> > Out of interest, how would this come about? > >> > >> ISPs are facing ever growing costs to continue providing IPv4 services. > > Could you please be more specific about which costs you are referring to? > > > > It's not like IP transit providers care if they deliver IPv4 or IPv6 bits to > > you. > > Have you priced blocks of IPv4 addresses lately? IPv4 address blocks have a fixed one-time cost, not an ongoing $X/month cost. - Jared
Re: RE: CGNAT scaling cost (was V6 still not supported)
Hi Eduard, Do I interpret your findings correctly, if this means that CGNAT costs scale more or less linearly with traffic growth over time? And as a corollary, that the cost of scaling CGNAT in itself isn't likely a primary driver for IPv6 adoption? - Jared Vasilenko Eduard wrote: > > CGNAT cost was very close to 3x compared to routers of the same performance. > Hence, 1 hop through CGNAT = 3 hops through routers. > 3 router hops maybe the 50% of overall hops in the particular Carrier (or > even less). > > DWDM is 3x more expensive per hop. Fiber is much more expensive (greatly > varies per situation and distance). > Hence, +50% for IP does not mean +50% for the whole infrastructure, not at > all. > > I was on all primary vendors for 2.5 decades. 3x cost of NAT was consistent > for all vendors and at all times. > Because it is a "Network processor" (really flexible one with a big memory) > against "specialized ASIC". COTS (x86) is much worse for the big scale - does > not make sense to compare. > It has started to decrease recently when SFPs have become the bigger part of > the router (up to 50% for single-mode). > Hence, I expect the decrease of the difference between router and CGNAT cost > to 2x long-term. > Optical vendors are more capable to protect their margins. > > It is a different situation in Mobile Carriers, where Packet Core and Gi-LAN > were never accelerated in hardware. > Everything else is so expensive (x86) per Gbps, that CGNAT is not visible in > the cost. > > Eduard > -Original Message- > From: NANOG [mailto:nanog-bounces+vasilenko.eduard=huawei@nanog.org] On > Behalf Of Jared Brown > Sent: Wednesday, March 30, 2022 6:33 PM > To: nanog@nanog.org > Subject: CGNAT scaling cost (was Re: V6 still not supported) > > An oft-cited driver of IPv6 adoption is the cost of scaling CGNAT or > equivalent infrastructure for IPv4. > > Those of you facing costs for scaling CGNAT, are your per unit costs rising > or declining faster or slower than your IPv4 traffic growth? > > I ask because I realize I am not fit to evaluate the issue on a general > level, as, most probably due to our insignificant scale, our CGNAT marginal > costs are zero. This is mainly because our CGNAT solution is oversized to our > needs. Even though scaling up our currently oversized system further would > lower per unit costs, I understand this may not be the case outside our > bubble. > > > - Jared >
Re: V6 still not supported
Randy Carpenter wrote: > >> >> >> Owen DeLong via NANOG wrote: > >> >> >> When your ISP starts charging $X/Month for legacy protocol support > >> >> > > >> >> > Out of interest, how would this come about? > >> >> > >> >> ISPs are facing ever growing costs to continue providing IPv4 services. > >> > Could you please be more specific about which costs you are referring > >> > to? > >> > > >> > It's not like IP transit providers care if they deliver IPv4 or IPv6 > >> > bits to > >> > you. > >> > >> Have you priced blocks of IPv4 addresses lately? > > IPv4 address blocks have a fixed one-time cost, not an ongoing $X/month > > cost. > > > > - Jared > > How, exactly, would you propose a company recoup the cost? There are many options, depending on the commercial relationship between ISP and customer. The ISP may simply charge a single one-time fee per IPv4. The customer may choose to bring their own IPv4 blocks as many BGP customers do. The ISP may chose not to charge separately per IPv4, as having those IPs enables them to charge $Y/month for Internet service. And so on and so forth. Furthermore IPv4 addresses do not wear out. IPs can be reused upon customer churn and excess blocks can be sold, if need be. - Jared
Re: V6 still not supported
Randy Carpenter wrote: > >> >> Owen DeLong via NANOG wrote: > >> >> When your ISP starts charging $X/Month for legacy protocol support > >> > > >> > Out of interest, how would this come about? > >> > >> ISPs are facing ever growing costs to continue providing IPv4 services. > > Could you please be more specific about which costs you are referring to? > > > > It's not like IP transit providers care if they deliver IPv4 or IPv6 bits > > to > > you. > > Have you priced blocks of IPv4 addresses lately? IPv4 address blocks have a fixed one-time cost, not an ongoing $X/month cost. - Jared
CGNAT scaling cost (was Re: V6 still not supported)
An oft-cited driver of IPv6 adoption is the cost of scaling CGNAT or equivalent infrastructure for IPv4. Those of you facing costs for scaling CGNAT, are your per unit costs rising or declining faster or slower than your IPv4 traffic growth? I ask because I realize I am not fit to evaluate the issue on a general level, as, most probably due to our insignificant scale, our CGNAT marginal costs are zero. This is mainly because our CGNAT solution is oversized to our needs. Even though scaling up our currently oversized system further would lower per unit costs, I understand this may not be the case outside our bubble. - Jared
Re: V6 still not supported
Owen DeLong via NANOG wrote: > >> When your ISP starts charging $X/Month for legacy protocol support > > > > Out of interest, how would this come about? > > ISPs are facing ever growing costs to continue providing IPv4 services. Could you please be more specific about which costs you are referring to? It's not like IP transit providers care if they deliver IPv4 or IPv6 bits to you. - Jared
Re: V6 still not supported
Doug McIntyre wrote: > > Jared Brown wrote: > > > Owen DeLong via NANOG wrote: > > > When your ISP starts charging $X/Month for legacy protocol support > > > > Out of interest, how would this come about? > > It already happens, more along the lines of "Business Class" vs. "Residential > Class". > > Ie. for Residential Class, you may get put onto CGNAT, and have no control > over that. > > While on x level of Business Class, you get to opt out of CGNAT, and > potentially even have a > static IP address assigned to your connection. I find this example to be somewhat contrived. Both classes have access to IPv4 and thus there is no surcharge for "legacy protocol support". Furthermore these are not operator services nor are the prices for these services tightly coupled to actual production costs. - Jared
MAP-T (was: Re: V6 still not supported)
Most IPv6 transition mechanisms involve some form of (CG)NAT. After watching a NANOG presentation on MAP-T, I have a question regarding this. Why isn't MAP-T more prevalent, given that it is (almost) stateless on the provider side? Is it CPE support, the headache of moving state to the CPE, vendor support, or something else? NANOG 2017 Mapping of Address and Port using Translation MAP T: Deployment at Charter Communications https://www.youtube.com/watch?v=ZmfYHCpfr_w - Jared
Re: V6 still not supported
Owen DeLong via NANOG wrote: > When your ISP starts charging $X/Month for legacy protocol support Out of interest, how would this come about? - Jared
Re: Russia to disconnect from global Internet
Accidentally put the wrong link for the translation. Here is the correct link to the machine translation: https://twitter.com/JiriVysin/status/1500560017640067077 -- According to Nexta (Belorussian media outlet: https://nexta.tv , https://en.wikipedia.org/wiki/Nexta ) Russia has begun active preparations to disconnection from the global Internet. No later than March 11, all servers and domains must be transferred to the Russian zone. In addition, detailed data on the network infrastructure of the sites is being collected. Source: https://twitter.com/nexta_tv/status/1500553480548892679 Machine translation of decree: https://twitter.com/nexta_tv/status/1500553480548892679 Cogent exiting the Russian market is probably not related, but interesting nevertheless. - Jared
Russia to disconnect from global Internet
According to Nexta (Belorussian media outlet: https://nexta.tv , https://en.wikipedia.org/wiki/Nexta ) Russia has begun active preparations to disconnection from the global Internet. No later than March 11, all servers and domains must be transferred to the Russian zone. In addition, detailed data on the network infrastructure of the sites is being collected. Source: https://twitter.com/nexta_tv/status/1500553480548892679 Machine translation of decree: https://twitter.com/nexta_tv/status/1500553480548892679 Cogent exiting the Russian market is probably not related, but interesting nevertheless. - Jared
Re: 25G SFP28 capable of rate-adaption down to 1G?
Mikrotik claims a multirate 1G / 10G / 25G SFP28 https://mikrotik.com/product/xs_31lc10d - Jared
Re: S.Korea broadband firm sues Netflix after traffic surge
I don't know what they are putting in the water in Korea, but strange things are reported from there. In addition to the SK Telecom shenanigans, apparently KT can't tell the difference between a DDoS and a routing error. https://en.yna.co.kr/view/AEN20211025006253320 - Jared
Re: S.Korea broadband firm sues Netflix after traffic surge
Not to be outdone, British Telecom joins the cephalopod games: “Every Tbps (terabit-per-second) of data consumed over and above current levels costs about £50m,” says Marc Allera, the chief executive of BT’s consumer division. “In the last year alone we’ve seen 4Tbps of extra usage and the cost to keep up with that growth is huge.” “When the rules were created 25 years ago I don’t think anyone would have envisioned four or five companies would be driving 80% of the traffic on the world’s internet. They aren’t making a contribution to the services they are being carried on; that doesn’t feel right.” “A lot of the principles of net neutrality are incredibly valuable, we are not trying to stop or marginalise players but there has to be more effective coordination of demand than there is today” https://www.theguardian.com/business/2021/oct/10/squid-games-success-reopens-debate-over-who-should-pay-for-rising-internet-traffic-netflix For reference British Telecom has about 10 million broadband subscribers, so apparently those £200m capacity upgrades are stinging. All in all, this raises an interesting question. Is British Telecom running their networks so hot, that just keeping the lights on requires capacity upgrades or are they just looking for freebies? - Jared
Re: S.Korea broadband firm sues Netflix after traffic surge
Doug Barton wrote: > One incentive I haven't seen anyone mention is that ISPs don't want to > charge customers what it really costs to provide them access. For the sake of argument, let's assume this is true. For this to work, I am really trying hard to ignore inconvenient facts like: "South Korea’s SK Telecom (SKT) has reported operating revenues of KRW4.818 trillion (USD4.2 billion) for the quarter ended 30 June 2021, up 4.7% year-on-year, with it saying that the increase was ‘due to continued solid growth trends in all business areas’. SKT’s operating income in Q2 2021 totalled KRW397 billion, up 10.8% on an annualised basis..." https://www.commsupdate.com/articles/2021/08/12/sk-telecom-reports-revenue-increase-in-2q21-as-5g-subscriber-numbers-rise/ Nevertheless, let's go with the hypothesis that service is provided below cost. Providing access is mostly fixed costs, as there are very few consumables in running a network. IP transit costs aren't an issue, since Netflix will do settlement free peering. This leaves the internal network of SK Telecom as the problem and cost center. There would have been no marginal cost if SK Telecom's own network was capable of handling the traffic of its customers. So basically SK Telecom is mad at Netflix for forcing equipment upgrades faster than budgeted. Should Netflix have to pay for SK Telecom sucking at traffic planning and budgeting? - Jared
Re: S.Korea broadband firm sues Netflix after traffic surge
Mark Tinka wrote: >> Well, yes. Or you could just stream content that is guaranteed to be >> compatible with the device used. > > People on this list would bother to check compatibility. > > Jane + Thatho just point & click. Since we aren't talking about random pirated content, but p2p streaming from a major content provider it would obviously be point & click. - Jared
Re: S.Korea broadband firm sues Netflix after traffic surge
Mark Tinka wrote: > Someone can correct me if I'm wrong, but the way I know BitTorrent to > work is the file is downloaded to disk, unarchived and then listed as > ready to watch. That's not how it works. Several streaming BitTorrent clients specifically request blocks in order so that you can start watching immediately. Not that you need a special client, it works pretty well with the standard client as well on a well seeded torrent, as blocks are generally requested more or less in order. > It also assumes the device has all the necessary apps > and codecs needed to render the file. Well, yes. Or you could just stream content that is guaranteed to be compatible with the device used. > On the other hand, BitTorrent could just make an Apple > TV/PS4/PS5/Xbox/whatever-device-you-use app as well. They could, and they might even have, I forget, but there is little demand for such a thing as a centralized CDN strategy works better. > But I doubt that > will work, unless someone can think up a clever way to modify BitTorrent > to suit today's network architectures. Unless network topology is somehow exposed, this isn't possible. All anybody can do is use latency, IP and ASN information as a proxy. Nothing is stopping a BitTorrent client from being selective about its peers. The current peer selection algorithm optimizes for throughput, not adjecency or topology. - Jared
Re: New minimum speed for US broadband connections
On Sunday, 30 May 2021 Mike Hammet wrote: > Why 100/100? Because subsidies should only be used for long term solutions. The definition of broadband is mainly relevant to determine who should receive subsidies. Commercial broadband has already far surpassed the minimums. - Jared
Muni broadband sucks (was: New minimum speed for US broadband connections)
On Mon, May 31, 2021 Mike Hammett wrote: > Muni broadband does suck, but that's another thread for another day. Excluding cases where muni broadband doesn't suck, why does muni broadband suck? Personally I wouldn't mind more access to dark fiber à la Stokab, much like the dry copper pairs of yesterday. If the default state of muni broadband of is suck, what is the root cause? Is it a people problem and/or can something be done to improve on the default state? Jared
Re: How to Fix IP GEO for google/youtube tv
Grant Taylor wrote: > The process takes multiple weeks. Out of interest, why does it take multiple weeks to edit a GEO IP entry? I wonder why Google even has this problem at all. If you've so much as looked at Google maps or used any app that uses location services then Google knows with a high certainty where you are. This is a problem Google actually *can* automate away. Jared
Re: CGNAT
Kevin, One of the presented options isn't like the others. As such the comparison isn't really fair, especially if you expect to run your business longer than 7 years. If you buy more IPv4 space you will neither have to deal with CGNAT nor worry about traffic growth. Both of those benefits are easily worth the (short term) premium. In the long term, buying more IPv4 blocks now is likely to be cheaper than running CGNAT for the foreseeable future. To echo Owen, in general, the economics today still work out to make purchasing addresses more favorable than CGNAT. - Jared Sent: Tue Feb 2314:36:48 UTC 2021 From: Kevin Burke kburke at burlingtontelecom.com To: nanog@nanog.org Subject: Re: CGNAT We are looking at implementing a similar solution with A10 for CGNAT. We've been in touch with A10. Just wondering if there are some alternative vendors that anyone would recommend. We'd probably be looking at a solution to support 5k to 15k customers and bandwidth up to around 30-40 gig as a starting point. A solution that is as transparent to user experience as possible is a priority. The numbers below are for a similar target of subscriber’s and peak bandwidth. We assumed a couple of numbers: Current Peak Bandwidth = 40G Remaining IPv4 traffic after migration = 20% (Seen references to 10% or 20% on this forum) Future Bandwidth Growth = 2x (no data behind this assumption) Future CGNAT’ed bandwidth = 15Gbps Equipment & budget lifecycle = 7Yr Getting that data led us to this price comparison: Solution Lifecycle/ Term Annual Cost/Sub Product Lifecycle Cost/Sub Lease IPv4 Cogent 7 $ 4.45 $ 31.13 A10 CGNAT 15Gb 7Yr 7 $ 1.21 $ 8.47 A10 CGNAT 40Gb 7Yr 7 $ 1.95 $ 13.68 Purchase @ $25 7Yr 7 $ 3.57 $ 25.00 The current plan is implement an A10 CGNAT solution after upgrading our network for IPv6. In the interim we will have to lease IPv4 to tide us over. I would be curious to see what other’s estimate the costs of various approaches. Feel free to ping me off-list for more specific numbers. Kevin Burke 802-540-0979 Burlington Telecom 200 Church St, Burlington, VT
Global Peer Exchange
Hello NANOG! Does anybody have anything, good or bad, to say about Cogent's Global Peer Exchange? Jared
Re: 100G over 100 km of dark fiber
Turns out I was wrong, again. There is at least one vendor that offers pluggable 100G QSFP28 optics that can reach 120 km with amplification. Thanks to everybody that reached out! I appreciate all the input and lessons learned. Jared --- Hello NANOG! I need to push 100G over 100 km of dark fiber. Since there are no 100G pluggable optics with this reach (~25 dB), I have been offered coherent transport systems to solve my problem. This is all good and well, except total system costs start from high five figures. So, my question is, do I have any other options? I can't help noticing that you can break out a 100G QSFP into four 25G QSFPs. 25G DWDM systems are relatively inexpensive (low five figures), but can you make 25G DWDM go 100 km? I only need the one 100G, so I don't really need a highly scalable DWDM system. I can't put anything midspan, or if I could it would cost more than just going with a coherent system. Jared
Re: 100G over 100 km of dark fiber
The 100 km leg completes a ring. Jared Sent: Friday, October 30, 2020 From: "Ben Cannon" To: "Jared Brown" Cc: nanog@nanog.org Subject: Re: 100G over 100 km of dark fiber You could break this into 10x 10g coherent lanes, but you’re going to end up back close to coherent 100g prices. You’re at the threshold distance where you’re past all the short range tech and are seriously pushing it - whereas the 100g coherent tech is just taking off. How important is this link? Ms. Benjamin PD Cannon, ASCE 6x7 Networks & 6x7 Telecom, LLC CEO b...@6by7.net[mailto:b...@6by7.net] "The only fully end-to-end encrypted global telecommunications company in the world.” FCC License KJ6FJJ On Oct 30, 2020, at 7:19 AM, Jared Brown wrote: Hello NANOG! I need to push 100G over 100 km of dark fiber. Since there are no 100G pluggable optics with this reach (~25 dB), I have been offered coherent transport systems to solve my problem. This is all good and well, except total system costs start from high five figures. So, my question is, do I have any other options? I can't help noticing that you can break out a 100G QSFP into four 25G QSFPs. 25G DWDM systems are relatively inexpensive (low five figures), but can you make 25G DWDM go 100 km? I only need the one 100G, so I don't really need a highly scalable DWDM system. I can't put anything midspan, or if I could it would cost more than just going with a coherent system. Jared
100G over 100 km of dark fiber
Hello NANOG! I need to push 100G over 100 km of dark fiber. Since there are no 100G pluggable optics with this reach (~25 dB), I have been offered coherent transport systems to solve my problem. This is all good and well, except total system costs start from high five figures. So, my question is, do I have any other options? I can't help noticing that you can break out a 100G QSFP into four 25G QSFPs. 25G DWDM systems are relatively inexpensive (low five figures), but can you make 25G DWDM go 100 km? I only need the one 100G, so I don't really need a highly scalable DWDM system. I can't put anything midspan, or if I could it would cost more than just going with a coherent system. Jared
Getting Fiber to My Town by Jared Mauch
I believe this belongs here: Getting Fiber to My Town by Jared Mauch https://www.youtube.com/watch?v=ASXJgvy3mEg (YouTube video of NLnog presentation) https://nlnog.net/static/live/nlnog_live_sep_2020_jared.pdf (slides for presentation) https://news.ycombinator.com/item?id=24424910#24430901 (discussion on Hacker News with Jared participating) https://washftth.com/ (project homepage) I find this an interesting description of how to apply skills that we normally only use at work to solve connectivity issues at home. Quite timely too, as home connectivity is needed more than ever. Highlights: - location: just outside of Ann Arbor - no fixed broadband since 2002 -> build own network - pre-wire neighbors with fiber drops and feed them off WISP first - lots of work to sign up customers, having to resort to snailmail to reach all - 70% of homes passed signed up - Have ASN, get IPv6 and IPv4 allocation, multihome and connect to local IXP - purchase equipment: fusion splicer, OTDR, materials, directional drill(!) - hire contractors, deal with all manner of problems, theft, stop work orders, unbudgeted costs, unmarked/badly marked utilities, hitting (own) utilities - build own fiber blower(!) for blowing in fiber in ducts - splice, OTDR, resplice, schedule installs ... which don't always go to plan - Upstream very helpful, offered a Cisco 6500 as CPE, however respectfuly declined and went with Arista - Link up! Network is now live with 17 subscribers hooked up. More waiting to be connected - Mixed Active Ethernet and GPON - latency drops from 30 ms to 8 ms, bandwidth from 20-30M to 730M, total commit 1.5G on 10G port, plans from 50M to 500M - SPAM! IPv4 brokers and the usual unsolicited contacts from bottom of the baller IP transit providers - Costs: $126k in 2020, $95k contractors, $32k materials and equipment. Total outlay ~$150k + years of sweat equity. Important to spread out costs over longer period of time to be able to afford. Offset costs by using pre-pay model (Can pay $5,000 up-front and receive $50 credit for 100 months) All in all it was an excellent presentation. I only wish Jared had spent some more time on how he had to become a telco and how this got him better access to the public right of way. Of course, some more details on his directional boring and some nice video of him running the drill would have been a cherry on top :) I'd like to congratulate Jared on lighting up his network and wish him success in running it. I did a similar build almost twenty years ago and regret I didn't have the forethought to document the effort better then. Not that there was a YouTube to put it all on then :) You can find the other presentations from NLnog live September 2020 at https://www.youtube.com/channel/UCVz78FbsOJ6v6xb6S2GpvWQ Jared, not the Mauch one
Re: Router Suggestions
Sent: Tuesday, June 16, 2020 From: "Matt Harris" >> On Tue, Jun 16, 2020 at 9:52 AM Jared Brown >> mailto:nanog-...@mail.com]> wrote: >> My no-effort quote from last month lists just the box at $13,000. Once you >> are all in the total is that 1.5 multiple Baldur mentioned compared to OP. >> >> However, if you google "mx204 price" the first hit wants very much to sell >> you one for <$11,000. Caveat emptor and YMMV. >> >> Jared >> > Not all MX204's are created equal, however. For edge applications, many folks > will want to go with the -IR model, and the -R model is the > fully-unrestricted one. > These will cost substantively more than the base model which has rib, fib, > and vrf limitations enforced. True enough. I was, however, under the impression you could upgrade the license at a later date. Jared
Re: Partial vs Full tables
From: Mike Hammett Date: Fri, 5 Jun 2020 08:17:26 -0500 (CDT) > I've been wondering a similar thing for how to take advantage of the 150k - > 250k hardware routes the CRS317 now has in > v7 beta. That many routes should cover the peering tables for most operators, > maybe even transit's customers. Perhaps the thing you are looking for is SIR - the SDN Internet Router https://github.com/dbarrosop/sir TLDR; use pmacct to grab top N ASN speakers and install them into the FIB. Rinse and repeat. Alternatively filter out anything not from ARIN. Conveniently fits into 250k: Prefixes being announced by ARIN Region ASes:238331 http://gregsowell.com/?p=5505 Should you want something that isn't quite so bleeding edge beta, perhaps pick up a used Arista 7050QX? It's about the same as a CRS317 and holds 144k routes. Jared
RE: Router Suggestions
My no-effort quote from last month lists just the box at $13,000. Once you are all in the total is that 1.5 multiple Baldur mentioned compared to OP. However, if you google "mx204 price" the first hit wants very much to sell you one for <$11,000. Caveat emptor and YMMV. Jared > Yes I too looked into that. And it was not near that price.. Please send me > and email off list. I would like to know where I might find that. > >> On Mon, Jun 15, 2020 at 2:58 PM Forrest Christian (List Account) >> wrote: >> >> We just got a MX204 quote and it was close to 2.5x the price you're quoting, >> with apparently the minimum license needed for full tables, and Next Day >> replacement. >> So if it's really $11K, please shoot me an email off list. Or if someone >> has a better place to get a decent quote for a MX204, or can clarify where >> this quote >> might have went wrong, that would be useful too. >> >> We're also looking at going the virtual router route where we put 2-3 >> servers in a HA cluster loaded up with 10Gb interfaces and running some sort >> of routing software. >> In case you didn't catch on, I'm >> fairly early in running this idea >> through the paces, although it seems like this is a pretty common thing >> nowadays. >> >>>On Mon, Jun 15, 2020 at 6:02 AM Colton Conor >>> wrote: >>> >>>For around $11,000 right now, you can get a brand new Juniper MX204 >>> router. Alternatively, you can get a used MX240 / MX480 with quad power >>> supplies, redundant quad core RE's, and 2 16X10G MIC cards >>>for around >>> $12,000. >>> >>>My question, is there anything else worth looking at in this price >>> range / port configuration? Open to both new and used options. Looking to >>> take full BGP routes. >>> >> >> >> >> -- >> - Forrest
An appeal for more bandwidth to the Internet Archive
Hello all! Last week the Internet Archive upgraded their bandwidth 30% from 47 Gbps to 62 Gbps. It was all gobbled up immediately. There's a lovely solid green graph showing how usage grows vertically as each interface comes online until it too is 100% saturated. Looking at the graph legend you can see that their usage for the past 24 hours averages 49.76G on their 50G of transport. To see the pretty pictures follow the below link: https://blog.archive.org/2020/05/11/thank-you-for-helping-us-increase-our-bandwidth/ Relevant parts from the blog post: "A year ago, usage was 30Gbits/sec. At the beginning of this year, we were at 40Gbits/sec, and we were handling it. ... Then Covid-19 hit and demand rocketed to 50Gbits/sec and overran our network infrastructure’s ability to handle it. So much so, our network statistics probes had difficulty collecting data (hence the white spots in the graphs). We bought a second router with new line cards, and got it installed and running (and none of this is easy during a pandemic), and increased our capacity from 47Gbits/sec peak to 62Gbits/sec peak. And we are handling it better, but it is still consumed." It is obvious that the Internet Archive needs more bandwidth to power the Wayback machine and to fulfill its mission of being the Internet library and the historic archive of our times. The Internet Archive is present at Digital Realty SFO (200 Paul) and a member of the San Francisco Metropolitan Internet Exchange (SFMIX). I appeal to all list members present or capable of getting to these facilities to peer with and/or donate bandwidth to the Internet Archive. I appeal to all vendors and others with equipment that they can donate to the Internet Archive to contact them so that they can scale their services and sustain their growth. The Internet Archive is currently running 10G equipment. If you can help them gain 100G connectivity, 100G routing, 100G switching and/or 100G DWDM capabilities, please reach out to them. They have the infrastructure and dark fiber to transition to 100G, but lack the equipment. You can find the Internet Archive's contact information below or you can contact Jonah at the Archive Org directly either by email or via the contact information available on his Twitter profile @jonahedwards. You can also donate at https://archive.org/donate/ The Internet Archive is a 501(c)(3) non-profit. Donations are tax-deductible. Contact information: https://archive.org/about/contact.php Volunteering: https://archive.org/about/volunteerpositions.php Disclaimer: I am not affiliated with the Internet Archive. Nobody asked me to write this post. If something angers you about this post, be angry at me. I merely think that the Internet Archive is a good thing and deserves our support. Jared
Cloudflare "Magic" IP Transit
Hello NANOG! Does anybody have any experience with Cloudflare's "Magic" IP Transit? Good, bad or ugly? Jared
Re: BGP prefix filter list
There are a few approaches to culling the routing table. You can do it either statically or dynamically, according to your needs. 1. Filtering based on upstream communities Slimming down the Internet routing table https://www.redpill-linpro.com/sysadvent/2016/12/09/slimming-routing-table.html 2. Filtering based on region BGP filter for North American routes http://gregsowell.com/?p=5505 Substitute prefixes for applicable region(s). Each region is about 200k prefixes. For more granularity use a geolocation service to select prefixes and/or ASNs. 3. Using flow information to install only top routes SDN Internet Router – Part 2 https://labs.spotify.com/2016/01/27/sdn-internet-router-part-2/ https://blog.ipspace.net/2015/01/sdn-router-spotify-on-software-gone-wild.html 4. Aggregate the routing table According to the weekly routing table report you can aggregate announcements to about half the number of prefixes. You need to roll your own software to preprocess the BGP feed. There are some tools out there, but I couldn't find a blog post about it with a quick search. If you have one, please share! Jared On 05/15/19 13:43 +0200, Baldur Norddahl wrote: >Hello > >This morning we apparently had a problem with our routers not handling >the full table. So I am looking into culling the least useful prefixes >from our tables. I can hardly be the first one to take on that kind of >project, and I am wondering if there is a ready made prefix list or >similar? > >Or maybe we have a list of worst offenders? I am looking for ASN that >announces a lot of unnecessary /24 prefixes and which happens to be >far away from us? I would filter those to something like /20 and then >just have a default route to catch all. > >Thanks, > >Baldur >
