Re: Peering and Network Cost
On 5/23/15 10:23 AM, Jay Ashworth wrote: - Original Message - From: Dave Taht dave.t...@gmail.com Two things I am curious about are 1) What is the measured benefit of moving a netflix server into your local ISP network and 2) does anyone measure cross town latency. If we lived in a world where skype/voip/etc transited the local town only, what sort of latencies would be see within an ISP and within a cross-connect from, say a gfiber to a comcast? Once upon a time I'd heard that most phone calls were within 6 miles of the person's home, but I don't remember the breakdown of those call percentages (?), and certainly the old-style phone system was achieving very low latencies for those kinds of traffic. The lack of decent geographic locality of reference on the Internet has bothered me for some time; it's often presented as an *effect* of the eyeballs/servers nature of the net, but I'm not at all sure it's not more a cause of it -- at least at this late date. if you're using DNS based GTM to localize access to an application service or CDN it's going to be localized to the resolver being employed. short of something like: https://tools.ietf.org/html/draft-ietf-dnsop-edns-client-subnet-00 The problem, of course, is that carriers make money off transit; it's not in their commercial best interest to unload those links; it's very similar to the reason my best friend's second semester pre-law textbooks cost her nearly $1000; the people selecting them have no interest in the price, since they don't pay it. Cheers, -- jra signature.asc Description: OpenPGP digital signature
Re: Peering and Network Cost
- Original Message - From: Dave Taht dave.t...@gmail.com Two things I am curious about are 1) What is the measured benefit of moving a netflix server into your local ISP network and 2) does anyone measure cross town latency. If we lived in a world where skype/voip/etc transited the local town only, what sort of latencies would be see within an ISP and within a cross-connect from, say a gfiber to a comcast? Once upon a time I'd heard that most phone calls were within 6 miles of the person's home, but I don't remember the breakdown of those call percentages (?), and certainly the old-style phone system was achieving very low latencies for those kinds of traffic. The lack of decent geographic locality of reference on the Internet has bothered me for some time; it's often presented as an *effect* of the eyeballs/servers nature of the net, but I'm not at all sure it's not more a cause of it -- at least at this late date. The problem, of course, is that carriers make money off transit; it's not in their commercial best interest to unload those links; it's very similar to the reason my best friend's second semester pre-law textbooks cost her nearly $1000; the people selecting them have no interest in the price, since they don't pay it. Cheers, -- jra -- Jay R. Ashworth Baylink j...@baylink.com Designer The Things I Think RFC 2100 Ashworth Associates http://www.bcp38.info 2000 Land Rover DII St Petersburg FL USA BCP38: Ask For It By Name! +1 727 647 1274
Re: Peering and Network Cost
James, curious to know... what size ISPs are they? In the last few years with the larger ones it has always been about lowering cost and increasing revenue, which throws the original idea of peering out the window (unless you are willing to pay). On Thu, May 21, 2015 at 4:52 AM, James Bensley jwbens...@gmail.com wrote: On 17 April 2015 at 16:53, Justin Wilson - MTIN li...@mtin.net wrote: Peering and peering on an exchange are two different things. Peering at an exchange has several benefits other than the simple cost of transit. If you are in a large data center which charges fees for cross connects a single cross connect to an exchange can save you money. Peering can also be a sales tool. If you buy from a VOIP provider and are peered with them your latency and such will go down. You also have more control over the QOS over that peer. This can be spun into marketing. Not to toot our own horn but we put together a list of benefits for our IX customers: http://www.midwest-ix.com/blog/?p=15 Also, a good article at: http://blog.webserver.com.my/index.php/the-benefits-of-hosting-at-internet-exchange-point/ I also have a similar working document that I'd welcome feedback on to improve; https://docs.google.com/document/d/1i2bPZDt75hAwcR4iKMqaNSGIeM-nJSWLZ6SLTTnuXNs/edit?usp=sharing I've used it once to help an ISP evalutate peering and started them in the world of public peering. I'm now going through that proces again with another ISP and again they will start public peering soon, having used this doc in both cases as an intro/FAQ for them. Cheers, James.
Re: Peering and Network Cost
As a small ISP, I'll peer with everybody possible. ;-) It's mostly about cost, but the quality goes up as well. Some of the people we're working with saw an increase in consumption the moment they joined IXes. The quality of the connections improved, so the streaming video (assumed) was able to flow at a higher bit-rate. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com - Original Message - From: Rafael Possamai raf...@gav.ufsc.br To: James Bensley jwbens...@gmail.com Cc: nanog@nanog.org Sent: Thursday, May 21, 2015 7:40:23 AM Subject: Re: Peering and Network Cost James, curious to know... what size ISPs are they? In the last few years with the larger ones it has always been about lowering cost and increasing revenue, which throws the original idea of peering out the window (unless you are willing to pay). On Thu, May 21, 2015 at 4:52 AM, James Bensley jwbens...@gmail.com wrote: On 17 April 2015 at 16:53, Justin Wilson - MTIN li...@mtin.net wrote: Peering and peering on an exchange are two different things. Peering at an exchange has several benefits other than the simple cost of transit. If you are in a large data center which charges fees for cross connects a single cross connect to an exchange can save you money. Peering can also be a sales tool. If you buy from a VOIP provider and are peered with them your latency and such will go down. You also have more control over the QOS over that peer. This can be spun into marketing. Not to toot our own horn but we put together a list of benefits for our IX customers: http://www.midwest-ix.com/blog/?p=15 Also, a good article at: http://blog.webserver.com.my/index.php/the-benefits-of-hosting-at-internet-exchange-point/ I also have a similar working document that I'd welcome feedback on to improve; https://docs.google.com/document/d/1i2bPZDt75hAwcR4iKMqaNSGIeM-nJSWLZ6SLTTnuXNs/edit?usp=sharing I've used it once to help an ISP evalutate peering and started them in the world of public peering. I'm now going through that proces again with another ISP and again they will start public peering soon, having used this doc in both cases as an intro/FAQ for them. Cheers, James.
Re: Peering and Network Cost
On 17 April 2015 at 16:53, Justin Wilson - MTIN li...@mtin.net wrote: Peering and peering on an exchange are two different things. Peering at an exchange has several benefits other than the simple cost of transit. If you are in a large data center which charges fees for cross connects a single cross connect to an exchange can save you money. Peering can also be a sales tool. If you buy from a VOIP provider and are peered with them your latency and such will go down. You also have more control over the QOS over that peer. This can be spun into marketing. Not to toot our own horn but we put together a list of benefits for our IX customers: http://www.midwest-ix.com/blog/?p=15 Also, a good article at: http://blog.webserver.com.my/index.php/the-benefits-of-hosting-at-internet-exchange-point/ I also have a similar working document that I'd welcome feedback on to improve; https://docs.google.com/document/d/1i2bPZDt75hAwcR4iKMqaNSGIeM-nJSWLZ6SLTTnuXNs/edit?usp=sharing I've used it once to help an ISP evalutate peering and started them in the world of public peering. I'm now going through that proces again with another ISP and again they will start public peering soon, having used this doc in both cases as an intro/FAQ for them. Cheers, James.
RE: Peering and Network Cost
We went that way too about 2 years ago. We usually pass around 25 to 40% of our North American traffic to the 4 IXes we're connected at a very low cost in Toronto and Montreal. One of the biggest IX we're connected to in New York is almost the same price per Mbps as some cheap transit providers but we're keeping our port for the connectivity improvement. Eric -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Mike Hammett Sent: May 21, 2015 8:50 AM To: nanog@nanog.org Subject: Re: Peering and Network Cost As a small ISP, I'll peer with everybody possible. ;-) It's mostly about cost, but the quality goes up as well. Some of the people we're working with saw an increase in consumption the moment they joined IXes. The quality of the connections improved, so the streaming video (assumed) was able to flow at a higher bit-rate. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com - Original Message - From: Rafael Possamai raf...@gav.ufsc.br To: James Bensley jwbens...@gmail.com Cc: nanog@nanog.org Sent: Thursday, May 21, 2015 7:40:23 AM Subject: Re: Peering and Network Cost James, curious to know... what size ISPs are they? In the last few years with the larger ones it has always been about lowering cost and increasing revenue, which throws the original idea of peering out the window (unless you are willing to pay). On Thu, May 21, 2015 at 4:52 AM, James Bensley jwbens...@gmail.com wrote: On 17 April 2015 at 16:53, Justin Wilson - MTIN li...@mtin.net wrote: Peering and peering on an exchange are two different things. Peering at an exchange has several benefits other than the simple cost of transit. If you are in a large data center which charges fees for cross connects a single cross connect to an exchange can save you money. Peering can also be a sales tool. If you buy from a VOIP provider and are peered with them your latency and such will go down. You also have more control over the QOS over that peer. This can be spun into marketing. Not to toot our own horn but we put together a list of benefits for our IX customers: http://www.midwest-ix.com/blog/?p=15 Also, a good article at: http://blog.webserver.com.my/index.php/the-benefits-of-hosting-at-internet-exchange-point/ I also have a similar working document that I'd welcome feedback on to improve; https://docs.google.com/document/d/1i2bPZDt75hAwcR4iKMqaNSGIeM-nJSWLZ6SLTTnuXNs/edit?usp=sharing I've used it once to help an ISP evalutate peering and started them in the world of public peering. I'm now going through that proces again with another ISP and again they will start public peering soon, having used this doc in both cases as an intro/FAQ for them. Cheers, James.
Re: Peering and Network Cost
On 21/May/15 18:59, Dave Taht wrote: Two things I am curious about are 1) What is the measured benefit of moving a netflix server into your local ISP network and 2) does anyone measure cross town latency. If we lived in a world where skype/voip/etc transited the local town only, what sort of latencies would be see within an ISP and within a cross-connect from, say a gfiber to a comcast? On average, 1ms for every 100km. We've seen this in practice - consistently - for any fibre deployed within the same town/city. Unless someone does something very wrong with the fibre, suffers terrible hardware issues, deliberately implements debilitating bandwidth management or does a piss-poor job of network design, it would be reasonably hard to go above +/- 1ms for traffic that originates and terminates within the same town, let alone 6 miles of speaking parties. Mark.
Re: Peering and Network Cost
On Thursday, May 21, 2015, Mark Tinka mark.ti...@seacom.mu wrote: On 21/May/15 18:59, Dave Taht wrote: Two things I am curious about are 1) What is the measured benefit of moving a netflix server into your local ISP network and 2) does anyone measure cross town latency. If we lived in a world where skype/voip/etc transited the local town only, what sort of latencies would be see within an ISP and within a cross-connect from, say a gfiber to a comcast? On average, 1ms for every 100km. We've seen this in practice - consistently - for any fibre deployed within the same town/city. Unless someone does something very wrong with the fibre, suffers terrible hardware issues, deliberately implements debilitating bandwidth management or does a piss-poor job of network design, it would be reasonably hard to go above +/- 1ms for traffic that originates and terminates within the same town, let alone 6 miles of speaking parties. Mark. -- Sent from Gmail Mobile
Re: Peering and Network Cost
On 21 May 2015 at 13:40, Rafael Possamai raf...@gav.ufsc.br wrote: James, curious to know... what size ISPs are they? In the last few years with the larger ones it has always been about lowering cost and increasing revenue, which throws the original idea of peering out the window (unless you are willing to pay). Yes agreed, I have seen the same behaviour too with larger companies although peering can lower costs as I will show below in the 2nd example. Typically though I hear what you are saying, if you are a larger transit consumer with larger commits you really have the weight to stand on your transit provider’s neck until they give you the price you want (which is pretty effective, transit really is dirt cheap these days if you have the traffic levels to back it up). With regards to your question I can't say too much as I'm not sure what I can and can't disclose. The first ISP was a small one with circa 1Gbps of total transit volume (at the time I carried them through the peering process, could be different now). They managed to peer off a third of their transit traffic requirement, so dropping a third of their transit made them a small but acceptable cost saving (since at the time they only had circa 1Gbps of total ingress/egress traffic). For them the marketing aspect of being at a big well know IX was/is very important. So from that rather small cost saving gained from reducing their transit commit with the overhead of peering, the value add for them was greatly boosted by being able to market their IX presence. In the period that followed joining their first IXP that ISP then gained further from a technical perspective as we managed to take direct peering’s across that IXP LAN to some VoIP upstreams and downstreams of theirs and a hosted service provider that ISP works with, and in all those cases that has reduced latency and packet loss which customers have directly noted on having a positive impact. The second ISP I'm now running this exercise for is a medium size ISP, they have about 5Gbps of transit requirements at present and are hoping to peer off half of that. They also intent to increase the transit requirements to circa 10G within the next couple of years, so if they peer off 50% ingress/egress traffic they stand to save quite a bit of money. A 10G peering port is usually a fixed priced so they will just see the ROI on that port grow over time hopefully. One important reason they will make a significant cost saving is due to legacy contracts such as some old PA space they can drop off which is very costly and old transit contracts still on high cost-per-Mbps tariffs My colleague on this expects to cut the transit bill literally in half by the end of the first year. Cheers, James.
Re: Peering and Network Cost
On Wed, Apr 15, 2015 at 10:50 AM, Max Tulyev max...@netassist.ua wrote: Hi Roderick, transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. Two things I am curious about are 1) What is the measured benefit of moving a netflix server into your local ISP network and 2) does anyone measure cross town latency. If we lived in a world where skype/voip/etc transited the local town only, what sort of latencies would be see within an ISP and within a cross-connect from, say a gfiber to a comcast? Once upon a time I'd heard that most phone calls were within 6 miles of the person's home, but I don't remember the breakdown of those call percentages (?), and certainly the old-style phone system was achieving very low latencies for those kinds of traffic. On 04/15/15 17:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. I thank you in advance for any insights. Regards, - R. Roderick Beck Sales Director/Europe and the Americas Hibernia Networks This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e-mail in error, please immediately telephone or e-mail the sender and permanently delete the original copy and any copy of this e-mail, and any printout thereof. All documents, contracts or agreements referred or attached to this e-mail are SUBJECT TO CONTRACT. The contents of an attachment to this e-mail may contain software viruses that could damage your own computer system. While Hibernia Networks has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage that you sustain as a result of software viruses. You should carry out your own virus checks before opening any attachment. -- Dave Täht Open Networking needs **Open Source Hardware** https://plus.google.com/u/0/+EricRaymond/posts/JqxCe2pFr67
Re: Peering and Network Cost
That's generally good idea, but average TCP session speed depends not only your side of connection, but another side as well. On 18.04.15 07:58, Mark Tinka wrote: On 17/Apr/15 15:05, Max Tulyev wrote: One more interesting thing. If you buy IP transit, mostly you are paying by exact bandwidth, per megabit. If you buy IX peering port, you are paying for port. This means Tranist ports are overloaded or close to it, while IX ports usually always have some extra free capacity. In practice, this mean if your customer download some file using IX way, speed will be much higher that same file reachable by IP transit. This depends entirely on how you run your network. If you run links hot, you can't guarantee anything (keeping in mind that your less congested exchange point ports does not mean other exchange point members are in the same position also). We, for example, buy transit or peer with a minimum of 10Gbps port, with the ability to push traffic at line rate if needed. We do not allow ports to run hot (typically upgrading them anywhere from between 50% - 70% utilization). I appreciate that not everyone can be in this position, while others can be even more aggressive with their over-engineering, but this kind of information is hard to quantify reliably. There is also backhaul from the interconnect point into the backbone to think about, but that follows a similar strategy. Mark.
Re: Peering and Network Cost
On 21/Apr/15 19:37, Max Tulyev wrote: That's generally good idea, but average TCP session speed depends not only your side of connection, but another side as well. It was always best effort :-). Mark.
Re: Peering and Network Cost
Choose another IX to peer. Or even make your own ;) Kiev have 3 major IXes, and price is about $100 for 10GE port. On 19.04.15 12:23, Baldur Norddahl wrote: So why is IX peering so expensive? Again if I look at my local IX (dix.dk) they have about 40 networks connected. Each network pays minimum 5800 USD a year. That gives them a budget of 24+ USD a year. But the only service is running an old layer 2 switch. Why do these guys deserve to be paid that much for so little? Recently we had a competitor show up in the form of Netnod. However the pricing is almost exactly the same, although Netnod tries to deliver slightly more service. Seems to me that this an unsound market. The 40 dix particants should donate 1000 USD once and get a new layer 2 switch. Why does that not happen? Does not look like it is a local phenomenon either. IX'es all over are way more expensive than they should be. Regards Baldur
Re: Peering and Network Cost
On 21/Apr/15 11:58, Max Tulyev wrote: Choose another IX to peer. Or even make your own ;) Kiev have 3 major IXes, and price is about $100 for 10GE port. Switch port costs will be governed by how the exchange point is run. Low or no running costs will, theoretically, yield cheaper ports. Mark.
Re: Peering and Network Cost
On Apr 19, 2015, at 2:34 PM, Mark Tinka mark.ti...@seacom.mu wrote: The age of the Ethernet switch has little to do with its performance, unless it has everything to do with its performance. Mark, you realize that this is what NANOG will make sure is engraved on your headstone, right? -Bill signature.asc Description: Message signed with OpenPGP using GPGMail
Re: Peering and Network Cost
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 20/Apr/15 08:32, Bill Woodcock wrote: Mark, you realize that this is what NANOG will make sure is engraved on your headstone, right? Only if I expire :-)... Mark. -BEGIN PGP SIGNATURE- iQIcBAEBAgAGBQJVNJ6JAAoJEGcZuYTeKm+GvuwP/1RNI91Ef3Et5LiQ0py9NyiT 7c2PXLlIa2FBqFiJ2rczsq96uVS8FqsHm0Vklk1e1G7cNsekrGB0Xr1IIS2Be3jC +NPurzflUYQ69LnDnXN5WMzPeUgHGNWOXGySd3WVrVSGdiXS9u24tIk1uCOZM5mZ YSK9yVezpCcYaynQFKBvyrNlVibr7jFS51/sVuxMstEVsySJacp3hNQCgTFMu2YJ k36kxV9VhvKiRVcMT9sfWo59dYS+jE3m90yGhQrgLT1OOfonyuZrVMKFDV4ZqjRK T3qt5svOL+4VXnd4ZUvOG8dHXB5N/ubcYXwXrv4J6iCFFwLKKvFwr49q8DETn52v 5F7W018ebQedYqZjdUxYiDD6xbmfVecDup0hjt9LUotv9ZueTvlf3cIK/VtLmzqd CHv8BO/H7bjoEehKp5Bs2wuvY6C8C6zUoYDOSzrF367oNX9IoLAyp2EMyv0K/lT5 qH6z+hYAa8BD1tstdjSWTwq8YJYjAtA3RYISJCrLDXBQJY8DrLequisYV7sV3HPq tlLvtbeojJBb3VvhYZftH6UQ+zIdj3DVTa85tHZZs6IWbRdCMUKBHtjTnaDRJx7A FGj9xDRJR23ssDHnZ0QCHPJdYrmjFe0t0euFGfu/jxbCng8y3MI72jvkE5i2npMc ty8NRJng8qYfGem5D1lK =rkAw -END PGP SIGNATURE-
Re: Peering and Network Cost
So why is IX peering so expensive? Again if I look at my local IX (dix.dk) they have about 40 networks connected. Each network pays minimum 5800 USD a year. That gives them a budget of 24+ USD a year. But the only service is running an old layer 2 switch. Why do these guys deserve to be paid that much for so little? Recently we had a competitor show up in the form of Netnod. However the pricing is almost exactly the same, although Netnod tries to deliver slightly more service. Seems to me that this an unsound market. The 40 dix particants should donate 1000 USD once and get a new layer 2 switch. Why does that not happen? Does not look like it is a local phenomenon either. IX'es all over are way more expensive than they should be. Regards Baldur
Re: Peering and Network Cost
On Sun, 19 Apr 2015 11:23:53 +0200, Baldur Norddahl baldur.nordd...@gmail.com said: So why is IX peering so expensive? But the only service is running an old layer 2 switch. The 40 dix particants should donate 1000 USD once and get a new layer 2 switch. Why does that not happen? This is something like how TORIX was operated at the beginning. The switch was donated by Cisco and rack space by a member with a cage at a convenient spot at 151 Front -- I think this was jlixfeld at look.ca. Fees were a $1/port/year peppercorn. It has been a long time since I was in any way involved in that, but today for a 1Gbps port TORIX charges $1200/year which is more but still not as much as you say for other IXPs. It would be interesting to hear from someone who was involved in TORIX at the time how this transition from $1 to $1200 went and the reasoning behind it. My guess would be moving to its own space and having to pay rent was a major part of it, and possibly acquiring staff? Also note that the LINX exchanges do not charge for the first 1Gbps port (or the n-th at the regional exchanges) though there is a membership fee which makes it roughly equivalent to what TORIX does today. From that point of view you guys in Denmark seem to be paying somewhat over the odds. Cheers, -w -- William Waites wwai...@tardis.ed.ac.uk | School of Informatics http://tardis.ed.ac.uk/~wwaites/ | University of Edinburgh http://www.hubs.net.uk/| HUBS AS60241 The University of Edinburgh is a charitable body, registered in Scotland, with registration number SC005336. pgpxHH_APz_go.pgp Description: PGP signature
Re: Peering and Network Cost
There is a revenue floor where it doesn't matter how much or how little service is provided, simply having a customer period requires a certain amount of revenue. Route servers, IXP Manager, AS112, route collectors, DNS, etc. all cost money. Maintenance costs money. The organization itself costs money. Upgrades cost money. Racks cost money. Power costs money. I'm sure I've left some things out. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Baldur Norddahl baldur.nordd...@gmail.com To: nanog@nanog.org Sent: Sunday, April 19, 2015 4:23:53 AM Subject: Re: Peering and Network Cost So why is IX peering so expensive? Again if I look at my local IX (dix.dk) they have about 40 networks connected. Each network pays minimum 5800 USD a year. That gives them a budget of 24+ USD a year. But the only service is running an old layer 2 switch. Why do these guys deserve to be paid that much for so little? Recently we had a competitor show up in the form of Netnod. However the pricing is almost exactly the same, although Netnod tries to deliver slightly more service. Seems to me that this an unsound market. The 40 dix particants should donate 1000 USD once and get a new layer 2 switch. Why does that not happen? Does not look like it is a local phenomenon either. IX'es all over are way more expensive than they should be. Regards Baldur
Re: Peering and Network Cost
On 19/Apr/15 11:23, Baldur Norddahl wrote: So why is IX peering so expensive? Again if I look at my local IX (dix.dk) they have about 40 networks connected. Each network pays minimum 5800 USD a year. That gives them a budget of 24+ USD a year. But the only service is running an old layer 2 switch. The age of the Ethernet switch has little to do with its performance, unless it has everything to do with its performance. Why do these guys deserve to be paid that much for so little? The exchange point operator do not typically interfere with what the members do. That said, while it is not their job to grow your peering traffic, it would make thier exchange point more successful if they managed to grow their membership. Recently we had a competitor show up in the form of Netnod. However the pricing is almost exactly the same, although Netnod tries to deliver slightly more service. Seems to me that this an unsound market. The 40 dix particants should donate 1000 USD once and get a new layer 2 switch. Why does that not happen? Unless it is the case, the old switch should not prevent growth of the exchange point, unless, of course, it currently does. Does not look like it is a local phenomenon either. IX'es all over are way more expensive than they should be. Most exchange point operators have staff that run the network, find members, e.t.c. These salaries need to be paid. If a member is unable to extract the most value from their peering setup, there is very little an exchange point can do about that. In short, it's not for everyone, despite its good intentions. Mark.
Re: Peering and Network Cost
On Apr 19, 2015, at 6:09 AM, William Waites wwai...@tardis.ed.ac.uk wrote: On Sun, 19 Apr 2015 11:23:53 +0200, Baldur Norddahl baldur.nordd...@gmail.com said: So why is IX peering so expensive? But the only service is running an old layer 2 switch. The 40 dix particants should donate 1000 USD once and get a new layer 2 switch. Why does that not happen? This is something like how TORIX was operated at the beginning. The switch was donated by Cisco and rack space by a member with a cage at a convenient spot at 151 Front -- I think this was jlixfeld at look.ca. Fees were a $1/port/year peppercorn. It has been a long time since I was in any way involved in that, but today for a 1Gbps port TORIX charges $1200/year which is more but still not as much as you say for other IXPs. It would be interesting to hear from someone who was involved in TORIX at the time how this transition from $1 to $1200 went and the reasoning behind it. My guess would be moving to its own space and having to pay rent was a major part of it, and possibly acquiring staff? To be clear, we asked for $1/port/year, but we never really bothered to pay attention to who actually paid :) Instead of addressing your questions directly, how about a brief and much abridged history of TorIX? ;) The recollection of Mr. Waites on our humble beginnings is pretty much bang-on. For the first 7 or so years, we were really ad-hoc, but we eventually decided that we needed to incorporate. That decision was simply due to the fact that we didn’t think we’d be taken very seriously by larger players (larger eyeball networks or large content networks (either nationally or internationally)) unless we moved away from an ad-hoc collection of nothing and no-one, and into an actual legal entity. Along with feedback from the participates of our little IX, those of us who made up the organizing body of this ad-hoc TorIX decided that while a legal organization was an important next step in our evolution, incorporating with non-profit status (as opposed to a full-blown commercial IX) was the most appropriate method of becoming legit. Bill Campbell (former owner Hostopia, former owner Internet Direct (later became Look)) put up 100% of the money to incorporate TorIX in early 2004. Second, up until about 2008’ish, whenever we needed gear, we’d usually have to pass the hat when we needed a GigE switch or something a little more high test than someone’s decommissioned FastE kit. The problem with passing the hat is that it rarely makes everyone happy because there’s always someone who gets left out. The cash in the hat would only give us enough to buy a 12 port switch, but inevitably, a few more than 12 participants all donated towards buying the switch. The last ones to offer up the cash had to be dropped until the next time the hat got passed around. We didn’t think asking all our participants to drop money into the hat was an appropriate course of action. Not everyone would contribute, for a multitude of reasons, but everyone would still expect the same level of service. Needless to say, it got messy. It was an inevitable part of our growth, sure. It might still be inevitable for any budding IX. After our incorporation, there were many offers from folks with skids of decommissioned 6500s, 6704s and SUP720s. These extremely generous donations made it possible for us to turn up our first 10G port, but it resulted in other challenges: who would be allowed to occupy the other three ports? Do we charge for them? We got the ports for free so HTF do we figure this one out, guys? These sorts of dilemmas would cause strife, so around 2008, the serving Board at the time decided that the next step in our evolution was to make the organization completely self-sufficient by introducing a reasonable port fee structure. Port fees could let us get space where we felt we needed it. We could buy our own gear so anyone would always be able to have any speed port they wanted. We could pay for the support contracts, hire lawyers and accountants, and also contribute to community initiatives like sponsoring the Canadian ISP Summit, NANOG and ARIN. We strive to keep our port fees low. 99% of folks never thought our port fees were too high. In fact, I can remember a few folks who laughed when we introduced port fees asking if they could pay for 5 years up front because the port fees were so cheap they were a joke. The Board introduced a reduced port fee structure across the board for 2015. Everyone[1] who contributes to TorIX still does so in a volunteer capacity; Board members, the operations group, even our book keeper :) [1]In 2014, the Board voted in favour of a motion to hire an Executive Director to further drive the growth of TorIX. In March, the Board announced that Bill Sandiford had accepted the role. In the 18 year history of TorIX, the Executive Director role is the first ever
Re: Peering and Network Cost
Getting networks to connect to an ix is Uber expensive in relation to the overall costs. Specifically before critical mass is reached. Getting the first X gig of traffic is a hard problem that takes money to fix. On Apr 19, 2015 7:51 AM, Mike Hammett na...@ics-il.net wrote: There is a revenue floor where it doesn't matter how much or how little service is provided, simply having a customer period requires a certain amount of revenue. Route servers, IXP Manager, AS112, route collectors, DNS, etc. all cost money. Maintenance costs money. The organization itself costs money. Upgrades cost money. Racks cost money. Power costs money. I'm sure I've left some things out. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Baldur Norddahl baldur.nordd...@gmail.com To: nanog@nanog.org Sent: Sunday, April 19, 2015 4:23:53 AM Subject: Re: Peering and Network Cost So why is IX peering so expensive? Again if I look at my local IX (dix.dk) they have about 40 networks connected. Each network pays minimum 5800 USD a year. That gives them a budget of 24+ USD a year. But the only service is running an old layer 2 switch. Why do these guys deserve to be paid that much for so little? Recently we had a competitor show up in the form of Netnod. However the pricing is almost exactly the same, although Netnod tries to deliver slightly more service. Seems to me that this an unsound market. The 40 dix particants should donate 1000 USD once and get a new layer 2 switch. Why does that not happen? Does not look like it is a local phenomenon either. IX'es all over are way more expensive than they should be. Regards Baldur
Re: Peering and Network Cost
One more interesting thing. If you buy IP transit, mostly you are paying by exact bandwidth, per megabit. If you buy IX peering port, you are paying for port. This means Tranist ports are overloaded or close to it, while IX ports usually always have some extra free capacity. In practice, this mean if your customer download some file using IX way, speed will be much higher that same file reachable by IP transit. So more peerings your uplink use - better performance of your network connection you have ;) On 04/15/15 23:12, Grzegorz Janoszka wrote: On 2015-04-15 19:50, Max Tulyev wrote: transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. Even in the three cities you mentioned peering on small scale is usually not cheaper at all or only very little. Please keep in mind that some companies peer despite it offers no savings for them and at the end of the day it might be even more expensive. They do it because of performance and reliability reasons.
Re: Peering and Network Cost
If you have so much difference in price of IX connectivity (in general, including cabling, DWDM to one of major IX, colo, etc) - this only mean you should have a long talk with your current IP transit sales. Or just change it to another one. On 04/15/15 21:45, Mike Hammett wrote: (Reply to thread, not necessarily myself.) If you can pull a third of your traffic off at the cost of a cross connect and another third at the cost of an IX port, now you can spend a buck or two a meg on what's left. Yes, I understand the cost of a cross connect or IX port is the $/megabit you're actually using and not $/megabit of capacity. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mike Hammett na...@ics-il.net To: Max Tulyev max...@netassist.ua Cc: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:33:35 PM Subject: Re: Peering and Network Cost Very true. I left it as I did given that I expect a similar profile from others in North America... on NANOG. Basically, wherever your region's streaming video or application updates come from. ;-) - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:27:45 PM Subject: Re: Peering and Network Cost Not actually Facebook net, but Akamai CDN. Not a Google (peer), but GCC node ;) It is varying from location to location. For example here in Ukraine we (still) have 1st place for traffic amount from Vkontakte (mostly music streams), second from EX.ua (movie store), but almost none NetFlix, Hulu or Amazon. And you can't get both of them in a good quality neither at IXes, nor at Tier1. I think in another locations, for example in India, traffic profile will be different from both of us, and have some local specific as well. On 04/15/15 20:58, Mike Hammett wrote: It also depends on traffic makeup. Huge amounts of eyeball traffic go to (well, come from) NetFlix (a third) and Google, FaceBook, Hulu, Amazon, etc. (another third). It's comparable price to peer off those few huge sources of traffic and buy better transit than you would have than to just buy cheap transit. A lot of people tend to forget there are thousands of independent ISPs out there, usually in areas where there aren't a breadth of providers in the first place. Most could get buy with a single GigE (or even less). - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 12:50:41 PM Subject: Re: Peering and Network Cost Hi Roderick, transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. On 04/15/15 17:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. I thank you in advance for any insights. Regards, - R. Roderick Beck Sales Director/Europe and the Americas Hibernia Networks This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e-mail in error, please immediately telephone or e-mail the sender and permanently delete the original copy and any copy of this e-mail, and any printout thereof. All documents, contracts or agreements referred or attached to this e-mail are SUBJECT TO CONTRACT. The contents of an attachment to this e-mail may contain software viruses that could damage your own computer system. While Hibernia Networks has taken every reasonable precaution to minimize this risk, we cannot accept
Re: Peering and Network Cost
For sure, that's the main reason of peering, not a cost saving ;) On 04/15/15 23:12, Grzegorz Janoszka wrote: Please keep in mind that some companies peer despite it offers no savings for them and at the end of the day it might be even more expensive. They do it because of performance and reliability reasons.
Re: Peering and Network Cost
Transit should cost more than peering and should never cost little more than the cost of a cross connect or a switch, given the load of additional responsibilities. I counter that if peering is cheaper than transit, you need to talk to your IX about it's cost models. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Friday, April 17, 2015 5:33:04 AM Subject: Re: Peering and Network Cost If you have so much difference in price of IX connectivity (in general, including cabling, DWDM to one of major IX, colo, etc) - this only mean you should have a long talk with your current IP transit sales. Or just change it to another one. On 04/15/15 21:45, Mike Hammett wrote: (Reply to thread, not necessarily myself.) If you can pull a third of your traffic off at the cost of a cross connect and another third at the cost of an IX port, now you can spend a buck or two a meg on what's left. Yes, I understand the cost of a cross connect or IX port is the $/megabit you're actually using and not $/megabit of capacity. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mike Hammett na...@ics-il.net To: Max Tulyev max...@netassist.ua Cc: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:33:35 PM Subject: Re: Peering and Network Cost Very true. I left it as I did given that I expect a similar profile from others in North America... on NANOG. Basically, wherever your region's streaming video or application updates come from. ;-) - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:27:45 PM Subject: Re: Peering and Network Cost Not actually Facebook net, but Akamai CDN. Not a Google (peer), but GCC node ;) It is varying from location to location. For example here in Ukraine we (still) have 1st place for traffic amount from Vkontakte (mostly music streams), second from EX.ua (movie store), but almost none NetFlix, Hulu or Amazon. And you can't get both of them in a good quality neither at IXes, nor at Tier1. I think in another locations, for example in India, traffic profile will be different from both of us, and have some local specific as well. On 04/15/15 20:58, Mike Hammett wrote: It also depends on traffic makeup. Huge amounts of eyeball traffic go to (well, come from) NetFlix (a third) and Google, FaceBook, Hulu, Amazon, etc. (another third). It's comparable price to peer off those few huge sources of traffic and buy better transit than you would have than to just buy cheap transit. A lot of people tend to forget there are thousands of independent ISPs out there, usually in areas where there aren't a breadth of providers in the first place. Most could get buy with a single GigE (or even less). - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 12:50:41 PM Subject: Re: Peering and Network Cost Hi Roderick, transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. On 04/15/15 17:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. I thank you in advance for any insights. Regards, - R. Roderick Beck Sales Director/Europe and the Americas Hibernia Networks This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission
Re: Peering and Network Cost
On 16/Apr/15 17:10, Edward Dore wrote: I don't have any quantifiable data on what has happened to IP transit costs over the same period, but for a point comparison I'd say that off the top of my head you can get a 1G CDR on a 10G port from a tier-1 provider in London for approximately the same cost as a 10G port at LINX these days, maybe slightly cheaper. Transit costs are certainly falling at a much faster rate than exchange point ports. However, because most major exchange points are pushing reasonably high ports (1Gbps and 10Gbps) to members, the challenge with filling those ports makes transit ports a more viable solution in the short term. If you're willing to stick it out long enough, peering ports can become as cheap as transit ports, but they will never give you 100% coverage like a transit port can. Mark.
Re: Peering and Network Cost
E, countering that if transit is cheaper than peering, you should talk to your IX. The effects of posting when I haven't been awake for hardy more than ten minutes - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mike Hammett na...@ics-il.net To: nanog@nanog.org Sent: Friday, April 17, 2015 6:51:09 AM Subject: Re: Peering and Network Cost Transit should cost more than peering and should never cost little more than the cost of a cross connect or a switch, given the load of additional responsibilities. I counter that if peering is cheaper than transit, you need to talk to your IX about it's cost models. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Friday, April 17, 2015 5:33:04 AM Subject: Re: Peering and Network Cost If you have so much difference in price of IX connectivity (in general, including cabling, DWDM to one of major IX, colo, etc) - this only mean you should have a long talk with your current IP transit sales. Or just change it to another one. On 04/15/15 21:45, Mike Hammett wrote: (Reply to thread, not necessarily myself.) If you can pull a third of your traffic off at the cost of a cross connect and another third at the cost of an IX port, now you can spend a buck or two a meg on what's left. Yes, I understand the cost of a cross connect or IX port is the $/megabit you're actually using and not $/megabit of capacity. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mike Hammett na...@ics-il.net To: Max Tulyev max...@netassist.ua Cc: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:33:35 PM Subject: Re: Peering and Network Cost Very true. I left it as I did given that I expect a similar profile from others in North America... on NANOG. Basically, wherever your region's streaming video or application updates come from. ;-) - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:27:45 PM Subject: Re: Peering and Network Cost Not actually Facebook net, but Akamai CDN. Not a Google (peer), but GCC node ;) It is varying from location to location. For example here in Ukraine we (still) have 1st place for traffic amount from Vkontakte (mostly music streams), second from EX.ua (movie store), but almost none NetFlix, Hulu or Amazon. And you can't get both of them in a good quality neither at IXes, nor at Tier1. I think in another locations, for example in India, traffic profile will be different from both of us, and have some local specific as well. On 04/15/15 20:58, Mike Hammett wrote: It also depends on traffic makeup. Huge amounts of eyeball traffic go to (well, come from) NetFlix (a third) and Google, FaceBook, Hulu, Amazon, etc. (another third). It's comparable price to peer off those few huge sources of traffic and buy better transit than you would have than to just buy cheap transit. A lot of people tend to forget there are thousands of independent ISPs out there, usually in areas where there aren't a breadth of providers in the first place. Most could get buy with a single GigE (or even less). - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 12:50:41 PM Subject: Re: Peering and Network Cost Hi Roderick, transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. On 04/15/15 17:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. I thank you in advance for any insights. Regards, - R. Roderick Beck Sales
Re: Peering and Network Cost
Peering and peering on an exchange are two different things. Peering at an exchange has several benefits other than the simple cost of transit. If you are in a large data center which charges fees for cross connects a single cross connect to an exchange can save you money. Peering can also be a sales tool. If you buy from a VOIP provider and are peered with them your latency and such will go down. You also have more control over the QOS over that peer. This can be spun into marketing. Not to toot our own horn but we put together a list of benefits for our IX customers: http://www.midwest-ix.com/blog/?p=15 Also, a good article at: http://blog.webserver.com.my/index.php/the-benefits-of-hosting-at-internet-exchange-point/ Justin Justin Wilson j...@mtin.net http://www.mtin.net Managed Services – xISP Solutions – Data Centers http://www.thebrotherswisp.com Podcast about xISP topics http://www.midwest-ix.com Peering – Transit – Internet Exchange On Apr 16, 2015, at 11:10 AM, Edward Dore edward.d...@freethought-internet.co.uk wrote: On 16 Apr 2015, at 08:00, Tore Anderson t...@fud.no wrote: * Mark Tinka mark.ti...@seacom.mu On 16/Apr/15 07:25, Tore Anderson wrote: We're in a similar situation here; transit prices has come down so much in recent years (while IX fees are indeed stagnant) that I am certain that if I were to cut all peering and buy everything from a regional tier-2 instead, I'd be lowering my total MRC somewhat, without really reducing connectivity quality to my (former) peers. I wouldn't say exchange point prices are stagnant, per se. They may remain the same, but what goes up is the port bandwidth. It's not directly linear, but you get my point. Again, the burden is on the peering members to extract the most out of their peering links by having as much peering as possible. You appear to be assuming that an IP transit port is more expensive then an IXP port with the same speed. That doesn't seem to always be the case anymore, at least not in all parts of the world, and I expect this trend to continue - transit prices seems to go down almost on a monthly basis, while the price lists of the two closest IXPs to where I'm sitting are dated 2011 and 2013, respectively. Even if the transit port itself remains slightly more expensive than the IXP port like in the example Baldur showed, the no-peering alternative might still be cheaper overall because even if you're peering most of your traffic you'll still need to pay a nonzero amount for a (smaller or less utilised) transit port anyway. Tore Pricing at LINX here in the UK has definitely dropped over the past few years. Back in 2011, the membership fee was £1500/year and it's now £1200/year. 1G ports were £391/month on the first London LAN and £335/month on the second London LAN. They're now free on both LANs for the first port and then £270/month and £180/month respectively for additional ports. You can also get a free 1G port on each of the Manchester UK, Cardiff UK, Edinburgh UK and North Virginia/Washington DC USA LANs as part of the same membership fee (none of these additional LANs existed in 2011). 10G ports were £1463/month on the first London LAN and £1250/month on the second London LAN. They're now £1030/month and £785/month respectively. So that's what, a 20% reduction in membership fees and a 30% or higher (depending on the service) reduction in port fees in 4 years? I don't have any quantifiable data on what has happened to IP transit costs over the same period, but for a point comparison I'd say that off the top of my head you can get a 1G CDR on a 10G port from a tier-1 provider in London for approximately the same cost as a 10G port at LINX these days, maybe slightly cheaper. Edward Dore Freethought Internet
Re: Peering and Network Cost
On 17/Apr/15 15:05, Max Tulyev wrote: One more interesting thing. If you buy IP transit, mostly you are paying by exact bandwidth, per megabit. If you buy IX peering port, you are paying for port. This means Tranist ports are overloaded or close to it, while IX ports usually always have some extra free capacity. In practice, this mean if your customer download some file using IX way, speed will be much higher that same file reachable by IP transit. This depends entirely on how you run your network. If you run links hot, you can't guarantee anything (keeping in mind that your less congested exchange point ports does not mean other exchange point members are in the same position also). We, for example, buy transit or peer with a minimum of 10Gbps port, with the ability to push traffic at line rate if needed. We do not allow ports to run hot (typically upgrading them anywhere from between 50% - 70% utilization). I appreciate that not everyone can be in this position, while others can be even more aggressive with their over-engineering, but this kind of information is hard to quantify reliably. There is also backhaul from the interconnect point into the backbone to think about, but that follows a similar strategy. Mark.
Re: Peering and Network Cost
On 16/Apr/15 07:25, Tore Anderson wrote: We're in a similar situation here; transit prices has come down so much in recent years (while IX fees are indeed stagnant) that I am certain that if I were to cut all peering and buy everything from a regional tier-2 instead, I'd be lowering my total MRC somewhat, without really reducing connectivity quality to my (former) peers. I wouldn't say exchange point prices are stagnant, per se. They may remain the same, but what goes up is the port bandwidth. It's not directly linear, but you get my point. Again, the burden is on the peering members to extract the most out of their peering links by having as much peering as possible. Route servers at the exchange points have played a huge role in facilitating this, but the final stretch involves getting in touch with a bunch of members to setup bi-lateral sessions, with no guarantee they will agree, or if they do, may not peer their entire network, not taking into account whether they will do this for free or not. Perhaps the next bunch of exchange point operators will be those who play a more active role in facilitating peering across all members, so as to keep traffic on their switch fabric and away from transit providers. But that is probably a zero-sum game, as their involvement will just mean more salaries that need to get paid, leading to an increase in membership fees. Mark.
Re: Peering and Network Cost
On 15/Apr/15 22:07, Baldur Norddahl wrote: Transit cost is down but IX cost remains the same. Therefore IX is longer cost effective for a small ISP. As an (non US) example, here in Copenhagen, Denmark we have two internet exchanges DIX and Netnod. We also have many major transit providers, including Hurricane Electric and Cogent. Netnod price for a 1 Gbps port is 4 SEK = 4500 USD / year http://www.netnod.se/ix/join/prices. DIX is 4 DKK = 5700 USD / year http://dix.dk/serviceinformation/ HE.net is offering 1 Gbps flatrate for 450 USD / month list price = 5400 USD /year. Cogent can match that. So why would a small ISP pay 4500 USD for a service with no guarantee of how much traffic they will be able to peer away? You need to get a 10 Gbps port and be able to peer at least 2-3 Gbps before it is even break even with the deals you get from the transit providers. But then you will notice that all the traffic is only with a few peers and you can just peer directly with those and skip the middleman. Even though you get more bandwidth at an exchange point for the price you pay a transit provider for less bandwidth, the value you extract from the more cost-efficient peering port is directly proportional to how much peering you can get off of it. If you are unable to offload enough of your traffic on to peering that an incremental transit service cost can justify, you're likely better off with a transit provider if budget is your constraint. Mark.
Re: Peering and Network Cost
* Mark Tinka mark.ti...@seacom.mu On 16/Apr/15 07:25, Tore Anderson wrote: We're in a similar situation here; transit prices has come down so much in recent years (while IX fees are indeed stagnant) that I am certain that if I were to cut all peering and buy everything from a regional tier-2 instead, I'd be lowering my total MRC somewhat, without really reducing connectivity quality to my (former) peers. I wouldn't say exchange point prices are stagnant, per se. They may remain the same, but what goes up is the port bandwidth. It's not directly linear, but you get my point. Again, the burden is on the peering members to extract the most out of their peering links by having as much peering as possible. You appear to be assuming that an IP transit port is more expensive then an IXP port with the same speed. That doesn't seem to always be the case anymore, at least not in all parts of the world, and I expect this trend to continue - transit prices seems to go down almost on a monthly basis, while the price lists of the two closest IXPs to where I'm sitting are dated 2011 and 2013, respectively. Even if the transit port itself remains slightly more expensive than the IXP port like in the example Baldur showed, the no-peering alternative might still be cheaper overall because even if you're peering most of your traffic you'll still need to pay a nonzero amount for a (smaller or less utilised) transit port anyway. Tore
Re: Peering and Network Cost
On 15/Apr/15 22:12, Grzegorz Janoszka wrote: Please keep in mind that some companies peer despite it offers no savings for them and at the end of the day it might be even more expensive. They do it because of performance and reliability reasons. And also to reduce AS hops. If you and your competition are pushing BGP routes to your multi-homed customers, the customers are more likely to choose paths with fewer AS hops. Traffic to you means $$ to you. In the long run, it could pay off, or at the very least, get you more customers. Mark.
Re: Peering and Network Cost
IX port pricing in the Chicago area has plummeted in the past two or three years. It's gone down... maybe 2/3. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Tore Anderson t...@fud.no To: Mark Tinka mark.ti...@seacom.mu Cc: nanog@nanog.org Sent: Thursday, April 16, 2015 2:00:53 AM Subject: Re: Peering and Network Cost * Mark Tinka mark.ti...@seacom.mu On 16/Apr/15 07:25, Tore Anderson wrote: We're in a similar situation here; transit prices has come down so much in recent years (while IX fees are indeed stagnant) that I am certain that if I were to cut all peering and buy everything from a regional tier-2 instead, I'd be lowering my total MRC somewhat, without really reducing connectivity quality to my (former) peers. I wouldn't say exchange point prices are stagnant, per se. They may remain the same, but what goes up is the port bandwidth. It's not directly linear, but you get my point. Again, the burden is on the peering members to extract the most out of their peering links by having as much peering as possible. You appear to be assuming that an IP transit port is more expensive then an IXP port with the same speed. That doesn't seem to always be the case anymore, at least not in all parts of the world, and I expect this trend to continue - transit prices seems to go down almost on a monthly basis, while the price lists of the two closest IXPs to where I'm sitting are dated 2011 and 2013, respectively. Even if the transit port itself remains slightly more expensive than the IXP port like in the example Baldur showed, the no-peering alternative might still be cheaper overall because even if you're peering most of your traffic you'll still need to pay a nonzero amount for a (smaller or less utilised) transit port anyway. Tore
Re: Peering and Network Cost
On 16 Apr 2015, at 08:00, Tore Anderson t...@fud.no wrote: * Mark Tinka mark.ti...@seacom.mu On 16/Apr/15 07:25, Tore Anderson wrote: We're in a similar situation here; transit prices has come down so much in recent years (while IX fees are indeed stagnant) that I am certain that if I were to cut all peering and buy everything from a regional tier-2 instead, I'd be lowering my total MRC somewhat, without really reducing connectivity quality to my (former) peers. I wouldn't say exchange point prices are stagnant, per se. They may remain the same, but what goes up is the port bandwidth. It's not directly linear, but you get my point. Again, the burden is on the peering members to extract the most out of their peering links by having as much peering as possible. You appear to be assuming that an IP transit port is more expensive then an IXP port with the same speed. That doesn't seem to always be the case anymore, at least not in all parts of the world, and I expect this trend to continue - transit prices seems to go down almost on a monthly basis, while the price lists of the two closest IXPs to where I'm sitting are dated 2011 and 2013, respectively. Even if the transit port itself remains slightly more expensive than the IXP port like in the example Baldur showed, the no-peering alternative might still be cheaper overall because even if you're peering most of your traffic you'll still need to pay a nonzero amount for a (smaller or less utilised) transit port anyway. Tore Pricing at LINX here in the UK has definitely dropped over the past few years. Back in 2011, the membership fee was £1500/year and it's now £1200/year. 1G ports were £391/month on the first London LAN and £335/month on the second London LAN. They're now free on both LANs for the first port and then £270/month and £180/month respectively for additional ports. You can also get a free 1G port on each of the Manchester UK, Cardiff UK, Edinburgh UK and North Virginia/Washington DC USA LANs as part of the same membership fee (none of these additional LANs existed in 2011). 10G ports were £1463/month on the first London LAN and £1250/month on the second London LAN. They're now £1030/month and £785/month respectively. So that's what, a 20% reduction in membership fees and a 30% or higher (depending on the service) reduction in port fees in 4 years? I don't have any quantifiable data on what has happened to IP transit costs over the same period, but for a point comparison I'd say that off the top of my head you can get a 1G CDR on a 10G port from a tier-1 provider in London for approximately the same cost as a 10G port at LINX these days, maybe slightly cheaper. Edward Dore Freethought Internet
Re: Peering and Network Cost
On 16/Apr/15 09:00, Tore Anderson wrote: You appear to be assuming that an IP transit port is more expensive then an IXP port with the same speed. That doesn't seem to always be the case anymore, at least not in all parts of the world, and I expect this trend to continue - transit prices seems to go down almost on a monthly basis, while the price lists of the two closest IXPs to where I'm sitting are dated 2011 and 2013, respectively. Agreed. Even if the transit port itself remains slightly more expensive than the IXP port like in the example Baldur showed, the no-peering alternative might still be cheaper overall because even if you're peering most of your traffic you'll still need to pay a nonzero amount for a (smaller or less utilised) transit port anyway. Agreed again. Mark.
Re: Peering and Network Cost
Hi Roderick, transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. On 04/15/15 17:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. I thank you in advance for any insights. Regards, - R. Roderick Beck Sales Director/Europe and the Americas Hibernia Networks This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e-mail in error, please immediately telephone or e-mail the sender and permanently delete the original copy and any copy of this e-mail, and any printout thereof. All documents, contracts or agreements referred or attached to this e-mail are SUBJECT TO CONTRACT. The contents of an attachment to this e-mail may contain software viruses that could damage your own computer system. While Hibernia Networks has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage that you sustain as a result of software viruses. You should carry out your own virus checks before opening any attachment.
Re: Peering and Network Cost
It also depends on traffic makeup. Huge amounts of eyeball traffic go to (well, come from) NetFlix (a third) and Google, FaceBook, Hulu, Amazon, etc. (another third). It's comparable price to peer off those few huge sources of traffic and buy better transit than you would have than to just buy cheap transit. A lot of people tend to forget there are thousands of independent ISPs out there, usually in areas where there aren't a breadth of providers in the first place. Most could get buy with a single GigE (or even less). - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 12:50:41 PM Subject: Re: Peering and Network Cost Hi Roderick, transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. On 04/15/15 17:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. I thank you in advance for any insights. Regards, - R. Roderick Beck Sales Director/Europe and the Americas Hibernia Networks This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e-mail in error, please immediately telephone or e-mail the sender and permanently delete the original copy and any copy of this e-mail, and any printout thereof. All documents, contracts or agreements referred or attached to this e-mail are SUBJECT TO CONTRACT. The contents of an attachment to this e-mail may contain software viruses that could damage your own computer system. While Hibernia Networks has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage that you sustain as a result of software viruses. You should carry out your own virus checks before opening any attachment.
Re: Peering and Network Cost
(Reply to thread, not necessarily myself.) If you can pull a third of your traffic off at the cost of a cross connect and another third at the cost of an IX port, now you can spend a buck or two a meg on what's left. Yes, I understand the cost of a cross connect or IX port is the $/megabit you're actually using and not $/megabit of capacity. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mike Hammett na...@ics-il.net To: Max Tulyev max...@netassist.ua Cc: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:33:35 PM Subject: Re: Peering and Network Cost Very true. I left it as I did given that I expect a similar profile from others in North America... on NANOG. Basically, wherever your region's streaming video or application updates come from. ;-) - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:27:45 PM Subject: Re: Peering and Network Cost Not actually Facebook net, but Akamai CDN. Not a Google (peer), but GCC node ;) It is varying from location to location. For example here in Ukraine we (still) have 1st place for traffic amount from Vkontakte (mostly music streams), second from EX.ua (movie store), but almost none NetFlix, Hulu or Amazon. And you can't get both of them in a good quality neither at IXes, nor at Tier1. I think in another locations, for example in India, traffic profile will be different from both of us, and have some local specific as well. On 04/15/15 20:58, Mike Hammett wrote: It also depends on traffic makeup. Huge amounts of eyeball traffic go to (well, come from) NetFlix (a third) and Google, FaceBook, Hulu, Amazon, etc. (another third). It's comparable price to peer off those few huge sources of traffic and buy better transit than you would have than to just buy cheap transit. A lot of people tend to forget there are thousands of independent ISPs out there, usually in areas where there aren't a breadth of providers in the first place. Most could get buy with a single GigE (or even less). - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 12:50:41 PM Subject: Re: Peering and Network Cost Hi Roderick, transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. On 04/15/15 17:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. I thank you in advance for any insights. Regards, - R. Roderick Beck Sales Director/Europe and the Americas Hibernia Networks This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e-mail in error, please immediately telephone or e-mail the sender and permanently delete the original copy and any copy of this e-mail, and any printout thereof. All documents, contracts or agreements referred or attached to this e-mail are SUBJECT TO CONTRACT. The contents of an attachment to this e-mail may contain software viruses that could damage your own computer system. While Hibernia Networks has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage that you sustain as a result of software viruses. You should carry out your own virus checks before opening any attachment.
Re: Peering and Network Cost
Not actually Facebook net, but Akamai CDN. Not a Google (peer), but GCC node ;) It is varying from location to location. For example here in Ukraine we (still) have 1st place for traffic amount from Vkontakte (mostly music streams), second from EX.ua (movie store), but almost none NetFlix, Hulu or Amazon. And you can't get both of them in a good quality neither at IXes, nor at Tier1. I think in another locations, for example in India, traffic profile will be different from both of us, and have some local specific as well. On 04/15/15 20:58, Mike Hammett wrote: It also depends on traffic makeup. Huge amounts of eyeball traffic go to (well, come from) NetFlix (a third) and Google, FaceBook, Hulu, Amazon, etc. (another third). It's comparable price to peer off those few huge sources of traffic and buy better transit than you would have than to just buy cheap transit. A lot of people tend to forget there are thousands of independent ISPs out there, usually in areas where there aren't a breadth of providers in the first place. Most could get buy with a single GigE (or even less). - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 12:50:41 PM Subject: Re: Peering and Network Cost Hi Roderick, transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. On 04/15/15 17:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. I thank you in advance for any insights. Regards, - R. Roderick Beck Sales Director/Europe and the Americas Hibernia Networks This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e-mail in error, please immediately telephone or e-mail the sender and permanently delete the original copy and any copy of this e-mail, and any printout thereof. All documents, contracts or agreements referred or attached to this e-mail are SUBJECT TO CONTRACT. The contents of an attachment to this e-mail may contain software viruses that could damage your own computer system. While Hibernia Networks has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage that you sustain as a result of software viruses. You should carry out your own virus checks before opening any attachment.
Re: Peering and Network Cost
Very true. I left it as I did given that I expect a similar profile from others in North America... on NANOG. Basically, wherever your region's streaming video or application updates come from. ;-) - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:27:45 PM Subject: Re: Peering and Network Cost Not actually Facebook net, but Akamai CDN. Not a Google (peer), but GCC node ;) It is varying from location to location. For example here in Ukraine we (still) have 1st place for traffic amount from Vkontakte (mostly music streams), second from EX.ua (movie store), but almost none NetFlix, Hulu or Amazon. And you can't get both of them in a good quality neither at IXes, nor at Tier1. I think in another locations, for example in India, traffic profile will be different from both of us, and have some local specific as well. On 04/15/15 20:58, Mike Hammett wrote: It also depends on traffic makeup. Huge amounts of eyeball traffic go to (well, come from) NetFlix (a third) and Google, FaceBook, Hulu, Amazon, etc. (another third). It's comparable price to peer off those few huge sources of traffic and buy better transit than you would have than to just buy cheap transit. A lot of people tend to forget there are thousands of independent ISPs out there, usually in areas where there aren't a breadth of providers in the first place. Most could get buy with a single GigE (or even less). - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 12:50:41 PM Subject: Re: Peering and Network Cost Hi Roderick, transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. On 04/15/15 17:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. I thank you in advance for any insights. Regards, - R. Roderick Beck Sales Director/Europe and the Americas Hibernia Networks This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e-mail in error, please immediately telephone or e-mail the sender and permanently delete the original copy and any copy of this e-mail, and any printout thereof. All documents, contracts or agreements referred or attached to this e-mail are SUBJECT TO CONTRACT. The contents of an attachment to this e-mail may contain software viruses that could damage your own computer system. While Hibernia Networks has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage that you sustain as a result of software viruses. You should carry out your own virus checks before opening any attachment.
Re: Peering and Network Cost
On 4/15/15 07:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. You might find https://www.nanog.org/meetings/nanog53/presentations/Tuesday/valancius.pdf and the concomitant http://conferences.sigcomm.org/sigcomm/2011/papers/sigcomm/p194.pdf interesting. -Scot I thank you in advance for any insights. Regards, - R. Roderick Beck Sales Director/Europe and the Americas Hibernia Networks This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e-mail in error, please immediately telephone or e-mail the sender and permanently delete the original copy and any copy of this e-mail, and any printout thereof. All documents, contracts or agreements referred or attached to this e-mail are SUBJECT TO CONTRACT. The contents of an attachment to this e-mail may contain software viruses that could damage your own computer system. While Hibernia Networks has taken every reasonable precaution to minimize this risk, we cannot accept liability for any damage that you sustain as a result of software viruses. You should carry out your own virus checks before opening any attachment.
Re: Peering and Network Cost
Transit cost is down but IX cost remains the same. Therefore IX is longer cost effective for a small ISP. As an (non US) example, here in Copenhagen, Denmark we have two internet exchanges DIX and Netnod. We also have many major transit providers, including Hurricane Electric and Cogent. Netnod price for a 1 Gbps port is 4 SEK = 4500 USD / year http://www.netnod.se/ix/join/prices. DIX is 4 DKK = 5700 USD / year http://dix.dk/serviceinformation/ HE.net is offering 1 Gbps flatrate for 450 USD / month list price = 5400 USD /year. Cogent can match that. So why would a small ISP pay 4500 USD for a service with no guarantee of how much traffic they will be able to peer away? You need to get a 10 Gbps port and be able to peer at least 2-3 Gbps before it is even break even with the deals you get from the transit providers. But then you will notice that all the traffic is only with a few peers and you can just peer directly with those and skip the middleman. Regards, Baldur
Re: Peering and Network Cost
On 2015-04-15 19:50, Max Tulyev wrote: transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. Even in the three cities you mentioned peering on small scale is usually not cheaper at all or only very little. Please keep in mind that some companies peer despite it offers no savings for them and at the end of the day it might be even more expensive. They do it because of performance and reliability reasons. -- Grzegorz Janoszka
Re: Peering and Network Cost
https://www.youtube.com/watch?v=Y43Fy4oU2XE There are reasons to peer other than cost reduction. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Baldur Norddahl baldur.nordd...@gmail.com To: nanog@nanog.org Sent: Wednesday, April 15, 2015 3:07:52 PM Subject: Re: Peering and Network Cost Transit cost is down but IX cost remains the same. Therefore IX is longer cost effective for a small ISP. As an (non US) example, here in Copenhagen, Denmark we have two internet exchanges DIX and Netnod. We also have many major transit providers, including Hurricane Electric and Cogent. Netnod price for a 1 Gbps port is 4 SEK = 4500 USD / year http://www.netnod.se/ix/join/prices. DIX is 4 DKK = 5700 USD / year http://dix.dk/serviceinformation/ HE.net is offering 1 Gbps flatrate for 450 USD / month list price = 5400 USD /year. Cogent can match that. So why would a small ISP pay 4500 USD for a service with no guarantee of how much traffic they will be able to peer away? You need to get a 10 Gbps port and be able to peer at least 2-3 Gbps before it is even break even with the deals you get from the transit providers. But then you will notice that all the traffic is only with a few peers and you can just peer directly with those and skip the middleman. Regards, Baldur
RE: Peering and Network Cost
Most cost models select a capacity figure that represents typical high-watermark utilization before the next cash outlay is triggered. By using your actual utilization, you might be penalizing your cost if you have low utilization and that low utilization is expected to be a temporary situation given the state of your business. That way your cost doesn't increase (for example) as a function of losing a large customer or other traffic shifting event. The only reason I would see some intentionally pick a lower figure is if the dynamic of their specific business suggests that low-utilization interconnect ports are typical for them. -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Mike Hammett Sent: Wednesday, April 15, 2015 12:45 PM To: nanog@nanog.org Subject: Re: Peering and Network Cost (Reply to thread, not necessarily myself.) If you can pull a third of your traffic off at the cost of a cross connect and another third at the cost of an IX port, now you can spend a buck or two a meg on what's left. Yes, I understand the cost of a cross connect or IX port is the $/megabit you're actually using and not $/megabit of capacity. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mike Hammett na...@ics-il.net To: Max Tulyev max...@netassist.ua Cc: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:33:35 PM Subject: Re: Peering and Network Cost Very true. I left it as I did given that I expect a similar profile from others in North America... on NANOG. Basically, wherever your region's streaming video or application updates come from. ;-) - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 1:27:45 PM Subject: Re: Peering and Network Cost Not actually Facebook net, but Akamai CDN. Not a Google (peer), but GCC node ;) It is varying from location to location. For example here in Ukraine we (still) have 1st place for traffic amount from Vkontakte (mostly music streams), second from EX.ua (movie store), but almost none NetFlix, Hulu or Amazon. And you can't get both of them in a good quality neither at IXes, nor at Tier1. I think in another locations, for example in India, traffic profile will be different from both of us, and have some local specific as well. On 04/15/15 20:58, Mike Hammett wrote: It also depends on traffic makeup. Huge amounts of eyeball traffic go to (well, come from) NetFlix (a third) and Google, FaceBook, Hulu, Amazon, etc. (another third). It's comparable price to peer off those few huge sources of traffic and buy better transit than you would have than to just buy cheap transit. A lot of people tend to forget there are thousands of independent ISPs out there, usually in areas where there aren't a breadth of providers in the first place. Most could get buy with a single GigE (or even less). - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Max Tulyev max...@netassist.ua To: nanog@nanog.org Sent: Wednesday, April 15, 2015 12:50:41 PM Subject: Re: Peering and Network Cost Hi Roderick, transit cost is lowering close to peering cost, so it is doubghtful economy on small channels. If you don't live in Amsterdam/Frankfurt/London - add the DWDM cost from you to one of major IX. That's the magic. In large scale peering is still efficient. It is efficient on local traffic which is often huge. On 04/15/15 17:28, Rod Beck wrote: Hi, As you all know, transit costs in the wholesale market today a few percent of what it did in 2000. I assume that most of that decline is due to a modified version of Moore's Law (I don't believe optics costs decline 50% every 18 months) and the advent of maverick players like Cogent that broker cozy oligopoly pricing. But I also wondering whether the advent of widespread peering (promiscuous?) among the Tier 2 players (buy transit and peer) has played a role. In 2000 peering was still an exclusive club and in contrast today Tier 2 players often have hundreds of peers. Peering should reduce costs and also demand in the wholesale IP market. Supply increases and demand falls. I thank you in advance for any insights. Regards, - R. Roderick Beck Sales Director/Europe and the Americas Hibernia Networks This e-mail and any attachments thereto is intended only for use by the addressee(s) named herein and may be proprietary and/or legally privileged. If you are not the intended recipient of this e-mail, you are hereby notified that any dissemination, distribution or copying of this email, and any attachments thereto, without the prior written permission of the sender is strictly prohibited. If you receive this e
Re: Peering and Network Cost
* Baldur Norddahl baldur.nordd...@gmail.com Transit cost is down but IX cost remains the same. Therefore IX is longer cost effective for a small ISP. As an (non US) example, here in Copenhagen, Denmark we have two internet exchanges DIX and Netnod. We also have many major transit providers, including Hurricane Electric and Cogent. Netnod price for a 1 Gbps port is 4 SEK = 4500 USD / year http://www.netnod.se/ix/join/prices. DIX is 4 DKK = 5700 USD / year http://dix.dk/serviceinformation/ HE.net is offering 1 Gbps flatrate for 450 USD / month list price = 5400 USD /year. Cogent can match that. So why would a small ISP pay 4500 USD for a service with no guarantee of how much traffic they will be able to peer away? We're in a similar situation here; transit prices has come down so much in recent years (while IX fees are indeed stagnant) that I am certain that if I were to cut all peering and buy everything from a regional tier-2 instead, I'd be lowering my total MRC somewhat, without really reducing connectivity quality to my (former) peers. For us, the primary reason that keeps us peering is DDoS prevention. Our traffic is mostly regional, so if a customer of mine gets hit with a volumetric DDoS attack that would saturate my IP transit lines and cause collateral damage, that's no big deal as we can just RTBH the customers prefix towards our transit providers. The customer is only mildly inconvenienced by this as, say, 90% of his traffic goes to our peers. Without peering the attack would succeed because my RTBH would completely offline my customer. Tore
