Re: Coop Peering Fabric??
On Mon, Aug 11, 2008 at 11:15:49PM -0400, Deepak Jain wrote: A coop, best-effort switch fabric colo'd at a few sites would allow participants to peer off traffic at a price of the order of a single cross-connect (~$500/month per 10G port is possible, maybe less), private-VLANs all-around, or to only-mutually approved peers (e.g. via an automated web interface, prior art) to avoid many of the /old/ issues. No requirement for multi-lateral peering. You could peer, sell transit, buy transit, multicast, etc. This has been working for years at http://www.torix.net , and on a smaller scale at http://www.ottix.net -- Jim Mercer[EMAIL PROTECTED]+971 55 410-5633 I'm Prime Minister of Canada, I live here and I'm going to take a leak. - Lester Pearson in 1967, during a meeting between himself and President Lyndon Johnson, whose Secret Service detail had taken over Pearson's cottage retreat. At one point, a Johnson guard asked Pearson, Who are you and where are you going?
Re: Coop Peering Fabric??
Deepak, If it were as easy as you make it sound, I can assure you people would be doing it. Also, does your Equinix MSA contain a non-compete clause, which could be interpreted to mean you can't run a competing IX (metro fabric, exchange, whatever) out of their facilities? I hear many do. Drive Slow, PAUL WALL On Mon, Aug 11, 2008 at 11:15 PM, Deepak Jain [EMAIL PROTECTED] wrote: Warning: This may actually be operational too. Given Cogent (and others) recent pursuit of sub $4/mb/s transit... and the relatively flat cost of a paid peering fabric (even at 10G) and the O(N) costs for cross-connects, the thought of revisiting the old peering coops presented itself again. Assuming 10G PNI model: Assuming even nominal cross-connect fees of $100-$300/month per fiber pair, plus router port costs for each private peer (assuming you aren't at 10% utilization on the port) at a commercial exchange, you are eating a pretty significant cost per megabit you are actually moving. (plug in your numbers here). Assumption: Above 1Gb/s utilization, this makes sense or you are counting on growth. Below 10% you would normally go to a paid peering fabric where you are paying cross connect + a flat port charge + router port for 1-N peers and hoping that enough utilization occurs that you get 10% utilization (to recover capex, opex, etc) and then whatever additional utilization you need to cover the flat port charge or you are counting on growth. A coop, best-effort switch fabric colo'd at a few sites would allow participants to peer off traffic at a price of the order of a single cross-connect (~$500/month per 10G port is possible, maybe less), private-VLANs all-around, or to only-mutually approved peers (e.g. via an automated web interface, prior art) to avoid many of the /old/ issues. No requirement for multi-lateral peering. You could peer, sell transit, buy transit, multicast, etc. The way I figure it, it removes approximately an order of magnitude from the operational cost of peering with more than a handful of your largest single talkers. Especially as 100G LAN Ethernet becomes production before 100G WAN connections become commonplace. Economic theory (assuming that worked on the Internet) suggests this would allow for the increase in number of peers by approximately an order of magnitude (maybe more). Does this actually improve the present-day rationale to peer, or are most operations' costs so far above (from long haul, etc) or so far below (since the cost of transit has dropped so much) that this is no longer a relevant part of the equation? Warning: This may actually be operational too. Deepak Jain AiNET
Re: Coop Peering Fabric??
On Aug 12, 2008, at 3:37 AM, Paul Wall wrote: If it were as easy as you make it sound, I can assure you people would be doing it. People are. I (and others) mentioned SIX TorIX, plus I mentioned PaNAP. Then there's AtlantaIX, although that recently got slurped by TelX. (Hrmmm, could one of the dangers of a coop be borg'ed by for- profit entity looking to rip out every cent they can? :) Tons of others exist, in big and little markets. There's one in 365 Main SF, there's KleyReX in the same building as DE-CIX, Big APE in 111 8th, NYCx there too, ChicagoIX just opened, etc., etc. Trust me, it _is_ being done. Also, does your Equinix MSA contain a non-compete clause, which could be interpreted to mean you can't run a competing IX (metro fabric, exchange, whatever) out of their facilities? I hear many do. So don't run it in an Equinix or SD cage. -- TTFN, patrick On Mon, Aug 11, 2008 at 11:15 PM, Deepak Jain [EMAIL PROTECTED] wrote: Warning: This may actually be operational too. Given Cogent (and others) recent pursuit of sub $4/mb/s transit... and the relatively flat cost of a paid peering fabric (even at 10G) and the O(N) costs for cross-connects, the thought of revisiting the old peering coops presented itself again. Assuming 10G PNI model: Assuming even nominal cross-connect fees of $100-$300/month per fiber pair, plus router port costs for each private peer (assuming you aren't at 10% utilization on the port) at a commercial exchange, you are eating a pretty significant cost per megabit you are actually moving. (plug in your numbers here). Assumption: Above 1Gb/s utilization, this makes sense or you are counting on growth. Below 10% you would normally go to a paid peering fabric where you are paying cross connect + a flat port charge + router port for 1-N peers and hoping that enough utilization occurs that you get 10% utilization (to recover capex, opex, etc) and then whatever additional utilization you need to cover the flat port charge or you are counting on growth. A coop, best-effort switch fabric colo'd at a few sites would allow participants to peer off traffic at a price of the order of a single cross-connect (~$500/month per 10G port is possible, maybe less), private-VLANs all-around, or to only-mutually approved peers (e.g. via an automated web interface, prior art) to avoid many of the /old/ issues. No requirement for multi-lateral peering. You could peer, sell transit, buy transit, multicast, etc. The way I figure it, it removes approximately an order of magnitude from the operational cost of peering with more than a handful of your largest single talkers. Especially as 100G LAN Ethernet becomes production before 100G WAN connections become commonplace. Economic theory (assuming that worked on the Internet) suggests this would allow for the increase in number of peers by approximately an order of magnitude (maybe more). Does this actually improve the present-day rationale to peer, or are most operations' costs so far above (from long haul, etc) or so far below (since the cost of transit has dropped so much) that this is no longer a relevant part of the equation? Warning: This may actually be operational too. Deepak Jain AiNET
Re: Coop Peering Fabric??
Yes you are absolutely correct. Smaller players doing this for fun and experimentation if not only a good idea, I believe it is critical for the internet to grow and change. Ask UUNET how long it takes them to get approval to implement something bigor even small. Two pts thought. First, the cross connects at most of these locations are still going to be a major monthly INVESTMENT unless the colo provider gets involved. As was stated earlier $500 MRC has to be justified be costs savings or other benefits. Second, I have heard a lot of talk about SIX over the last year or so and there is no guarantees that situation won't change. Telx and others can do a fine job. I have not heard Any2 mentioned and their traffic levels have been very good while keeping ports cost effective. Can that model scale? Basically it's about the community deciding to support something. Perhaps it's more about the players then the best model. This business is still run significantly on trust and reputation of the people running the infrastructure. No? David On Tue, Aug 12, 2008 at 10:11 AM, Patrick W. Gilmore [EMAIL PROTECTED]wrote: On Aug 12, 2008, at 9:58 AM, David Diaz wrote: Love the Borg comment. Thanx. Great thread. Old topic. It recycles every couple of years. Not to speak for telx or Mike L but I do not think anyone was motivated to Borg anything but to support AIX. 10Gig ports are expensive. I like the idea of more exchange points in that they usually provide more recovery pts and redundancy, allow the sharing of skills and knowledge in the local community, and provide flexibility for growth and change of the internet. How many COs do we have? There has long been the argument of how many IXs are needed, would it be 1 per state? What happens with Voip, IPtv etc. As for coops I think the argument is would the larger traffic players feel comfortable connecting and making it a part of their networks? Who are the anchors and 1st movers? What are the guarantees that any investment in infrastructure needed to get there will be recovered over X years... Will the coop fold before that pt? Wll it have the resources to upgrade. Who said anything about larger traffic players? What's wrong with a bunch of little guys getting together to trade traffic, for fun and profit? The smaller guys might have a better focus on performance in the local area (gamers anyone?), plus they tend to pay more per Mbps because they don't have scale, which makes moving a little traffic off more economical. All that said, Akamai is a pretty big network and they're present at a lot of these small IXen. Ditto for local eyeball networks, e.g. Shaw @ SIX, Rogers @ TorIX, etc. I so not think a poison pill is needed. Perhaps just a group or company championing Coops and giving them booth-space at events, sponsoring conference travels, providing rack space etc. But if it's in the BEST interest of the members to have a larger group come in and take over then what is the harm? What is the alternative, have members pay membership fees? Corp Sponsorship? I agree on much of this. But as with most things it comes down to money. Do members have a financial incentive to join and what is the financial model to keep the Coop moving forward as a success. Several small IXes have grown quite a bit with no or very small membership fees. Look at the ones I mentioned. I think SIX is the largest, but they're all not that tiny. -- TTFN, patrick
Re: Coop Peering Fabric??
On Tue, Aug 12, 2008 at 10:11:13AM -0400, Patrick W. Gilmore wrote: Several small IXes have grown quite a bit with no or very small membership fees. Look at the ones I mentioned. I think SIX is the largest, but they're all not that tiny. TorIX, for many years, was financed by announcing an upcoming expediture, and waiting to see if one of the members stepped up (or usually, the member suggesting the expenditure, also covering its cost), and if no-one was willing to foot the entire bill, the hat was passed around until it filled sufficiently. they have since formalized into a not-for-profit (i stepped away, physically and involvement-wise), but my understanding is that financially, it is using the same funding model. TorIX was initially founded by driving a stake (a single Cisco 2900 as i recall) in the ground and inviting all-comers (each having to simply pay to drag connectivity to the stake). the initial membership was small to medium (quasi-large) ISP's, the largest of which were finding they were locked out of the incumbent IX (CanIX) for various financial and political reasons. (that CanIX appears to have vaporized, and its name now taken by some colo provider) some joined for monetary reasons, some for the fun of it, others because it became a cost effective way to shunt packets (even when weighed against the best-effort management) TorIX is now sustaining 10Gbps across some 90+ peers, with a decent spectrum of eyeballs, content-only providers and transit providers. i would bet that if someone analyzed the data, that it has maintained 5 9's reliability too, or pretty damn close for a best-effort facility. -- Jim Mercer[EMAIL PROTECTED]+971 55 410-5633 I'm Prime Minister of Canada, I live here and I'm going to take a leak. - Lester Pearson in 1967, during a meeting between himself and President Lyndon Johnson, whose Secret Service detail had taken over Pearson's cottage retreat. At one point, a Johnson guard asked Pearson, Who are you and where are you going?
Re: Coop Peering Fabric??
I guess they would be more interesting deployed in Ashburn or some place similar because you could exclude the cost of bringing traffic to the exchange if the equipment (and bits) are already transported through that facility. Certainly there are some of us who would see this as advantageous. The cost of going through the Equinix public switch is relatively high, high enough that at the point we could justify it, it's cheaper and easier to just run a private connection or ten, and have more peering capacity, which turns into an argument against the Equinix service. Were it just the cost of a cross-connect plus a modest membership fee, with at least some other participants that had a relatively open peering policy, it would be quite interesting. Bonus points for being able to buy transit or routes. I had been working towards doing something like this in the Milwaukee area years ago, but the volume and interest wasn't quite there. I can't easily see it failing in the same way in Ashburn... there are a bunch of people who we exchange traffic with that are in the XXMbps range, maybe not enough to justify a private cross connect, but certainly good enough for a shared switch. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: Coop Peering Fabric??
Speaking of AtlantaIX, the new business model seems less attractive for customers than the old one. Can anyone speak to why it got sold? Was it failing financially or someone just wanted to cash out? On 8/12/08, Patrick W. Gilmore [EMAIL PROTECTED] wrote: On Aug 12, 2008, at 3:37 AM, Paul Wall wrote: If it were as easy as you make it sound, I can assure you people would be doing it. People are. I (and others) mentioned SIX TorIX, plus I mentioned PaNAP. Then there's AtlantaIX, although that recently got slurped by TelX. (Hrmmm, could one of the dangers of a coop be borg'ed by for-profit entity looking to rip out every cent they can? :) Tons of others exist, in big and little markets. There's one in 365 Main SF, there's KleyReX in the same building as DE-CIX, Big APE in 111 8th, NYCx there too, ChicagoIX just opened, etc., etc. Trust me, it _is_ being done. Also, does your Equinix MSA contain a non-compete clause, which could be interpreted to mean you can't run a competing IX (metro fabric, exchange, whatever) out of their facilities? I hear many do. So don't run it in an Equinix or SD cage. -- TTFN, patrick On Mon, Aug 11, 2008 at 11:15 PM, Deepak Jain [EMAIL PROTECTED] wrote: Warning: This may actually be operational too. Given Cogent (and others) recent pursuit of sub $4/mb/s transit... and the relatively flat cost of a paid peering fabric (even at 10G) and the O(N) costs for cross-connects, the thought of revisiting the old peering coops presented itself again. Assuming 10G PNI model: Assuming even nominal cross-connect fees of $100-$300/month per fiber pair, plus router port costs for each private peer (assuming you aren't at 10% utilization on the port) at a commercial exchange, you are eating a pretty significant cost per megabit you are actually moving. (plug in your numbers here). Assumption: Above 1Gb/s utilization, this makes sense or you are counting on growth. Below 10% you would normally go to a paid peering fabric where you are paying cross connect + a flat port charge + router port for 1-N peers and hoping that enough utilization occurs that you get 10% utilization (to recover capex, opex, etc) and then whatever additional utilization you need to cover the flat port charge or you are counting on growth. A coop, best-effort switch fabric colo'd at a few sites would allow participants to peer off traffic at a price of the order of a single cross-connect (~$500/month per 10G port is possible, maybe less), private-VLANs all-around, or to only-mutually approved peers (e.g. via an automated web interface, prior art) to avoid many of the /old/ issues. No requirement for multi-lateral peering. You could peer, sell transit, buy transit, multicast, etc. The way I figure it, it removes approximately an order of magnitude from the operational cost of peering with more than a handful of your largest single talkers. Especially as 100G LAN Ethernet becomes production before 100G WAN connections become commonplace. Economic theory (assuming that worked on the Internet) suggests this would allow for the increase in number of peers by approximately an order of magnitude (maybe more). Does this actually improve the present-day rationale to peer, or are most operations' costs so far above (from long haul, etc) or so far below (since the cost of transit has dropped so much) that this is no longer a relevant part of the equation? Warning: This may actually be operational too. Deepak Jain AiNET
Re: Coop Peering Fabric??
On Aug 12, 2008, at 10:23 AM, David Diaz wrote: Second, I have heard a lot of talk about SIX over the last year or so and there is no guarantees that situation won't change. As a board member of SIX, I can tell you that we are not going away any time soon. -- TTFN, patrick
Re: Coop Peering Fabric??
On 12 Aug 2008, at 04:15, Deepak Jain wrote: A coop, best-effort switch fabric colo'd at a few sites would allow participants to peer off traffic at a price of the order of a single cross-connect (~$500/month per 10G port is possible, maybe less) Most of the Internet Exchanges in Europe that quickly spring to mind as successful, are run as co-operative entities, similar to what you describe. Specifically, most (all?) of the larger ones over here run as independent bodies that are owned mutually -- that is to say, owned by all of the participators at the exchange. The model is popular, and many hundreds of GB/s of traffic is exchanged on switches run by mutual organisations in Europe. This works really well because it means there is no commercial/profit motivation to operate significantly above cost-recovery levels. Here, costs mean the CapEx, OpEx, and any community/member sanctioned projects. Where it breaks is when we have to tell a network with lots of traffic that in order to participate at the exchange, they have to become a member (part owner) of the organisation. Due to organisational or even regulatory issues, it may not be legal to sell services (exchange ports) to non members/owners. This doesn't frighten the engineer asking for a connection, but it causes some concern at C*O level (err, I might have to declare this to shareholders/regulators...) I think my message to you would be that if you have a bunch of colleagues at other organisations near you that want to start peering ... configure a switch, peer, and take it from there as you grow ! I hope your new exchange is successful ! Best wishes Andy Davidson Declared hat - www.lonap.net (London, UK based mutual IX)
Re: Coop Peering Fabric??
On Tue, 12 Aug 2008, Patrick W. Gilmore wrote: Could one of the dangers of a coop be borg'ed by for-profit entity looking to rip out every cent they can? That's one of the reasons many of them incorporate as non-profits... Under the tax laws of most countries, the U.S. and Canada included, non-profits are legaly protected against acquisition by for-profits. -Bill
RE: Coop Peering Fabric??
-Original Message- From: Bill Woodcock [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 12, 2008 12:33 PM To: Patrick W. Gilmore Cc: NANOG list Subject: Re: Coop Peering Fabric?? On Tue, 12 Aug 2008, Patrick W. Gilmore wrote: Could one of the dangers of a coop be borg'ed by for-profit entity looking to rip out every cent they can? That's one of the reasons many of them incorporate as non-profits... Under the tax laws of most countries, the U.S. and Canada included, non-profits are legaly protected against acquisition by for-profits. Do any of these operations post their tax returns online? -M
Re: Coop Peering Fabric??
On Tue, 12 Aug 2008, Paul Wall wrote: If it were as easy as you make it sound, I can assure you people would be doing it. Yup, they are. There are a bit over three hundred IXPs in the world, about eighty of them in the U.S., and the vast majority of them were built by ISPs solving problems for themselves, as Deepak is suggesting. -Bill
Re: Coop Peering Fabric??
On Tue, Aug 12, 2008 at 8:32 AM, Patrick W. Gilmore [EMAIL PROTECTED] wrote: Tons of others exist, in big and little markets. There's one in 365 Main SF, there's KleyReX in the same building as DE-CIX, Big APE in 111 8th, NYCx there too, ChicagoIX just opened, etc., etc. Excellent point on Europe. Not so much in the United States. Do SFMIX, BIG APE, NYCX, etc 1) have more than a half dozen participants 2) exchange any traffic other than BGP keep-alives and ARP? :) I think not. When you look at why not, it's usually always predatory practices on the part of various collo and IX operators preventing widespread adoptation. If CHIX were doing real traffic, do you think Equinix would allow them to remain accessible from their suites, and in a cost-effective manner? Trust me, it _is_ being done. It's being done, just not on a large scale in the United States outside of the SIX. Paul
Re: Coop Peering Fabric??
Paul Wall wrote: On Tue, Aug 12, 2008 at 8:32 AM, Patrick W. Gilmore [EMAIL PROTECTED] wrote: Tons of others exist, in big and little markets. There's one in 365 Main SF, there's KleyReX in the same building as DE-CIX, Big APE in 111 8th, NYCx there too, ChicagoIX just opened, etc., etc. Excellent point on Europe. Not so much in the United States. Do SFMIX, BIG APE, NYCX, etc 1) have more than a half dozen participants 2) exchange any traffic other than BGP keep-alives and ARP? :) I think not. When you look at why not, it's usually always predatory practices on the part of various collo and IX operators preventing widespread adoptation. If CHIX were doing real traffic, do you think Equinix would allow them to remain accessible from their suites, and in a cost-effective manner? Trust me, it _is_ being done. It's being done, just not on a large scale in the United States outside of the SIX. Is there a more appropriate place for interested parties to discuss the possible creation of such a beast in the WDC area? I know we have about a lot of optical capacity we could help contribute to a stake in the ground between Equinix/Ash and a facility less than 1ms away if there is interest. Deepak
Re: Coop Peering Fabric??
On Tue, Aug 12, 2008 at 5:06 PM, Deepak Jain [EMAIL PROTECTED] wrote: Is there a more appropriate place for interested parties to discuss the possible creation of such a beast in the WDC area? I know we have about a lot of optical capacity we could help contribute to a stake in the ground between Equinix/Ash and a facility less than 1ms away if there is interest. And people in the Equinix campus would connect to this exchange how exactly? I'm not trying to downplay your generous offer, though I'm afraid you're missing the underlying problem. Drive Slow, Paul
Re: Coop Peering Fabric??
Paul Wall wrote: On Tue, Aug 12, 2008 at 5:06 PM, Deepak Jain [EMAIL PROTECTED] wrote: Is there a more appropriate place for interested parties to discuss the possible creation of such a beast in the WDC area? I know we have about a lot of optical capacity we could help contribute to a stake in the ground between Equinix/Ash and a facility less than 1ms away if there is interest. And people in the Equinix campus would connect to this exchange how exactly? I'm not trying to downplay your generous offer, though I'm afraid you're missing the underlying problem. Cross-connects to a cabinet @ Equinix same as if the switch were on-site? If Equinix were to block cross-connects inside their facility, that would seem a little farther reaching than a non-compete. Deepak
Re: Coop Peering Fabric??
On Aug 12, 2008, at 5:06 PM, Deepak Jain wrote: Is there a more appropriate place for interested parties to discuss the possible creation of such a beast in the WDC area? I know we have about a lot of optical capacity we could help contribute to a stake in the ground between Equinix/Ash and a facility less than 1ms away if there is interest. I don't know anything about your optical capacity, but it sure does seem like ANY2 DC has everything you are looking for except for easy access from Ashburn. It seems to me an organization (coop, non-profit, etc) that could enable access to Any2 from Ashburn would be quite interesting. CRG might even help. -Matt
Re: Coop Peering Fabric??
Matt Liotta wrote: On Aug 12, 2008, at 5:06 PM, Deepak Jain wrote: Is there a more appropriate place for interested parties to discuss the possible creation of such a beast in the WDC area? I know we have about a lot of optical capacity we could help contribute to a stake in the ground between Equinix/Ash and a facility less than 1ms away if there is interest. I don't know anything about your optical capacity, but it sure does seem like ANY2 DC has everything you are looking for except for easy access from Ashburn. It seems to me an organization (coop, non-profit, etc) that could enable access to Any2 from Ashburn would be quite interesting. CRG might even help. There are lots of providers that can do connectivity between Any2 and Equinix. It has been suggested privately that some Equinix MSAs may prevent this sort of thing. In fact, to prevent this sort of thing, I suggested providing x-connects from 1275 AND Equinix to another facility to prevent Borg'ing ops in the future. I am not aware of Any2 pricing, but I'm sure the 6 members of CRG K Street's Any2 would be happy to join any new initiative (either a larger Any2 or something new). Deepak
Re: Coop Peering Fabric??
On Aug 12, 2008, at 6:17 PM, Deepak Jain wrote: There are lots of providers that can do connectivity between Any2 and Equinix. It has been suggested privately that some Equinix MSAs may prevent this sort of thing. In fact, to prevent this sort of thing, I suggested providing x-connects from 1275 AND Equinix to another facility to prevent Borg'ing ops in the future. I am not aware of Any2 pricing, but I'm sure the 6 members of CRG K Street's Any2 would be happy to join any new initiative (either a larger Any2 or something new). My understanding is that ports are currently free on Any2. I think I remember that normally they are $1000 annually for GigE. CRG has also indicated that they plan to interconnect their Any2 fabrics in NYC and Miami with DC much like they have done in California. -Matt
RE: Coop Peering Fabric??
Matt Any2 is open to support any initiative that will reinforce development of networks and creativity within the Internet-connected community. There have been somewhat successful initiatives at locations such as the SIX to interconnect exchange points, and Any2 is open to contributing to similar projects. In locations such as California and Washington DC, Equinix and CRG West have many common facility-based and services networks. A tenant in either location should find it fairly easy to interconnect with a 3rd party between the facilities. To my knowledge CRG West, Equinix, SD, Savvis, nor any other collocation or IXP provider prejudices tenants for interconnections terminating beyond their demarcation point. We certainly do not prevent cross-connects outside of our properties to competitor sites. In a couple of our properties we even facility-manage multiple IXPs within the same building No desire to BORG operations! John -Original Message- From: Matt Liotta [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 12, 2008 4:30 PM To: NANOG list Subject: Re: Coop Peering Fabric?? On Aug 12, 2008, at 6:17 PM, Deepak Jain wrote: There are lots of providers that can do connectivity between Any2 and Equinix. It has been suggested privately that some Equinix MSAs may prevent this sort of thing. In fact, to prevent this sort of thing, I suggested providing x-connects from 1275 AND Equinix to another facility to prevent Borg'ing ops in the future. I am not aware of Any2 pricing, but I'm sure the 6 members of CRG K Street's Any2 would be happy to join any new initiative (either a larger Any2 or something new). My understanding is that ports are currently free on Any2. I think I remember that normally they are $1000 annually for GigE. CRG has also indicated that they plan to interconnect their Any2 fabrics in NYC and Miami with DC much like they have done in California. -Matt
RE: Coop Peering Fabric??
On Tue, 12 Aug 2008, Martin Hannigan wrote: On Tue, 12 Aug 2008, Bill Woodcock wrote: On Tue, 12 Aug 2008, Patrick W. Gilmore wrote: Could one of the dangers of a coop be borg'ed by for-profit entity looking to rip out every cent they can? That's one of the reasons many of them incorporate as non-profits... Under the tax laws of most countries, the U.S. and Canada included, non-profits are legaly protected against acquisition by for-profits. Do any of these operations post their tax returns online? The Seattle IX (SIX) filings, along with financial reports to the membership, are openly maintained at: http://www.seattleix.net/docs/ Chris
Re: Coop Peering Fabric??
That's one of the reasons many of them incorporate as non-profits... Under the tax laws of most countries, the U.S. and Canada included, non-profits are legaly protected against acquisition by for-profits. Do any of these operations post their tax returns online? -M They might be posted at http://www.guidestar.org/ -- _ Nachman Yaakov Ziskind, FSPA, LLM [EMAIL PROTECTED] Attorney and Counselor-at-Law http://ziskind.us Economic Group Pension Services http://egps.com Actuaries and Employee Benefit Consultants
Re: Coop Peering Fabric??
That's one of the reasons many of them incorporate as non-profits... Under the tax laws of most countries, the U.S. and Canada included, non-profits are legaly protected against acquisition by for-profits. Do any of these operations post their tax returns online? In the US, every non-profit has to file an annual financial report on form 990 or, for small ones, 990-EZ. These are by law open to public inspection, and if you call, write, fax or e-mail them and ask for a copy they better send you one. The Foundation Center has a fairly good online 990 database: http://tfcny.fdncenter.org/990s/990search/esearch.php If you're wondering what my signature looks like, search for Domain Assurance Council R's, John
Re: Coop Peering Fabric??
On Aug 12, 2008, at 4:48 PM, Paul Wall wrote: On Tue, Aug 12, 2008 at 8:32 AM, Patrick W. Gilmore [EMAIL PROTECTED] wrote: Tons of others exist, in big and little markets. There's one in 365 Main SF, there's KleyReX in the same building as DE-CIX, Big APE in 111 8th, NYCx there too, ChicagoIX just opened, etc., etc. Excellent point on Europe. Not so much in the United States. Do SFMIX, BIG APE, NYCX, etc 1) have more than a half dozen participants 2) exchange any traffic other than BGP keep-alives and ARP? :) I think not. When you look at why not, it's usually always predatory practices on the part of various collo and IX operators preventing widespread adoptation. If CHIX were doing real traffic, do you think Equinix would allow them to remain accessible from their suites, and in a cost-effective manner? I'm guessing the answer to 1 2 is yes. Proof of at least participant count: http://www.ny6ix.net/. Trust me, it _is_ being done. It's being done, just not on a large scale in the United States outside of the SIX. Define large. For instance, Atlanta IX had more traffic than PAIX in the same building last I checked. And how large does it need to be to save a network $300/month? -- TTFN, patrick
Coop Peering Fabric??
Warning: This may actually be operational too. Given Cogent (and others) recent pursuit of sub $4/mb/s transit... and the relatively flat cost of a paid peering fabric (even at 10G) and the O(N) costs for cross-connects, the thought of revisiting the old peering coops presented itself again. Assuming 10G PNI model: Assuming even nominal cross-connect fees of $100-$300/month per fiber pair, plus router port costs for each private peer (assuming you aren't at 10% utilization on the port) at a commercial exchange, you are eating a pretty significant cost per megabit you are actually moving. (plug in your numbers here). Assumption: Above 1Gb/s utilization, this makes sense or you are counting on growth. Below 10% you would normally go to a paid peering fabric where you are paying cross connect + a flat port charge + router port for 1-N peers and hoping that enough utilization occurs that you get 10% utilization (to recover capex, opex, etc) and then whatever additional utilization you need to cover the flat port charge or you are counting on growth. A coop, best-effort switch fabric colo'd at a few sites would allow participants to peer off traffic at a price of the order of a single cross-connect (~$500/month per 10G port is possible, maybe less), private-VLANs all-around, or to only-mutually approved peers (e.g. via an automated web interface, prior art) to avoid many of the /old/ issues. No requirement for multi-lateral peering. You could peer, sell transit, buy transit, multicast, etc. The way I figure it, it removes approximately an order of magnitude from the operational cost of peering with more than a handful of your largest single talkers. Especially as 100G LAN Ethernet becomes production before 100G WAN connections become commonplace. Economic theory (assuming that worked on the Internet) suggests this would allow for the increase in number of peers by approximately an order of magnitude (maybe more). Does this actually improve the present-day rationale to peer, or are most operations' costs so far above (from long haul, etc) or so far below (since the cost of transit has dropped so much) that this is no longer a relevant part of the equation? Warning: This may actually be operational too. Deepak Jain AiNET
Re: Coop Peering Fabric??
As a big ra-ra guy around peering, I thought this might be interesting, but I do not think I agree with the numbers. On Aug 11, 2008, at 11:15 PM, Deepak Jain wrote: Given Cogent (and others) recent pursuit of sub $4/mb/s transit... and the relatively flat cost of a paid peering fabric (even at 10G) and the O(N) costs for cross-connects, the thought of revisiting the old peering coops presented itself again. The $4/Mbps under prices is usually reserved for very large CIR or full pipe. With a full pipe, assuming you are very, very, very good, you still pay $5 or more. (I'm assuming a max of 8 Gbps on a 10G pipe, which seems overly optimistic IMHO.) But that's not important to the discussion. Assuming 10G PNI model: Assuming even nominal cross-connect fees of $100-$300/month per fiber pair, plus router port costs for each private peer (assuming you aren't at 10% utilization on the port) at a commercial exchange, you are eating a pretty significant cost per megabit you are actually moving. (plug in your numbers here). Assumption: Above 1Gb/s utilization, this makes sense or you are counting on growth. Define significant? Taking 1 Gbps (10%), and assuming even 20K per 10G port over 2 years, adding in $300/month for the x-conn, you are still looking at barely over $1/Mbps. If you have more than 10% utilization, that number goes down. Is that significant? Compared to what? Transit? I would say a 75% price reduction is pretty significant. Plus you haven't considered CapEx cost for the transit ports. Below 10% you would normally go to a paid peering fabric where you are paying cross connect + a flat port charge + router port for 1-N peers and hoping that enough utilization occurs that you get 10% utilization (to recover capex, opex, etc) and then whatever additional utilization you need to cover the flat port charge or you are counting on growth. Here we agree. The port fee even on european IXes is measured in 1000s of $$ per month. And don't get me started on US or Japanese ports A coop, best-effort switch fabric colo'd at a few sites would allow participants to peer off traffic at a price of the order of a single cross-connect (~$500/month per 10G port is possible, maybe less), private-VLANs all-around, or to only-mutually approved peers (e.g. via an automated web interface, prior art) to avoid many of the /old/ issues. No requirement for multi-lateral peering. You could peer, sell transit, buy transit, multicast, etc. The way I figure it, it removes approximately an order of magnitude from the operational cost of peering with more than a handful of your largest single talkers. Especially as 100G LAN Ethernet becomes production before 100G WAN connections become commonplace. Economic theory (assuming that worked on the Internet) suggests this would allow for the increase in number of peers by approximately an order of magnitude (maybe more). Sorry, I can't get there. First, the largest single talkers would not be on a shared fabric, they'd be on dedicated ports, so this idea doesn't help. For the medium to small guys, I think it's a great idea. Look at SIX, TorIX, PaNAP, etc. But shaving an _order of magnitude_ off? No, I don't see it. CapEx alone is more than 10% of your cost. (Well, unless you get Japanese IX ports or the most expensive US IX ports.) Perhaps I'm lost or confused? Can someone help me understand? -- TTFN, patrick
Re: Coop Peering Fabric??
On Mon, 11 Aug 2008, Deepak Jain wrote: A coop, best-effort switch fabric colo'd at a few sites would allow participants to peer off traffic at a price of the order of a single cross-connect (~$500/month per 10G port is possible, maybe less), $0/month per 10G port is common enough. https://www.seattleix.net/faq.htm Why pay someone else to let you use an Ethernet switch? Presumably if you can configure BGP, plugging into an Ethernet switch is well within your core competency. -Bill
Re: Coop Peering Fabric??
Bill Woodcock wrote: On Mon, 11 Aug 2008, Deepak Jain wrote: A coop, best-effort switch fabric colo'd at a few sites would allow participants to peer off traffic at a price of the order of a single cross-connect (~$500/month per 10G port is possible, maybe less), $0/month per 10G port is common enough. https://www.seattleix.net/faq.htm Why pay someone else to let you use an Ethernet switch? Presumably if you can configure BGP, plugging into an Ethernet switch is well within your core competency. The only point of a fee would be to provide better than when-we-get-around-to-it support. Obviously there are ways to achieve this without a fee. There are other benefits too [like the ability to have a real non-profit structure, insurances, and others to address the inevitable subpoena, wire-tapping request, CNN reporter, etc]. The economization of cross-connects (a large percentage of a certain colo provider's gross revenues and profit growth) does NOT occur when each provider sets up its own L2 switch for its peers to connect to -- unless they can peer with each other over that switch too. Which brings you back to a coop. Deepak
