Yes that’s exactly it (the major IX but some others contributed as well)…. And certain peers we turned up additional PNI connections with (which typically are preferred over IX connections) ….
-p From: Af [mailto:[email protected]] On Behalf Of Mike Hammett Sent: June 20, 2016 4:07 PM To: [email protected] Subject: Re: [AFMUG] Peering - WAS: Seen this Brocade before? "However – the biggest reason for the traffic drop isn’t the additional peering locations as much as our largest peers are now closer to us than ever on the most cost effective peering IX we are connected to, and several of them employ cold potato routing so win/win for us all the way around." I had a tough read of that. Meaning you're on TorIX and more of your peers came to TorIX leaving the American ones less needed? ----- Mike Hammett <http://www.ics-il.com/> Intelligent Computing Solutions <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> <http://www.midwest-ix.com/> Midwest Internet Exchange <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> <http://www.thebrotherswisp.com/> The Brothers WISP <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> _____ From: "Paul Stewart" <[email protected] <mailto:[email protected]> > To: [email protected] <mailto:[email protected]> Sent: Monday, June 20, 2016 3:01:34 PM Subject: [AFMUG] Peering - WAS: Seen this Brocade before? My caffeine is really working well still so I’ll continue to take us further off track lol ;) Peering is an interesting concept and I’m a big advocate of it before anyone jumps on me about this topic – done it internationally and regionally in my career with different companies. The value proposition of peering is dropping anymore … I still very much think it’s a great thing but the volumes you need to build a business case are getting more difficult. I’ll give an example – at $$job, we have a site in NYC specifically for peering. When this site was built out, it was a tough sell but at that time it was our first peering POP in the US and it was going to save us a limited amount of money and increase performance … the latter being the big sell. Capital costs are one thing (Juniper MX480, management network, out of band etc – pretty typical stuff) but the MRC is always the biggest concern (OPEX vs CAPEX). We lit some 10G ports on NYIIX and also on DECIX-NY and traffic has been “ok” but over time the amount of traffic has actually dropped considerably…. Two reasons for the drops – one is the we, just last year, turned up peering at Chicago Equinix IX and this caused some traffic to drop off from NYC… and ironically we are turning up new connections to Ashburn soon too ….. this leaves a significant amount of capacity sitting idle, which on a Mb/s cost basis is quite high suddenly. However – the biggest reason for the traffic drop isn’t the additional peering locations as much as our largest peers are now closer to us than ever on the most cost effective peering IX we are connected to, and several of them employ cold potato routing so win/win for us all the way around. If we dropped NYC, Ashburn, and CHI from our peering we would actually save money at this point … and performance would only be mildly impacted. I still believe peering on IX’s (or better yet via PNI’s) are the right thing to do, but if you sometimes look at pure economy portion of it that part is really hurting due to extremely low transit costs (from quality providers)…. Paul From: Af [mailto:[email protected]] On Behalf Of Erich Kaiser Sent: June 20, 2016 1:11 PM To: [email protected] <mailto:[email protected]> Subject: Re: [AFMUG] Seen this Brocade before? I think what I am confused about, is that it seems like Midwest-IX(Maybe just Mike LOL) is spearheading this thing when really there are multiple entities involved. Can you explain to everyone how this is all going to go down. I know Faisal and some others run an IX in Atlanta already. Plus there are existing IXs in some of these other locations, will they all be connecting to the magical network ring :) We just need to know realistic plans timframes on where to be to pick them up so we can pass downstream to WISPs and plan Fusion expansion appropriately. BTW from Fusion perspective, we already have built to most major peering POPs in the US. Ashburn, NYC and MIN are all that are left. Yes this has gotten off topic kinda. My coffee buzz is wearing out..... On Mon, Jun 20, 2016 at 11:03 AM, Mike Hammett <[email protected] <mailto:[email protected]> > wrote: Some think remote peering is critical, while some think it's a travesty. Customers ask for it and there was no viable alternative so we are doing it. Transit provider, no. Contradictory, no. Transit is full routes. This would be far from full routes. Pull out the big players and you're talking 4 digit, maybe 5 digit number of prefixes. It wouldn't be contradictory in that it's keeping the traffic as local as possible. It's not bouncing among however many upstream providers that neither end customer have any control over.It's customer, IX, remote peering, IX, customer. The primary intent is to provide a presence on our IXes to additional markets. There is no chance of a major player coming in and joining. They simply don't do that. Other than big content, most of the networks joining are going to be unique to the peering scene. Looking to leverage the value that peering brings (reduced costs, better performance, etc.) to people that can't normally avail themselves of it. Say Chuck breaks out his Chicago - Kentucky link in Indy and he wants to join the Indy IX. Say Nathan joins the St. Louis IX. They may want to talk to each other for whatever reason (say mailing list performance is imperative to Nathan). Getting transport between Chuck and Nathan for whatever amount of traffic they'd have between (likely not much) wouldn't be worth whatever a transport company would charge for it. At some point it's more cost effective to DIY, but those aren't the target market. The network looking to pickup 10, 50, 100 megs to one of the other IXes is. I don't see this in any way diminishing the value of an IX, only increasing it. Surely it would cost Fusion a pretty penny to POP into Indy, St. Louis, Cleveland, Columbus, Cincinnati, Richmond, Madison, Minneapolis, Buffalo, Detroit, etc., etc. just to pick up those unique peers themselves. That's opposed to Fusion POPing at one place and picking up additional VLANs for whatever additional IXes they'd wish to pick up there. ----- Mike Hammett <http://www.ics-il.com/> Intelligent Computing Solutions <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> <http://www.midwest-ix.com/> Midwest Internet Exchange <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> <http://www.thebrotherswisp.com/> The Brothers WISP <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> _____ From: "Erich Kaiser" <[email protected] <mailto:[email protected]> > To: [email protected] <mailto:[email protected]> Sent: Monday, June 20, 2016 9:46:02 AM Subject: Re: [AFMUG] Seen this Brocade before? Can you elaborate on why remote peering is necessary, isn't that just making the IX a transit provider and would be somewhat contradictory of an IX "keeping it local"? Just want to know what I am missing here? I think everyone will agree that if we go out of our way connect to an IX, it needs to be worth it. On Mon, Jun 20, 2016 at 9:38 AM, Mike Hammett <[email protected] <mailto:[email protected]> > wrote: Yes, the current Rise cab. We began the process for a new cabinet back in October, but didn't get a cabinet provisioned until the end of May. We're still migrating things over. Each market has an independent IX fabric. There will just be remote peering and cloud exchange services available at each of them. The remote peering will connect to many IXes, not just our own and largely focused on markets the big guys of the remote peering sector have left behind. We don't build a market until we know we won't lose money doing it. Once DRT completes the same of the STL buildings, we'll be able to get that going again. Months of negotiation with DRT only to have them drop talks as a sale was imminent... 7 months ago. Joining us in STL when it's ready will be worth while. ----- Mike Hammett <http://www.ics-il.com/> Intelligent Computing Solutions <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> <http://www.midwest-ix.com/> Midwest Internet Exchange <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> <http://www.thebrotherswisp.com/> The Brothers WISP <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> _____ From: "Erich Kaiser" <[email protected] <mailto:[email protected]> > To: [email protected] <mailto:[email protected]> Sent: Monday, June 20, 2016 9:28:47 AM Subject: Re: [AFMUG] Seen this Brocade before? You mean the Rise cabinet(My old cab? ) is full, I thought you guys already moved? You know our offer is still open to Midwest-IX. Just gotta work with us Brotha... How come you guys are expanding Midwest to several Cities? I thought STL was going to have its own IX? Do you have enough popular peers to get traction in the other cities? On Mon, Jun 20, 2016 at 9:09 AM, Mike Hammett <[email protected] <mailto:[email protected]> > wrote: The reason for being at Cermak and not Elk Grove was to be able to do WDM instead of using their WDM transport extended cross connects. However, you could only do that if you had space. Our previous space had filled up, no room to grow (cross connects or boxes), so had to get a new space. No space was available. None. The people charging $800 for a riser cross connect are just going to let you run your own? That doesn't seem likely. Yes, frustrated with just about everyone after Josh Reynold's message. Yes, I see the common point here... the room. ;-) ----- Mike Hammett <http://www.ics-il.com/> Intelligent Computing Solutions <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> <http://www.midwest-ix.com/> Midwest Internet Exchange <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> <http://www.thebrotherswisp.com/> The Brothers WISP <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> _____ From: "Erich Kaiser" <[email protected] <mailto:[email protected]> > To: [email protected] <mailto:[email protected]> Sent: Monday, June 20, 2016 9:00:41 AM Subject: Re: [AFMUG] Seen this Brocade before? Mike you are thinking small scale. On a large scale look, you would do DWDM or something between DCs or possibly build conduit and even pull your own fiber between the two facilities. Mike are you frustrated with all of us... On Mon, Jun 20, 2016 at 8:26 AM, Mike Hammett <[email protected] <mailto:[email protected]> > wrote: Getting the space in TelX and then getting multiple cross connects into Equinix for connects that are unavailable anywhere outside of CH1, CH2 or CH4 at a retail of now $1,150/month/each isn't an effective use of anything other than DRT\Equinix shares. Need a couple of those and now you're beholden to Equinix, well, if cabinets were available, which they usually aren't. I don't know how many other ways I can break this down without repeating myself, so I'll resort to that. It was joked that 1G ports are expensive. I joked that 1G ports get real expensive when attached to an Equinix $350/month cross connect. Someone said don't use Equinix. I said in markets like Chicago, you only really have one choice to go to and that's Equinix. Then someone pointed out, which I think most of us in this space (carrier hotels, big boy networks, etc.) already know that Equinix doesn't own 350 E. Cermak. Enter the $800/month riser fee. Enter a bunch of back and forth about a bunch of other ancillary stuff, again, from people that should all know better. ----- Mike Hammett <http://www.ics-il.com/> Intelligent Computing Solutions <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> <http://www.midwest-ix.com/> Midwest Internet Exchange <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> <http://www.thebrotherswisp.com/> The Brothers WISP <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> _____ From: "Erich Kaiser" <[email protected] <mailto:[email protected]> > To: [email protected] <mailto:[email protected]> Sent: Monday, June 20, 2016 7:56:05 AM Subject: Re: [AFMUG] Seen this Brocade before? Having a cabinet in Telx and Equinix covers probably 99% of providers and major IX connectivity. So where is the problem? Erich Kaiser North Central Tower [email protected] <mailto:[email protected]> Office: 630-621-4804 <tel:630-621-4804> Cell: 630-777-9291 <tel:630-777-9291> On Mon, Jun 20, 2016 at 7:19 AM, Mike Hammett <[email protected] <mailto:[email protected]> > wrote: If you want to have Equinix IX connectivity as well as connectivity to the variety of carriers present, that means you have to be in Equinix's CH1, CH2 or CH4. Those datacenters are routinely out of power, so you can't just buy a cabinet whenever you feel like it. They're only available when someone else has canceled theirs. Given the riser charge, you can't just take a cabinet in TelX, Steadfast, CME or any of the others due to the $800 riser fee. ----- Mike Hammett <http://www.ics-il.com/> Intelligent Computing Solutions <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> <http://www.midwest-ix.com/> Midwest Internet Exchange <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> <http://www.thebrotherswisp.com/> The Brothers WISP <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> _____ From: "Erich Kaiser" <[email protected] <mailto:[email protected]> > To: [email protected] <mailto:[email protected]> Sent: Sunday, June 19, 2016 9:19:53 PM Subject: Re: [AFMUG] Seen this Brocade before? Whats the problem with the riser(xtended x-connect) charge if they give you a set of strands? If you are trying to bring circuits over to DRT for lower x-connect, there are economies of scale like anything, but long term it makes sense. That's if you can not negotiate with Equinix. We will be in dual DCs in a few POPs by the end of they year for this reason. Erich Kaiser The Fusion Network [email protected] <mailto:[email protected]> Office: 630-621-4804 <tel:630-621-4804> Cell: 630-777-9291 <tel:630-777-9291> <https://docs.google.com/uc?export=download&id=0B12mNmVrr3-PVWFkTjVnNnh2czg&revid=0B12mNmVrr3-PVlJBWlZ1Z3kyeS80RUJjSWZWaFBUa0NCV1lBPQ> On Sun, Jun 19, 2016 at 4:45 PM, Mike Hammett <[email protected] <mailto:[email protected]> > wrote: Two different things and at one point three different companies. The Equinix rep said to get lower per XC price, you'd need 500 XCs. I know that isn't true because we got lower pricing. TelX manages the MMR and the riser, while DRT owns the building. Yes, I know DRT bought TelX. That riser charge was $800. ----- Mike Hammett <http://www.ics-il.com/> Intelligent Computing Solutions <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> <http://www.midwest-ix.com/> Midwest Internet Exchange <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> <http://www.thebrotherswisp.com/> The Brothers WISP <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> _____ From: "Paul Stewart" <[email protected] <mailto:[email protected]> > To: [email protected] <mailto:[email protected]> Sent: Sunday, June 19, 2016 4:35:42 PM Subject: Re: [AFMUG] Seen this Brocade before? Have never paid riser fees in any location ever FYI, although 111 8th in NYC you are paying for it per say between floors but it’s not broken out (at least for me)… and it’s definitely a lot less than 500 XC’s – it’s related to MRC derived from power/space/XC’s combined with negotiation skills and the market too. Paul From: Af [mailto:[email protected] <mailto:[email protected]> ] On Behalf Of Mike Hammett Sent: June 19, 2016 11:55 AM To: [email protected] <mailto:[email protected]> Subject: Re: [AFMUG] Seen this Brocade before? Right. One rep told us we'd need to be around 500 XCs to get a lower rate (which I know isn't true). Even our TelX rep said that the difference in space pricing wasn't enough to make up for the riser fee. ----- Mike Hammett <http://www.ics-il.com/> Intelligent Computing Solutions <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> <http://www.midwest-ix.com/> Midwest Internet Exchange <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> <http://www.thebrotherswisp.com/> The Brothers WISP <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> _____ From: "Erich Kaiser" <[email protected] <mailto:[email protected]> > To: [email protected] <mailto:[email protected]> Sent: Sunday, June 19, 2016 10:50:46 AM Subject: Re: [AFMUG] Seen this Brocade before? Equinix will negotiate if you have enough business with them and also having an alternate option to go to (Telx/DR runs the building MMR @ 350 Cermak). This DC/Bldg Mgmt war will go on forever. You do what fits right for the time. Hence why we are doing what we are doing, it helps everyone save in the long run. Can't stress this enough......... Every major POP we go into we automatically take a port or two to the local exchange and if has the major peers (google,netflix, etc..) So all you IXPs we want major peers!! :) Erich Kaiser North Central Tower [email protected] <mailto:[email protected]> Office: 630-621-4804 <tel:630-621-4804> Cell: 630-777-9291 <tel:630-777-9291> <https://docs.google.com/uc?export=download&id=0B12mNmVrr3-PVWFkTjVnNnh2czg&revid=0B12mNmVrr3-PVlJBWlZ1Z3kyeS80RUJjSWZWaFBUa0NCV1lBPQ> On Sun, Jun 19, 2016 at 7:03 AM, Mike Hammett <[email protected] <mailto:[email protected]> > wrote: Of course. Leaving Equinix is even worse as you have a $350/month cross connect and then an $800/month riser fee. That's why I specified Equinix at 350 E. Cermak. ----- Mike Hammett <http://www.ics-il.com/> Intelligent Computing Solutions <https://www.facebook.com/ICSIL> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> <https://www.linkedin.com/company/intelligent-computing-solutions> <https://twitter.com/ICSIL> <http://www.midwest-ix.com/> Midwest Internet Exchange <https://www.facebook.com/mdwestix> <https://www.linkedin.com/company/midwest-internet-exchange> <https://twitter.com/mdwestix> <http://www.thebrotherswisp.com/> The Brothers WISP <https://www.facebook.com/thebrotherswisp> <https://www.youtube.com/channel/UCXSdfxQv7SpoRQYNyLwntZg> _____ From: "Eric Kuhnke" <[email protected] <mailto:[email protected]> > To: [email protected] <mailto:[email protected]> Sent: Sunday, June 19, 2016 12:47:03 AM Subject: Re: [AFMUG] Seen this Brocade before? You know that Equinix doesn't own or run 350 E. Cermak, right? They're just one tenant. It's owned by Digital Realty Trust. On Sat, Jun 18, 2016 at 2:52 PM, Mike Hammett <[email protected] <mailto:[email protected]> > wrote: In some markets, that's the only viable place to go. Chicago, for instance. Equinix at 350 E. Cermak has a commanding lead over anywhere else for connectivity. With AMS-IX, Megaport and United-IX here now, we'll likely see some of that shift around a bit, but it'll take time. ----- Mike Hammett <http://www.ics-il.com/> Intelligent Computing Solutions <https://www.facebook.com/ICSIL> Midwest Internet Exchange The Brothers WISP <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> _____ <https://www.facebook.com/ICSIL> From: "Eric Kuhnke" <[email protected]> To: [email protected] Sent: Friday, June 17, 2016 11:02:51 PM Subject: Re: [AFMUG] Seen this Brocade before? Smart people don't start leases in Equinix facilities unless they have absolutely, positively no other option. <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> On Fri, Jun 17, 2016 at 8:17 AM, Mike Hammett <https://www.facebook.com/ICSIL> <[email protected]> wrote: <https://www.facebook.com/ICSIL> In an Equinix facility, 1G ports are very expensive... at $350/month per cross connect. :-) <https://www.facebook.com/ICSIL> ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> _____ <https://www.facebook.com/ICSIL> From: "Josh Reynolds" <[email protected]> To: [email protected] Sent: Friday, June 17, 2016 10:10:58 AM Subject: Re: [AFMUG] Seen this Brocade before? <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> Once upon a time, that was probably said about 10G links. Before then, about 1G links. Those 1G ports are expensive you know. <https://www.facebook.com/ICSIL> On Fri, Jun 17, 2016 at 8:14 AM, Paul Stewart <https://www.facebook.com/ICSIL> <[email protected]> wrote: Had a demo of that platform a month or so ago .. wasn’t public obviously - pretty impressive but a lot jammed into a “small package” … tough sell for folks who need that kind of density, personal opinion is to keep things more “spread out” on 100G links reducing areas of risk, but I digress as that’s more design/resiliency related :) <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> On Jun 16, 2016, at 5:25 PM, Mike Hammett <https://www.facebook.com/ICSIL> <[email protected]> wrote: <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> Don't worry, no one has. First public pictures. https://www.facebook.com/thebrotherswisp/posts/910364895755630 <https://www.facebook.com/ICSIL> ----- Mike Hammett Intelligent Computing Solutions Midwest Internet Exchange The Brothers WISP <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL> <https://www.facebook.com/ICSIL>
