Note: I did get everything for pennies though, and went for n+1 redundancy with hot and cold spares, including full chassis. On Jan 15, 2016 9:29 AM, "Josh Reynolds" <[email protected]> wrote:
> See, I'm in a somewhat of a unique boat on this, so I'll give some > perspective. > > Brand new Fiber ISP right outside KC Metro. 144ct going in everywhere with > spare duct. Combo above ground PED w/ below ground slack storage. Redundant > ring, half on poles, half buried. Two providers feeding us, with hospitals > and school districts excited about turning up 1-10000Mbps service with us > over the next couple of months. Netflix and Google peering happening. Two > TV content providers getting ready to feed us, plus local TV rebroadcast > rights. Upstreams want to skip 40Gbps feeds and move right to 100Gbps when > we get past 10Gbps on our feeds (projections show 18-24mo). > > I needed CGNAT due to the state of IPv4. Also deploying IPv6 along side > it. Needed mpls for certain handoffs, and MEF services for the gpon side > and other features and classes of service, as well as decent routing table > capability and BGP Flowspec. > > I tried to go with a smaller chassis like that, even tried to get a > netiron XMR to work, but I simply couldn't get the feature sets and > expandability I needed to make everything work right out of the gate. :( > On Jan 15, 2016 9:18 AM, "Eric Kuhnke" <[email protected]> wrote: > >> >> >> *The difference with the Junipers and Ciscos and Brocades of the world is >> that they often have a higher tier of products for those entities that can >> afford it that will support their needs* >> >> The great thing about buying used/refurb Cisco and Juniper is that there >> is a VAST quantity of stuff on the market which is no longer fast or >> capable enough for a datacenter/hosting/virtualization/purely fiber ISP >> environment, but is real overkill for the average medium sized WISP use. >> I'll use an example of a to-remain-nameless adult content hosting >> colocation ISP located near a major IX point. They push probably an >> aggregate of 120-150Gbps of traffic to their peers and upstreams on a >> fairly regular basis and need core routers capable of handling such >> volumes, and even more in the future. >> >> You can buy an identical matched pair of used 7604 or 7606 with RSP720 + >> various 10GbE and 1GbE optical interfaces for less than $4500-5000. Such a >> configuration for a WISP can handle a port-channeled 2 x 10Gbps link to an >> upstream. Is there a WISP on the list that moves more than 12-15Gbps of >> traffic? Doing 10GbE with fiber is easy, doing real 1GbE full duplex with >> PTP part101 links is *hard and expensive*. The bottleneck in many cases >> is the PTP microwave backhaul connection in and out of towers and rooftop >> sites, not the router capabilities. It takes a lot of money and a lot of >> users to support building a resilient ring of Exalt ExtremeAir 18 GHz 1 >> Gbps links serving a county sized area. Huge numbers of WISPs have traffic >> patterns to the Internet as a whole that look like 180 to 350Mbps in a >> daily cyclic sine wave pattern that fit nicely inside a 1GbE handoff from >> their upstream. >> >> Take that 7604 for example, you can put in a few 24-port SFP linecards >> that will do 24 Gbps line rate. And you have a couple of blades each with 4 >> x 10GbE. You'd need to be a *big *WISP to outgrow that any time soon. >> More likely you'd outgrow the total 1 million routing table entry size >> (combined IPV4 + IPV6 FIB) before the forwarding ability of the ASICs. >> >> >> >> On Fri, Jan 15, 2016 at 6:52 AM, Josh Reynolds <[email protected]> >> wrote: >> >>> That's not quite true. >>> >>> For example, on Juniper with a standard switching control board, a >>> 16x10Gbps line card has a 4:1 oversub ratio. That interface line card >>> retails for something around $160,000 MSRP. With the replacement of the >>> standard SCB for an enhanced on, you can get full line rate on all ports. >>> >>> On an MX240 you might have a 4x10Gbps card, but with firewall acl and >>> cgnat enabled you might be limited to 8-9Gbps total throughput through it. >>> >>> So to say "it just works" on the higher end of vendors and products is >>> baloney. The difference with the Junipers and Ciscos and Brocades of the >>> world is that they often have a higher tier of products for those entities >>> that can afford it that will support their needs. Also their support is >>> more highly trained (and there's more of them), and companies will fix your >>> shit pretty quick when you buy a few hundred million worth of their >>> products. >>> On Jan 15, 2016 8:19 AM, "Erich Kaiser" <[email protected]> >>> wrote: >>> >>>> I agree there is a huge price difference between a Mikrotik and a >>>> Juniper/Cisco/Brocade router, but you also need to look at the facts. >>>> Those routers are made to support a Full routed 10Gbps/40Gbps/100Gbps of >>>> throughput per port. With Mikrotik it is a MAYBE it will if I don't do >>>> this or do that.. Obviously there are many models out there, but if you >>>> look at the true carrier grade router lines, it is a full routed port. >>>> >>>> I think each one has its place. >>>> >>>> Just my opinion.. >>>> >>>> Erich Kaiser >>>> North Central Tower >>>> [email protected] >>>> Office: 630-621-4804 >>>> Cell: 630-777-9291 >>>> >>>> >>>> On Fri, Jan 15, 2016 at 7:37 AM, Mike Hammett <[email protected]> wrote: >>>> >>>>> The #1 thing I learned at NANOG was that the big vendors are just as >>>>> full of shit as the little vendors. Cisco, Juniper, AlcaLu, etc. are all >>>>> just as likely to ignore problems and create bugs as Mikrotik, Ubiquiti, >>>>> etc. >>>>> >>>>> When sitting at a table at about 1 AM one night with a head engineer >>>>> with one international operator and one national operator I said something >>>>> to the effect of, "I've just largely been listening to you guys comparing >>>>> notes on different products and issues you've had. I use Mikrotik routers >>>>> in my network and people often say they are junk and that Cisco, Juniper, >>>>> etc. are the way to go. Those vendors are exactly who you guys use and yet >>>>> here you are with the same type of complaints as we have with our >>>>> vendors." >>>>> The one at the international vendor chimed in, "and we pay 10x more for >>>>> that right." >>>>> >>>>> Don't lose sight of that. >>>>> >>>>> For example, Cisco is known for having Ethernet chipset >>>>> incompatibility issues.... which is exactly what this sub-topic has been >>>>> about. >>>>> >>>>> >>>>> To think otherwise is simply blind arrogance. >>>>> >>>>> >>>>> >>>>> ----- >>>>> Mike Hammett >>>>> Intelligent Computing Solutions >>>>> http://www.ics-il.com >>>>> >>>>> <https://www.facebook.com/ICSIL> >>>>> <https://plus.google.com/+IntelligentComputingSolutionsDeKalb> >>>>> <https://www.linkedin.com/company/intelligent-computing-solutions> >>>>> <https://twitter.com/ICSIL> >>>>> >>>>> Midwest Internet Exchange >>>>> http://www.midwest-ix.com >>>>> >>>>> <https://www.facebook.com/mdwestix> >>>>> <https://www.linkedin.com/company/midwest-internet-exchange> >>>>> <https://twitter.com/mdwestix> >>>>> ------------------------------ >>>>> *From: *"Eric Kuhnke" <[email protected]> >>>>> *To: *[email protected] >>>>> *Sent: *Thursday, January 14, 2016 10:49:12 PM >>>>> *Subject: *Re: [AFMUG] Weird Problem with download >>>>> >>>>> This is sort of an illustration of why I am still very wary of >>>>> Mikrotik for routers. >>>>> >>>>> There's a myriad of Cisco routers available with 12 to 20 SFP-based >>>>> GbE interfacess (used/refurb) that are just as capable, if somewhat bigger >>>>> and hotter, that run very stable versions of IOS. It's totally >>>>> unacceptable >>>>> for Cisco to break basic functionality for $HUGE_ISP and >>>>> $HUGE_ENTERPRISE_CUSTOMER , which are usually using the same IOS trains I >>>>> use for service provider functions. >>>>> >>>>> Mikrotik OS software support is pretty much "hey go over to this forum >>>>> and hope somebody answers your question", if something is broken you are >>>>> just supposed to wait for a new version release in a month or two. If >>>>> something is fundamentally broken in a version of IOS used by a mature (6+ >>>>> year old) Cisco platform it's almost certain that somebody much bigger and >>>>> more important than yourself has already run into the issue, escalated it >>>>> with Cisco, and a new build fixing the bug is available. >>>>> >>>>> If the space and power are available I would much rather have a 6503E >>>>> with dual supervisor than a CCR.... >>>>> >>>>> On Wed, Jan 13, 2016 at 7:34 PM, Rory Conaway <[email protected]> >>>>> wrote: >>>>> >>>>>> This just gets even better. Worked 24 hours, then breaks again. Up >>>>>> to Level 4 tech support. >>>>>> >>>>>> >>>>>> >>>>>> Rory >>>>>> >>>>>> >>>>>> >>>>>> *From:* Af [mailto:[email protected]] *On Behalf Of *Rory Conaway >>>>>> *Sent:* Wednesday, January 13, 2016 7:56 AM >>>>>> >>>>>> *To:* [email protected] >>>>>> *Subject:* Re: [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> We are taking it to the Mikrotik forum. This makes no sense since we >>>>>> have 12 of the already running with no issues. >>>>>> >>>>>> >>>>>> >>>>>> Rory >>>>>> >>>>>> >>>>>> >>>>>> *From:* Af [mailto:[email protected] <[email protected]>] *On >>>>>> Behalf Of *Glen Waldrop >>>>>> *Sent:* Wednesday, January 13, 2016 7:54 AM >>>>>> *To:* [email protected] >>>>>> *Subject:* Re: [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> I’m not terribly happy with ROS since 6.29. >>>>>> >>>>>> >>>>>> >>>>>> Maybe it’s just still not 100% yet. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *From:* Rory Conaway <[email protected]> >>>>>> >>>>>> *Sent:* Tuesday, January 12, 2016 8:36 PM >>>>>> >>>>>> *To:* [email protected] >>>>>> >>>>>> *Subject:* Re: [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> Replaced the Mikrotik routers with an Airrouter, it works. Replaced >>>>>> with an Edgerouter, still works. This goes under the category of WTF. >>>>>> >>>>>> >>>>>> >>>>>> Rory >>>>>> >>>>>> >>>>>> >>>>>> *From:* Af [mailto:[email protected] <[email protected]>] *On >>>>>> Behalf Of *Rory Conaway >>>>>> *Sent:* Monday, January 11, 2016 9:13 PM >>>>>> *To:* [email protected] >>>>>> *Subject:* Re: [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> We tried 3 Mikrotik routers, 2 cable modems, and two radios from two >>>>>> different manufacturers. >>>>>> >>>>>> >>>>>> >>>>>> Rory >>>>>> >>>>>> >>>>>> >>>>>> *From:* Af [mailto:[email protected] <[email protected]>] *On >>>>>> Behalf Of *Glen Waldrop >>>>>> *Sent:* Monday, January 11, 2016 8:52 PM >>>>>> *To:* [email protected] >>>>>> *Subject:* Re: [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> MTU? >>>>>> >>>>>> Last time I had that problem was an inconsistent Ethernet connection. >>>>>> It wasn’t showing errors, but UDP was dropping packets. TCP >>>>>> retransmitted. >>>>>> >>>>>> UDP was screwy, but TCP was downloading at the limit of the wireless >>>>>> feed. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *From:* Rory Conaway <[email protected]> >>>>>> >>>>>> *Sent:* Monday, January 11, 2016 9:47 PM >>>>>> >>>>>> *To:* [email protected] >>>>>> >>>>>> *Subject:* Re: [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> Nailed the problem down to the Mikrotik router having a problem with >>>>>> DNS caching whether it’s Cox or Google. Not sure where to go from there >>>>>> since the other 12 don’t have a problem. >>>>>> >>>>>> >>>>>> >>>>>> Rory >>>>>> >>>>>> >>>>>> >>>>>> *From:* Af [mailto:[email protected] <[email protected]>] *On >>>>>> Behalf Of *Rory Conaway >>>>>> *Sent:* Monday, January 11, 2016 4:44 PM >>>>>> *To:* [email protected] >>>>>> *Subject:* Re: [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> We had raons bit no dangling cables. Level 3 says no errors to the >>>>>> modem and we had a tech out already. We are changing the 2011 to a 450 >>>>>> just to confirm before we eacalate further. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> Rory Conaway >>>>>> >>>>>> Triad Wireless >>>>>> >>>>>> >>>>>> >>>>>> Typed on my phone with one finger so please excuse typos and >>>>>> abbreviations. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -------- Original message -------- >>>>>> >>>>>> From: That One Guy /sarcasm <[email protected]> >>>>>> >>>>>> Date: 1/11/2016 10:38 AM (GMT-07:00) >>>>>> >>>>>> To: [email protected] >>>>>> >>>>>> Subject: Re: [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> If it just started and youre in an area hit by storms, some of the >>>>>> neighbors, mostly the empty houses with no tenants at this point, cable >>>>>> was >>>>>> busted down by tree limbs and is dangling terminated byt the mud now >>>>>> >>>>>> >>>>>> >>>>>> On Mon, Jan 11, 2016 at 10:58 AM, Glen Waldrop < >>>>>> [email protected]> wrote: >>>>>> >>>>>> Likely an RF issue on the cable side. >>>>>> >>>>>> I see this sort of thing with the company I consult for. They’ll have >>>>>> me going through the router, trying to shut down the “virus” or peer to >>>>>> peer when it is thousands and thousands of retransmits and broken >>>>>> packets. >>>>>> >>>>>> >>>>>> >>>>>> Just like our wireless it can have good quality until loaded, then >>>>>> watch the quality drop to nearly nothing and loose connection. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *From:* Rory Conaway <[email protected]> >>>>>> >>>>>> *Sent:* Sunday, January 10, 2016 8:25 PM >>>>>> >>>>>> *To:* [email protected] >>>>>> >>>>>> *Subject:* Re: [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> The Netgear router has nothing to do with it, I was just using their >>>>>> website as an example of downloading a file. It’s computer, 2011, cable >>>>>> modem, the world. I just can’t download anything from any sites. What’s >>>>>> weird is they just drop. >>>>>> >>>>>> >>>>>> >>>>>> Rory >>>>>> >>>>>> >>>>>> >>>>>> *From:* Af [mailto:[email protected]] *On Behalf Of *Ken Hohhof >>>>>> *Sent:* Sunday, January 10, 2016 5:12 PM >>>>>> *To:* [email protected] >>>>>> *Subject:* Re: [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> You mention Netgear, is this a Netgear router? They used to have a >>>>>> problem with downloads stalling, but that was quite a few years ago. The >>>>>> fix was to disable SPI Firewall. >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> *From:* Rory Conaway <[email protected]> >>>>>> >>>>>> *Sent:* Sunday, January 10, 2016 6:06 PM >>>>>> >>>>>> *To:* [email protected] >>>>>> >>>>>> *Subject:* [AFMUG] Weird Problem with download >>>>>> >>>>>> >>>>>> >>>>>> We have a cable circuit that can do everything except download. >>>>>> There is a Mikrotik router between the computer and the cable modem. >>>>>> Haven’t tested taking the router out yet but we have tried 2 routers, >>>>>> same >>>>>> results. Basically, downloads either don’t start or start and then just >>>>>> crash. Game machines can’t update, even downloading a 20MB file from >>>>>> netgear fails. Everything else seems to work. Any ideas would be >>>>>> helpful. >>>>>> >>>>>> >>>>>> >>>>>> *Rory Conaway **• Triad Wireless •** CEO* >>>>>> >>>>>> *4226 S. 37th Street • Phoenix • AZ 85040* >>>>>> >>>>>> *602-426-0542 <602-426-0542>* >>>>>> >>>>>> *[email protected] <[email protected]>* >>>>>> >>>>>> *www.triadwireless.net <http://www.triadwireless.net/>* >>>>>> >>>>>> >>>>>> >>>>>> “Life is full of unfair calls, missed plays, and bad catches. But >>>>>> true baseball players keep on playing” - Lessons from Baseball >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> If you only see yourself as part of the team but you don't see your >>>>>> team as part of yourself you have already failed as part of the team. >>>>>> >>>>> >>>>> >>>>> >>>> >>
