Re: Comcast Support (from NANOG Digest, Vol 84, Issue 24)
Dear John, On 24/01/2015 10:00, nanog-requ...@nanog.org wrote: (...) Date: Fri, 23 Jan 2015 17:14:11 + From: Brzozowski, John john_brzozow...@cable.comcast.com To: nanog@nanog.org nanog@nanog.org Subject: Comcast Support (from NANOG Digest, Vol 84, Issue 23) Message-ID: d0e7e8e3.21d5aa%john_brzozow...@cable.comcast.com Content-Type: text/plain; charset=utf-8 (...) For customers where you bring your own cable modem or have one of the above in bridge mode we have enabled IPv6 support for you as well. However, your router behind the modem must be running software and configured with IPv6 support. Specifically, your router needs to be support stateful DHCPv6 for IPv6 address and prefix acquisition. We have received a number of reports from customers that the Juniper SRX does not appear to properly support IPv6. We are working with Juniper and also recommend that you reach out to Juniper as well. (...) Care to share scenarios where the SRXs do not perform well with DHCPv6? Any specific model? Thanks in advance, -- Rafael de Oliveira Ribeiro DAERO - Gerencia de Operacoes RNP - Rede Nacional de Ensino e Pesquisa Tel.: +55 21 2102 9659 - iNOC: 1916*767
Re: Comcast Support (from NANOG Digest, Vol 84, Issue 24)
Thanks John and Ron, We'll definitely reach out to Juniper. Best regards, -- Rafael de Oliveira Ribeiro DAERO - Gerencia de Operacoes RNP - Rede Nacional de Ensino e Pesquisa Tel.: +55 21 2102 9659 - iNOC: 1916*767
Re: REMINDER: Leap Second
I'm pretty sure University College, London (UCL) had a 360/195 on the net in the late 1970s. I remember it had open login to I guess it was TSO? I'd play with it but couldn't really figure out anything interesting to do lacking all documentation and by and large motivation other than it was kind of cool in like 1978 to be typing at a computer in London even if it was just saying do something or go away! I guess you had to be there. -Barry Shein The World | b...@theworld.com | http://www.TheWorld.com Purveyors to the Trade | Voice: 800-THE-WRLD| Dial-Up: US, PR, Canada Software Tool Die| Public Access Internet | SINCE 1989 *oo* On January 26, 2015 at 03:36 bar...@databus.com (Barney Wolff) wrote: On Sun, Jan 25, 2015 at 06:42:51PM -0500, TR Shaw wrote: That made the transformers smaller/cooler and more efficient. I seem to remember a 195 as well but maybe it is just CRS. Google says the 360/195 did exist. But my baby was the 360/95, where the first megabyte of memory was flat-film at 60ns, which made it faster than the 195 for some things. It was incredibly expensive to build - we heard rumors of $30 million in 1967 dollars, and sold to NASA at a huge loss, which is why there were only two built. I used to amuse myself by climbing into the flats memory cabinet, and was amused again some years later when I could have ingested a megabyte without harm. Ours sat directly above Tom's Restaurant, of Seinfeld fame. Very early climate modeling was done on that machine, along with a lot of astrophysics.
Re: Comcast Support (from NANOG Digest, Vol 84, Issue 24)
From the looks of it, there is no IPv6 PD support per RFC3633. = John Jason Brzozowski Comcast Cable p) 484-962-0060 w) www.comcast6.net e) john_brzozow...@cable.comcast.com = -Original Message- From: Rafael de Oliveira Ribeiro rafael.ribe...@rnp.br Organization: Rede Nacional de ensino e Pesquisa Date: Monday, January 26, 2015 at 11:00 To: John Brzozowski john_brzozow...@cable.comcast.com, NANOG nanog@nanog.org Subject: Re: Comcast Support (from NANOG Digest, Vol 84, Issue 24) Dear John, On 24/01/2015 10:00, nanog-requ...@nanog.org wrote: (...) Date: Fri, 23 Jan 2015 17:14:11 + From: Brzozowski, John john_brzozow...@cable.comcast.com To: nanog@nanog.org nanog@nanog.org Subject: Comcast Support (from NANOG Digest, Vol 84, Issue 23) Message-ID: d0e7e8e3.21d5aa%john_brzozow...@cable.comcast.com Content-Type: text/plain; charset=utf-8 (...) For customers where you bring your own cable modem or have one of the above in bridge mode we have enabled IPv6 support for you as well. However, your router behind the modem must be running software and configured with IPv6 support. Specifically, your router needs to be support stateful DHCPv6 for IPv6 address and prefix acquisition. We have received a number of reports from customers that the Juniper SRX does not appear to properly support IPv6. We are working with Juniper and also recommend that you reach out to Juniper as well. (...) Care to share scenarios where the SRXs do not perform well with DHCPv6? Any specific model? Thanks in advance, -- Rafael de Oliveira Ribeiro DAERO - Gerencia de Operacoes RNP - Rede Nacional de Ensino e Pesquisa Tel.: +55 21 2102 9659 - iNOC: 1916*767
Re: REMINDER: Leap Second
Barney Wolff bar...@databus.com wrote: On Sun, Jan 25, 2015 at 06:42:51PM -0500, TR Shaw wrote: That made the transformers smaller/cooler and more efficient. I seem to remember a 195 as well but maybe it is just CRS. Google says the 360/195 did exist. But my baby was the 360/95, where the first megabyte of memory was flat-film at 60ns, which made it faster than the 195 for some things. ... The /95 was a /91 with a megabyte of thin film memory, which was both much faster than core (120 vs 780 ns cycle time) and much more expensive (7c rather than 1.6c per bit.) The /195 was a /91 reimplemented in slightly faster logic with a 54ns rather than 60ns cycle time, and a cache adapted from the /85. I can easily believe that for programs that didn't cache well, the /95 with the fast memory would be faster. IBM lost money on all of them and eventually stopped trying to compete with CDC in that niche. See alt.folklore.computers (yes, usenet, reports of its death are premature) for endless discussion of topics like this. R's, John
Re: Comcast Support (from NANOG Digest, Vol 84, Issue 24)
Sorry Ron, just replied with the same information. = John Jason Brzozowski Comcast Cable p) 484-962-0060 w) www.comcast6.net e) john_brzozow...@cable.comcast.com = -Original Message- From: Ron Broersma r...@dren.mil Date: Monday, January 26, 2015 at 13:15 To: Rafael de Oliveira Ribeiro rafael.ribe...@rnp.br Cc: John Brzozowski john_brzozow...@cable.comcast.com, NANOG nanog@nanog.org Subject: Re: Comcast Support (from NANOG Digest, Vol 84, Issue 24) On Jan 26, 2015, at 8:00 AM, Rafael de Oliveira Ribeiro rafael.ribe...@rnp.br wrote: Care to share scenarios where the SRXs do not perform well with DHCPv6? Any specific model? As one example, there is no support for DHCPv6-relay in the SRX, so we never use them for edge routers (in our enterprise networks). —Ron
Requesting Consolidated Communications (AS5742) contact
Requesting to speak with Consolidated Communications (AS5742) regarding routing in Illinois region towards Gaikai (AS33353). Please contact me offline. Thank you, Chris Costa Gaikai cco...@gaikai.com
Re: ATT uVerse blocking SIP?
Yes. If you move to another port, e.g.: 5061 it works fine. If you’re running on a Linux based system, you can do this: /sbin/iptables -A PREROUTING -t nat -i eth1 -p udp --dport 5061 -j REDIRECT --to-port 5060 on the host to remap 5061 - 5060 with no application change. - Jared On Jan 26, 2015, at 4:26 PM, Brad Bendy b...@1stclasshosting.com wrote: Has anyone seen issues where a end user on uVerse trying to connect to either another provider or ATT non uVerse (in this case DIA) is having SIP blocked? SIP leaving the uVerse network going to another uVerse DSL account is fine, but it appears soon as it leave the uVerse network all SIP traffic is blocked? It appears others have seen this problem, some say it's a modem issue, some say they are truly blocking it. Ive yet to call uVerse support yet as im guessing ill get no where. Thanks for any insight on this. -- This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. 1st Class Hosting, LLC. 1712 Pioneer Ave, Suite 1854, Cheyenne, WY 82001
Re: scaling linux-based router hardware recommendations
On Tue, 27 Jan 2015 11:10:54 +0900, Paul S. said: Like Mike mentioned, the feature list in RouterOS is nothing short of impressive -- problem is that pretty much everything in there is inherently buggy. That and one hell of a painful syntax-schema to work with too. Latvian grammar is.. somewhat unusual. Just be glad the development team wasn't Finnish. :) (Sorry, I couldn't resist. :) pgp2auUp1yykM.pgp Description: PGP signature
Re: Facebook outage?
On 1/27/2015 00:58, Larry Sheldon wrote: On 1/27/2015 00:47, Damien Burke wrote: Facebook outage? Everyone panic! https://twitter.com/search?q=facebooksrc=typd Let the record show that I noticed it quite a while ago, but did NOT go for first NANOG mention. It is back up in Omaha. -- The unique Characteristics of System Administrators: The fact that they are infallible; and, The fact that they learn from their mistakes. Quis custodiet ipsos custodes
Re: scaling linux-based router hardware recommendations
Aren't most of the new whitebox\open source platforms based on switching and not routing? I'd assume that the cloud-scale data centers deploying this stuff still have more traditional big iron at their cores. The small\medium sized ISP usually is left behind. They're not big enough to afford the big new hardware, but all of their user's NetFlix and porn and whatever else they do is chewing up bandwidth. For example, the small\medium ISPs are at the Nx10GigE stage now. The new hardware is expensive, the old hardware (besides being old) is likely in a huge chassis if you can get any sort of port density at all. 48 port GigE switches with a couple 10GigE can be had for $100. A minimum of 24 port 10GigE switches (except for the occasional IBM switch ) is 30x to 40x times that. Routers (BGP, MPLS, etc.) with that more than just a couple 10GigEs are even more money, I'd assume. I thought vMX was going to save the day, but it's pricing for 10 gigs of traffic (licensed by throughput and standard\advanced licenses) is really about 5x - 10x what I'd be willing to pay for it. Haven't gotten a quote from AlcaLu yet. Vyatta (last I checked, which was admittedly some time ago) doesn't have MPLS. The FreeBSD world can bring zero software cost and a stable platform, but no MPLS. Mikrotik brings most (though not all) of the features one would want... a good enough feature set, let's say... but is a non-stop flow of bugs. I don't think a week or two goes by where one of my friends doesn't submit some sort of reproducible bug to Mikrotik. They've also been looking into DPDK for 2.5 years now. hasn't shown up yet. I've used MT for 10 years and I'm always left wanting just a little more, but it may be the best balance between the features and performance I want and the ability to pay for it. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mehmet Akcin meh...@akcin.net To: micah anderson mi...@riseup.net Cc: nanog@nanog.org Sent: Monday, January 26, 2015 6:06:53 PM Subject: Re: scaling linux-based router hardware recommendations Cumulus Networks has some stuff, http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal-2014-final.pdf Pretty decent presentation with more details you like. Mehmet On Jan 26, 2015, at 8:53 PM, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
Re: scaling linux-based router hardware recommendations
On 1/26/15 5:43 PM, Mike Hammett wrote: Aren't most of the new whitebox\open source platforms based on switching and not routing? I'd assume that the cloud-scale data centers deploying this stuff still have more traditional big iron at their cores. A L3 ethernet switch and a router are effectively indistinguishable. the actual feature set you need drives what platforms are appropiate. A signficant push for DCs particularly those with CLOS archectures is away from modular chassis based switches towards dense but fixed configuration switches. This drives the complexity and a signficant chunk of the cost out of these switches. The small\medium sized ISP usually is left behind. They're not big enough to afford the big new hardware, but all of their user's NetFlix and porn and whatever else they do is chewing up bandwidth. Everyone in the industry is under margin pressure. Done well every subsequent generation of your infrastrucuture is less costly per bit delivered while also being faster. For example, the small\medium ISPs are at the Nx10GigE stage now. The new hardware is expensive, the old hardware (besides being old) is likely in a huge chassis if you can get any sort of port density at all. If you're a small consumer based ISP how many routers do you actually need the have a full table (the customer access network doesn't need it). 48 port GigE switches with a couple 10GigE can be had for $100. I'm not aware of that being the case. With respect to merchant silicon there a limited number of comon l3 switch asic building blocks which all switch/router vendors can avail themselves of. broadcom trident+ trident 2 and arad, intel fm6000, marvell prestera etc. A minimum of 24 port 10GigE switches (except for the occasional IBM switch ) is 30x to 40x times that. Routers (BGP, MPLS, etc.) with that more than just a couple 10GigEs are even more money, I'd assume. a 64 port 10 or mixed 10/40Gb/s switch can forward more half a Tb/s worth of 64byte packets, do so with cut-through forwarding and in a thermal enevelope of 150 watts. device like that retail for ~20k, in reality you need more than one. the equivalent gigabit product is 15 or 20% of the price. you mention mpls support so that dictates appropriate support which is available in some platforms and asics. I thought vMX was going to save the day, but it's pricing for 10 gigs of traffic (licensed by throughput and standard\advanced licenses) is really about 5x - 10x what I'd be willing to pay for it. The servers capable of relatively high-end forwarding feats aren't free either nor are the equivalent. Haven't gotten a quote from AlcaLu yet. Vyatta (last I checked, which was admittedly some time ago) doesn't have MPLS. The FreeBSD world can bring zero software cost and a stable platform, but no MPLS. mpls implementions have abundant ipr, which among other things prevents practical merging with the linux kernel. Mikrotik brings most (though not all) of the features one would want... a good enough feature set, let's say... but is a non-stop flow of bugs. I don't think a week or two goes by where one of my friends doesn't submit some sort of reproducible bug to Mikrotik. They've also been looking into DPDK for 2.5 years now. hasn't shown up yet. I've used MT for 10 years and I'm always left wanting just a little more, but it may be the best balance between the features and performance I want and the ability to pay for it. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mehmet Akcin meh...@akcin.net To: micah anderson mi...@riseup.net Cc: nanog@nanog.org Sent: Monday, January 26, 2015 6:06:53 PM Subject: Re: scaling linux-based router hardware recommendations Cumulus Networks has some stuff, http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal-2014-final.pdf Pretty decent presentation with more details you like. Mehmet On Jan 26, 2015, at 8:53 PM, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if
Re: scaling linux-based router hardware recommendations
Under 30sec (more like 15 to 20) on an i7 based Mikrotik for full BGP Tables. Faisal Imtiaz - Original Message - From: Ken Chase m...@sizone.org To: nanog@nanog.org Sent: Monday, January 26, 2015 10:29:28 PM Subject: Re: scaling linux-based router hardware recommendations Hows convergence time on these mikrotik/ubiquity/etc units for a full table? /kc -- Ken Chase - m...@sizone.org Toronto
Re: scaling linux-based router hardware recommendations
Like Mike mentioned, the feature list in RouterOS is nothing short of impressive -- problem is that pretty much everything in there is inherently buggy. That and one hell of a painful syntax-schema to work with too. On 1/27/2015 午前 10:57, Tony Wicks wrote: And the solution to this issue is - http://routerboard.com/ or http://www.mikrotik.com/software# on x86 hardware, plus any basic layer2 switch. Don't scoff until you have tried it, the price/performance is pretty staggering if you are in the sub 20gig space. -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Mike Hammett Sent: Tuesday, 27 January 2015 2:44 p.m. To: nanog@nanog.org Subject: Re: scaling linux-based router hardware recommendations Aren't most of the new whitebox\open source platforms based on switching and not routing? I'd assume that the cloud-scale data centers deploying this stuff still have more traditional big iron at their cores. The small\medium sized ISP usually is left behind. They're not big enough to afford the big new hardware, but all of their user's NetFlix and porn and whatever else they do is chewing up bandwidth. For example, the small\medium ISPs are at the Nx10GigE stage now. The new hardware is expensive, the old hardware (besides being old) is likely in a huge chassis if you can get any sort of port density at all. 48 port GigE switches with a couple 10GigE can be had for $100. A minimum of 24 port 10GigE switches (except for the occasional IBM switch ) is 30x to 40x times that. Routers (BGP, MPLS, etc.) with that more than just a couple 10GigEs are even more money, I'd assume. I thought vMX was going to save the day, but it's pricing for 10 gigs of traffic (licensed by throughput and standard\advanced licenses) is really about 5x - 10x what I'd be willing to pay for it. Haven't gotten a quote from AlcaLu yet. Vyatta (last I checked, which was admittedly some time ago) doesn't have MPLS. The FreeBSD world can bring zero software cost and a stable platform, but no MPLS. Mikrotik brings most (though not all) of the features one would want... a good enough feature set, let's say... but is a non-stop flow of bugs. I don't think a week or two goes by where one of my friends doesn't submit some sort of reproducible bug to Mikrotik. They've also been looking into DPDK for 2.5 years now. hasn't shown up yet. I've used MT for 10 years and I'm always left wanting just a little more, but it may be the best balance between the features and performance I want and the ability to pay for it. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mehmet Akcin meh...@akcin.net To: micah anderson mi...@riseup.net Cc: nanog@nanog.org Sent: Monday, January 26, 2015 6:06:53 PM Subject: Re: scaling linux-based router hardware recommendations Cumulus Networks has some stuff, http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal-2014-final.pdf Pretty decent presentation with more details you like. Mehmet On Jan 26, 2015, at 8:53 PM, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
Re: scaling linux-based router hardware recommendations
Hows convergence time on these mikrotik/ubiquity/etc units for a full table? /kc -- Ken Chase - m...@sizone.org Toronto
Re: scaling linux-based router hardware recommendations
On 27/01/2015, at 4:29 pm, Ken Chase m...@sizone.org wrote: Hows convergence time on these mikrotik/ubiquity/etc units for a full table? For the CCR1036-12G-4S with one full table, one domestic table (NZ - ~26k entries) some peering and iBGP full convergence took about three minutes forty seconds last time I timed it from cold. I may do some new timing as they have been working hard to improve the multi core support (currently BGP still only single core however they been doing some work on efficient allocation of other tasks to cores. /kc -- Ken Chase - m...@sizone.org Toronto
Re: scaling linux-based router hardware recommendations
On Tue, Jan 27, 2015 at 04:59:12PM +1300, Alexander Neilson said: For the CCR1036-12G-4S with one full table, one domestic table (NZ - ~26k entries) some peering and iBGP full convergence took about three minutes forty seconds last time I timed it from cold. That's terrible. I dont know what model that is or appropriate deploys but I think a couple of my peers use these and report similar times for their models for 500k+ routes. Still too slow. I think the single threaded nature of the routing table manip is at fault with the 36-but-slow cores (mikrotic). Im not sure how you get around this without drastically rewriting the kernel, which puts you out on your own developing new fundamental tech. I'd be more comfortable with full-cpu models (like xeon based for eg.) From: Faisal Imtiaz fai...@snappytelecom.net Under 30sec (more like 15 to 20) on an i7 based Mikrotik for full BGP Tables. Ya, that. /kc -- Ken Chase - m...@sizone.org Toronto
Re: scaling linux-based router hardware recommendations
Must not have read my whole e-mail. ;-) There aren't very many people outside of my group that know more about Mikrotik. Trainers, MUM presenters, direct-line-to-Janis guys, etc. Still can't make those Latvians produce what we want. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Tony Wicks t...@wicks.co.nz To: Mike Hammett na...@ics-il.net, nanog@nanog.org Sent: Monday, January 26, 2015 7:57:44 PM Subject: RE: scaling linux-based router hardware recommendations And the solution to this issue is - http://routerboard.com/ or http://www.mikrotik.com/software# on x86 hardware, plus any basic layer2 switch. Don't scoff until you have tried it, the price/performance is pretty staggering if you are in the sub 20gig space. -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Mike Hammett Sent: Tuesday, 27 January 2015 2:44 p.m. To: nanog@nanog.org Subject: Re: scaling linux-based router hardware recommendations Aren't most of the new whitebox\open source platforms based on switching and not routing? I'd assume that the cloud-scale data centers deploying this stuff still have more traditional big iron at their cores. The small\medium sized ISP usually is left behind. They're not big enough to afford the big new hardware, but all of their user's NetFlix and porn and whatever else they do is chewing up bandwidth. For example, the small\medium ISPs are at the Nx10GigE stage now. The new hardware is expensive, the old hardware (besides being old) is likely in a huge chassis if you can get any sort of port density at all. 48 port GigE switches with a couple 10GigE can be had for $100. A minimum of 24 port 10GigE switches (except for the occasional IBM switch ) is 30x to 40x times that. Routers (BGP, MPLS, etc.) with that more than just a couple 10GigEs are even more money, I'd assume. I thought vMX was going to save the day, but it's pricing for 10 gigs of traffic (licensed by throughput and standard\advanced licenses) is really about 5x - 10x what I'd be willing to pay for it. Haven't gotten a quote from AlcaLu yet. Vyatta (last I checked, which was admittedly some time ago) doesn't have MPLS. The FreeBSD world can bring zero software cost and a stable platform, but no MPLS. Mikrotik brings most (though not all) of the features one would want... a good enough feature set, let's say... but is a non-stop flow of bugs. I don't think a week or two goes by where one of my friends doesn't submit some sort of reproducible bug to Mikrotik. They've also been looking into DPDK for 2.5 years now. hasn't shown up yet. I've used MT for 10 years and I'm always left wanting just a little more, but it may be the best balance between the features and performance I want and the ability to pay for it. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mehmet Akcin meh...@akcin.net To: micah anderson mi...@riseup.net Cc: nanog@nanog.org Sent: Monday, January 26, 2015 6:06:53 PM Subject: Re: scaling linux-based router hardware recommendations Cumulus Networks has some stuff, http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal-2014-final.pdf Pretty decent presentation with more details you like. Mehmet On Jan 26, 2015, at 8:53 PM, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
RE: Facebook outage?
Dead here at AS11404 from all locations where we PNI or public peer... must be bad over there, v4 dies at their edge, v6 makes it in but no page loads. John
Re: Facebook outage?
On 1/27/2015 00:47, Damien Burke wrote: Facebook outage? Everyone panic! https://twitter.com/search?q=facebooksrc=typd Let the record show that I noticed it quite a while ago, but did NOT go for first NANOG mention. -- The unique Characteristics of System Administrators: The fact that they are infallible; and, The fact that they learn from their mistakes. Quis custodiet ipsos custodes
Re: Facebook outage?
And it appears to be back for me. - Tim On 27 Jan 2015, at 3:08 pm, Tim Raphael raphael.timo...@gmail.com wrote: Instagram used to use Amazon AWS before being purchased by Facebook. There has been a slow migration onto FB infrastructure, so yes, a mixture of addresses like that makes sense. - Tim On 27 Jan 2015, at 2:58 pm, Christopher Morrow morrowc.li...@gmail.com wrote: On Tue, Jan 27, 2015 at 1:56 AM, Jason Canady ja...@unlimitednet.us wrote: Instagram appears to be down as well, but that would make sense since they are part of Facebook. $ dig +short facebook.com 173.252.120.6 NetRange: 173.252.64.0 - 173.252.127.255 CIDR: 173.252.64.0/18 NetName:FACEBOOK-INC but $ dig +short instagram.com 54.209.14.128 107.23.173.176 54.175.77.206 54.208.246.103 107.23.166.70 54.236.148.28 54.209.197.196 54.236.177.12 those are amazon addresses... err, not sure the connection makes sense though? -chris
Re: scaling linux-based router hardware recommendations
Depends on the hardware. 30 - 45 seconds for the higher end stuff? I'm not sure how long it is on an RB750 (list price of like $40). ;-) - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Ken Chase m...@sizone.org To: nanog@nanog.org Sent: Monday, January 26, 2015 9:29:28 PM Subject: Re: scaling linux-based router hardware recommendations Hows convergence time on these mikrotik/ubiquity/etc units for a full table? /kc -- Ken Chase - m...@sizone.org Toronto
RE: Facebook outage?
I hear that AIM and hipchat is also having issues. Any other major company down too? -Original Message- From: John van Oppen [mailto:jvanop...@spectrumnet.us] Sent: Monday, January 26, 2015 10:49 PM To: Damien Burke; nanog@nanog.org Subject: RE: Facebook outage? Dead here at AS11404 from all locations where we PNI or public peer... must be bad over there, v4 dies at their edge, v6 makes it in but no page loads. John
Re: Facebook outage?
Seems unlikely, probably taking credit for someone tripping over a cable. On Jan 27, 2015 2:01 AM, Trent Farrell tfarr...@riotgames.com wrote: https://twitter.com/LizardMafia/status/559963134006292481 On Mon, Jan 26, 2015 at 10:50 PM, Damien Burke dam...@supremebytes.com wrote: I hear that AIM and hipchat is also having issues. Any other major company down too? -Original Message- From: John van Oppen [mailto:jvanop...@spectrumnet.us] Sent: Monday, January 26, 2015 10:49 PM To: Damien Burke; nanog@nanog.org Subject: RE: Facebook outage? Dead here at AS11404 from all locations where we PNI or public peer... must be bad over there, v4 dies at their edge, v6 makes it in but no page loads. John -- *Trent Farrell* *Riot Games* *IP Network Engineer* E: tfarr...@riotgames.com | IE: +353 83 446 6809 | US: +1 424 285 9825 Summoner name: Foro
RE: scaling linux-based router hardware recommendations
And the solution to this issue is - http://routerboard.com/ or http://www.mikrotik.com/software# on x86 hardware, plus any basic layer2 switch. Don't scoff until you have tried it, the price/performance is pretty staggering if you are in the sub 20gig space. -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Mike Hammett Sent: Tuesday, 27 January 2015 2:44 p.m. To: nanog@nanog.org Subject: Re: scaling linux-based router hardware recommendations Aren't most of the new whitebox\open source platforms based on switching and not routing? I'd assume that the cloud-scale data centers deploying this stuff still have more traditional big iron at their cores. The small\medium sized ISP usually is left behind. They're not big enough to afford the big new hardware, but all of their user's NetFlix and porn and whatever else they do is chewing up bandwidth. For example, the small\medium ISPs are at the Nx10GigE stage now. The new hardware is expensive, the old hardware (besides being old) is likely in a huge chassis if you can get any sort of port density at all. 48 port GigE switches with a couple 10GigE can be had for $100. A minimum of 24 port 10GigE switches (except for the occasional IBM switch ) is 30x to 40x times that. Routers (BGP, MPLS, etc.) with that more than just a couple 10GigEs are even more money, I'd assume. I thought vMX was going to save the day, but it's pricing for 10 gigs of traffic (licensed by throughput and standard\advanced licenses) is really about 5x - 10x what I'd be willing to pay for it. Haven't gotten a quote from AlcaLu yet. Vyatta (last I checked, which was admittedly some time ago) doesn't have MPLS. The FreeBSD world can bring zero software cost and a stable platform, but no MPLS. Mikrotik brings most (though not all) of the features one would want... a good enough feature set, let's say... but is a non-stop flow of bugs. I don't think a week or two goes by where one of my friends doesn't submit some sort of reproducible bug to Mikrotik. They've also been looking into DPDK for 2.5 years now. hasn't shown up yet. I've used MT for 10 years and I'm always left wanting just a little more, but it may be the best balance between the features and performance I want and the ability to pay for it. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mehmet Akcin meh...@akcin.net To: micah anderson mi...@riseup.net Cc: nanog@nanog.org Sent: Monday, January 26, 2015 6:06:53 PM Subject: Re: scaling linux-based router hardware recommendations Cumulus Networks has some stuff, http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal-2014-final.pdf Pretty decent presentation with more details you like. Mehmet On Jan 26, 2015, at 8:53 PM, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
Re: Facebook outage?
Instagram appears to be down as well, but that would make sense since they are part of Facebook. On Jan 27, 2015, at 1:50, Damien Burke dam...@supremebytes.com wrote: I hear that AIM and hipchat is also having issues. Any other major company down too? -Original Message- From: John van Oppen [mailto:jvanop...@spectrumnet.us] Sent: Monday, January 26, 2015 10:49 PM To: Damien Burke; nanog@nanog.org Subject: RE: Facebook outage? Dead here at AS11404 from all locations where we PNI or public peer... must be bad over there, v4 dies at their edge, v6 makes it in but no page loads. John
Re: Facebook outage?
Hacking group Lizard Sqaud claims to have taken down @facebook https://twitter.com/facebook/@instagram https://twitter.com/instagram/@Tinder https://twitter.com/Tinder/@aim https://twitter.com/aim/@Myspace https://twitter.com/Myspace/
Re: Facebook outage?
https://twitter.com/LizardMafia/status/559963134006292481 On Mon, Jan 26, 2015 at 10:50 PM, Damien Burke dam...@supremebytes.com wrote: I hear that AIM and hipchat is also having issues. Any other major company down too? -Original Message- From: John van Oppen [mailto:jvanop...@spectrumnet.us] Sent: Monday, January 26, 2015 10:49 PM To: Damien Burke; nanog@nanog.org Subject: RE: Facebook outage? Dead here at AS11404 from all locations where we PNI or public peer... must be bad over there, v4 dies at their edge, v6 makes it in but no page loads. John -- *Trent Farrell* *Riot Games* *IP Network Engineer* E: tfarr...@riotgames.com | IE: +353 83 446 6809 | US: +1 424 285 9825 Summoner name: Foro
Re: Facebook outage?
Dead here from a close peering link. 1 10ge11-3.core1.lax1.he.net (65.49.27.149) [AS 6939] 4 msec 12 msec 0 msec 2 10ge1-3.core1.lax2.he.net (72.52.92.122) [AS 6939] 0 msec 8 msec 4 msec 3 any2ix.coresite.com (206.72.210.161) 0 msec 4 msec 0 msec 4 be2.bb01.lax1.tfbnw.net (31.13.30.24) [AS 32934] [MPLS: Label 20180 Exp 2] 64 msec 68 msec be2.bb02.lax1.tfbnw.net (31.13.30.26) [AS 32934] [MPLS: Label 18881 Exp 2] 56 msec 5 ae31.bb01.atl1.tfbnw.net (204.15.20.76) [AS 32934] [MPLS: Label 688077 Exp 2] 100 msec ae12.bb01.atl1.tfbnw.net (31.13.28.109) [AS 32934] [MPLS: Label 706669 Exp 2] 56 msec ae31.bb01.atl1.tfbnw.net (204.15.20.76) [AS 32934] [MPLS: Label 687613 Exp 2] 76 msec 6 be25.bb01.frc3.tfbnw.net (204.15.23.53) [AS 32934] [MPLS: Label 16526 Exp 2] 60 msec 64 msec 72 msec 7 ae60.dr03.frc3.tfbnw.net (74.119.79.11) [AS 32934] 96 msec ae60.dr01.frc3.tfbnw.net (204.15.23.247) [AS 32934] 56 msec ae60.dr03.frc3.tfbnw.net (74.119.79.11) [AS 32934] 64 msec 8 * * * 9 * * * On Mon, Jan 26, 2015 at 11:57 PM, Christopher Morrow morrowc.li...@gmail.com wrote: snow, it's a terrible thing. On Tue, Jan 27, 2015 at 1:50 AM, Damien Burke dam...@supremebytes.com wrote: I hear that AIM and hipchat is also having issues. Any other major company down too? -Original Message- From: John van Oppen [mailto:jvanop...@spectrumnet.us] Sent: Monday, January 26, 2015 10:49 PM To: Damien Burke; nanog@nanog.org Subject: RE: Facebook outage? Dead here at AS11404 from all locations where we PNI or public peer... must be bad over there, v4 dies at their edge, v6 makes it in but no page loads. John -- eSited LLC (701) 390-9638
Re: Facebook outage?
Instagram used to use Amazon AWS before being purchased by Facebook. There has been a slow migration onto FB infrastructure, so yes, a mixture of addresses like that makes sense. - Tim On 27 Jan 2015, at 2:58 pm, Christopher Morrow morrowc.li...@gmail.com wrote: On Tue, Jan 27, 2015 at 1:56 AM, Jason Canady ja...@unlimitednet.us wrote: Instagram appears to be down as well, but that would make sense since they are part of Facebook. $ dig +short facebook.com 173.252.120.6 NetRange: 173.252.64.0 - 173.252.127.255 CIDR: 173.252.64.0/18 NetName:FACEBOOK-INC but $ dig +short instagram.com 54.209.14.128 107.23.173.176 54.175.77.206 54.208.246.103 107.23.166.70 54.236.148.28 54.209.197.196 54.236.177.12 those are amazon addresses... err, not sure the connection makes sense though? -chris
Re: scaling linux-based router hardware recommendations
I'm also in the research stage of building our own router. I'm interested in reading more if you can post links to some of this research and/or testing. David Sent from my iPad On Jan 26, 2015, at 6:45 PM, Phil Bedard bedard.p...@gmail.com wrote: Kind of unsurprisingly, the traditional network vendors are somewhat at the forefront of pushing what an x86 server can do as well. Brocade (Vyatta), Juniper, and Alcatel-Lucent all have virtualized routers using Intel's DPDK pushing 5M+ PPS at this point. They are all also tweaking what Intel is providing, and they are the ones with lots of software developers with a lot of hardware and network programming experience. ALU claims to be able to get 160Gbps full duplex through a 2RU server with 16x10G interfaces and two 10-core latest-gen Xeon processors. Of course that's probably at 9000 byte packet sizes, but at Imix type traffic it's probably still pushing 60-70Gbps. They have a demo of lots of them in a single rack managed as a single router pushing Tbps. A commerical offering you are going to pay for that kind of performance and the control plane software. Over time though you'll see the DPDK type enhancements make it into standard OS stacks. Other options include servers with integrated network processors or NPs on a PCI card, there is a whole rash of those type of devices out there now and coming out. Phil On 1/26/15, 22:53, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
Facebook outage?
Facebook outage? Everyone panic! https://twitter.com/search?q=facebooksrc=typd -Damien
Re: Facebook outage?
snow, it's a terrible thing. On Tue, Jan 27, 2015 at 1:50 AM, Damien Burke dam...@supremebytes.com wrote: I hear that AIM and hipchat is also having issues. Any other major company down too? -Original Message- From: John van Oppen [mailto:jvanop...@spectrumnet.us] Sent: Monday, January 26, 2015 10:49 PM To: Damien Burke; nanog@nanog.org Subject: RE: Facebook outage? Dead here at AS11404 from all locations where we PNI or public peer... must be bad over there, v4 dies at their edge, v6 makes it in but no page loads. John
Re: Facebook outage?
On Tue, Jan 27, 2015 at 1:56 AM, Jason Canady ja...@unlimitednet.us wrote: Instagram appears to be down as well, but that would make sense since they are part of Facebook. $ dig +short facebook.com 173.252.120.6 NetRange: 173.252.64.0 - 173.252.127.255 CIDR: 173.252.64.0/18 NetName:FACEBOOK-INC but $ dig +short instagram.com 54.209.14.128 107.23.173.176 54.175.77.206 54.208.246.103 107.23.166.70 54.236.148.28 54.209.197.196 54.236.177.12 those are amazon addresses... err, not sure the connection makes sense though? -chris
Re: scaling linux-based router hardware recommendations
On 1/26/15 14:53, micah anderson wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! DPDK is your friend here. -Scott micah
Re: scaling linux-based router hardware recommendations
One thing to note about Ubiquiti's EdgeMax products is that they are not Intel based. They use Cavium Octeon's (at least that's what my EdgeRouter Lite has in it). Oliver - Oliver Garraux Check out my blog: blog.garraux.net Follow me on Twitter: twitter.com/olivergarraux On Mon, Jan 26, 2015 at 4:18 PM, Joe Greco jgr...@ns.sol.net wrote: I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. 10-15 years ago, we were seeing early Pentium 4 boxes capable of moving 100Kpps+ on FreeBSD. See for example http://info.iet.unipi.it/~luigi/polling/ Luigi moved on to Netmap, which looks promising for this sort of thing. https://www.usenix.org/system/files/conference/atc12/atc12-final186.pdf I was under the impression that some people have been using this for 10G routing. Also I'll note that Ubiquiti has some remarkable low-power gear capable of 1Mpps+. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: scaling linux-based router hardware recommendations
Kind of unsurprisingly, the traditional network vendors are somewhat at the forefront of pushing what an x86 server can do as well. Brocade (Vyatta), Juniper, and Alcatel-Lucent all have virtualized routers using Intel's DPDK pushing 5M+ PPS at this point. They are all also tweaking what Intel is providing, and they are the ones with lots of software developers with a lot of hardware and network programming experience. ALU claims to be able to get 160Gbps full duplex through a 2RU server with 16x10G interfaces and two 10-core latest-gen Xeon processors. Of course that's probably at 9000 byte packet sizes, but at Imix type traffic it's probably still pushing 60-70Gbps. They have a demo of lots of them in a single rack managed as a single router pushing Tbps. A commerical offering you are going to pay for that kind of performance and the control plane software. Over time though you'll see the DPDK type enhancements make it into standard OS stacks. Other options include servers with integrated network processors or NPs on a PCI card, there is a whole rash of those type of devices out there now and coming out. Phil On 1/26/15, 22:53, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
Re: scaling linux-based router hardware recommendations
It really depends on the application that you are interested in beyond forwarding, but not knowing that and to scale forwarding ³at a reasonable price, things have to come off cpu and become more customized for forwarding, especially for low latency forwarding. The optimization comes in minimizing packet tuple copies, off load to co-processors and network coprocessors (some of which can be in NICs) and parallel processing with some semblance of shared memory across, all of which takes customization beyond CPU and Kernel which in itself needs to be stripped down bare and embedded. Ultimately that¹s what appliance vendors do with different levels of hardware/firmware customization depending on ROI of features, speeds and price. A generic OpenSource compatible OEM product with multi-gig ports will generally be at least half to 5th the price of a high end latest server architecture server product with ability to support 10 gig interfaces in the same forwarding performance range (which are in the market for a different scale problem in compute and net I/O but exist at a price point that make them exorbitant to solve forwarding speed). Cheers, Sudeep Khuraijam On 1/26/15, 2:53 PM, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
Re: scaling linux-based router hardware recommendations
Different (configuration) strokes for different folks. I look at a Cisco interface now and say, Who the hell would use this? despite my decade old Cisco training. I was corrected offlist that Vyatta does do MPLS now... but I can't find anything on it doing VPLS, so I guess that's still out. The 5600's license (according to their SDNCentral performance report) appears to be near $7k whereas MT you can get a license for $80. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Paul S. cont...@winterei.se To: nanog@nanog.org Sent: Monday, January 26, 2015 8:10:54 PM Subject: Re: scaling linux-based router hardware recommendations Like Mike mentioned, the feature list in RouterOS is nothing short of impressive -- problem is that pretty much everything in there is inherently buggy. That and one hell of a painful syntax-schema to work with too. On 1/27/2015 午前 10:57, Tony Wicks wrote: And the solution to this issue is - http://routerboard.com/ or http://www.mikrotik.com/software# on x86 hardware, plus any basic layer2 switch. Don't scoff until you have tried it, the price/performance is pretty staggering if you are in the sub 20gig space. -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Mike Hammett Sent: Tuesday, 27 January 2015 2:44 p.m. To: nanog@nanog.org Subject: Re: scaling linux-based router hardware recommendations Aren't most of the new whitebox\open source platforms based on switching and not routing? I'd assume that the cloud-scale data centers deploying this stuff still have more traditional big iron at their cores. The small\medium sized ISP usually is left behind. They're not big enough to afford the big new hardware, but all of their user's NetFlix and porn and whatever else they do is chewing up bandwidth. For example, the small\medium ISPs are at the Nx10GigE stage now. The new hardware is expensive, the old hardware (besides being old) is likely in a huge chassis if you can get any sort of port density at all. 48 port GigE switches with a couple 10GigE can be had for $100. A minimum of 24 port 10GigE switches (except for the occasional IBM switch ) is 30x to 40x times that. Routers (BGP, MPLS, etc.) with that more than just a couple 10GigEs are even more money, I'd assume. I thought vMX was going to save the day, but it's pricing for 10 gigs of traffic (licensed by throughput and standard\advanced licenses) is really about 5x - 10x what I'd be willing to pay for it. Haven't gotten a quote from AlcaLu yet. Vyatta (last I checked, which was admittedly some time ago) doesn't have MPLS. The FreeBSD world can bring zero software cost and a stable platform, but no MPLS. Mikrotik brings most (though not all) of the features one would want... a good enough feature set, let's say... but is a non-stop flow of bugs. I don't think a week or two goes by where one of my friends doesn't submit some sort of reproducible bug to Mikrotik. They've also been looking into DPDK for 2.5 years now. hasn't shown up yet. I've used MT for 10 years and I'm always left wanting just a little more, but it may be the best balance between the features and performance I want and the ability to pay for it. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Mehmet Akcin meh...@akcin.net To: micah anderson mi...@riseup.net Cc: nanog@nanog.org Sent: Monday, January 26, 2015 6:06:53 PM Subject: Re: scaling linux-based router hardware recommendations Cumulus Networks has some stuff, http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal-2014-final.pdf Pretty decent presentation with more details you like. Mehmet On Jan 26, 2015, at 8:53 PM, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old
ATT uVerse blocking SIP?
Has anyone seen issues where a end user on uVerse trying to connect to either another provider or ATT non uVerse (in this case DIA) is having SIP blocked? SIP leaving the uVerse network going to another uVerse DSL account is fine, but it appears soon as it leave the uVerse network all SIP traffic is blocked? It appears others have seen this problem, some say it's a modem issue, some say they are truly blocking it. Ive yet to call uVerse support yet as im guessing ill get no where. Thanks for any insight on this. -- This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. 1st Class Hosting, LLC. 1712 Pioneer Ave, Suite 1854, Cheyenne, WY 82001
Re: ATT uVerse blocking SIP?
I think this is due to the CPE using a particular ALG ... (from recollection having never been a UVerse customer, but having sat through a long, long, long set of discussions about the merits/demerits of sip blocking) On Mon, Jan 26, 2015 at 10:22 PM, Jared Mauch ja...@puck.nether.net wrote: Yes. If you move to another port, e.g.: 5061 it works fine. If you’re running on a Linux based system, you can do this: /sbin/iptables -A PREROUTING -t nat -i eth1 -p udp --dport 5061 -j REDIRECT --to-port 5060 on the host to remap 5061 - 5060 with no application change. - Jared On Jan 26, 2015, at 4:26 PM, Brad Bendy b...@1stclasshosting.com wrote: Has anyone seen issues where a end user on uVerse trying to connect to either another provider or ATT non uVerse (in this case DIA) is having SIP blocked? SIP leaving the uVerse network going to another uVerse DSL account is fine, but it appears soon as it leave the uVerse network all SIP traffic is blocked? It appears others have seen this problem, some say it's a modem issue, some say they are truly blocking it. Ive yet to call uVerse support yet as im guessing ill get no where. Thanks for any insight on this. -- This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. E-mail transmission cannot be guaranteed to be secure or error-free as information could be intercepted, corrupted, lost, destroyed, arrive late or incomplete, or contain viruses. The sender therefore does not accept liability for any errors or omissions in the contents of this message, which arise as a result of e-mail transmission. If verification is required please request a hard-copy version. 1st Class Hosting, LLC. 1712 Pioneer Ave, Suite 1854, Cheyenne, WY 82001
Re: Facebook outage?
Some busted links there, but http://www.bbc.co.uk/newsbeat/30306319 /kc On Mon, Jan 26, 2015 at 10:58:21PM -0800, Chaim Rieger said: Hacking group Lizard Sqaud claims to have taken down @facebook https://twitter.com/facebook/@instagram https://twitter.com/instagram/@Tinder https://twitter.com/Tinder/@aim https://twitter.com/aim/@Myspace https://twitter.com/Myspace/ -- Ken Chase - m...@sizone.org Toronto
Re: Facebook outage?
cable was replugged, insta/fb back up here. /kc On Tue, Jan 27, 2015 at 02:04:58AM -0500, Zachary said: Seems unlikely, probably taking credit for someone tripping over a cable. -- Ken Chase - m...@sizone.org Toronto
Re: Facebook outage?
down from toronto. instagram too, of course. /kc -- Ken Chase - m...@sizone.org Toronto
Re: scaling linux-based router hardware recommendations
Hello! Looks like somebody want to build Linux soft router!) Nice idea for routing 10-30 GBps. I route about 5+ Gbps in Xeon E5-2620v2 with 4 10GE cards Intel 82599 and Debian Wheezy 3.2 (but it's really terrible kernel, everyone should use modern kernels since 3.16 because buggy linux route cache). My current processor load on server is about: 15%, thus I can route about 15 GE on my Linux server. Surely, you should deploy backup server too if master server fails. On Tue, Jan 27, 2015 at 1:53 AM, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah -- Sincerely yours, Pavel Odintsov
Re: Facebook outage?
It is back now fwiw On Jan 27, 2015 12:18 PM, Damien Burke dam...@supremebytes.com wrote: Facebook outage? Everyone panic! https://twitter.com/search?q=facebooksrc=typd -Damien
Re: scaling linux-based router hardware recommendations
A Maxxwave Routermxx MW-RM1300-i7 (x86 mikrotik router) pulls full tables from two peers and converges in about 40 seconds. On Mon, Jan 26, 2015 at 9:51 PM, Mike Hammett na...@ics-il.net wrote: Depends on the hardware. 30 - 45 seconds for the higher end stuff? I'm not sure how long it is on an RB750 (list price of like $40). ;-) - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Ken Chase m...@sizone.org To: nanog@nanog.org Sent: Monday, January 26, 2015 9:29:28 PM Subject: Re: scaling linux-based router hardware recommendations Hows convergence time on these mikrotik/ubiquity/etc units for a full table? /kc -- Ken Chase - m...@sizone.org Toronto -- Adair Winter VP, Network Operations / Owner Amarillo Wireless | 806.316.5071 C: 806.231.7180 http://www.amarillowireless.net
Re: Facebook outage?
The js console for hipchat shows tons of connection errors and 503s to facebook.com. It's like just a facebook outage and breaking sites that have facebook login options. On Mon, Jan 26, 2015 at 10:58 PM, Chaim Rieger chaim.rie...@gmail.com wrote: Hacking group Lizard Sqaud claims to have taken down @facebook https://twitter.com/facebook/@instagram https://twitter.com/instagram/@Tinder https://twitter.com/Tinder/@aim https://twitter.com/aim/@Myspace https://twitter.com/Myspace/
Re: REMINDER: Leap Second
On Sun, Jan 25, 2015 at 06:42:51PM -0500, TR Shaw wrote: That made the transformers smaller/cooler and more efficient. I seem to remember a 195 as well but maybe it is just CRS. Google says the 360/195 did exist. But my baby was the 360/95, where the first megabyte of memory was flat-film at 60ns, which made it faster than the 195 for some things. It was incredibly expensive to build - we heard rumors of $30 million in 1967 dollars, and sold to NASA at a huge loss, which is why there were only two built. I used to amuse myself by climbing into the flats memory cabinet, and was amused again some years later when I could have ingested a megabyte without harm. Ours sat directly above Tom's Restaurant, of Seinfeld fame. Very early climate modeling was done on that machine, along with a lot of astrophysics.
Peering Track: Peering Personals - any new peers out there?
Everyone: As the NANOG 63 Peering Track moderator, I would like to make a call for Peering Personals. This time around, I would like to limit the Personals to new - networks new to peering, existing networks with new locations, changes to peering policies, turning up v6 peering, etc. If you or your network would like to announce something new at the N63 Peering Track, please ping me off-list and I'll ensure there is time for your presentation. Thank you! -- TTFN, patrick
Re: scaling linux-based router hardware recommendations
Hi Micah, There is a segment in the Hardware Side of the industry that produces Network Appliances. (Folks such as Axiomtek, Lanner Electronics, Caswell Networks, Portwell etc etc) These appliances are commonly used as a commercial (OEM) platform for a variety of uses.. Routers, Firewalls, Specialized network applications etc. Our internal testing ( informal), matches up with the commonly quoted PPS handling by the different product vendors who incorporate these appliances in their network product offerings. i3/i5/i7 (x86) based network appliances will forward traffic as long as pps does not exceed 1.4million (In our testing we found the pps to be limiting factor and not the amount of traffic being moved) (will easily handle 6G to 10G of traffic Core2duo (x86) based network appliances will forward traffic as long as pps does not exceed 600, pps (will easily handle 1.5G to 2G of traffic) Atom based (x86) network appliances will forward traffic as long as pps does not exceed 250,000 pps. Of course, if you start to bog down the router with lots of NAT/ACL/ Bridge Rules (i.e. the CPU has to get involved in traffic management) then your actual performance will be degraded. Regards. Faisal Imtiaz Snappy Internet Telecom 7266 SW 48 Street Miami, FL 33155 Tel: 305 663 5518 x 232 Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net - Original Message - From: micah anderson mi...@riseup.net To: nanog@nanog.org Sent: Monday, January 26, 2015 5:53:54 PM Subject: scaling linux-based router hardware recommendations Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
scaling linux-based router hardware recommendations
Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
Re: scaling linux-based router hardware recommendations
Has anyone tested these setups with something more beefy like dual Xeons of Sandybridge or later vintage? Waiting to hear back from one NIC vendor (HotLava) what they think can be done on larger hardware setups. Put in two big Xeons and you're looking at 24 cores to work with as opposed to the 8 on the desktop versions. The newer ones would also have PCIe 3, which would overcome bus speed limitations in PCIe 2. Realistic to put 6x - 12x 10GigEs into a server with that much beef and expect it to perform well? What vintage of core ix do you run, Faisal? - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com - Original Message - From: Faisal Imtiaz fai...@snappytelecom.net To: micah anderson mi...@riseup.net Cc: nanog@nanog.org Sent: Monday, January 26, 2015 5:27:55 PM Subject: Re: scaling linux-based router hardware recommendations Hi Micah, There is a segment in the Hardware Side of the industry that produces Network Appliances. (Folks such as Axiomtek, Lanner Electronics, Caswell Networks, Portwell etc etc) These appliances are commonly used as a commercial (OEM) platform for a variety of uses.. Routers, Firewalls, Specialized network applications etc. Our internal testing ( informal), matches up with the commonly quoted PPS handling by the different product vendors who incorporate these appliances in their network product offerings. i3/i5/i7 (x86) based network appliances will forward traffic as long as pps does not exceed 1.4million (In our testing we found the pps to be limiting factor and not the amount of traffic being moved) (will easily handle 6G to 10G of traffic Core2duo (x86) based network appliances will forward traffic as long as pps does not exceed 600, pps (will easily handle 1.5G to 2G of traffic) Atom based (x86) network appliances will forward traffic as long as pps does not exceed 250,000 pps. Of course, if you start to bog down the router with lots of NAT/ACL/ Bridge Rules (i.e. the CPU has to get involved in traffic management) then your actual performance will be degraded. Regards. Faisal Imtiaz Snappy Internet Telecom 7266 SW 48 Street Miami, FL 33155 Tel: 305 663 5518 x 232 Help-desk: (305)663-5518 Option 2 or Email: supp...@snappytelecom.net - Original Message - From: micah anderson mi...@riseup.net To: nanog@nanog.org Sent: Monday, January 26, 2015 5:53:54 PM Subject: scaling linux-based router hardware recommendations Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
Re: scaling linux-based router hardware recommendations
I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. 10-15 years ago, we were seeing early Pentium 4 boxes capable of moving 100Kpps+ on FreeBSD. See for example http://info.iet.unipi.it/~luigi/polling/ Luigi moved on to Netmap, which looks promising for this sort of thing. https://www.usenix.org/system/files/conference/atc12/atc12-final186.pdf I was under the impression that some people have been using this for 10G routing. Also I'll note that Ubiquiti has some remarkable low-power gear capable of 1Mpps+. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples.
Re: scaling linux-based router hardware recommendations
Cumulus Networks has some stuff, http://www.bigswitch.com/sites/default/files/presentations/onug-baremetal-2014-final.pdf Pretty decent presentation with more details you like. Mehmet On Jan 26, 2015, at 8:53 PM, micah anderson mi...@riseup.net wrote: Hi, I know that specially programmed ASICs on dedicated hardware like Cisco, Juniper, etc. are going to always outperform a general purpose server running gnu/linux, *bsd... but I find the idea of trying to use proprietary, NSA-backdoored devices difficult to accept, especially when I don't have the budget for it. I've noticed that even with a relatively modern system (supermicro with a 4 core 1265LV2 CPU, with a 9MB cache, Intel E1G44HTBLK Server adapters, and 16gig of ram, you still tend to get high percentage of time working on softirqs on all the CPUs when pps reaches somewhere around 60-70k, and the traffic approaching 600-900mbit/sec (during a DDoS, such hardware cannot typically cope). It seems like finding hardware more optimized for very high packet per second counts would be a good thing to do. I just have no idea what is out there that could meet these goals. I'm unsure if faster CPUs, or more CPUs is really the problem, or networking cards, or just plain old fashioned tuning. Any ideas or suggestions would be welcome! micah
