Re: Low Cost 10G Router
For the lists benefit, there is a 6 X 10GBE option for the ASR1000 series it seems. No idea on pricing though. http://www.cisco.com/c/en/us/products/collateral/application-networking-services/wide-area-application-services-waas-software/data-sheet-c78-729778.pdf Cheers, Mark On Wed, May 20, 2015 at 3:59 PM, Mark Tinka mark.ti...@seacom.mu wrote: On 19/May/15 20:46, Ray Soucy wrote: An ASR1K might do the trick, but more likely than not you're looking at an ASR9K if you want full tables; I don't have any experience with the 1K personally so I can't speak to that. The ASR 9K is a really great platform and is what we use for BGP here, but it's pretty much the opposite of cheap. The ASR1000 is a very good box, but I tend to prefer them for low-speed services, which are generally non-Ethernet in nature, e.g., downstream customers coming in via SDH. They do support 10Gbps ports, but that is a 1-port SPA; and the most you can have in today's SIP's (carrier cards) would be 4x 1-port SPA's. So not very dense. Their forwarding planes start at 2.5Gbps (fixed) all the way to 200Gbps (13-slot chassis). But you're more likely to run out of high-speed ports before you stress a 200Gbps forwarding plane on that chassis. So if the applications are purely Ethernet, I'd not consider the ASR1000. But if there is a mix-and-match for Ethernet and non-Ethernet ports, it's the perfect box. That and the MX104. Mark. -- Regards, Mark L. Tees
Re: Low Cost 10G Router
On 20/May/15 08:54, Jeff Tantsura wrote: ASR1K (XE) has great BGP implementation, go for it if you are OK with density/throughput. I second that. BGP for IOS XE is very mature (except RPKI, which has just got a fix). Mark.
Re: Low Cost 10G Router
On 19/May/15 23:59, Rodrigo 1telecom wrote: I know if is not possible to have a full routing on ex3300(low memory for it) , but i never tried to do a default router on it( with EFL licence and software above version 12) I have many bgp session with cisco 3750 switchs.. Traffic about 2gb on it... Have a peer( ebgp customer) with a acx2000( i know it have 10gb port) we send to this router a default route only... And it have 1.5gb with us and more 1gb with other link provider... If you need a full table in FIB, then you're stuffed with any switch vendor out there. But if your switch vendor is able to hold the full table in RIB, and allow you to selectively hold chosen routes in FIB, then you could get away with lots of 10Gbps-capable switches at a reasonable price. Mark.
Re: Low Cost 10G Router
On 19/May/15 20:46, Ray Soucy wrote: An ASR1K might do the trick, but more likely than not you're looking at an ASR9K if you want full tables; I don't have any experience with the 1K personally so I can't speak to that. The ASR 9K is a really great platform and is what we use for BGP here, but it's pretty much the opposite of cheap. The ASR1000 is a very good box, but I tend to prefer them for low-speed services, which are generally non-Ethernet in nature, e.g., downstream customers coming in via SDH. They do support 10Gbps ports, but that is a 1-port SPA; and the most you can have in today's SIP's (carrier cards) would be 4x 1-port SPA's. So not very dense. Their forwarding planes start at 2.5Gbps (fixed) all the way to 200Gbps (13-slot chassis). But you're more likely to run out of high-speed ports before you stress a 200Gbps forwarding plane on that chassis. So if the applications are purely Ethernet, I'd not consider the ASR1000. But if there is a mix-and-match for Ethernet and non-Ethernet ports, it's the perfect box. That and the MX104. Mark.
Re: Low Cost 10G Router
ASR1K (XE) has great BGP implementation, go for it if you are OK with density/throughput. Regards, Jeff On May 19, 2015, at 11:35 PM, Mark Tees markt...@gmail.com wrote: For the lists benefit, there is a 6 X 10GBE option for the ASR1000 series it seems. No idea on pricing though. http://www.cisco.com/c/en/us/products/collateral/application-networking-services/wide-area-application-services-waas-software/data-sheet-c78-729778.pdf Cheers, Mark On Wed, May 20, 2015 at 3:59 PM, Mark Tinka mark.ti...@seacom.mu wrote: On 19/May/15 20:46, Ray Soucy wrote: An ASR1K might do the trick, but more likely than not you're looking at an ASR9K if you want full tables; I don't have any experience with the 1K personally so I can't speak to that. The ASR 9K is a really great platform and is what we use for BGP here, but it's pretty much the opposite of cheap. The ASR1000 is a very good box, but I tend to prefer them for low-speed services, which are generally non-Ethernet in nature, e.g., downstream customers coming in via SDH. They do support 10Gbps ports, but that is a 1-port SPA; and the most you can have in today's SIP's (carrier cards) would be 4x 1-port SPA's. So not very dense. Their forwarding planes start at 2.5Gbps (fixed) all the way to 200Gbps (13-slot chassis). But you're more likely to run out of high-speed ports before you stress a 200Gbps forwarding plane on that chassis. So if the applications are purely Ethernet, I'd not consider the ASR1000. But if there is a mix-and-match for Ethernet and non-Ethernet ports, it's the perfect box. That and the MX104. Mark. -- Regards, Mark L. Tees
Re: Measuring DNS Performance Graphing Logs
I was wondering which tool(s) can I use to measure the performance of my 3 DNS servers (1 primary, 1 secondary, 1 solely cacheDNS)? From the stats I would like to know if my DNS server is serving as it should be or if any of it's options are set inappropriately and others alike. Perhaps http://dns.measurement-factory.com/tools/dsc/ (used by AS112) can help. Denis
Re: Measuring DNS Performance Graphing Logs
Smokeping (http://oss.oetiker.ch/smokeping/) can graph DNS response latency via dig. ThousandEyes (https://www.thousandeyes.com/) has some commercial options for monitoring DNS server responsiveness, and zone performance from different vantage points throughout the globe. On Tue, May 19, 2015 at 12:34 PM, Zayed Mahmud zayed.mah...@gmail.com wrote: Hello! This is my first message to NANOG's mailing list. I hope someone can help me. I was wondering which tool(s) can I use to measure the performance of my 3 DNS servers (1 primary, 1 secondary, 1 solely cacheDNS)? From the stats I would like to know if my DNS server is serving as it should be or if any of it's options are set inappropriately and others alike. I looked for a while but could not find any. Any help would be highly appreciated. I am running bind9 on UNIX platform. Question 2) I would also like to know how can I graph my DNS logs? And how can I integrate it to my CACTI server as well? I couldn't find any suitable plugin. Any suggestion? -- -- Best Regards, *Zayed Mahmud* *Senior Core IP Network Team,* *Banglalion Communications Limited, Bangladesh.*
Re: Low Cost 10G Router
You're right I dropped down to the v2 for pricing reasons: - Supermicro SuperServer 5017R-MTRF - 4x SATA - 8x DDR3 - 400W Redundant - Eight-Core Intel Xeon Processor E5-2640 v2 2.00GHz 20MB Cache (95W) - 4 x SAMSUNG 2GB PC3-12800 DDR3-160 - 2 x 500GB SATA 6.0Gb/s 7200RPM - 3.5 - Western Digital RE4 WD5003ABYZ - Supermicro System Cabinet Front Bezel CSE-PTFB-813B with Lock and Filter (Black) - No Windows Operating System (Hardware Warranty Only, No Software Support) - Three Year Warranty with Advanced Parts Replacement FWIW I used Sourcecode as the system builder. They've been great to work with. On Tue, May 19, 2015 at 4:46 PM, Joe Greco jgr...@ns.sol.net wrote: How cheap is cheap and what performance numbers are you looking for? About as cheap as you can get: For about $3,000 you can build a Supermicro OEM system with an 8-core Xeon E5 V3 and 4-port 10G Intel SFP+ NIC with 8G of RAM running VyOS. The pro is that BGP convergence time will be good (better than a 7200 VXR), and number of tables likely won't be a concern since RAM is cheap. The con is that you're not doing things in hardware, so you'll have higher latency, and your PPS will be lower. What 8 core Xeon E5 v3 would that be? The 26xx's are hideously pricey, and for a router, you're probably better off with something like a Supermicro X10SRn fsvo n with a Xeon E5-1650v3. Board is typically around $300, 1650 is around $550, so total cost I'm guessing closer to $1500-$2000 that route. The edge you get there is the higher clock on the CPU. Only six cores and only 15M cache, but 3.5GHz. The E5-2643v3 is three times the cost for very similar performance specs. Costwise, E5 single socket is the way to go unless you *need* more. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples. -- Ray Patrick Soucy Network Engineer University of Maine System T: 207-561-3526 F: 207-561-3531 MaineREN, Maine's Research and Education Network www.maineren.net
Re: Low Cost 10G Router
On Tuesday, May 19, 2015, Warsaw wrote: On May 19, 2015, at 10:22, Colton Conor colton.co...@gmail.com javascript:; wrote: What options are available for a small, low cost router that has at least four 10G ports, and can handle full BGP routes? All that I know of are the Juniper MX80, and the Brocade CER line. What does Cisco and others have that compete with these two? Any other vendors besides Juniper, Brocade, and Cisco to look at? I have two ServerU L-800 boxes routing BGP and OSPF, one of those has 4x10G SFP+ port and the I'm good w/ ServerU L-800 as well running BGP with FreeBSD in a location and VyOS in a couple other. I still dont know how much traffic Mr Conor needs to forward, if it's a 10G base or just needs 10G ports. Without Chelsio ASICS I route 4Gb/s on this router and I second the suggestion for L-800 if the desired forwarding rate is around 4Gbit. I didnt know Chelsio expansions could do forwarding directly on the card. just heard about its low rate of interruption requests. Sounds like it worths further investigation thanks on that.. As for L-800 I run it for over one year now doing BGP and firewalling. Great value for a twelve hundred bucks purchase. It's a 1,200 USD starting cost for a very decent router which promisses to delivery a good pps and bps rate specially when compared to Mikrotik's CCR and other Cisco/Brocade routers on this same grade. Add to it a couple hundred extra bucks to have a very decent Chelsio T5 ASICS expansion to L800 chassis and you pretty much have a system that, according to Chelsion data sheet, promisses to delivery 27 milion packets per second filtered and forwarded. Pretty much Line Rate for 10G ports. I don't know about the expected 27Mpps per port, but I can confirm 4.8Mpps peaking / 4.2Mpps avging on my rack everyday, and for the price I pay on this ServerU + FreeBSD setup I can't avoid to suggest it worths pretty much a try! http://www.serveru.us/en/netmapl800 If you buy a Chelsio card or already have it, or have it at a better price (sometimes we find very good 300.00 USD deals on chelsio T5, while their list price is ~900.00 USD) talk to 'em first, they have Chelsio front expansions by default but if you buy a Chelsio x8 PCIe card your own they need to arrange ServerU L-800 to have it perfectly fitted in their L-800 chassis, and usually it requires rear raiser replacement in their router, so talk to them first... I learned it the bad way ;] bought the chelsio card myself and found out I could not use it, since this L-800 router comes with raisers for front expansions. They were gentle enough to upgrade the raiser for free but I had to ship the box back to Florida. So talk to them... -- === Eduardo Meyer pessoal: dudu.me...@gmail.com profissional: ddm.farmac...@saude.gov.br
Re: Low Cost 10G Router
P.S I went through HotLava Systems for the Intel-based SFP+ NICs to add to those, http://hotlavasystems.com/ (not trying to plug; these are just hard to find) On Wed, May 20, 2015 at 9:08 AM, Ray Soucy r...@maine.edu wrote: You're right I dropped down to the v2 for pricing reasons: - Supermicro SuperServer 5017R-MTRF - 4x SATA - 8x DDR3 - 400W Redundant - Eight-Core Intel Xeon Processor E5-2640 v2 2.00GHz 20MB Cache (95W) - 4 x SAMSUNG 2GB PC3-12800 DDR3-160 - 2 x 500GB SATA 6.0Gb/s 7200RPM - 3.5 - Western Digital RE4 WD5003ABYZ - Supermicro System Cabinet Front Bezel CSE-PTFB-813B with Lock and Filter (Black) - No Windows Operating System (Hardware Warranty Only, No Software Support) - Three Year Warranty with Advanced Parts Replacement FWIW I used Sourcecode as the system builder. They've been great to work with. On Tue, May 19, 2015 at 4:46 PM, Joe Greco jgr...@ns.sol.net wrote: How cheap is cheap and what performance numbers are you looking for? About as cheap as you can get: For about $3,000 you can build a Supermicro OEM system with an 8-core Xeon E5 V3 and 4-port 10G Intel SFP+ NIC with 8G of RAM running VyOS. The pro is that BGP convergence time will be good (better than a 7200 VXR), and number of tables likely won't be a concern since RAM is cheap. The con is that you're not doing things in hardware, so you'll have higher latency, and your PPS will be lower. What 8 core Xeon E5 v3 would that be? The 26xx's are hideously pricey, and for a router, you're probably better off with something like a Supermicro X10SRn fsvo n with a Xeon E5-1650v3. Board is typically around $300, 1650 is around $550, so total cost I'm guessing closer to $1500-$2000 that route. The edge you get there is the higher clock on the CPU. Only six cores and only 15M cache, but 3.5GHz. The E5-2643v3 is three times the cost for very similar performance specs. Costwise, E5 single socket is the way to go unless you *need* more. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples. -- Ray Patrick Soucy Network Engineer University of Maine System T: 207-561-3526 F: 207-561-3531 MaineREN, Maine's Research and Education Network www.maineren.net -- Ray Patrick Soucy Network Engineer University of Maine System T: 207-561-3526 F: 207-561-3531 MaineREN, Maine's Research and Education Network www.maineren.net
Re: Low Cost 10G Router
Hello! Ray, I could suggest switch from multi physical CPU configuration to single. Like Intel Xeon E5-1650/1660/1680 or even Xeon E3 platforms. Because multi processor systems need really huge amount of knowledge for NUMA configuration and PCI-E devices assignment for each NUMA. Secondly, I could vote many times for Supermicro! :) Dell or HP are really ugly systems for soft routers. CPU frequency tuning, PCM debugging are real nightmare on this systems. Please beware of they! Supermicro is very clear and do not block useful functions of platform. On Wed, May 20, 2015 at 4:08 PM, Ray Soucy r...@maine.edu wrote: You're right I dropped down to the v2 for pricing reasons: - Supermicro SuperServer 5017R-MTRF - 4x SATA - 8x DDR3 - 400W Redundant - Eight-Core Intel Xeon Processor E5-2640 v2 2.00GHz 20MB Cache (95W) - 4 x SAMSUNG 2GB PC3-12800 DDR3-160 - 2 x 500GB SATA 6.0Gb/s 7200RPM - 3.5 - Western Digital RE4 WD5003ABYZ - Supermicro System Cabinet Front Bezel CSE-PTFB-813B with Lock and Filter (Black) - No Windows Operating System (Hardware Warranty Only, No Software Support) - Three Year Warranty with Advanced Parts Replacement FWIW I used Sourcecode as the system builder. They've been great to work with. On Tue, May 19, 2015 at 4:46 PM, Joe Greco jgr...@ns.sol.net wrote: How cheap is cheap and what performance numbers are you looking for? About as cheap as you can get: For about $3,000 you can build a Supermicro OEM system with an 8-core Xeon E5 V3 and 4-port 10G Intel SFP+ NIC with 8G of RAM running VyOS. The pro is that BGP convergence time will be good (better than a 7200 VXR), and number of tables likely won't be a concern since RAM is cheap. The con is that you're not doing things in hardware, so you'll have higher latency, and your PPS will be lower. What 8 core Xeon E5 v3 would that be? The 26xx's are hideously pricey, and for a router, you're probably better off with something like a Supermicro X10SRn fsvo n with a Xeon E5-1650v3. Board is typically around $300, 1650 is around $550, so total cost I'm guessing closer to $1500-$2000 that route. The edge you get there is the higher clock on the CPU. Only six cores and only 15M cache, but 3.5GHz. The E5-2643v3 is three times the cost for very similar performance specs. Costwise, E5 single socket is the way to go unless you *need* more. ... JG -- Joe Greco - sol.net Network Services - Milwaukee, WI - http://www.sol.net We call it the 'one bite at the apple' rule. Give me one chance [and] then I won't contact you again. - Direct Marketing Ass'n position on e-mail spam(CNN) With 24 million small businesses in the US alone, that's way too many apples. -- Ray Patrick Soucy Network Engineer University of Maine System T: 207-561-3526 F: 207-561-3531 MaineREN, Maine's Research and Education Network www.maineren.net -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
I haven't tried myself but some of the stuff Cumulus Linux is doing is pretty amazing, not certain quagga can or should handle full bgp table but you could probably get a Penguin 10gbe for less than 8k. On Tue, May 19, 2015, 10:25 AM Colton Conor colton.co...@gmail.com wrote: What options are available for a small, low cost router that has at least four 10G ports, and can handle full BGP routes? All that I know of are the Juniper MX80, and the Brocade CER line. What does Cisco and others have that compete with these two? Any other vendors besides Juniper, Brocade, and Cisco to look at?
Re: Low Cost 10G Router
On 20/05/2015 14:32, Cody Grosskopf wrote: I haven't tried myself but some of the stuff Cumulus Linux is doing is pretty amazing, not certain quagga can or should handle full bgp table but you could probably get a Penguin 10gbe for less than 8k. quagga (or whatever RIB manager you want, e.g. bird) isn't the issue. The issue is that these switches have limited hardware FIB capacity and if you attempt to put a full table on them, they won't accept it. Nick
Re: Low Cost 10G Router
We could cut full BGP and select only important prefixes with ExaBGP. On Wed, May 20, 2015 at 4:41 PM, Nick Hilliard n...@foobar.org wrote: On 20/05/2015 14:32, Cody Grosskopf wrote: I haven't tried myself but some of the stuff Cumulus Linux is doing is pretty amazing, not certain quagga can or should handle full bgp table but you could probably get a Penguin 10gbe for less than 8k. quagga (or whatever RIB manager you want, e.g. bird) isn't the issue. The issue is that these switches have limited hardware FIB capacity and if you attempt to put a full table on them, they won't accept it. Nick -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
On 20 May 2015 at 15:00, Pavel Odintsov pavel.odint...@gmail.com wrote: Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Aled
Re: Low Cost 10G Router
I have tried Cumulus. It's awesome! :) You definitely could run Quagga, Bird or even ExaBGP https://github.com/Exa-Networks/exabgp and build full feature router from 10GE switch. On Wed, May 20, 2015 at 4:32 PM, Cody Grosskopf codygrossk...@gmail.com wrote: I haven't tried myself but some of the stuff Cumulus Linux is doing is pretty amazing, not certain quagga can or should handle full bgp table but you could probably get a Penguin 10gbe for less than 8k. On Tue, May 19, 2015, 10:25 AM Colton Conor colton.co...@gmail.com wrote: What options are available for a small, low cost router that has at least four 10G ports, and can handle full BGP routes? All that I know of are the Juniper MX80, and the Brocade CER line. What does Cisco and others have that compete with these two? Any other vendors besides Juniper, Brocade, and Cisco to look at? -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
On 20/05/2015 14:46, Pavel Odintsov wrote: We could cut full BGP and select only important prefixes with ExaBGP. exabgp is rib mgmt only and doesn't program the fib. you will need quagga / bird / etc for this. Nick
Re: Low Cost 10G Router
Yes, right! But ExaBGP could receive full BGP table, drop some rules and reflect they to Quagga which could load FIB on the Cumulus. On Wed, May 20, 2015 at 4:53 PM, Nick Hilliard n...@foobar.org wrote: On 20/05/2015 14:46, Pavel Odintsov wrote: We could cut full BGP and select only important prefixes with ExaBGP. exabgp is rib mgmt only and doesn't program the fib. you will need quagga / bird / etc for this. Nick -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
On 20/05/2015 14:56, Pavel Odintsov wrote: Yes, right! But ExaBGP could receive full BGP table, drop some rules and reflect they to Quagga which could load FIB on the Cumulus. or you could not bother with exabgp and do your route filtering on quagga. Nothing wrong with exabgp, btw. Great product. It's just the wrong tool for the job here. Nick
Re: Low Cost 10G Router
Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. On Wed, May 20, 2015 at 4:57 PM, Nick Hilliard n...@foobar.org wrote: On 20/05/2015 14:56, Pavel Odintsov wrote: Yes, right! But ExaBGP could receive full BGP table, drop some rules and reflect they to Quagga which could load FIB on the Cumulus. or you could not bother with exabgp and do your route filtering on quagga. Nothing wrong with exabgp, btw. Great product. It's just the wrong tool for the job here. Nick -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
There will *not* be multi-threaded BGP in RouterOS. I was going to refer you to the post I made last night, but due to the unique way the e-mail list is setup, I replied directly to Colton instead of the list. I resent it again to the list. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com - Original Message - From: Aled Morris al...@qix.co.uk To: Colton Conor colton.co...@gmail.com Cc: North American Network Operators Group nanog@nanog.org Sent: Wednesday, May 20, 2015 11:59:04 AM Subject: Re: Low Cost 10G Router On 20 May 2015 at 17:44, Colton Conor colton.co...@gmail.com wrote: So are the rest of the processes in Mikrotik OS multi threaded? I would hope so to take advantage of 36 cores! The forthcoming new major software release from Mikrotik apparently will have multi-threaded BGP - it is targetted at their (also forthcoming) 72 core 8x10GE router, the CCR1072 I would treat this as speculation until you can order it though - it's been promised for 18 months now. Aled
Re: Low Cost 10G Router
So are the rest of the processes in Mikrotik OS multi threaded? I would hope so to take advantage of 36 cores! What is up with all of these network vendors not supporting more than one core in their OS? I just don't get it. On Tue, May 19, 2015 at 9:49 PM, Josh Baird joshba...@gmail.com wrote: The BGP daemon on the CCR routers is not multi-threaded; it only will use one core. Josh On Tue, May 19, 2015 at 10:06 PM, Colton Conor colton.co...@gmail.com wrote: So this new $1295 Mikrotik CCR1036-8G-2S+EM has a 36 core Tilera CPU with 16GB of ram. Each core is running at 1.2Ghz? I assume that Mikrotik is multicore in software, so why does this box not outperform these intel boxes that everyone is recommending? Is it just a limitation of ports? On Tue, May 19, 2015 at 6:03 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: I've seen serious, unusual performance bottlenecks in Mikrotik CCR, in some cases not even achieving a gigabit speeds on 10G interfaces. Performance drops more rapidly then Cisco with smaller packet sizes. -mel beckman Folks often forget that Mikrotik ROS can also run on x86 machines. Size your favorite hardware (server) or network appliance with appropriate ports, add MT ROS on a CF card, and you are good to go. We use i7 based network appliance with dual 10g cards (you can use a quad 10g card, such as those made by hotlav). with a 2gig of ram, you can easily do multiple (4-5 or more full bgp peers), and i7 are good for approx 1.2mill pps. Best of luck. Faisal Imtiaz Snappy Internet Telecom
Re: Low Cost 10G Router
On 20 May 2015 at 17:44, Colton Conor colton.co...@gmail.com wrote: So are the rest of the processes in Mikrotik OS multi threaded? I would hope so to take advantage of 36 cores! The forthcoming new major software release from Mikrotik apparently will have multi-threaded BGP - it is targetted at their (also forthcoming) 72 core 8x10GE router, the CCR1072 I would treat this as speculation until you can order it though - it's been promised for 18 months now. Aled
Re: Low Cost 10G Router
ZTE M6000-3S. It is what we use. Works well for us. Just remember to get a memory upgrade to 8 GB memory or you will run out of RIB space. Regards Baldur Den 20/05/2015 18.43 skrev Colton Conor colton.co...@gmail.com: So, from the sounds of it most are saying for low cost, the way to go would be a software router, which I was trying to avoid. To answer the bandwidth question, we would have three 10G ports with three different carriers and at max push 10Gbps of total traffic to start. I think this leaves me with hardware routers that can support full BGP tables. So, who actually sells full bgp routers. So far on my list I have: Juniper MX Series Brocade MLXe or CER Cisco ASR 9K Huawei NE40E-X1-M4 ZTE, not sure which model? ALU 7750 Besides the above, am I missing anyone else that makes a true carrier grade hardware router? On Wed, May 20, 2015 at 9:54 AM, Pavel Odintsov pavel.odint...@gmail.com wrote: Hello! Yes, we could run route add / route del when we got any announce from external world with ExaBGP directly. I have implemented custom custom Firewall (netmap-ipfw) management tool which implement in similar manner. But I'm working with BGP flow spec. It's so complex, standard BGP is much times simpler. And I could share my ExaBGP configuration and hook scripts. ExaBGP config: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_firewall.conf Hook script which put all announces to Redis Queue: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_queue_writer.py But full BGP route table is enough big and need external processing. But yes, with some Python code is possible to implement route server with ExaBGP. On Wed, May 20, 2015 at 5:25 PM, Aled Morris al...@qix.co.uk wrote: On 20 May 2015 at 15:00, Pavel Odintsov pavel.odint...@gmail.com wrote: Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Aled -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
Well, the cores on a many-core CPU aren't going to have the torque that a Xeon would. They're also still working on the software. It has gotten a ton better over the life of the CCRs thus far. BGP is still atrocious on the CCRs, but that's because the route update process isn't multithreaded. It won't be multithreaded in the next major version either, but they will have done some programming voodoo (all programming is voodoo to me) to reign in the poor performance issues with full tables. https://youtu.be/ihZiAC-Rox8?t=37m8s - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com - Original Message - From: Colton Conor colton.co...@gmail.com To: Faisal Imtiaz fai...@snappytelecom.net Cc: North American Network Operators Group nanog@nanog.org Sent: Tuesday, May 19, 2015 9:06:26 PM Subject: Re: Low Cost 10G Router So this new $1295 Mikrotik CCR1036-8G-2S+EM has a 36 core Tilera CPU with 16GB of ram. Each core is running at 1.2Ghz? I assume that Mikrotik is multicore in software, so why does this box not outperform these intel boxes that everyone is recommending? Is it just a limitation of ports? On Tue, May 19, 2015 at 6:03 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: I've seen serious, unusual performance bottlenecks in Mikrotik CCR, in some cases not even achieving a gigabit speeds on 10G interfaces. Performance drops more rapidly then Cisco with smaller packet sizes. -mel beckman Folks often forget that Mikrotik ROS can also run on x86 machines. Size your favorite hardware (server) or network appliance with appropriate ports, add MT ROS on a CF card, and you are good to go. We use i7 based network appliance with dual 10g cards (you can use a quad 10g card, such as those made by hotlav). with a 2gig of ram, you can easily do multiple (4-5 or more full bgp peers), and i7 are good for approx 1.2mill pps. Best of luck. Faisal Imtiaz Snappy Internet Telecom
Re: Low Cost 10G Router
Hello Pavel, Using ExaBGP as an SDN already has been done (and in a very large scale). But I would agree with Nick; It is not something I would recommend to everyone. Once more to echo Nick, to add/remove route/fw entries on Linux please do use netlink. The lastest ExaBGP master has some start of code to implement NetLink in python but I recently found a python module for it: https://github.com/svinota/pyroute2 Before ExaBGP can become a route server, I must complete a number of pieces (like the CLI which I am currently coding). I have spoken with the IX community about making ExaBGP a RR/RS and the idea was not badly received, but no one offered to help so it is on the back burner. Thomas On 20 May 2015, at 15:54, Pavel Odintsov wrote: Hello! Yes, we could run route add / route del when we got any announce from external world with ExaBGP directly. I have implemented custom custom Firewall (netmap-ipfw) management tool which implement in similar manner. But I'm working with BGP flow spec. It's so complex, standard BGP is much times simpler. And I could share my ExaBGP configuration and hook scripts. ExaBGP config: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_firewall.conf Hook script which put all announces to Redis Queue: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_queue_writer.py But full BGP route table is enough big and need external processing. But yes, with some Python code is possible to implement route server with ExaBGP. On Wed, May 20, 2015 at 5:25 PM, Aled Morris al...@qix.co.uk wrote: On 20 May 2015 at 15:00, Pavel Odintsov pavel.odint...@gmail.com wrote: Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Aled -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
So, from the sounds of it most are saying for low cost, the way to go would be a software router, which I was trying to avoid. To answer the bandwidth question, we would have three 10G ports with three different carriers and at max push 10Gbps of total traffic to start. I think this leaves me with hardware routers that can support full BGP tables. So, who actually sells full bgp routers. So far on my list I have: Juniper MX Series Brocade MLXe or CER Cisco ASR 9K Huawei NE40E-X1-M4 ZTE, not sure which model? ALU 7750 Besides the above, am I missing anyone else that makes a true carrier grade hardware router? On Wed, May 20, 2015 at 9:54 AM, Pavel Odintsov pavel.odint...@gmail.com wrote: Hello! Yes, we could run route add / route del when we got any announce from external world with ExaBGP directly. I have implemented custom custom Firewall (netmap-ipfw) management tool which implement in similar manner. But I'm working with BGP flow spec. It's so complex, standard BGP is much times simpler. And I could share my ExaBGP configuration and hook scripts. ExaBGP config: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_firewall.conf Hook script which put all announces to Redis Queue: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_queue_writer.py But full BGP route table is enough big and need external processing. But yes, with some Python code is possible to implement route server with ExaBGP. On Wed, May 20, 2015 at 5:25 PM, Aled Morris al...@qix.co.uk wrote: On 20 May 2015 at 15:00, Pavel Odintsov pavel.odint...@gmail.com wrote: Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Aled -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
good, cheap, built by someone else pick 2 On Wed, May 20, 2015 at 9:42 AM, Colton Conor colton.co...@gmail.com wrote: So, from the sounds of it most are saying for low cost, the way to go would be a software router, which I was trying to avoid. To answer the bandwidth question, we would have three 10G ports with three different carriers and at max push 10Gbps of total traffic to start. I think this leaves me with hardware routers that can support full BGP tables. So, who actually sells full bgp routers. So far on my list I have: Juniper MX Series Brocade MLXe or CER Cisco ASR 9K Huawei NE40E-X1-M4 ZTE, not sure which model? ALU 7750 Besides the above, am I missing anyone else that makes a true carrier grade hardware router? On Wed, May 20, 2015 at 9:54 AM, Pavel Odintsov pavel.odint...@gmail.com wrote: Hello! Yes, we could run route add / route del when we got any announce from external world with ExaBGP directly. I have implemented custom custom Firewall (netmap-ipfw) management tool which implement in similar manner. But I'm working with BGP flow spec. It's so complex, standard BGP is much times simpler. And I could share my ExaBGP configuration and hook scripts. ExaBGP config: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_firewall.conf Hook script which put all announces to Redis Queue: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_queue_writer.py But full BGP route table is enough big and need external processing. But yes, with some Python code is possible to implement route server with ExaBGP. On Wed, May 20, 2015 at 5:25 PM, Aled Morris al...@qix.co.uk wrote: On 20 May 2015 at 15:00, Pavel Odintsov pavel.odint...@gmail.com wrote: Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Aled -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
Add Alcatel-Lucent 7750? I have no experience but this list seems to love them. On Wed, May 20, 2015, 9:44 AM Colton Conor colton.co...@gmail.com wrote: So, from the sounds of it most are saying for low cost, the way to go would be a software router, which I was trying to avoid. To answer the bandwidth question, we would have three 10G ports with three different carriers and at max push 10Gbps of total traffic to start. I think this leaves me with hardware routers that can support full BGP tables. So, who actually sells full bgp routers. So far on my list I have: Juniper MX Series Brocade MLXe or CER Cisco ASR 9K Huawei NE40E-X1-M4 ZTE, not sure which model? ALU 7750 Besides the above, am I missing anyone else that makes a true carrier grade hardware router? On Wed, May 20, 2015 at 9:54 AM, Pavel Odintsov pavel.odint...@gmail.com wrote: Hello! Yes, we could run route add / route del when we got any announce from external world with ExaBGP directly. I have implemented custom custom Firewall (netmap-ipfw) management tool which implement in similar manner. But I'm working with BGP flow spec. It's so complex, standard BGP is much times simpler. And I could share my ExaBGP configuration and hook scripts. ExaBGP config: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_firewall.conf Hook script which put all announces to Redis Queue: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_queue_writer.py But full BGP route table is enough big and need external processing. But yes, with some Python code is possible to implement route server with ExaBGP. On Wed, May 20, 2015 at 5:25 PM, Aled Morris al...@qix.co.uk wrote: On 20 May 2015 at 15:00, Pavel Odintsov pavel.odint...@gmail.com wrote: Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Aled -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
Since you are considering multiple options, I'd build a decision matrix. You can put down all the requirements, score each option, and then normalize it to give each a final score. After that you can calculate some other things such as throughput per dollar, etc. http://asq.org/learn-about-quality/decision-making-tools/overview/decision-matrix.html Regarding the Mikrotik, there's a difference between Multithreading and Multiprocessing. On Wed, May 20, 2015 at 11:44 AM, Colton Conor colton.co...@gmail.com wrote: So are the rest of the processes in Mikrotik OS multi threaded? I would hope so to take advantage of 36 cores! What is up with all of these network vendors not supporting more than one core in their OS? I just don't get it. On Tue, May 19, 2015 at 9:49 PM, Josh Baird joshba...@gmail.com wrote: The BGP daemon on the CCR routers is not multi-threaded; it only will use one core. Josh On Tue, May 19, 2015 at 10:06 PM, Colton Conor colton.co...@gmail.com wrote: So this new $1295 Mikrotik CCR1036-8G-2S+EM has a 36 core Tilera CPU with 16GB of ram. Each core is running at 1.2Ghz? I assume that Mikrotik is multicore in software, so why does this box not outperform these intel boxes that everyone is recommending? Is it just a limitation of ports? On Tue, May 19, 2015 at 6:03 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: I've seen serious, unusual performance bottlenecks in Mikrotik CCR, in some cases not even achieving a gigabit speeds on 10G interfaces. Performance drops more rapidly then Cisco with smaller packet sizes. -mel beckman Folks often forget that Mikrotik ROS can also run on x86 machines. Size your favorite hardware (server) or network appliance with appropriate ports, add MT ROS on a CF card, and you are good to go. We use i7 based network appliance with dual 10g cards (you can use a quad 10g card, such as those made by hotlav). with a 2gig of ram, you can easily do multiple (4-5 or more full bgp peers), and i7 are good for approx 1.2mill pps. Best of luck. Faisal Imtiaz Snappy Internet Telecom
Re: Low Cost 10G Router
As mentioned by others on the list, a properly configured ASR1004 and up can do this. --Blake Colton Conor wrote on 5/20/2015 11:42 AM: So, from the sounds of it most are saying for low cost, the way to go would be a software router, which I was trying to avoid. To answer the bandwidth question, we would have three 10G ports with three different carriers and at max push 10Gbps of total traffic to start. I think this leaves me with hardware routers that can support full BGP tables. So, who actually sells full bgp routers. So far on my list I have: Juniper MX Series Brocade MLXe or CER Cisco ASR 9K Huawei NE40E-X1-M4 ZTE, not sure which model? ALU 7750 Besides the above, am I missing anyone else that makes a true carrier grade hardware router? On Wed, May 20, 2015 at 9:54 AM, Pavel Odintsov pavel.odint...@gmail.com wrote: Hello! Yes, we could run route add / route del when we got any announce from external world with ExaBGP directly. I have implemented custom custom Firewall (netmap-ipfw) management tool which implement in similar manner. But I'm working with BGP flow spec. It's so complex, standard BGP is much times simpler. And I could share my ExaBGP configuration and hook scripts. ExaBGP config: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_firewall.conf Hook script which put all announces to Redis Queue: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_queue_writer.py But full BGP route table is enough big and need external processing. But yes, with some Python code is possible to implement route server with ExaBGP. On Wed, May 20, 2015 at 5:25 PM, Aled Morris al...@qix.co.uk wrote: On 20 May 2015 at 15:00, Pavel Odintsov pavel.odint...@gmail.com wrote: Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Aled -- Sincerely yours, Pavel Odintsov
[no subject]
This post was from a subscriber whose From: address domain has a DMARC policy of reject or quarantine. The NANOG mailing list has automatically wrapped this message to prevent other subscribers mail systems from rejecting it.---BeginMessage--- It was resolved at around 2015-05-20 17:18 UTC Regards, Marty Strong -- CloudFlare - AS13335 Network Engineer ma...@cloudflare.com +44 20 3514 6970 UK (Office) +44 7584 906 055 UK (Mobile) +1 888 993 5273 US (Office) smartflare (Skype) http://www.peeringdb.com/view.php?asn=13335 On 20 May 2015, at 19:00, Mel Beckman m...@beckman.org wrote: There is a massive fiber cut in Santa Barbara affecting coastal paths for some carriers. That might be a factor. -mel beckman On May 20, 2015, at 7:42 AM, Tyler Applebaum appleba...@ochin.org wrote: Still seeing this as of 7:40AM PST. Looks isolated to ATT and Telia in Seattle. HOST: PC-002Loss% Snt LastAvg Best Wrst StDev 1.|-- 172.31.255.1 0.0% 10 00.803 0.9 2.|-- 10.98.0.4 0.0% 10 11.514 1.1 3.|-- 67.51.253.17 0.0% 10 62.826 1.2 4.|-- 67.51.253.10.0% 10 21.412 0.5 5.|-- 67.51.253.30.0% 10 21.312 0.5 6.|-- v202.core1.pdx1.he.net 0.0% 10 12.014 1.2 7.|-- 10ge12-4.core1.sea1.he.net 0.0% 10 9 10.99 13 1.0 8.|-- sea-b1-link.telia.net 50.0% 1042 42.0 42 42 0.0 9.|-- att-ic-153030-sea-b1.c.telia.net 50.0% 1046 44.8 43 46 1.3 10.|-- cr84.st0wa.ip.att.net 40.0% 1071 73.8 71 76 1.8 11.|-- cr2.st6wa.ip.att.net 40.0% 1074 73.7 72 75 1.2 12.|-- 12.122.158.14670.0% 1074 73.7 73 74 0.6 13.|-- 12.122.158.15750.0% 1071 71.0 71 71 0.0 14.|-- 12.248.207.6 20.0% 1071 71.0 71 71 0.0 15.|-- ancr-5-1-12-12.attalascom.net 30.0% 1071 71.0 71 71 0.0 16.|-- 66-2-12-12.attalascom.net 30.0% 1085 85.3 85 86 0.5 17.|-- KCHC-42-7-12-12.attalascom.net30.0% 1095 95.6 95 96 0.5 -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Tyler Applebaum Sent: Tuesday, May 19, 2015 4:20 PM To: nanog@nanog.org Subject: ATT/Telia issue Seeing this on AS7018 to AS1299. Anyone out there at either provider know anything about this? HOST: PC-002 Loss% Snt LastAvg Best Wrst StDev 1.|-- 172.31.255.1 0.0% 10 10.7030.9 2.|-- 10.98.0.30.0% 10 11.0110.0 3.|-- 67.51.253.17 0.0% 10 22.5240.7 4.|-- 67.51.253.3 0.0% 10 11.2120.4 5.|-- v202.core1.pdx1.he.net 0.0% 10 7 10.57 121.9 6.|-- 10ge12-4.core1.sea1.he.net 0.0% 10 55.0550.0 7.|-- sea-b1-link.telia.net0.0% 10 55.85 122.2 8.|-- den-b1-link.telia.net0.0% 10 108 107.3 106 1080.7 9.|-- sjo-b21-link.telia.net 20.0% 10 137 134.9 134 1371.0 10.|-- 192.205.33.45 40.0% 10 136 136.2 135 1381.2 11.|-- cr1.sffca.ip.att.net10.0% 10 141 141.9 139 1451.9 12.|-- 12.122.2.77 20.0% 10 140 140.1 137 1422.0 13.|-- 12.122.160.149 10.0% 10 138 141.1 137 1648.6 14.|-- 12.117.131.214 30.0% 10 139 141.0 139 1451.9 15.|-- 199.103.47.230.0% 1051 128.0 51 142 34.0 HOST: PC-002 Loss% Snt Last Avg Best Wrst StDev 1.|-- 172.31.255.1 0.0% 20 1 1.1030.6 2.|-- 10.98.0.40.0% 20 1 1.3140.7 3.|-- 67.51.253.17 0.0% 20 3 4.92 48 10.2 4.|-- 67.51.253.1 0.0% 20 2 1.1120.3 5.|-- 67.51.253.11 0.0% 20 1 1.4120.5 6.|-- v202.core1.pdx1.he.net 0.0% 20 6 9.11 123.2 7.|-- 10ge12-4.core1.sea1.he.net 0.0% 20 5 6.55 111.7 8.|-- sea-b1-link.telia.net0.0% 20 5 5.1560.3 9.|-- att-ic-153030-sea-b1.c.telia.net 0.0% 20 9 7.7691.2 10.|-- cr83.st0wa.ip.att.net
Re: Spamhaus BGP feed experiences?
At dnswl.org http://dnswl.org/ we check our data against the DROP list every once in a while. The overlap of DROP with legitimate sources of SMTP traffic is very, very small: a low single-digit number, and most of them are crappy to start with (so we don’t publish them, but only keep them in our database for reference purposes). — Matthias Am 19.05.2015 um 20:38 schrieb Max Tulyev max...@netassist.ua: How much false positives (i.e. blackholing traffic users want to reach)? On 18.05.15 21:04, Marco d'Itri wrote: On May 17, Mike Lyon mike.l...@gmail.com wrote: Any ISPs out there (big or small) ever used the Spamhaus BGP feed to prevent against botnet, spam, etc? If so, how has your experience been? Is it worthwhile? Has it helped? On / off list responses are appreciated in advance. We use Spamhaus DROP (not the BGP version: our software asks a human to review each change). The benefits are not obvious since we do not have access customers, but it will blackhole some networks you obviously do not want to talk to, and it has not caused any troubles either. smime.p7s Description: S/MIME cryptographic signature
Re: ATT/Telia issue
There is a massive fiber cut in Santa Barbara affecting coastal paths for some carriers. That might be a factor. -mel beckman On May 20, 2015, at 7:42 AM, Tyler Applebaum appleba...@ochin.org wrote: Still seeing this as of 7:40AM PST. Looks isolated to ATT and Telia in Seattle. HOST: PC-002Loss% Snt LastAvg Best Wrst StDev 1.|-- 172.31.255.1 0.0% 10 00.803 0.9 2.|-- 10.98.0.4 0.0% 10 11.514 1.1 3.|-- 67.51.253.17 0.0% 10 62.826 1.2 4.|-- 67.51.253.10.0% 10 21.412 0.5 5.|-- 67.51.253.30.0% 10 21.312 0.5 6.|-- v202.core1.pdx1.he.net 0.0% 10 12.014 1.2 7.|-- 10ge12-4.core1.sea1.he.net 0.0% 10 9 10.99 13 1.0 8.|-- sea-b1-link.telia.net 50.0% 1042 42.0 42 42 0.0 9.|-- att-ic-153030-sea-b1.c.telia.net 50.0% 1046 44.8 43 46 1.3 10.|-- cr84.st0wa.ip.att.net 40.0% 1071 73.8 71 76 1.8 11.|-- cr2.st6wa.ip.att.net 40.0% 1074 73.7 72 75 1.2 12.|-- 12.122.158.14670.0% 1074 73.7 73 74 0.6 13.|-- 12.122.158.15750.0% 1071 71.0 71 71 0.0 14.|-- 12.248.207.6 20.0% 1071 71.0 71 71 0.0 15.|-- ancr-5-1-12-12.attalascom.net 30.0% 1071 71.0 71 71 0.0 16.|-- 66-2-12-12.attalascom.net 30.0% 1085 85.3 85 86 0.5 17.|-- KCHC-42-7-12-12.attalascom.net30.0% 1095 95.6 95 96 0.5 -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Tyler Applebaum Sent: Tuesday, May 19, 2015 4:20 PM To: nanog@nanog.org Subject: ATT/Telia issue Seeing this on AS7018 to AS1299. Anyone out there at either provider know anything about this? HOST: PC-002 Loss% Snt LastAvg Best Wrst StDev 1.|-- 172.31.255.1 0.0% 10 10.7030.9 2.|-- 10.98.0.30.0% 10 11.0110.0 3.|-- 67.51.253.17 0.0% 10 22.5240.7 4.|-- 67.51.253.3 0.0% 10 11.2120.4 5.|-- v202.core1.pdx1.he.net 0.0% 10 7 10.57 121.9 6.|-- 10ge12-4.core1.sea1.he.net 0.0% 10 55.0550.0 7.|-- sea-b1-link.telia.net0.0% 10 55.85 122.2 8.|-- den-b1-link.telia.net0.0% 10 108 107.3 106 1080.7 9.|-- sjo-b21-link.telia.net 20.0% 10 137 134.9 134 1371.0 10.|-- 192.205.33.45 40.0% 10 136 136.2 135 1381.2 11.|-- cr1.sffca.ip.att.net10.0% 10 141 141.9 139 1451.9 12.|-- 12.122.2.77 20.0% 10 140 140.1 137 1422.0 13.|-- 12.122.160.149 10.0% 10 138 141.1 137 1648.6 14.|-- 12.117.131.214 30.0% 10 139 141.0 139 1451.9 15.|-- 199.103.47.230.0% 1051 128.0 51 142 34.0 HOST: PC-002 Loss% Snt LastAvg Best Wrst StDev 1.|-- 172.31.255.1 0.0% 20 1 1.1030.6 2.|-- 10.98.0.40.0% 20 1 1.3140.7 3.|-- 67.51.253.17 0.0% 20 3 4.92 48 10.2 4.|-- 67.51.253.1 0.0% 20 2 1.1120.3 5.|-- 67.51.253.11 0.0% 20 1 1.4120.5 6.|-- v202.core1.pdx1.he.net 0.0% 20 6 9.11 123.2 7.|-- 10ge12-4.core1.sea1.he.net 0.0% 20 5 6.55 111.7 8.|-- sea-b1-link.telia.net0.0% 20 5 5.1560.3 9.|-- att-ic-153030-sea-b1.c.telia.net 0.0% 20 9 7.7691.2 10.|-- cr83.st0wa.ip.att.net5.0% 20 118 119.7 117 1231.5 11.|-- cr2.ptdor.ip.att.net 0.0% 20 119 120.1 118 1221.4 12.|-- cr2.sffca.ip.att.net 0.0% 20 120 119.2 117 1211.4 13.|-- cr2.sc1ca.ip.att.net 0.0% 20 119 121.1 118 1496.6 14.|-- 12.122.151.129 0.0% 20 118 119.8 117 1221.5 15.|-- ???100.0% 20 0 0.0000.0 16.|-- 71.157.120.39 75.0% 20
Re: Low Cost 10G Router
Yep, thats what I meant be ALU 7750 :) On Wed, May 20, 2015 at 12:17 PM, Cody Grosskopf codygrossk...@gmail.com wrote: Add Alcatel-Lucent 7750? I have no experience but this list seems to love them. On Wed, May 20, 2015, 9:44 AM Colton Conor colton.co...@gmail.com wrote: So, from the sounds of it most are saying for low cost, the way to go would be a software router, which I was trying to avoid. To answer the bandwidth question, we would have three 10G ports with three different carriers and at max push 10Gbps of total traffic to start. I think this leaves me with hardware routers that can support full BGP tables. So, who actually sells full bgp routers. So far on my list I have: Juniper MX Series Brocade MLXe or CER Cisco ASR 9K Huawei NE40E-X1-M4 ZTE, not sure which model? ALU 7750 Besides the above, am I missing anyone else that makes a true carrier grade hardware router? On Wed, May 20, 2015 at 9:54 AM, Pavel Odintsov pavel.odint...@gmail.com wrote: Hello! Yes, we could run route add / route del when we got any announce from external world with ExaBGP directly. I have implemented custom custom Firewall (netmap-ipfw) management tool which implement in similar manner. But I'm working with BGP flow spec. It's so complex, standard BGP is much times simpler. And I could share my ExaBGP configuration and hook scripts. ExaBGP config: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_firewall.conf Hook script which put all announces to Redis Queue: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_queue_writer.py But full BGP route table is enough big and need external processing. But yes, with some Python code is possible to implement route server with ExaBGP. On Wed, May 20, 2015 at 5:25 PM, Aled Morris al...@qix.co.uk wrote: On 20 May 2015 at 15:00, Pavel Odintsov pavel.odint...@gmail.com wrote: Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Aled -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
Well, in my experience, which is limited to small iron mostly. Juniper MX104 Do not forget to get a second RE (Routine Engine) for software upgrade, and be prepare to accept to pay a license to use the 10Gbps ports on top of buying the IO cards. (1 license per 2 ports). Don't forget to set aside some times to port your configuration into it, if you are used to Cisco/Brocade style config. And that I'm too stupid to figure out a way to make 'test policy' do the same thing as show ip bgp route-map XYZ CER2K (latest revision) Has plenty of RAM for 6 full routing table (and maybe more) and 1.5M RIB compared to the ~524k from the first gen. ( Got burned on those ) MLX Juniper MX104 where cheaper for about the same platform using MLX products. Cisco I don't know about the licensing for the ASR but I mostly deal with second hand devices. They are not flashy but do the job. Huawei, ZTE I didn't touch those and mostly won't beside looking into some security concern some people are having. PS: With almost 130k prefixes polluting the routing table you could use a software route server and feed an auto-summary of the full route into a router/switch that can handle the RIB/FIB. I have yet to test Bird but I heard good things about using it for that function. ( By pollution, I mean, it was a test made on 6 peers where I found ~130k prefixes where using the same path as their larger subnet, I have to put up more time on that bench thou ) - Alain Hebertaheb...@pubnix.net PubNIX Inc. 50 boul. St-Charles P.O. Box 26770 Beaconsfield, Quebec H9W 6G7 Tel: 514-990-5911 http://www.pubnix.netFax: 514-990-9443 On 05/20/15 12:42, Colton Conor wrote: So, from the sounds of it most are saying for low cost, the way to go would be a software router, which I was trying to avoid. To answer the bandwidth question, we would have three 10G ports with three different carriers and at max push 10Gbps of total traffic to start. I think this leaves me with hardware routers that can support full BGP tables. So, who actually sells full bgp routers. So far on my list I have: Juniper MX Series Brocade MLXe or CER Cisco ASR 9K Huawei NE40E-X1-M4 ZTE, not sure which model? ALU 7750 Besides the above, am I missing anyone else that makes a true carrier grade hardware router? On Wed, May 20, 2015 at 9:54 AM, Pavel Odintsov pavel.odint...@gmail.com wrote: Hello! Yes, we could run route add / route del when we got any announce from external world with ExaBGP directly. I have implemented custom custom Firewall (netmap-ipfw) management tool which implement in similar manner. But I'm working with BGP flow spec. It's so complex, standard BGP is much times simpler. And I could share my ExaBGP configuration and hook scripts. ExaBGP config: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_firewall.conf Hook script which put all announces to Redis Queue: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_queue_writer.py But full BGP route table is enough big and need external processing. But yes, with some Python code is possible to implement route server with ExaBGP. On Wed, May 20, 2015 at 5:25 PM, Aled Morris al...@qix.co.uk wrote: On 20 May 2015 at 15:00, Pavel Odintsov pavel.odint...@gmail.com wrote: Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Aled -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
On Wed, May 20, 2015 at 2:07 PM, Mike Hammett na...@ics-il.net wrote: Well, the cores on a many-core CPU aren't going to have the torque that a Xeon would. They're also still working on the software. It has gotten a ton better over the life of the CCRs thus far. BGP is still atrocious on the CCRs, but that's because the route update process isn't multithreaded. It won't be multithreaded in the next major version either, but they will have done some programming voodoo (all programming is voodoo to me) to reign in the poor performance issues with full tables. https://youtu.be/ihZiAC-Rox8?t=37m8s I honestly don't know why most people gets impressed by the number of Tylera cores on CCR and think it's a good thing. Your torque point makes much sense to me. A few cores with decent clock and Xeon or Rangeley torque is just better. Adding that much weak tylera cores with low clock only results in much more context switching, much more CPU Affinity needs. Multithreading the relevant grained bit of code will also lead to more context switching, but for threads now instead of processes. As I understand the architecture of those solutions, I don't see why a bgp daemon mono threaded is a problem. Ok, multithreaded would give a better full routing convergence. But once the routing table is loaded it does not matter how many threads the bgp process will use. The dirty work on Linux (RouterOS kernel for that matter) will be done on the forward information table, on the packet forwarding code and specially on softirq (interrupt requests). This is where the bottleneck seems to be, IMHO. Linux is not good at multithreaded packet forwarding and not good specially at handling interrupt requests on multi-queue NICs. So, RouterOS is not good as well. Therefore that several dozens cheap and weak tylera cores powering CCR boxes is absolutely not friendly for Linux core and RouterOS itself. I'm better served off with a smaller amount of cores with better clock and better torque as Mr Hammett mentioned (I liked the expression usage yes) and that's why a Linux or a BSD box with a couple Xeon CPUs will perform better than CCR. Sometimes as someone mentioned a couple i7 cores will outperform a CCR box as well. More torque, yeah. Less context switching and time sharing wasted. However this horizontal scalar number of tylera cores on the CCR is good for marketing. After all you are buying a 36 CPU box paying a couple hundred bucks. Impressive, hum? Well not for me. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com - Original Message - From: Colton Conor colton.co...@gmail.com To: Faisal Imtiaz fai...@snappytelecom.net Cc: North American Network Operators Group nanog@nanog.org Sent: Tuesday, May 19, 2015 9:06:26 PM Subject: Re: Low Cost 10G Router So this new $1295 Mikrotik CCR1036-8G-2S+EM has a 36 core Tilera CPU with 16GB of ram. Each core is running at 1.2Ghz? I assume that Mikrotik is multicore in software, so why does this box not outperform these intel boxes that everyone is recommending? Is it just a limitation of ports? On Tue, May 19, 2015 at 6:03 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: I've seen serious, unusual performance bottlenecks in Mikrotik CCR, in some cases not even achieving a gigabit speeds on 10G interfaces. Performance drops more rapidly then Cisco with smaller packet sizes. -mel beckman Folks often forget that Mikrotik ROS can also run on x86 machines. Size your favorite hardware (server) or network appliance with appropriate ports, add MT ROS on a CF card, and you are good to go. We use i7 based network appliance with dual 10g cards (you can use a quad 10g card, such as those made by hotlav). with a 2gig of ram, you can easily do multiple (4-5 or more full bgp peers), and i7 are good for approx 1.2mill pps. Best of luck. Faisal Imtiaz Snappy Internet Telecom
Re: Low Cost 10G Router
On 5/19/15 1:22 PM, Colton Conor wrote: What options are available for a small, low cost router that has at least four 10G ports, and can handle full BGP routes? All that I know of are the Juniper MX80, and the Brocade CER line. What does Cisco and others have that compete with these two? Any other vendors besides Juniper, Brocade, and Cisco to look at? In the same price range as the MX80 there is the Alcatel SRa-4/8 router. These will do 100g in and out, and handle full tables. You get redundant control modules vs. a single on the juniper. BGP is multi-threaded on the box, does RPKI for route verification, and it's got extensive HQoS functionality amongst other features. -- Bryan Fields 727-409-1194 - Voice 727-214-2508 - Fax http://bryanfields.net
Re: Low Cost 10G Router
On Wed, May 20, 2015 at 1:42 PM, Colton Conor colton.co...@gmail.com wrote: So, from the sounds of it most are saying for low cost, the way to go would be a software router, which I was trying to avoid. To answer the bandwidth question, we would have three 10G ports with three different carriers and at max push 10Gbps of total traffic to start. I think this leaves me with hardware routers that can support full BGP tables. So, who actually sells full bgp routers. So far on my list I have: Juniper MX Series Brocade MLXe or CER Cisco ASR 9K Huawei NE40E-X1-M4 ZTE, not sure which model? ALU 7750 Besides the above, am I missing anyone else that makes a true carrier grade hardware router? right now I'm pushing 11G/s 1.2Mpps, ServerU L-800 + Chelsio T580-CR, see below although you can ssh in, it's definitely not a software router since it's essentially T5 ASICS hardware pushing the packets % sudo rate -i cxgbe0 -R -b = Currently 11.08 Gbps/1199.50 kpps, Average: 11.08 Gbps/1199.50 kpps = Currently 11.13 Gbps/1206.68 kpps, Average: 11.10 Gbps/1203.08 kpps = Currently 11.11 Gbps/1202.70 kpps, Average: 11.10 Gbps/1202.95 kpps = Currently 11.13 Gbps/1206.54 kpps, Average: 11.11 Gbps/1203.85 kpps = Currently 11.24 Gbps/1207.24 kpps, Average: 11.12 Gbps/1204.53 kpps = Currently 11.12 Gbps/1208.79 kpps, Average: 11.12 Gbps/1205.24 kpps = Currently 11.22 Gbps/1208.03 kpps, Average: 11.12 Gbps/1205.63 kpps = Currently 11.12 Gbps/1207.79 kpps, Average: 11.12 Gbps/1205.90 kpps = Currently 11.23 Gbps/1207.76 kpps, Average: 11.12 Gbps/1206.11 kpps = Currently 11.24 Gbps/1207.46 kpps, Average: 11.12 Gbps/1206.24 kpps = Currently 11.32 Gbps/1207.82 kpps, Average: 11.12 Gbps/1206.39 kpps = Currently 11.03 Gbps/1207.04 kpps, Average: 11.12 Gbps/1206.44 kpps btw this is a 40G QSFP SR4 port it's a thousand dollar card on top of a thousand dollar router + a penny for their x8 raiser card you won't find anything like that below 3k USD for your 10G routing low cost needs, I'm guessing
Re: Low Cost 10G Router
Well said Eddie, It would be worth pointing out that on CCR's each port also has a core dedicated to it, a benefit of such a design is that each port is able to handle a much higher PPS rate, and if there is a DDOS attack on one port, it will not bring down the rest of the ports / router etc. (disclaimer, if the router is setup properly, without all traffic going thru the CPU etc etc). Faisal Imtiaz Snappy Internet Telecom - Original Message - From: Eddie Tardist edtard...@gmail.com To: North American Network Operators Group nanog@nanog.org Sent: Wednesday, May 20, 2015 6:34:11 PM Subject: Re: Low Cost 10G Router On Wed, May 20, 2015 at 2:07 PM, Mike Hammett na...@ics-il.net wrote: Well, the cores on a many-core CPU aren't going to have the torque that a Xeon would. They're also still working on the software. It has gotten a ton better over the life of the CCRs thus far. BGP is still atrocious on the CCRs, but that's because the route update process isn't multithreaded. It won't be multithreaded in the next major version either, but they will have done some programming voodoo (all programming is voodoo to me) to reign in the poor performance issues with full tables. https://youtu.be/ihZiAC-Rox8?t=37m8s I honestly don't know why most people gets impressed by the number of Tylera cores on CCR and think it's a good thing. Your torque point makes much sense to me. A few cores with decent clock and Xeon or Rangeley torque is just better. Adding that much weak tylera cores with low clock only results in much more context switching, much more CPU Affinity needs. Multithreading the relevant grained bit of code will also lead to more context switching, but for threads now instead of processes. As I understand the architecture of those solutions, I don't see why a bgp daemon mono threaded is a problem. Ok, multithreaded would give a better full routing convergence. But once the routing table is loaded it does not matter how many threads the bgp process will use. The dirty work on Linux (RouterOS kernel for that matter) will be done on the forward information table, on the packet forwarding code and specially on softirq (interrupt requests). This is where the bottleneck seems to be, IMHO. Linux is not good at multithreaded packet forwarding and not good specially at handling interrupt requests on multi-queue NICs. So, RouterOS is not good as well. Therefore that several dozens cheap and weak tylera cores powering CCR boxes is absolutely not friendly for Linux core and RouterOS itself. I'm better served off with a smaller amount of cores with better clock and better torque as Mr Hammett mentioned (I liked the expression usage yes) and that's why a Linux or a BSD box with a couple Xeon CPUs will perform better than CCR. Sometimes as someone mentioned a couple i7 cores will outperform a CCR box as well. More torque, yeah. Less context switching and time sharing wasted. However this horizontal scalar number of tylera cores on the CCR is good for marketing. After all you are buying a 36 CPU box paying a couple hundred bucks. Impressive, hum? Well not for me. - Mike Hammett Intelligent Computing Solutions http://www.ics-il.com Midwest Internet Exchange http://www.midwest-ix.com - Original Message - From: Colton Conor colton.co...@gmail.com To: Faisal Imtiaz fai...@snappytelecom.net Cc: North American Network Operators Group nanog@nanog.org Sent: Tuesday, May 19, 2015 9:06:26 PM Subject: Re: Low Cost 10G Router So this new $1295 Mikrotik CCR1036-8G-2S+EM has a 36 core Tilera CPU with 16GB of ram. Each core is running at 1.2Ghz? I assume that Mikrotik is multicore in software, so why does this box not outperform these intel boxes that everyone is recommending? Is it just a limitation of ports? On Tue, May 19, 2015 at 6:03 PM, Faisal Imtiaz fai...@snappytelecom.net wrote: I've seen serious, unusual performance bottlenecks in Mikrotik CCR, in some cases not even achieving a gigabit speeds on 10G interfaces. Performance drops more rapidly then Cisco with smaller packet sizes. -mel beckman Folks often forget that Mikrotik ROS can also run on x86 machines. Size your favorite hardware (server) or network appliance with appropriate ports, add MT ROS on a CF card, and you are good to go. We use i7 based network appliance with dual 10g cards (you can use a quad 10g card, such as those made by hotlav). with a 2gig of ram, you can easily do multiple (4-5 or more full bgp peers), and i7 are good for approx 1.2mill pps. Best of luck. Faisal Imtiaz Snappy Internet Telecom
Re: Low Cost 10G Router
2015-05-20 20:54 GMT-03:00 BPNoC Group bpnoc.li...@gmail.com: right now I'm pushing 11G/s 1.2Mpps, ServerU L-800 + Chelsio T580-CR, see below although you can ssh in, it's definitely not a software router since it's essentially T5 ASICS hardware pushing the packets % sudo rate -i cxgbe0 -R -b = Currently 11.08 Gbps/1199.50 kpps, Average: 11.08 Gbps/1199.50 kpps = Currently 11.13 Gbps/1206.68 kpps, Average: 11.10 Gbps/1203.08 kpps = Currently 11.11 Gbps/1202.70 kpps, Average: 11.10 Gbps/1202.95 kpps = Currently 11.13 Gbps/1206.54 kpps, Average: 11.11 Gbps/1203.85 kpps = Currently 11.24 Gbps/1207.24 kpps, Average: 11.12 Gbps/1204.53 kpps = Currently 11.12 Gbps/1208.79 kpps, Average: 11.12 Gbps/1205.24 kpps = Currently 11.22 Gbps/1208.03 kpps, Average: 11.12 Gbps/1205.63 kpps = Currently 11.12 Gbps/1207.79 kpps, Average: 11.12 Gbps/1205.90 kpps = Currently 11.23 Gbps/1207.76 kpps, Average: 11.12 Gbps/1206.11 kpps = Currently 11.24 Gbps/1207.46 kpps, Average: 11.12 Gbps/1206.24 kpps = Currently 11.32 Gbps/1207.82 kpps, Average: 11.12 Gbps/1206.39 kpps = Currently 11.03 Gbps/1207.04 kpps, Average: 11.12 Gbps/1206.44 kpps How much routes in the FIB? Thanks. -- Eduardo Schoedler
Re: Low Cost 10G Router
Bryan, Very interesting. Doesn't ALU mainly compare the new Alcatel SRa-4/8 router vs a MX104 though? Besides no redundancy, what limitations does the MX80 and MX104 have? I am assume the Juniper does not have BGP is multi-threaded on the box, does RPKI for route verification, and it's got extensive HQoS functionality? I heard the MX80 was limited on QoS, but never looked into it. On Wed, May 20, 2015 at 7:03 PM, Bryan Fields br...@bryanfields.net wrote: On 5/19/15 1:22 PM, Colton Conor wrote: What options are available for a small, low cost router that has at least four 10G ports, and can handle full BGP routes? All that I know of are the Juniper MX80, and the Brocade CER line. What does Cisco and others have that compete with these two? Any other vendors besides Juniper, Brocade, and Cisco to look at? In the same price range as the MX80 there is the Alcatel SRa-4/8 router. These will do 100g in and out, and handle full tables. You get redundant control modules vs. a single on the juniper. BGP is multi-threaded on the box, does RPKI for route verification, and it's got extensive HQoS functionality amongst other features. -- Bryan Fields 727-409-1194 - Voice 727-214-2508 - Fax http://bryanfields.net
Re: Low Cost 10G Router
On Wed, May 20, 2015 at 9:16 PM, Eduardo Schoedler lis...@esds.com.br wrote: 2015-05-20 20:54 GMT-03:00 BPNoC Group bpnoc.li...@gmail.com: right now I'm pushing 11G/s 1.2Mpps, ServerU L-800 + Chelsio T580-CR, see below although you can ssh in, it's definitely not a software router since it's essentially T5 ASICS hardware pushing the packets % sudo rate -i cxgbe0 -R -b = Currently 11.08 Gbps/1199.50 kpps, Average: 11.08 Gbps/1199.50 kpps = Currently 11.13 Gbps/1206.68 kpps, Average: 11.10 Gbps/1203.08 kpps = Currently 11.11 Gbps/1202.70 kpps, Average: 11.10 Gbps/1202.95 kpps = Currently 11.13 Gbps/1206.54 kpps, Average: 11.11 Gbps/1203.85 kpps = Currently 11.24 Gbps/1207.24 kpps, Average: 11.12 Gbps/1204.53 kpps = Currently 11.12 Gbps/1208.79 kpps, Average: 11.12 Gbps/1205.24 kpps = Currently 11.22 Gbps/1208.03 kpps, Average: 11.12 Gbps/1205.63 kpps = Currently 11.12 Gbps/1207.79 kpps, Average: 11.12 Gbps/1205.90 kpps = Currently 11.23 Gbps/1207.76 kpps, Average: 11.12 Gbps/1206.11 kpps = Currently 11.24 Gbps/1207.46 kpps, Average: 11.12 Gbps/1206.24 kpps = Currently 11.32 Gbps/1207.82 kpps, Average: 11.12 Gbps/1206.39 kpps = Currently 11.03 Gbps/1207.04 kpps, Average: 11.12 Gbps/1206.44 kpps How much routes in the FIB? Thanks. actually it makes no difference, the relevant route entries are stored in the T5 chip cxgbetool tells me I have 532447 entries right now for fib 0 anyway, I have a similar number of entries (a couple more due to pinned ipv6 not triggered to the card), but other than management port for ssh, snmp, webgui and netflow, only 180kpps for a trunked copper dmz segment is actually forwarded at fib. everything else is done on the card -- Eduardo Schoedler
Re: Low Cost 10G Router
On 20/05/2015 15:25, Aled Morris wrote: Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Yes, you could probably do this. No, you probably wouldn't want to do this. Pls see the netlink interface modules in bird and quagga to understand why. Nick
Re: Low Cost 10G Router
Hello! Yes, we could run route add / route del when we got any announce from external world with ExaBGP directly. I have implemented custom custom Firewall (netmap-ipfw) management tool which implement in similar manner. But I'm working with BGP flow spec. It's so complex, standard BGP is much times simpler. And I could share my ExaBGP configuration and hook scripts. ExaBGP config: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_firewall.conf Hook script which put all announces to Redis Queue: https://github.com/FastVPSEestiOu/fastnetmon/blob/master/src/scripts/exabgp_queue_writer.py But full BGP route table is enough big and need external processing. But yes, with some Python code is possible to implement route server with ExaBGP. On Wed, May 20, 2015 at 5:25 PM, Aled Morris al...@qix.co.uk wrote: On 20 May 2015 at 15:00, Pavel Odintsov pavel.odint...@gmail.com wrote: Yes, you could do filtering with Quagga. But Quagga is pretty old tool without multiple dynamic features. But with ExaBGP you could do really any significant route table transformations with Python in few lines of code. But it's definitely add additional point of failure/bug. Couldn't your back-end scripts running under ExaBGP also manage the FIB, using standard Unix tools/APIs? Managing the FIB is basically just route add and route delete right? Aled -- Sincerely yours, Pavel Odintsov
Re: Low Cost 10G Router
On 2015-05-20 08:17, Pavel Odintsov wrote: Hello! Ray, I could suggest switch from multi physical CPU configuration to single. Like Intel Xeon E5-1650/1660/1680 or even Xeon E3 platforms. Because multi processor systems need really huge amount of knowledge for NUMA configuration and PCI-E devices assignment for each NUMA. Not really. Well that's opinion I suppose. It didn't seem like that steep of a learning curve. Just need to play with taskset and do some reading. If you are just starting out and experimenting, then sure a single CPU system would probably be the way to go. Secondly, I could vote many times for Supermicro! :) Dell or HP are really ugly systems for soft routers. CPU frequency tuning, PCM debugging are real nightmare on this systems. And why is that any different on a supermicro system? Isn't it all the same hardware? I personally would recommend buying from Dell or HP, as they things like 4hr turn around times (at least in the major urban centers, usually it's about an hour). I don't know how good Supermicro purchase/procurement system is. Dell has some neat things for asset management, support etc. HP probably has the same. Please beware of they! Supermicro is very clear and do not block useful functions of platform. What don't they block? What vendors block things, and what things do they block?
RE: ATT/Telia issue
Still seeing this as of 7:40AM PST. Looks isolated to ATT and Telia in Seattle. HOST: PC-002Loss% Snt LastAvg Best Wrst StDev 1.|-- 172.31.255.1 0.0% 10 00.803 0.9 2.|-- 10.98.0.4 0.0% 10 11.514 1.1 3.|-- 67.51.253.17 0.0% 10 62.826 1.2 4.|-- 67.51.253.10.0% 10 21.412 0.5 5.|-- 67.51.253.30.0% 10 21.312 0.5 6.|-- v202.core1.pdx1.he.net 0.0% 10 12.014 1.2 7.|-- 10ge12-4.core1.sea1.he.net 0.0% 10 9 10.99 13 1.0 8.|-- sea-b1-link.telia.net 50.0% 1042 42.0 42 42 0.0 9.|-- att-ic-153030-sea-b1.c.telia.net 50.0% 1046 44.8 43 46 1.3 10.|-- cr84.st0wa.ip.att.net 40.0% 1071 73.8 71 76 1.8 11.|-- cr2.st6wa.ip.att.net 40.0% 1074 73.7 72 75 1.2 12.|-- 12.122.158.14670.0% 1074 73.7 73 74 0.6 13.|-- 12.122.158.15750.0% 1071 71.0 71 71 0.0 14.|-- 12.248.207.6 20.0% 1071 71.0 71 71 0.0 15.|-- ancr-5-1-12-12.attalascom.net 30.0% 1071 71.0 71 71 0.0 16.|-- 66-2-12-12.attalascom.net 30.0% 1085 85.3 85 86 0.5 17.|-- KCHC-42-7-12-12.attalascom.net30.0% 1095 95.6 95 96 0.5 -Original Message- From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Tyler Applebaum Sent: Tuesday, May 19, 2015 4:20 PM To: nanog@nanog.org Subject: ATT/Telia issue Seeing this on AS7018 to AS1299. Anyone out there at either provider know anything about this? HOST: PC-002 Loss% Snt LastAvg Best Wrst StDev 1.|-- 172.31.255.1 0.0% 10 10.7030.9 2.|-- 10.98.0.30.0% 10 11.0110.0 3.|-- 67.51.253.17 0.0% 10 22.5240.7 4.|-- 67.51.253.3 0.0% 10 11.2120.4 5.|-- v202.core1.pdx1.he.net 0.0% 10 7 10.57 121.9 6.|-- 10ge12-4.core1.sea1.he.net 0.0% 10 55.0550.0 7.|-- sea-b1-link.telia.net0.0% 10 55.85 122.2 8.|-- den-b1-link.telia.net0.0% 10 108 107.3 106 1080.7 9.|-- sjo-b21-link.telia.net 20.0% 10 137 134.9 134 1371.0 10.|-- 192.205.33.45 40.0% 10 136 136.2 135 1381.2 11.|-- cr1.sffca.ip.att.net10.0% 10 141 141.9 139 1451.9 12.|-- 12.122.2.77 20.0% 10 140 140.1 137 1422.0 13.|-- 12.122.160.149 10.0% 10 138 141.1 137 1648.6 14.|-- 12.117.131.214 30.0% 10 139 141.0 139 1451.9 15.|-- 199.103.47.230.0% 1051 128.0 51 142 34.0 HOST: PC-002 Loss% Snt LastAvg Best Wrst StDev 1.|-- 172.31.255.1 0.0% 20 11.1 030.6 2.|-- 10.98.0.40.0% 20 11.3 140.7 3.|-- 67.51.253.17 0.0% 20 34.9 2 48 10.2 4.|-- 67.51.253.1 0.0% 20 21.1 120.3 5.|-- 67.51.253.11 0.0% 20 11.4 120.5 6.|-- v202.core1.pdx1.he.net 0.0% 20 69.1 1 123.2 7.|-- 10ge12-4.core1.sea1.he.net 0.0% 20 56.5 5 111.7 8.|-- sea-b1-link.telia.net0.0% 20 55.1 560.3 9.|-- att-ic-153030-sea-b1.c.telia.net 0.0% 20 97.7 691.2 10.|-- cr83.st0wa.ip.att.net5.0% 20 118 119.7 117 1231.5 11.|-- cr2.ptdor.ip.att.net 0.0% 20 119 120.1 118 1221.4 12.|-- cr2.sffca.ip.att.net 0.0% 20 120 119.2 117 1211.4 13.|-- cr2.sc1ca.ip.att.net 0.0% 20 119 121.1 118 1496.6 14.|-- 12.122.151.129 0.0% 20 118 119.8 117 1221.5 15.|-- ???100.0% 20 00.0 000.0 16.|-- 71.157.120.39 75.0% 20 119 118.6 118 1190.5 17.|-- 108-248-29-59.lightspeed.renonv.sbcglobal.net5.0% 20 139 137.1 135 1462.5 18.|-- 108-241-228-42.lightspeed.renonv.sbcglobal.net 5.0% 20 143 139.2 135 1524.9 Attention: Information contained in this message
