Re: Calling Crown Castle Fibre sales

[Jason Lixfeld]

Thanks for all the replies.  I’m in contact with them now, thanks to the 
support here.

> On Apr 9, 2020, at 11:01 AM, Jason Lixfeld  wrote:
> 
> Can someone from Crown Castle Fibre sales ping me?  I haven’t heard back 
> after submitting on your contact form.
> 
> Thanks.

Calling Crown Castle Fibre sales

[Jason Lixfeld]

Can someone from Crown Castle Fibre sales ping me?  I haven’t heard back after 
submitting on your contact form.

Thanks.

WTR: 1-2RU @ Equinix Ashburn

[Jason Lixfeld]

Hi,

I’m wondering if anyone is looking to subsidize their Equinix Ashburn colo 
costs by way of carving out 1-2 RU to a friendly for a low density networking 
application.  If so, I’d love to hear from you!

Thanks in advance!

Re: SFP oraganizers / storage recommendations

[Jason Lixfeld]

We have ones from FS with our own logo on them and don’t pay any more (or 
marginally so), so I’m sure it’d trivial for FS to make a label that included a 
bar code for the PN or whatever you wanted, really.

> On Oct 30, 2019, at 10:10 AM, Luke Guillory  wrote:
> 
> Barcodes on FS.com is the serial, so you'd need to receive them in or enter 
> them with PN and SN.
> 
> 
> 
> Ns
> 
> 
> 
> 
> 
> 
> -Original Message-
> From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Jason Lixfeld
> Sent: Wednesday, October 30, 2019 9:03 AM
> To: Warren Kumari
> Cc: NANOG mailing list
> Subject: Re: SFP oraganizers / storage recommendations
> 
> I’m wondering if the barcodes on the SFPs would let you simplify things a bit 
> more vs. updating a spreadsheet.  IE:  Some sort of barcode scanner app for 
> your phone that could automagically add/remove from some sort of document or 
> database?
> 
>> On Oct 30, 2019, at 9:53 AM, Warren Kumari  wrote:
>> 
>> If you buy your SFPs from fs.com, they come in a nice organizer -- and
>> if you buy less than a tray full, you still get a tray.
>> I keep spares in the trays, labeled on the outside -- I then put the
>> trays in a cheap toolbox / fishing tackle box, and list what's in each
>> one in a Google spreadsheet.
>> 
>> Whenever I'm actually at the cage / rack and have a few minutes I
>> compare the spreadsheet to reality, and update accordingly (SFPs, and
>> XFPs in particular evaporate over time...)
>> 
>> W
>> 
>> On Wed, Oct 30, 2019 at 9:36 AM Matthew Huff  wrote:
>>> 
>>> Any recommendations to keep track of different SFP and keep them organized? 
>>> Any storage boxes / trays designed for SFPs?
>> 
>> 
>> 
>> --
>> I don't think the execution is relevant when it was obviously a bad
>> idea in the first place.
>> This is like putting rabid weasels in your pants, and later expressing
>> regret at having chosen those particular rabid weasels and that pair
>> of pants.
>>  ---maf
> 
> 

Re: SFP oraganizers / storage recommendations

[Jason Lixfeld]

I’m wondering if the barcodes on the SFPs would let you simplify things a bit 
more vs. updating a spreadsheet.  IE:  Some sort of barcode scanner app for 
your phone that could automagically add/remove from some sort of document or 
database?

> On Oct 30, 2019, at 9:53 AM, Warren Kumari  wrote:
> 
> If you buy your SFPs from fs.com, they come in a nice organizer -- and
> if you buy less than a tray full, you still get a tray.
> I keep spares in the trays, labeled on the outside -- I then put the
> trays in a cheap toolbox / fishing tackle box, and list what's in each
> one in a Google spreadsheet.
> 
> Whenever I'm actually at the cage / rack and have a few minutes I
> compare the spreadsheet to reality, and update accordingly (SFPs, and
> XFPs in particular evaporate over time...)
> 
> W
> 
> On Wed, Oct 30, 2019 at 9:36 AM Matthew Huff  wrote:
>> 
>> Any recommendations to keep track of different SFP and keep them organized? 
>> Any storage boxes / trays designed for SFPs?
> 
> 
> 
> -- 
> I don't think the execution is relevant when it was obviously a bad
> idea in the first place.
> This is like putting rabid weasels in your pants, and later expressing
> regret at having chosen those particular rabid weasels and that pair
> of pants.
>   ---maf

Re: sfps from fs dot com

[Jason Lixfeld]

> On Sep 20, 2019, at 8:54 AM, Bryan Holloway  wrote:
> 
> In my travels we see a high failure rate -- higher than I'd like to see --, 
> but $boss likes the price, and, as Jason pointed out below, for the price, it 
> can be a "successful" business model.

> For someone who doesn't want to deal with outages, call-volume, hands 
> tickets, truck-rolls, and has money to burn, there are better alternatives.

Indeed.  My tune would be very different if we had 1:50 fail in the field.

Re: sfps from fs dot com

[Jason Lixfeld]

We have maybe 1:50 DOA, but they’re so cheap, we just throw them out because 
it’s not worth the RMA.

That said, I can’t remember the last time we’ve had any of these fail in the 
field, or have had any issues with variablity in TX/RX power.  We have tens of 
thousands of these in the field, from 100Mb Bidi to 100G-LR and everything in 
between, including *WDM up to 80KM.

> On Sep 20, 2019, at 8:31 AM, Nicholas Warren  
> wrote:
> 
> Anyone have experience with fs.com's lasers? Are they reliable?

Real-world MPLS P/LSR experience on BCM T3 (X5/X7) vs T2+

[Jason Lixfeld]

Hey all,

In the role of an MPLS P/LSR, I’m curious if there have been any gotchas (or 
fixes) revealed with BCM T3 vs. T2+.  I remember reading somewhere some years 
ago that there were oddities on the T2+ that I’d like to believe have been 
addressed on T3, but does anyone have any real-world experience with T3 in an 
MPLS core?  (IS-IS, LDP, rLFA, 2-3 labels wide, likely 
SR/Seamless/BGP-LU/whatever down the road)

I’m sure J, C, A, etc. all have their own challenges wrapping their code around 
the APIs, so would be curious to hear anything anyone has to share along those 
lines as well.

Thanks in advance.

BGP person from Bell Canada/AS577

[Jason Lixfeld]

Hello,

I’m looking to make contact with someone at Bell Canada/AS577 who is able to 
perform BGP prefix filtering facing their on-prem Akamai caches.  Normal sales 
rep and NOC channels are not producing any meaningful results so far.

Thanks in advance!

Re: Flexible OTN / fractional 100GbE

[Jason Lixfeld]

> On May 28, 2019, at 6:41 AM, Jérôme Nicolle  wrote:
> 
> Hi NaNOG !
> 
> I'm looking for a muxponder that would take OTU4s on the network side
> and provide 10/40/100GbE on the client side, with some kind of
> oversubscription, as to provide a "fractional 100GbE" e.g. starting with
> 30-60Gbps commit that could be upgraded to 100GbE when network capacity
> is available.
> 
> Is that something feasible at a decent price ?
> 
> I've read that Broadcom' StrataDNX (Qumran / Jericho) chips have OTN
> support in addition to ethernet now, is there some vendor who leverages
> this, preferably with OCP gear ?

Hi,

IP Infusion’s OcNOS is geared towards OCP gear, and while it’s not exactly what 
you’re looking for, they recently published[1] a note pertaining to IPoDWDM, so 
I could see them having maybe already done something along the lines of what 
you’re asking about, or they may have plans to.

[1] 
https://www.ipinfusion.com/news-events/ip-infusion-qualifies-inphi-colorz-in-its-latest-release-of-the-ocnos-network-operating-system/

Re: Free Program to take netflow

[Jason Lixfeld]

I loved using ElastiFlow, but we didn’t quite work out in the end.  Here’s my 
$0.02 -

- ElastiFlow setup is easy-ish.
- ELK setup is easy-ish.
- Scaling ELK is not easy unless you know what you’re doing.

If you’ve got enough flows that you need to scale ELK, you’re probably also 
using multiple flow exporters, at which point this[1] could bite you and if ELK 
scaling was hard for you, dealing with this might not be trivial until Rob 
decides how best to bake a fix into EF.

I learned ELK because I wanted to use EF, but I only learned enough about ELK 
to get me by.  Having to also learn about REDIS and having to learn more about 
ELK to make it work with REDIS and EF was a show stopper; I just didn’t have 
the time. 

[1] https://github.com/robcowart/elastiflow/issues/205

> On May 18, 2019, at 12:19 AM, Crist Clark  wrote:
> 
> Been loving Elastiflow. Way overkill for what you need, but it's
> actually pretty easy to setup.
> 
> https://github.com/robcowart/elastiflow
> 
> 
> On Fri, May 17, 2019 at 7:25 AM Dennis Burgess via NANOG
>  wrote:
>> 
>> I am looking for a free program to take netflow and output what the top 
>> traffic ASes to and from my AS are.   Something that we can look at every 
>> once in a while, and/or spin up and get data then shutdown..  Just have two 
>> ports need netflow from currently.
>> 
>> 
>> 
>> Thanks in advance.
>> 
>> 
>> 
>> 
>> 
>> Dennis Burgess, Mikrotik Certified Trainer
>> 
>> Author of "Learn RouterOS- Second Edition”
>> 
>> Link Technologies, Inc -- Mikrotik & WISP Support Services
>> 
>> Office: 314-735-0270  Website: http://www.linktechs.net
>> 
>> Create Wireless Coverage’s with www.towercoverage.com
>> 
>> 

Optical routes from MI-OH regionals

[Jason Lixfeld]

Hi,

Looking for someone who might have routes (lit or dark) from Detroit, MI to 
Columbus, OH preferably using a straight’ish shot from Toledo to Columbus.  
Most routes I’ve seen from the larger providers tend to run Toledo - Lima - 
Columbus or Toledo - Cleveland - Columbus, so I’m hoping a smaller regional 
player may have something more direct.

Thanks in advance!

Re: Free Open Source Network Operating Systems

[Jason Lixfeld]

I could be making this up, but my understanding is that the Broadcom SDK is not 
free, and without the SDK, hardware interaction is limited.

At one time ONL was a free ONIE NOS but sans SDK.

https://github.com/opencomputeproject/OpenNetworkLinux 
 ?

Sent from my iPhone

On Mar 9, 2019, at 11:08 AM, Colton Conor mailto:colton.co...@gmail.com>> wrote:

> What free, opensouce, network operating systems currently exist that run on 
> whitebox broadcom or other merchant silicon switches?
> 
> I know Cumulus is very popular, but I don't believe they have a free version 
> that runs on whitebox switches right? Only on a virtual machine from what I 
> can tell. 
> 
> I think if one of these vendors would release a free and truly opensource 
> network operating system, with the option for paid support if needed, then 
> whitebox switching would really take off. This would be similar to the Redhat 
> model, but for the networking world. 
> 
> Right now, the cost of the whitebox plus a paid network operating system 
> seems to equal the same cost as a discounted Juniper, Cisco, or Arista. I am 
> not seeing the savings on paper. 
> 
> If we could just buy the whitebox hardware, and have a free operating system 
> on there, then financially whitebox switches would be half the cost of a 
> similar Cisco switch after discount.
> 
> Am I missing something?
> 
> 

Re: BGP topological vs centralized route reflector

[Jason Lixfeld]

Hi Adam,

> On Feb 19, 2019, at 10:28 AM,  
>  wrote:
> 
> -Type-1 RDs will help you simulate full-mesh.  

By “Type-1 RD”, are you referring to a unique RD per PE?

Calling LinkedIn, Amazon and Akamai @ DE-CIX NY

[Jason Lixfeld]

Hi,

In late October 2018, DE-CIX announced that they would be renumbering their 
IPv4 address block in New York between 01-28-19 and 01-30-19.

This was followed by numerous reminders in months, weeks and even days leading 
up to the renumbering activity.

The renumbering activity has come and gone, but LinkedIn, Amazon and Akamai are 
still using the old IPs.

If three months has gone by and the numerous reminders that have been sent have 
resulted in these organizations still living on the old IP space, it seems to 
me that there may be some sort of a disconnect between who receives the 
notifications from IXPs and how they are filtered upstream.

I’m hopeful that the eyeballs who read this list are some of those folks who 
should have received the notifications from DE-CIX, or can at least filter the 
info back downstream to whoever can perform the renumbering activity.

Thanks.

Re: Amazon Peering

[Jason Lixfeld]

We circled back with them yesterday on a request we made in late November where 
at the time they said they wouldn’t be turned up until 2019 due to holiday 
network change freeze.

They responded within about 4 hours, thanked us for our patience and 
understanding and said we should expect them to be turned up in about 6 weeks, 
which is apparently their typical timing.

> On Jan 24, 2019, at 2:13 PM, Tom Beecher  wrote:
> 
> I hate to necro-thread , but has anyone seen any movement from Amazon on 
> this? I just got a Strongly Worded Message about it, and according to my 
> peering team , it's been radio silence for months. 
> 
> 
> On Sat, Nov 24, 2018 at 12:32 PM JASON BOTHE via NANOG  > wrote:
> This is a note I received on Oct18 when checking on a peering request 
> submitted on Aug7.. 
> 
> “Apologies for the delays here. We have temporarily frozen IX peering as we 
> revise some of our automation processes. I’m hopeful this will be unblocked 
> by early November. Thank you for your continued patience.”
> 
> On Nov 24, 2018, at 10:59, Darin Steffl  > wrote:
> 
>> It seems wasteful for Amazon to connect to an IX but then ignore peering 
>> requests for a year.
>> 
>> They have 40G of connectivity but are unresponsive. I'll try emailing all 
>> the other contacts listed in peeringdb.
>> 
>> Thanks 
>> 
>> On Sat, Nov 24, 2018, 10:38 AM Mike Hammett >  wrote:
>> I've e-mailed my contacts there a couple times on people's behalf. No 
>> response yet.
>> 
>> It seems like a lot of organizations need 1 more person in their peering 
>> departments.
>> 
>> 
>> 
>> -
>> Mike Hammett
>> Intelligent Computing Solutions
>> http://www.ics-il.com 
>> 
>> Midwest-IX
>> http://www.midwest-ix.com 
>> 
>> From: "Darin Steffl" > >
>> To: "North American Network Operators' Group" > >
>> Sent: Friday, November 23, 2018 10:21:51 PM
>> Subject: Amazon Peering
>> 
>> Hey all,
>> 
>> Does anyone have a direct contact to get a peering session established with 
>> Amazon at an IX? I sent a peering request Dec 2017 and two more times this 
>> Sept and Nov with no response.
>> 
>> I sent to peer...@amazon.com  and received one 
>> automated response back so I know they received my email but nothing since.
>> 
>> 
>> 
>> -- 
>> Darin Steffl
>> Minnesota WiFi
>> www.mnwifi.com 
>> 507-634-WiFi
>>   Like us on Facebook 
>> 

Re: Waves between Buffalo and Manhattan

[Jason Lixfeld]

Hi Mehmet,

Indeed Windstream has dark on an appropriate route, and we’re talking to them 
about that.  However they don’t seem to have lit services on that route.

> On Jan 18, 2019, at 12:59 PM, Mehmet Akcin  wrote:
> 
> hi Jason
> 
> https://dev.networkatlas.org <https://dev.networkatlas.org/> shows Zayo, 
> Windstream (and Earthlink) there. 
> 
> We are working with Charter , Level3/CL to load their fiber routes , but it 
> will be there and you will be able to connect with the sales teams directly 
> by clicking on a route. . In addition to that there are other small players 
> in this region.
> 
> Mehmet
> 
> On Fri, Jan 18, 2019 at 9:49 AM Jason Lixfeld  <mailto:jason%2bna...@lixfeld.ca>> wrote:
> Hello,
> 
> Does anyone have knowledge of carriers who are able to deliver 10G or 100G 
> waves between Buffalo and Manhattan that *do not* touch Albany?
> 
> I know Zayo is one.  I’m waiting to hear back from CenturyLink.  I’ve 
> reviewed networkatlas.org <http://networkatlas.org/>, to see if anything pops 
> up there, but otherwise I’m coming up empty.
> 
> Thanks in advance.

Waves between Buffalo and Manhattan

[Jason Lixfeld]

Hello,

Does anyone have knowledge of carriers who are able to deliver 10G or 100G 
waves between Buffalo and Manhattan that *do not* touch Albany?

I know Zayo is one.  I’m waiting to hear back from CenturyLink.  I’ve reviewed 
networkatlas.org , to see if anything pops up there, 
but otherwise I’m coming up empty.

Thanks in advance.

Re: Non-profit IX vs. neutral for-profit IX

[Jason Lixfeld]

New rates for 2019 just posted yesterday!  Get yer ports while they’re hot!

> On Dec 21, 2018, at 9:14 AM, Clayton Zekelman  wrote:
> 
> 
> TorIX is a great example of a not for profit IX that is very successful.
> 
> https://www.torix.ca/  
> 
> A very dedicated team of people provide an incredible level of service.
> 
> Thave a very transparent process.  Their pricing is listed up front on their 
> website:
> 
> https://www.torix.ca/peering/#pricing 
> 
> 
> 
> At 09:03 AM 21/12/2018, Mike Hammett wrote:
>> As far as neutral, I meant separate from the datacenters in which they're 
>> housed. People in NA seem to think there are only two kinds of IXes, 
>> Equinix, DRT, Coresite types and NWAX, SIX, MICE types.
>> 
>> 
>> 
>> -
>> Mike Hammett
>> Intelligent Computing Solutions 
>> 
>> Midwest Internet Exchange 
>> 
>> The Brothers WISP 
>> 
>> From: "Tim Raphael" 
>> To: "NANOG Mailing List" 
>> Sent: Thursday, December 20, 2018 8:39:42 PM
>> Subject: Re: Non-profit IX vs. neutral for-profit IX
>> 
>> The other point to consider is that a NFP can justify more locations and 
>> offer services (such as extended reach) that don’t have the same profit 
>> margins or ROI as for-profits.
>> This often leads to greater value to those with smaller networks and fewer 
>> customers allowing them to grow and expand without increased aggregation or 
>> transit costs. This in-turn leads to a richer array of providers and chips 
>> away at the monopolies in niche markets.
>> 
>> The NFP IXP I work for focuses on providing value to the broader community 
>> and the Internet as a whole - especially somewhere like Australia which has 
>> unique constraints.
>> 
>> Additionally, “Neutral” and For-Profit doesn’t always compute in my 
>> mind, there will always be commercial alliances that lead to not-total 
>> neutrality.
>> When a NFP is owned by it’s members there has to be 100% transparency in 
>> organisational decisions around member funds and resources which ensures 
>> accountability reliability.
>> 
>> - Tim
>> 
>> 
>> > On 21 Dec 2018, at 3:58 am, Brielle Bruns  wrote:
>> > 
>> > On 12/20/2018 12:51 PM, Aaron wrote:
>> >> Probably price.  Also perception of value.  If you're a for profit 
>> >> enterprise then they're paying for interconnection plus your bump.  If 
>> >> you're non-profit the perception is that there is a larger value because 
>> >> there's no bump.  Whether that's true or not, who knows but that's the 
>> >> perception I've heard.
>> > 
>> > Depending on the size of the non-profit, I'd almost compare it to how the 
>> > hospitals are here in Boise.
>> > 
>> > The non-profits are oversized, monopolistic, price gouging, etc.  Their 
>> > care can be pretty meh, esp since they bought up all the little 
>> > independent clinics (yay, ER pricing for a basic family clinic visit).
>> > 
>> > The for-profit smaller clinics and hospitals run a pretty tight ship, 
>> > better value for their money, service is very good, and compete with one 
>> > another for who has the best service.
>> > 
>> > People think they are getting 'better' because they are going to a place 
>> > that is supposed to be run to benefit people over profit, but alas, you'd 
>> > be very very wrong.
>> > -- 
>> > Brielle Bruns
>> > The Summit Open Source Development Group
>> > http://www.sosdg.org  / http://www.ahbl.org 
>> >  
>> > 
>> 
>> 
> -- 
> 
> Clayton Zekelman
> Managed Network Systems Inc. (MNSi)
> 3363 Tecumseh Rd. E
> Windsor, Ontario
> N8W 1H4
> 
> tel. 519-985-8410
> fax. 519-985-8409
> 

Re: A few GPON questions...

[Jason Lixfeld]

> On Dec 11, 2018, at 11:32 AM, Ben Cannon  wrote:
> 
> Rip it out and run 9/125 SMF fiber home runs. Use BiDi SFPs to re-use your 
> existing (likely SMF thankfully) cable plant.  My opinion.

There’s only so much space in conduits, risers and ducts.  At some point, scale 
would press this up against physical infrastructure realities depending on how 
far the active gear at the head end is from the subscriber.

350 E Cermak

[Jason Lixfeld]

Hey all,

Looking for some clue on how things work, and who’s who for colo at 350 E 
Cermak.  Looking at possibly putting a rack in somewhere there for a 
Peering/Transit/PNI POP.  Is there a list somewhere of colo facilities in that 
building?

Also, how does it work there in terms if inter-colo, intra-building 
connectivity, or is that a mixed bag?

Thanks!

Peering management software

[Jason Lixfeld]

Hello all,

I’m researching various peering management software options, commercial or 
otherwise, geared towards network operators.  Wondering if folks might be able 
to help add to my list -

https://github.com/loopodoopo/pms
https://www.6connect.com/peering-manager/
https://github.com/respawner/peering-manager
http://www.lacnic.net/innovaportal/file/2569/1/mnovakovic_pivo_1.7.pdf [1]
https://github.com/ipcjk/ixgen

[1] If anyone from LinkedIn is around who might be able to contact me with info 
on Pivo, I’d be grateful!

Thanks!

Re: NAT on a Trident/Qumran(/or other?) equipped whitebox?

[Jason Lixfeld]

Indeed, however there are some other features currently missing from the Arista 
stack that sort of take it off the table (granted, those features have been 
promised early-ish next year).

> On Oct 9, 2018, at 11:52 AM, Edward Dore 
>  wrote:
> 
> Not sure if you count Arista as whitebox given their use of merchant silicon 
> but running their own NOS, however they were touting the 7170 series as being 
> able to do NAT recently. That's a Barefoot Tofino chip under the hood.
> 
> I've no idea how well it can do NAT or what the limitations are mind you, but 
> it was a specific selling point that they were pushing ...
> 
> Edward Dore 
> Freethought Internet
> 
> On 09/10/2018, 16:38, "NANOG on behalf of Jason Lixfeld" 
>  wrote:
> 
>Has anyone played around with this?  Curious if the BCM (or whatever other 
> chip) can do this, and if not, if any of the box vendors have tried to find a 
> way to get these things to do a bunch of NAT - say some flavour of NAT, 
> line-rate @ 10G.  If so, anyone know of a NOS that has support for it?  
> OcNOS, Cumulus Linux, PicOS and Switch Light OS seem to have none, but not 
> sure if there are others out there.
> 
>Thanks!
> 

NAT on a Trident/Qumran(/or other?) equipped whitebox?

[Jason Lixfeld]

Has anyone played around with this?  Curious if the BCM (or whatever other 
chip) can do this, and if not, if any of the box vendors have tried to find a 
way to get these things to do a bunch of NAT - say some flavour of NAT, 
line-rate @ 10G.  If so, anyone know of a NOS that has support for it?  OcNOS, 
Cumulus Linux, PicOS and Switch Light OS seem to have none, but not sure if 
there are others out there.

Thanks!

Re: Not announcing (to the greater internet) loopbacks/PTP/infra - how ?

[Jason Lixfeld]

> On Oct 4, 2018, at 3:07 PM, Brandon Applegate  wrote:
> 
> Thanks in advance for insights on this.

If you’re MPLS enabled, one implementation could see place the loop/infra/p2p 
in the global table and customer/internet traffic inside a VRF.

Re: Towards an RPKI-rich Internet (and the appropriate allocation of responsibility in the event an RIR RPKI CA outage)

[Jason Lixfeld]

> On Oct 1, 2018, at 4:36 AM, Mark Tinka  wrote:
> 
> On 1/Oct/18 10:26, John Curran wrote:
> 
>> Indeed… Hence the question of liability during a RIR CA outage, should the 
>> liability for misconfigured ISPs (those handful of ISPs who do not properly 
>> fall back to using state NotFound routes) be the responsibility of each ISP, 
>> or perhaps those who announce ROAs, or should be with the RIR?
> 
> Any equipment misconfigurations should be the responsibility of the operator.

^^

> Responsibility for ROA's should lie with the resource holder, in ensuring 
> that not only is the information true, but that also all announced prefixes 
> are covered by a ROA.

^^

I need to swap out the wheels on my car.  I think I know better than to read 
the manual to, say, understand how much torque I should apply to each bolt, or 
what pattern I should use when tightening the bolts.  Or, I read the manual but 
decide it’s too hard to understand, and I don’t ask for help in clearing up 
some of the grey areas.

I change the wheels anyway.  In the end, it looks right.  They roll.  Meh.  All 
good.

Then the wheels fall off.

There is absolutely no one to blame for any of that but me.

In my view, I see no difference here.

Re: What NMS do you use and why?

[Jason Lixfeld]

(resending with really, really the correct from:)

Here’s a snapshot of what tends to work for me, along with my $0.02 of thoughts:

- Observium handles polling, graphing and alerting for SNMP exposed objects on 
network devices,
- I feel that a visual representation of the physical network topology is 
extremely helpful for many aspects of day-to-day operations, so InterMapper 
handles that,
- Syslog and SNMPTRAP collection, correlation and alerting is handled by Splunk,
- Netflow collection and graphing is handled by nfsen,
- Smokeping for what smokeping does (but I just discovered vaping this morning, 
which looks awesome and will get some love).

I believe that LibraNMS has at some capability to use more robust graphing 
engines, which for me would be great; I find rrd is a little limiting these 
days.  I think it also has (better?) support for weathermap, so I could 
technically replace InterMapper with weathermap and collapse the tool chain a 
bit.

With streaming telemetry becoming more of a thing, there will definitely be a 
shift away from SNMP for things that are polled for statistics.

There are interesting Netflow tools like Elastiflow and pmacct that are more 
robust than nfsen.  The latter has a ton of functionality that can produce some 
interesting data for purposes of traffic engineering, among other things.  The 
former uses ELK so it’s inherently gorgeous and fast, but it requires a ton of 
resources depending on the number of flows/sec that you’re collecting.

Hope that helps.

Sent from my iPhone

> On Aug 15, 2018, at 9:49 AM, Colton Conor  wrote:
> 
> We are looking for a new network monitoring system. Since there are so many 
> operators on this list, I would like to know which NMS do you use and why? Is 
> there one that you really like, and others that you hate? 
> 
> For free options (opensouce), LibreNMS and NetXMS come highly recommended by 
> many wireless ISPs on low budgets. However, I am not sure the commercial 
> options available nor their price points.
> 
> 

Re: How are you configuring BFD timers?

[Jason Lixfeld]

Thanks to everyone who has responded so far.  Enlightening!

My understanding around the origins of BFD is that it was developed in part to 
try and bring SONET like switchover times to an Ethernet world.  What I’m 
reading is for those who do run BFD, no one seems to be dialing it down to try 
and achieve those times.  Some folks explained why they chose the values they 
did, but others didn’t.  So my follow up question is “Why don’t you dial them 
down?”.  Are achieving those switchover times not important for your use case?  
Do you not trust that it won’t be reliable based on the gear you’re using, or 
the quality/reliability of the underlying circuit you’re trying to protect?  
Something else?

Also, interesting to read about why some folks don’t care much about BFD at all.

> On Mar 21, 2018, at 9:10 AM, Jason Lixfeld <jason+na...@lixfeld.ca> wrote:
> 
> Hey,
> 
> For those running BFD on your land-based point-to-point links, I’m interested 
> in hearing about what factors you consider when deciding how to configure 
> your timers and multiplier.
> 
> On paper, BFD between two devices over a local or metro dark fibre or wave 
> seems pretty trivial:  Assuming your gear can a) support echo mode b) 
> hardware offloads echo processing c) automatically treats echos as vital and 
> puts them into the appropriate high priority queue, then setting the timers 
> down to their lowest possible values (3ms on some of the gear that I’ve seen) 
> and some low multiplier seems more than reasonable.  But?
> 
> From another angle, your link isn’t dark fibre or a wave but, for example, 
> ethernet over some sort of IP based L2 Transport, and is still a low (sub 
> 1ms) one-way latency local or metro link.  How do you set your timers, and 
> what do you base that on?
> 
> From yet another angle, what if your link is a long-haul wave, or for that 
> matter a wave of any distance that imposes a one-way latency that is higher 
> than the minimum tx and rx timers that are supported by your gear?  We’ll 
> assume an unprotected wave, because I’m sure if it’s protected, you have no 
> choice but to consider the one-way latency of the longest of the two segments.
> 
> I made some assumptions above about support for echo mode and hardware 
> offload, but what if (some of) your gear doesn’t support some or all of that 
> stuff?  How do you factor your configuration decisions?
> 
> Thanks!

Re: How are you configuring BFD timers?

[Jason Lixfeld]

They were ME3600s.  AFAIR it was two of these things in a lab connected back to 
back with two links between them, one metric higher than the other.  Some sort 
of traffic generator running between the two that would generate fixed size UDP 
frames at some tens of milliseconds interval, yanking the preferred link, 
counting how many packets were lost and doing some math, correlating with 
various logs debugs on the boxes.

> On Mar 21, 2018, at 1:34 PM, Youssef Bengelloun-Zahr <benge...@gmail.com> 
> wrote:
> 
> Which platform ? What context ?
> 
> Best regards.
> 
> 
> 
>> Le 21 mars 2018 à 18:10, Jason Lixfeld <jason+na...@lixfeld.ca> a écrit :
>> 
>> A few years ago I did some testing and found that the time between the 
>> transceiver detecting LOS and the routing protocol (ISIS in this case) being 
>> informed that the link was down (triggering the recalculation) took longer 
>> than it took BFD to signal ISIS to recalculate.
>> 
>>> On Mar 21, 2018, at 12:35 PM, Bryan Holloway <br...@shout.net> wrote:
>>> 
>>> Wouldn't any tangible problem on a dark-fiber link result in an interface 
>>> shutdown, ostensibly creating the trigger one would need to begin 
>>> re-convergence?
>>> 
>>> 
>>>> On 3/21/18 11:31 AM, Alex Lembesis wrote:
>>>> To speed up BGP routing convergence.  The (2x) dark fiber links from PA to 
>>>> FL are being used as Layer3 datacenter interconnects, where each 
>>>> datacenter has its own AS.  The DF is also carrying FCIP traffic, so we 
>>>> need failover to be as fast as possible.
>>>> Best regards,
>>>> Alex
>>>> -Original Message-
>>>> From: Job Snijders (External) [mailto:j...@instituut.net]
>>>> Sent: Wednesday, March 21, 2018 12:25 PM
>>>> To: Youssef Bengelloun-Zahr
>>>> Cc: Alex Lembesis; NANOG
>>>> Subject: Re: How are you configuring BFD timers?
>>>> Silly question perhaps, but why would you do BFD on dark fiber?
>>>> Kind regards,
>>>> Job
>>>> This message is intended solely for the designated recipient(s). It may 
>>>> contain confidential or proprietary information and may be subject to 
>>>> attorney-client privilege or other confidentiality protections. If you are 
>>>> not a designated recipient you may not review, copy or distribute this 
>>>> message. If you receive this in error, please notify the sender by reply 
>>>> e-mail and delete this message. Thank you.
>> 

Re: How are you configuring BFD timers?

[Jason Lixfeld]

A few years ago I did some testing and found that the time between the 
transceiver detecting LOS and the routing protocol (ISIS in this case) being 
informed that the link was down (triggering the recalculation) took longer than 
it took BFD to signal ISIS to recalculate.

> On Mar 21, 2018, at 12:35 PM, Bryan Holloway  wrote:
> 
> Wouldn't any tangible problem on a dark-fiber link result in an interface 
> shutdown, ostensibly creating the trigger one would need to begin 
> re-convergence?
> 
> 
> On 3/21/18 11:31 AM, Alex Lembesis wrote:
>> To speed up BGP routing convergence.  The (2x) dark fiber links from PA to 
>> FL are being used as Layer3 datacenter interconnects, where each datacenter 
>> has its own AS.  The DF is also carrying FCIP traffic, so we need failover 
>> to be as fast as possible.
>> Best regards,
>> Alex
>> -Original Message-
>> From: Job Snijders (External) [mailto:j...@instituut.net]
>> Sent: Wednesday, March 21, 2018 12:25 PM
>> To: Youssef Bengelloun-Zahr
>> Cc: Alex Lembesis; NANOG
>> Subject: Re: How are you configuring BFD timers?
>> Silly question perhaps, but why would you do BFD on dark fiber?
>> Kind regards,
>> Job
>> This message is intended solely for the designated recipient(s). It may 
>> contain confidential or proprietary information and may be subject to 
>> attorney-client privilege or other confidentiality protections. If you are 
>> not a designated recipient you may not review, copy or distribute this 
>> message. If you receive this in error, please notify the sender by reply 
>> e-mail and delete this message. Thank you.

How are you configuring BFD timers?

[Jason Lixfeld]

Hey,

For those running BFD on your land-based point-to-point links, I’m interested 
in hearing about what factors you consider when deciding how to configure your 
timers and multiplier.

On paper, BFD between two devices over a local or metro dark fibre or wave 
seems pretty trivial:  Assuming your gear can a) support echo mode b) hardware 
offloads echo processing c) automatically treats echos as vital and puts them 
into the appropriate high priority queue, then setting the timers down to their 
lowest possible values (3ms on some of the gear that I’ve seen) and some low 
multiplier seems more than reasonable.  But?

From another angle, your link isn’t dark fibre or a wave but, for example, 
ethernet over some sort of IP based L2 Transport, and is still a low (sub 1ms) 
one-way latency local or metro link.  How do you set your timers, and what do 
you base that on?

From yet another angle, what if your link is a long-haul wave, or for that 
matter a wave of any distance that imposes a one-way latency that is higher 
than the minimum tx and rx timers that are supported by your gear?  We’ll 
assume an unprotected wave, because I’m sure if it’s protected, you have no 
choice but to consider the one-way latency of the longest of the two segments.

I made some assumptions above about support for echo mode and hardware offload, 
but what if (some of) your gear doesn’t support some or all of that stuff?  How 
do you factor your configuration decisions?

Thanks!

Any Telus (AS852) BGP customers in the house?

[Jason Lixfeld]

Hello!

As a fellow AS852 BGP customer, I’m looking to chat with other AS852 BGP 
customers on the topic of modifications to the prefix filters attached to your 
BGP session(s).  Specifically, understanding the procedure Telus wants you to 
follow to request any changes, how long it takes the requested changes to be 
implemented, and whether or not there are any costs associated with those 
changes.

Off-list is probably appropriate.

Thanks in advance!

Re: What's the point of prepend communities?

[Jason Lixfeld]

> On Oct 26, 2017, at 2:55 PM, Job Snijders  wrote:
> 
> If Network B offers some kind of “Prepend to Network C” BGP community, 
> network A will be able to utilize all of network B except the pieces that 
> perform less well. (This is ofcourse assuming that Network C picks some 
> alternative path because of the prepends)

Absolutely.  I understand the "Prepend to Network blah” use case.  The case I 
don’t get is where the ISP makes no distinction in their policy document about 
how the prepending of their own AS is applied to their upstream announcements, 
implying that it’s announced to everyone.

Re: What's the point of prepend communities?

[Jason Lixfeld]

Hi Bill,

> On Oct 26, 2017, at 2:37 PM, William Herrin  wrote:
> 
> BGP routing is based on "distance". Distance in BGP is primarily calculated 
> as the number of ASNs in the AS Path. Prepends make a path more distance, 
> encouraging routers to choose a different path if one is available.

I understand how prepends fit in the context of best path selection, but my 
question was more the difference between a customer signalling the ISP to 
prepend their AS using a BGP community stamped to a prefix vs. the customer 
prepending their own AS instead.

What's the point of prepend communities?

[Jason Lixfeld]

Hi,

Of all the ISPs that I am familiar with that have a BGP community structure 
usable by their peering partners and/or downstream customers, among other 
things, they allow the customer to signal the ISP to prepend their own AS to 
the as-path of a particular prefix announcement.

What functionality does a provider prepend support that is otherwise lost in 
the absence of such a feature, but all the while, the customer would be able to 
prepend their own AS to the same prefix announcement anyway?

Is this a relic from before ISPs allowed for local preference adjustment, or is 
there actually a use case for this?

Thanks!

Re: Allstream/Zayo in the house?

[Jason Lixfeld]

Thanks for all the private responses.  Contact made, and the issue has been 
resolved.  Thank you all.

> On Oct 21, 2017, at 10:00 AM, Jason Lixfeld <jason+na...@lixfeld.ca> wrote:
> 
> Having an issue where you’re caching announcements for my AS via a peering 
> session that was turned down hours ago causing * * *, and my Saturday to suck 
> :)
> 
> Emails out to NOC/Peering contacts on peeringdb haven’t had a response yet.  
> Hoping someone here can poke and/or prod.
> 
> Thanks in advance.

Allstream/Zayo in the house?

[Jason Lixfeld]

Having an issue where you’re caching announcements for my AS via a peering 
session that was turned down hours ago causing * * *, and my Saturday to suck :)

Emails out to NOC/Peering contacts on peeringdb haven’t had a response yet.  
Hoping someone here can poke and/or prod.

Thanks in advance.

Re: MPLS in the campus Network?

[Jason Lixfeld]

> On Oct 20, 2016, at 12:23 PM, Mark Tinka <mark.ti...@seacom.mu> wrote:
> 
> 
> 
> On 20/Oct/16 17:12, Jason Lixfeld wrote:
> 
>> 
>> It’s only more expensive the more big vendor products you use.  Sometimes 
>> you need to (i.e.: Boxes with big RIB/FIBs for DFZ, or deep buffers), but 
>> more and more, people are looking to OCP/White Box Switches [1][2].
> 
> It doesn't sound like the OP needs massive FIB space, so he could implement 
> FIB filtering and run the smaller boxes that have all the features but lack 
> the FIB real estate of the larger routers/switches.
> 
> This is what we do for our Metro-E Access networks.
> 
> Mark.

Likely not at the PE, true, but he did say Internet access, so I err’d on the 
side of assuming DFZ, somewhere.  If that assumption is true, FIB resources for 
the SP interconnect nodes and filtering towards the PEs, absolutely.

Re: MPLS in the campus Network?

[Jason Lixfeld]

Hi,

> On Oct 20, 2016, at 9:43 AM, steven brock  wrote:
> 
> Compared to MPLS, a L2 solution with 100 Gb/s interfaces between
> core switches and a 10G connection for each buildings looks so much
> cheaper. But we worry about future trouble using Trill, SPB, or other
> technologies, not only the "open" ones, but specifically the proprietary
> ones based on central controller and lots of magic (some colleagues feel
> the debug nightmare are garanteed).

From my perspective, in this day and age, no service provider or campus should 
really be using any sort of layer 2 protection mechanism in their backbone, if 
they can help it.

> If you had to make such a choice recently, did you choose an MPLS design
> even at lower speed ?

Yup.  5 or so years ago, and never looked back.  Actually, this was in 
conjunction with upgrading our 1G backbone to a 10G backbone, so it was an 
upgrade for us in all senses of the word.

> How would you convince your management that MPLS is the best solution for
> your campus network ?

You already did:


> We are not satisfied with the current backbone design ; we had our share
> of problems in the past:
> - high CPU load on the core switches due to multiple instances of spanning
> tree slowly converging when a topology change happens (somehow fixed
> with a few instances of MSTP)
> - spanning tree interoperability problems and spurious port blocking
> (fixed by BPDU filtering)
> - loops at the edge and broadcast/multicast storms (fixed with traffic
> limits and port blocking based on threshhold)
> - some small switches at the edge are overloaded with large numbers of
> MAC addresses (fixed with reducing broadcast domain size and subnetting)
> 
> This architecture doesn't feel very solid.
> Even if the service provisionning seems easy from an operational point
> of view (create a VLAN and it is immediately available at any point of the
> L2 backbone), we feel the configuration is not always consistent.
> We have to rely on scripts pushing configuration elements and human
> discipline (and lots of duct-tape, especially for QoS and VRFs).



> How would you justify the cost or speed difference ?

It’s only more expensive the more big vendor products you use.  Sometimes you 
need to (i.e.: Boxes with big RIB/FIBs for DFZ, or deep buffers), but more and 
more, people are looking to OCP/White Box Switches [1][2].

For example, assuming a BCM Trident II based board with 48 SFP+ cages and 6 
QSFP+ cages, you get a line-rate, MPLS capable 10G port for $65.  Or, if you’re 
like me and hate the idea of breakout cables, you’re at about $100/SFP+ cage, 
at which points the QSPF+ cages are pretty much free.

Software wise, there are lots of vendors.  One that I like is IPInfusion’s 
OcNOS[3] codebase.  They are putting a lot of resources into building a service 
provider feature set (full-blown MPLS/VPLS/EVPN, etc.) for OCP switches.  There 
are others, but last time I looked a couple of years ago, they were less 
focused on MPLS and more focused on SDN: Cumulus Networks[4], PICA8[5], Big 
Switch Networks[6].

> Thanks for your insights!

[1] 
https://www.linx.net/communications/press-releases/lon2-revolutionary-development
[2] 
http://www.ipinfusion.com/about/press/london-internet-exchange-use-ip-infusion’s-ocnos™-network-operating-system-new-london-in
[3] http://www.ipinfusion.com/products/ocnos
[4] https://cumulusnetworks.com
[5] http://www.pica8.com
[6] http://www.bigswitch.com

Customers announcing communities to SP of SP

[Jason Lixfeld]

Hi,

Consider the following scenario:

- Customer A is a customer of SP A
- SP A is a customer of SP B
- SP B has a traffic engineering community implementation 

With regards to using BGP communities for TE:

- Does SP A write their own community implementation that maps to (some portion 
of) the community implementation of SP B?
- Does SP A write their own community implementation that has no mappings at 
all to the community implementation of SP B; any TE that is required to be 
pushed to SP B is done by some dialog and coordination between Customer A and 
SP A?
- Does SP A allow Customer A to announce prefixes tagged with SP B’s 
communities[1][2]
- Is this sort of thing really complicated today, but one of the goals of 
draft-heitz-idr-large-community?

[1] Customer A has knowledge of SP A’s upstream SP B
[2] This opens up a can of worms where SP A or SP B implements some communities 
prefixed with reserved ASes, so we’ll assume that SP A implements some method 
of allowing communities prefixed with ASes of SP A and SP B only.

Thanks!

Re: Any ISPs using AS852 for IP Transit?

[Jason Lixfeld]

Sure.  My question was whether every TELUS BGP customer was being charged for 
these too, or if I’m the only one.  If I’m the only one, then I’m obviously 
caught in some administrative black hole there that I would like to get myself 
out of.  This is something that has only started happening in the last 6 months 
or so.  Prior to that, we were never charged by them for these requests.  
Unfortunately, my sales rep has been less than helpful in trying to understand 
what changed to make us susceptible to these new charges.

> On Sep 15, 2016, at 3:15 PM, Hugo Slabbert <h...@slabnet.com> wrote:
> 
> So, to be blunt, I would cast this as their charging you NRC for manual work 
> because of their failure to automate this.
> 
> -- 
> Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
> pgp key: B178313E   | also on Signal
> 
> On Thu 2016-Sep-15 15:09:33 -0400, Jason Lixfeld <jason+na...@lixfeld.ca> 
> wrote:
> 
>> Last time I asked, that wasn’t something that they had implemented, and had 
>> no definite plans to do so within any timeframe that was on their radar.
>> 
>>> On Sep 15, 2016, at 2:50 PM, Steven Schecter <schec...@gmail.com> wrote:
>>> 
>>> I question their motivation here and would follow up by asking if they 
>>> support filtering by IRRdb and are merely trying to encourage the practice?
>>> 
>>> 
>>> /Steve
>>> 
>>> On Thu, Sep 15, 2016 at 2:07 PM, Jason Lixfeld <jason+na...@lixfeld.ca> 
>>> wrote:
>>> If there are any ISPs who use TELUS/AS852 for IP Transit over BGP, I’d be 
>>> interested in hearing from you.
>>> 
>>> I’d like to compare notes to see if you are also paying $250 for each BGP 
>>> prefix filter updated request, or if we’re the only ones…
>>> 
>>> Thanks in advance!
>>> 
>>> 
>>> 
>>> --
>>> Steven J. Schecter
>>> (m) 917.676.1646
>> 

Re: Any ISPs using AS852 for IP Transit?

[Jason Lixfeld]

Last time I asked, that wasn’t something that they had implemented, and had no 
definite plans to do so within any timeframe that was on their radar.

> On Sep 15, 2016, at 2:50 PM, Steven Schecter <schec...@gmail.com> wrote:
> 
> I question their motivation here and would follow up by asking if they 
> support filtering by IRRdb and are merely trying to encourage the practice?
> 
> 
> /Steve
> 
> On Thu, Sep 15, 2016 at 2:07 PM, Jason Lixfeld <jason+na...@lixfeld.ca> wrote:
> If there are any ISPs who use TELUS/AS852 for IP Transit over BGP, I’d be 
> interested in hearing from you.
> 
> I’d like to compare notes to see if you are also paying $250 for each BGP 
> prefix filter updated request, or if we’re the only ones…
> 
> Thanks in advance!
> 
> 
> 
> -- 
> Steven J. Schecter
> (m) 917.676.1646

Any ISPs using AS852 for IP Transit?

[Jason Lixfeld]

If there are any ISPs who use TELUS/AS852 for IP Transit over BGP, I’d be 
interested in hearing from you.

I’d like to compare notes to see if you are also paying $250 for each BGP 
prefix filter updated request, or if we’re the only ones…

Thanks in advance!

Recommendations for used satellite decoder resellers

[Jason Lixfeld]

Hello,

I’m wondering if anyone can refer me to a company they’ve used in the past who 
may have (access to) used satellite decoder equipment.  I’m in the market for 
some used Sencore kit.

Thanks in advance!

Re: Network traffic simulator

[Jason Lixfeld]

I’m in the process of building a box using MoonGen [1] and a supported Intel 
82599 6 port SFP+ NIC [2] that is coming in at just under US$3800 all-in.  
Supposed to be able to drive at least the entire card at line rate for that 
price and have enough CPU and memory slots free to fill the box up with as many 
of these NICs as it will take if need be.

[1] https://github.com/emmericp/MoonGen
[2] 
http://www.interfacemasters.com/index.php?option=com_content=article=153=103

> On May 24, 2016, at 8:17 AM, Mitchell Lewis  
> wrote:
> 
> Hi,I am looking to validate the performance specs of a core router. I am 
> looking for a network traffic simulator which can simulate 40 gbps of 
> traffic. I am looking for a simulator with sfp+ ports.
> I am interested in any input as to brands to look at, build one myself etc.
> Thanks,Mitchell 

Perspectives about customer M/A/C in triple play environments

[Jason Lixfeld]

Hello,

I think it’s fair to say that most broadband/FTTx customers don’t have to think 
very much or need to have a very high degree of understanding if they want to 
move their wired Internet device from one room or another in their house.  

Maybe to keep things simple, let’s assume that we’re talking about a relatively 
modern MDU unit where a customer has some sort of provider CPE in their 
in-suite telecom demark closet/box/what have you with some number of switched 
'LAN’ ports on it, and each of those LAN ports would be wired to a wall jack 
somewhere.  Mr. or Ms. User can move their Internet device anywhere there is a 
wall jack and Bob’s your uncle.

My question is around how this landscape changes in triple play environments.  
As I understand it, most triple play deployments separate (in some cases VoIP,) 
TV and Internet traffic onto VLANs (Internet would be presented to the customer 
untagged).  The CPE would then allow the ISP to switch the video traffic onto a 
coax port, or maybe onto the CPE’s embedded switch, or maybe both.  For the 
sake of argument, let’s assume the provider is supplying an Ethernet based 
set-top-box, so customer should be able to connect the STB to any wall jack and 
it should just work.  And they should be able to connect their provider 
supplied ATA to any wall jack, and it should just work.  And they should be 
able to connect their Internet device to any wall jack and it should just work.

Or should it?

Are most CPEs that are provided by ISPs sophisticated enough to be able to put 
all service tags on all ports, and have those same ports act as untagged LAN 
ports as well?  If not, how do providers deal with this?  Do they dedicate one 
port for an IPTV STB?  One port for an ATA (assuming no built-in POTS on the 
CPE)?  And the rest of the ports for untagged Internet?  What if the customer 
has 2+ TVs?  Do they need to call in and have the provider remote in and 
provision another port for TV at the expense of some other service that might 
be running on that port already?  Do they need to install a switch that does 
IGMP snooping?

I feel like this all has the potential to become very complicated for the 
customer, and maybe the provider and their installers.  To me, the customer 
should continue to be dumb and unassuming.  They should be able to put whatever 
they want wherever they want and have it just work.  Is that how things 
actually are in the real world or are customers and providers making silent 
sacrifices for the sake of all this new fangled technology?

Duplex negotiation over 100Base fibre

[Jason Lixfeld]

Hello,

My understanding is that for 1G and 10G optical networks, there is no concept 
of half-duplex mode, but I’m unclear about half duplex in the 100M optical 
world.  Specifically, if I connect two 100Base-LX (or BX) transceivers 
together, is there a requirement for the controller(s on either side) to select 
between full or half duplex, either by static configuration, or 
auto-negotiation?

Thanks!

BCM SOC based IPTV STBs

[Jason Lixfeld]

Howdy,

What are folks using for BCM SOC (7424 gen, or newer) based IPTV STBs?[1]  I’m 
looking for something pretty simple - HDMI, S/PDIF (optical and 1/8” digital 
coax) and an Ethernet port is all I’m really interested in.  No RCA audio, 
component video or Wifi.  Nice and simple.  If GreenPeak RF4CE were baked in, 
that’d be cool too.

Informir and Airties have some interesting stuff, but I’m curious as to what 
other are using.

Thanks in advance.

[1] I’m assuming that *IP*TV based discussion is considered relevant 
conversation?  If not, forgive the intrusion, but I’d welcome any pointers on 
where one might go to discuss this sort of thing if this is considered 
off-topic.

Re: configuration sanity check

[Jason Lixfeld]

Either of these might come in handy..

https://www.nanog.org/meetings/abstract?id=2673
https://www.nanog.org/meetings/abstract?id=2678

> On Oct 29, 2015, at 4:16 AM, marcel.durega...@yahoo.fr wrote:
> 
> Hi Nanogers,
> 
> Any recommendation about a software which check the live config of 
> cisco/juniper devices against some templates ?
> 
> The goal is to have a template about different function device, like:
> - CORE device must have this bloc and this clock
> - PE device must have at least that and that
> - CPE must have this and that
> - Distrib switch block 1 and block2
> - etc...
> 
> And the software run once every day to check which device do not comply with 
> those rules and generate an alert.
> 
> Thank,
> - Marcel

Segment Routing for L2VPN?

[Jason Lixfeld]

Hello!

I've been doing some reading recently on Segment Routing.  By all accounts, it 
seems that the (only?) implementation for SR supports L3VPN.  Am I dumb and 
just missing the L2VPN bits, or is L3VPN simply the extent of the first 
generation?

Sent from my iPhone

Mac compatible SFP+/XFP programmer

[Jason Lixfeld]

Does anyone know where I might find a SFP+/XFP programmer with a Mac compatible 
programmer application?

Thanks!

In-rack DC distribution

[Jason Lixfeld]

Mornin’,

I’ve been looking for the holy grail of in-rack A/B DC distribution, 
unfortunately without much success so far.

In a perfect world, I’d have the equivalent of what we know and love about 0U 
vertical AC distribution rails (i.e.: APC), except with A/B feeds - lots of 
outlets, takes up no space in the rack, managed, remotely switched, etc.  I’ve 
only found one vendor who makes such a beast for the DC world, but the breakers 
are manual - if it trips, or if you need to cycle it, you're rolling a truck.  
The beauty of this though is that it can take the gambit of breakers from 
5A-200A.

I’ve also been looking for something in the 1U world with A/B feeds.  GMT 
panels are super high density, but GMT fuses are limited to 15A, so something 
that had small breakers (solid state, maybe??) 5-40A would be the holy grail 
there.  Again, managed, remotely monitored per input/per output, remotely 
switched or reset would be the holy grail.

Does anyone know of anything like this out there anywhere?

Infinera sales contact?

[Jason Lixfeld]

Might someone have an Infinera sales contact handy for Canada?  Information 
submitted via their web form doesn’t seem to be getting much attention.

Thanks!

Re: Peering and Network Cost

[Jason Lixfeld]

 On Apr 19, 2015, at 6:09 AM, William Waites wwai...@tardis.ed.ac.uk wrote:
 
 On Sun, 19 Apr 2015 11:23:53 +0200, Baldur Norddahl 
 baldur.nordd...@gmail.com said:
 
 So why is IX peering so expensive?
 
 But the only service is running an old layer 2 switch.
 
 The 40 dix particants should donate 1000 USD once and get a new
 layer 2 switch. Why does that not happen?
 
 This is something like how TORIX was operated at the beginning. The
 switch was donated by Cisco and rack space by a member with a cage at
 a convenient spot at 151 Front -- I think this was jlixfeld at
 look.ca. Fees were a $1/port/year peppercorn.
 
 It has been a long time since I was in any way involved in that, but
 today for a 1Gbps port TORIX charges $1200/year which is more but still
 not as much as you say for other IXPs. It would be interesting to hear
 from someone who was involved in TORIX at the time how this transition
 from $1 to $1200 went and the reasoning behind it. My guess would be
 moving to its own space and having to pay rent was a major part of it,
 and possibly acquiring staff?

To be clear, we asked for $1/port/year, but we never really bothered to pay 
attention to who actually paid :)

Instead of addressing your questions directly, how about a brief and much 
abridged history of TorIX? ;)

The recollection of Mr. Waites on our humble beginnings is pretty much bang-on. 
 For the first 7 or so years, we were really ad-hoc, but we eventually decided 
that we needed to incorporate.  That decision was simply due to the fact that 
we didn’t think we’d be taken very seriously by larger players (larger eyeball 
networks or large content networks (either nationally or internationally)) 
unless we moved away from an ad-hoc collection of nothing and no-one, and into 
an actual legal entity.  Along with feedback from the participates of our 
little IX, those of us who made up the organizing body of this ad-hoc TorIX 
decided that while a legal organization was an important next step in our 
evolution, incorporating with non-profit status (as opposed to a full-blown 
commercial IX) was the most appropriate method of becoming legit.  Bill 
Campbell (former owner Hostopia, former owner Internet Direct (later became 
Look)) put up 100% of the money to incorporate TorIX in early 2004.

Second, up until about 2008’ish, whenever we needed gear, we’d usually have to 
pass the hat when we needed a GigE switch or something a little more high test 
than someone’s decommissioned FastE kit.  The problem with passing the hat is 
that it rarely makes everyone happy because there’s always someone who gets 
left out.  The cash in the hat would only give us enough to buy a 12 port 
switch, but inevitably, a few more than 12 participants all donated towards 
buying the switch.  The last ones to offer up the cash had to be dropped until 
the next time the hat got passed around.  We didn’t think asking all our 
participants to drop money into the hat was an appropriate course of action.  
Not everyone would contribute, for a multitude of reasons, but everyone would 
still expect the same level of service.  Needless to say, it got messy.  It was 
an inevitable part of our growth, sure.  It might still be inevitable for any 
budding IX.

After our incorporation, there were many offers from folks with skids of 
decommissioned 6500s, 6704s and SUP720s.  These extremely generous donations 
made it possible for us to turn up our first 10G port, but it resulted in other 
challenges: who would be allowed to occupy the other three ports?  Do we charge 
for them?  We got the ports for free so HTF do we figure this one out, guys?  
These sorts of dilemmas would cause strife, so around 2008, the serving Board 
at the time decided that the next step in our evolution was to make the 
organization completely self-sufficient by introducing a reasonable port fee 
structure.  Port fees could let us get space where we felt we needed it.  We 
could buy our own gear so anyone would always be able to have any speed port 
they wanted.  We could pay for the support contracts, hire lawyers and 
accountants, and also contribute to community initiatives like sponsoring the 
Canadian ISP Summit, NANOG and ARIN.  We strive to keep our port fees low.  99% 
of folks never thought our port fees were too high.  In fact, I can remember a 
few folks who laughed when we introduced port fees asking if they could pay for 
5 years up front because the port fees were so cheap they were a joke.

The Board introduced a reduced port fee structure across the board for 2015.

Everyone[1] who contributes to TorIX still does so in a volunteer capacity; 
Board members, the operations group, even our book keeper :)

[1]In 2014, the Board voted in favour of a motion to hire an Executive Director 
to further drive the growth of TorIX.  In March, the Board announced that Bill 
Sandiford had accepted the role.  In the 18 year history of TorIX, the 
Executive Director role is the first ever 

Ixia or Spirent around?

[Jason Lixfeld]

I tried the contact form on their respective websites a couple of weeks back, 
but have not heard back.

If anyone from there is lurking about or if anyone has a sales contact that 
covers Toronto and could send me off-list, I'd be grateful.

Thank you!

Sent from my iPhone

802.11 based WISP hardware

[Jason Lixfeld]

Hi all,

I’m looking to gather some public opinion, links and pointers around the 
current landscape of WISP hardware vendors.  I’m familiar with Cisco, Ruckus, 
AdTran, Motorola and Aruba (HP) but I’m wondering who else is out there that 
folks have used with success.  My main areas of interest are around controller 
based (hardware or virtual (in-house, not off-net cloud based)) systems that 
have a range of indoor  outdoor 802.11AC PoE capable APs.  The controller(s) 
would be capable of tunnelling traffic from the APs for one or more SSIDs, 
support per-SSID captive portals and unique, intra-SSID captive portals.  In a 
perfect world, an on-board DHCP server would be super handy too.  The system 
should support CAPWAP, but some proprietary alternative is also fine, the usual 
suite of security protocols per SSID, reliable intra-SSID AP roaming algorithms 
and multi-SSID capable.

Thanks in advance.

Google served from non-google IPs?

[Jason Lixfeld]

So today, I saw this:

BlackBox:~ jlixfeld$ host google.ca 8.8.8.8
Using domain server:
Name: 8.8.8.8
Address: 8.8.8.8#53
Aliases:

google.ca has address 206.126.112.166
google.ca has address 206.126.112.177
google.ca has address 206.126.112.172
google.ca has address 206.126.112.187
google.ca has address 206.126.112.151
google.ca has address 206.126.112.158
google.ca has address 206.126.112.157
google.ca has address 206.126.112.173
google.ca has address 206.126.112.181
google.ca has address 206.126.112.155
google.ca has address 206.126.112.147
google.ca has address 206.126.112.185
google.ca has address 206.126.112.143
google.ca has address 206.126.112.170
google.ca has address 206.126.112.162
google.ca has IPv6 address 2607:f8b0:4006:808::100f
google.ca mail is handled by 50 alt4.aspmx.l.google.com.
google.ca mail is handled by 30 alt2.aspmx.l.google.com.
google.ca mail is handled by 20 alt1.aspmx.l.google.com.
google.ca mail is handled by 10 aspmx.l.google.com.
google.ca mail is handled by 40 alt3.aspmx.l.google.com.
BlackBox:~ jlixfeld$

That is not Google IPv4 address space, and those IPv4 IPs are not being 
announced by 15169.

Am I dumb in thinking that this is weird or is this sort of thing commonplace?

Is there a case for storm control and/or unknown traffic flood control in 'protected' bridge-domain?

[Jason Lixfeld]

Greetings,

Conceptually, a layer 2 port that is configured for either port protect mode 
(a’la Cisco 2950 vintage), UNI port-type (a’la Cisco ME3400 vintage) or EVC + 
split-horizon (a’la ME3600 vintage) should negate any requirement for features 
such as storm control or unknown traffic flood control to be configured in 
conjunction with either of those port modes.  In theory then, either of the 
three aforementioned configuration modes would prevent any and all cross-talk 
between ports, in the same bridge-domain, notwithstanding traffic hitting the 
‘trusted’ port, be it the trunk or uplink port, SVI, routed BD or whatever name 
your hardware uses to define that trusted port.

Assuming that’s an accurate theory, is there a case that I might be missing 
where one would need to use storm control or unknown traffic flood control in 
this sort of environment?

Private ASNs in the wild

[Jason Lixfeld]

I just fat fingered a regex that was intented to show how many private ASNs 
we’re using on our network for various things.  The results of the fat fingers 
showed that there are an astronomical number of private ASNs in the wild.  I 
checked the CIDR report, and those ASNs are shown there in a specific Bogon ASN 
report, but I’m surprised that as far as I can recall, there haven’t been any 
efforts made by the good netizens around these parts to bring awareness to this 
issue.

Do we feel that it’s not that big of a deal?  Have we not really been paying 
attention?  Some other reason this seems to be a rather muted topic?

Re: 10Gb iPerf kit?

[Jason Lixfeld]

I gotta wonder.  How reliable is iPerf over something like RFC2544 or Y.1564?  
Especially at those speeds?

I just picked up a couple of Accedian’s RFC2544/Y.1564 boxes to use as 
loopbacks to our field Exfos.  We’ll probably wind up buying a few more 
Accedian boxes for the field where we don’t need to spend the money on an Exfo.

One of the Accedian boxes is arguably less than what you’d pay for a TB MBP and 
that Sonnet adapter.

 On Nov 10, 2014, at 7:38 PM, Christopher Morrow morrowc.li...@gmail.com 
 wrote:
 
 why doesn't a tbird do this for you?
 
 On Mon, Nov 10, 2014 at 7:35 PM, Randy Carpenter rcar...@network1.net wrote:
 
 I have not tried doing that myself, but the only thing that would even be 
 possible that I know of is thunderbolt.
 
 A new MacBook Pro and one of these maybe: 
 http://www.sonnettech.com/product/echoexpresssel_10gbeadapter.html
 
 
 -Randy
 
 
 - On Nov 10, 2014, at 7:26 PM, Daniel Rohan dro...@gmail.com wrote:
 
 We're looking for a semi-portable solution to validate 10Gb customer
 circuits and hitting walls surrounding PCI lanes and the amount of data
 laptops can push via their busses. We'd prefer to not have techs lugging
 around server equipment for these tests.
 
 Anyone out there testing 10gbE with iPerf?  If so, what are you using?
 
 Thanks,
 
 
 Dan

Multi-port RFC2544/EtherSAM loopback appliance

[Jason Lixfeld]

Group,

I'm looking for options and opinions on a cost effective, multi-port (6'ish 
port SFP/SFP+) RFC2544/EtherSAM rack appliance that can act as the 
remote/loopback for our field installers' portable RFC2544/EtherSAM enabled 
Exfo test sets.

I came across XenaNetworks XenaCompact which looks like it would fit the bill, 
but I'm sure there are others (save Ixia, Exfo and Fluke, which I'm pretty much 
excluding by default because I imagine they are likely completely out to lunch 
on price relative to the extremely simplified feature set we require).

Thanks in advance.

AM dust filters

[Jason Lixfeld]

Hi,

I'm interested in knowing what sorts of material folks use to make after-market 
dust filters for their various devices which wouldn't normally have any.  This 
seems to almost be a necessity when these kinds of devices are deployed in 
environments that are overly dusty and dirty (it should also be implied that 
these environments are all in-doors and would have less than ideal airflow and 
climate control).

A material that is too dense will hider airflow and cause an immediate increase 
in inlet temperature, which would exacerbate a potentially threatening 
temperature situation in environments where the ambient temperature is already 
in the mid to high twenties and above (that's 77 - 86F+ for my American friends 
;)).  A material that is not dense enough won't do a very good job at filtering.

Do folks just hack up HEPA filters or something?

Re: AM dust filters

[Jason Lixfeld]

On Aug 12, 2014, at 3:09 PM, Tom Morris bluen...@gmail.com wrote:

 One important question: how often is the equipment accessed for maintenance?

Who knows :)  Maybe it becomes someone's full time job to go do regular checks 
and maintenances of every POP?  Maybe after an appropriate filter is found, a 
relatively low temperature threshold monitor is set up in an NMS.  When this 
threshold is reached, it would probably be safe to assume a dirty filter (or 
some other condition that would require a visit) and someone could be 
dispatched to replace it.

 I've had reasonably good luck with air filter media coated with a tackifier, 
 similar to the Dustlok media here 
 http://www.filtersales.com/pagout.htm?id=Pad%20Media
 It seems like what happens with it is heavier airborne fibers (lint, hair) 
 get caught up in the first few fibers of the media, not obstructing airflow, 
 and allow the finer dust to travel deeper into the media where it sticks to 
 the tacky layer at the back. It lasts a good long while. It's single use 
 though, so it has to be replenlished every now and then.
 
 Foam rubber media tends to have trouble with surface/airflow area vs pore 
 size.
 
 The best option, though, will be to enclose the equipment in a cabinet that 
 can be pressurized by one or more fan forced+filtered inlets. Middle Atlantic 
 makes rack cabinets and fan panels that can be used to pressurize them that 
 way. If you get a cabinet that takes a standard furnace filter, I've had good 
 luck with the off the shelf 3M Filtrete Ultra Allergen filters, they have a 
 TON of surface area with great fine dust capture and very low airflow 
 resistance, even when you're drawing the air through them really way too 
 fast. :)

Unfortunately a cabinet isn't possible due to a variety of issues.

 
 On Tue, Aug 12, 2014 at 2:19 PM, Jason Lixfeld ja...@lixfeld.ca wrote:
 Hi,
 
 I'm interested in knowing what sorts of material folks use to make 
 after-market dust filters for their various devices which wouldn't normally 
 have any.  This seems to almost be a necessity when these kinds of devices 
 are deployed in environments that are overly dusty and dirty (it should also 
 be implied that these environments are all in-doors and would have less than 
 ideal airflow and climate control).
 
 A material that is too dense will hider airflow and cause an immediate 
 increase in inlet temperature, which would exacerbate a potentially 
 threatening temperature situation in environments where the ambient 
 temperature is already in the mid to high twenties and above (that's 77 - 
 86F+ for my American friends ;)).  A material that is not dense enough won't 
 do a very good job at filtering.
 
 Do folks just hack up HEPA filters or something?
 
 
 
 -- 
 --
 Tom Morris, KG4CYX
 Mad Scientist and Operations Manager, WDNA-FM 88.9 Miami - Serious Jazz!
 786-228-7087
 151.820 Megacycles

Re: AM dust filters

[Jason Lixfeld]

On Aug 12, 2014, at 3:22 PM, Doug Barton do...@dougbarton.us wrote:

 On 08/12/2014 11:19 AM, Jason Lixfeld wrote:
 Hi,
 
 I'm interested in knowing what sorts of material folks use to make 
 after-market dust filters for their various devices which wouldn't normally 
 have any.  This seems to almost be a necessity when these kinds of devices 
 are deployed in environments that are overly dusty and dirty (it should also 
 be implied that these environments are all in-doors and would have less than 
 ideal airflow and climate control).
 
 A material that is too dense will hider airflow and cause an immediate 
 increase in inlet temperature, which would exacerbate a potentially 
 threatening temperature situation in environments where the ambient 
 temperature is already in the mid to high twenties and above (that's 77 - 
 86F+ for my American friends ;)).  A material that is not dense enough won't 
 do a very good job at filtering.
 
 Do folks just hack up HEPA filters or something?
 
 It sort of depends on what kind of stuff you're trying to filter out.

Small-ish stuff.  Your every day, run of the mill fine grain dust, tracked-in 
dirt  sand, some construction particulate (metal shavings, etc).

 Panty hose actually makes a reasonably good filter for larger stuff, but 
 Tom's question about how often are you going to service it comes into play, 
 since you need to remove the debris that it catches periodically in order to 
 avoid obstructing the air flow excessively.

Yup.  Depending, either a vacuum or a straight-up replacement of the 'filter', 
I'd suspect.  Or maybe just a good shake in some cases.

 OTOH, you also have to have some thought towards what are the benefits of not 
 having the internals of the system coated with dust, vs. slightly reduced air 
 flow.

Indeed.  The internals can definitely handle non-metalic dust, as well as a 
pretty wide temperature range (caused by either reduced airflow or an increase 
in ambient temperature, or both), so I'd imagine it would be a appropriate 
balance between the two.

 Tom's suggestion of a pressurized cabinet is a good one of course, but that's 
 not possible in all situations.

Re: Best practice for BGP session/ full routes for customer

[Jason Lixfeld]

1.  You already know that multihop is very ugly.  If it's for a one-off, it's 
probably fine.  But building a product around multi-hop wouldn't be my first 
choice.

2.  Most of the router/switch vendors that can support a full table are pretty 
expensive, per port.  Your best bet here might be to look into some way of 
transparently dragging customer traffic from the PE to the BGP speaker, which 
leads me to:

3.  If your network is MPLS enabled, you can do a routed pseudowire from a BGP 
speaking router with a full table to the access router (PE).  Other tunnelling 
technologies can probably do the same thing; GRE, L2TPv3 and also a plain'ol 
VLAN can do it too, depending on your network topology.  Do some sort of OAM 
over top of either of those (if your platform supports it) and it looks just 
like a wire to the end customer.

On Jul 7, 2014, at 2:33 PM, Anurag Bhatia m...@anuragbhatia.com wrote:

 Hello everyone!
 
 
 I have quick question on how you provide full BGP table to downstream
 customers?
 
 
 Most of large networks have few border routers (Internet gateways) which
 get full table feed and then they have Access routers on which customers
 are terminated. Now I don't think it makes sense to push full routing table
 on the access routers and simply their default points to border routers.
 
 
 In this scenario what is best practice for giving full table to downstream?
 
 
   1. Having multi-hop BGP session with a loopback on border router for
   injecting full table in customer router and another BGP session with access
   router for receiving routes? (messy!)
 
 
   2. Injecting full table in just all access routers so that it can be
   provided whenever needed?
 
   3. Any other?
 
 
 
 
 Thanks in advance!
 
 -- 
 
 
 Anurag Bhatia
 anuragbhatia.com
 
 Linkedin http://in.linkedin.com/in/anuragbhatia21 | Twitter
 https://twitter.com/anurag_bhatia
 Skype: anuragbhatia.com
 
 PGP Key Fingerprint: 3115 677D 2E94 B696 651B 870C C06D D524 245E 58E2

Anyone from AS577 and AS852 in the house?

[Jason Lixfeld]

Bell and Telus, if you're listening - I need to inquire about BGP community 
support on your respective networks that cannot be addressed by info published 
in RADB, by our assigned AM, SE, your NOC or any support documentation on your 
respective websites on the subject.

Please hit me up off-list.

Thanks in advance!

Who uses ARIN's IRR?

[Jason Lixfeld]

I don't need to use it much, but when I do, it's an ever-increasing royal pain 
in the ass.

My current plight revolves around not being able to get full dumps of objects.  
Certain mandatory fields in objects are 'filtered' and/or replaced with dummy 
data.  This poses a problem because one can no longer simply cut and paste the 
output, change the necessary bits and fire it off to r...@arin.net for 
processing.  WhoisRWS doesn't seem to have hooks into the IRR database like 
RIPE seems to have gotten right.

So how do people tend to get around this?  Is there something that I'm missing 
or do people just throw their hands up and move their IRR data to RADB or 
something?

Re: Who uses ARIN's IRR?

[Jason Lixfeld]

On Mar 7, 2014, at 12:01 PM, Koch, Andrew andrew.k...@tdstelecom.com wrote:

 You will notice right at the top of the output there is a hint on getting an 
 unfiltered object.  Try using the -B flag on your query to get around this.

Indeed, however that doesn't help with the dummy objects that also make it 
impossible to use the output as a new template like everyone has been doing for 
a hundred years.

Cogeco in the house?

[Jason Lixfeld]

If someone from Cogeco could ping me, I'd like to have a chat about something 
odd and intermittent:

It works:

BlackBox:~ jlixfeld$ mtr -c 1 -rw 162.243.142.155
Start: Fri Feb  7 18:46:06 2014
HOST: BlackBox.localLoss% Drop   Rcv   Snt  
Last  Best   Avg
 1.|-- 192.168.69.1   0.0%0 1 1   
4.0   4.0   4.0
 2.|-- 96-45-207-217.beanfield.net0.0%0 1 1   
9.3   9.3   9.3
 3.|-- gi0-1-0-2.bfr01.77mowatav01.yyz.beanfield.com  0.0%0 1 1   
9.9   9.9   9.9
 4.|-- be2.bfr01.60hudsonst01.jfk.beanfield.com   0.0%0 1 1  
20.8  20.8  20.8
 5.|-- nyk-b3-link.telia.net  0.0%0 1 1  
19.3  19.3  19.3
 6.|-- nyk-bb1-link.telia.net 0.0%0 1 1  
19.5  19.5  19.5
 7.|-- sjo-bb1-link.telia.net 0.0%0 1 1  
92.5  92.5  92.5
 8.|-- digitalocean-ic-302451-sjo-bb1.c.telia.net 0.0%0 1 1  
93.9  93.9  93.9
 9.|-- 198.199.99.238 0.0%0 1 1  
94.6  94.6  94.6
10.|-- streetscapeplus.com0.0%0 1 1  
94.2  94.2  94.2
BlackBox:~ jlixfeld$

Now it doesn't:

BlackBox:~ jlixfeld$ mtr -c 1 -r 162.243.142.155
Start: Fri Feb  7 18:42:54 2014
HOST: BlackBox.local  Loss% Drop   Rcv   Snt  Last  Best   Avg
 1.|-- 192.168.69.1   0.0%0 1 1   4.2   4.2   4.2
 2.|-- 96-45-207-217.beanfield.n  0.0%0 1 1   9.0   9.0   9.0
 3.|-- gi0-1-0-2.bfr01.77mowatav  0.0%0 1 1   9.8   9.8   9.8
 4.|-- te0-0-0-1.bfr01.151fronts  0.0%0 1 1   9.5   9.5   9.5
 5.|-- gw-mto.torontointernetxch  0.0%0 1 1   9.0   9.0   9.0
 6.|-- tge-1-1.ar1.mtrlpq07.coge  0.0%0 1 1  17.1  17.1  17.1
 7.|-- 206.223.224.2250.0%0 1 1  16.8  16.8  16.8
 8.|-- ???   100.01 0 1   0.0   0.0   0.0
BlackBox:~ jlixfeld$


Thanks!

Re: BRAS

[Jason Lixfeld]

What's so interesting about a guy asking for info on a Broadband Remote Access 
Server for DSL aggregation?

On Dec 11, 2013, at 1:11 AM, Nick Cameo sym...@gmail.com wrote:

 Sir whatever that is an acronym for, you have my undivided.
 
 This is going to make for an interesting thread in about 6 hours.
 

Re: bgp traceroute tool?

[Jason Lixfeld]

It would be slick if someone could patch mtr to do this too.

Sent from my iPhone

 On Nov 30, 2013, at 7:19 PM, Rene Wilhelm wilh...@ripe.net wrote:
 
 
 On 11/30/13 1:18 AM, Lee Clark wrote:
 The traceroute variant  included with CentOS 6.4  Mint 13 has an -A
  flag which does ASN lookups. ntraceroute on FreeBSD supports it as
  well. I believe the Linux port is traceroute-nanog.
 
  Lee
 
 traceroute -A  consults the internet routing registry which is know
 to beincomplete and at times incorrect when it comes to IP to
 BGP origin AS mapping. For this reason we developed riswhois.ripe.net,
 a whois style interface to the BGP data collected by RIPE NCC's Routing
 information service (http://www.ripe.net/data-tools/stats/ris)
 
 Reporting in the same format as the IRR, riswhois is plugin
 compatible with whois.radb.net. If your linux traceroutederives
 from http://traceroute.sourceforge.net/ all it takes to switch to
 using true BGP info in traceroute is setting the environment variable
 RA_SERVER to riswhois.ripe.net
 
 
 -- Rene
 
 P.S. the LFT tool metioned earlier in this thread can also use RISwhois
 to lookup ASNs; just pass it the -r option on the command line.
 
 
 
 
 [user@box ~]# traceroute -V Modern traceroute for  Linux, version
 2.0.14, Nov 11 2010 Copyright (c) 2008  Dmitry  Butskoy,   License:
 GPL v2 or any later
 
 [user@box ~]# traceroute -A www.google.ca traceroute  to www.google.ca
 (74.125.226.127), 30 hops max, 60 byte packets  snip 6  72.14.197.33
 (72.14.197.33) [AS15169]  73.927 ms  69.254 ms69.305 ms 7
 209.85.254.130 (209.85.254.130) [AS15169]  69.436 ms  209.85.254.122
 (209.85.254.122) [AS15169]  79.554 ms  64.269 ms 872.14.237.130
 (72.14.237.130) [AS15169]  64.979 ms  65.975 ms  209.85.254.238
 (209.85.254.238) [AS15169]  66.700 ms 9216.239.46.161
 (216.239.46.161) [AS15169]  71.293 ms  72.251 ms73.521 ms 10
 209.85.250.207 (209.85.250.207) [AS15169]  74.454  ms  74.920 ms
 75.889 ms 11  yyz08s13-in-f31.1e100.net  (74.125.226.127) [AS15169]
 76.628 ms  77.105 ms  70.928 ms
 
 
  -Original Message- From: John Conner
  [mailto:bs7...@gmail.com] Sent: Friday, November 29, 2013 5:04 PM To:
  nanog@nanog.org Subject: bgp traceroute tool?
 
  Hi there, is there any tools available under linux which can do bgp
  traceroute? (print bgp AS numbers for each traceroute hop ) , i
  googled and found nothing.
 
  thanks
 
  John
 
 
 
 
 

Re: Internet Surveillance and Boomerang Routing: A Call for Canadian Network Sovereignty

[Jason Lixfeld]

That notwithstanding, it's stupid to send traffic to/from one of the large 
$your_region/country incumbents via $not_your_region/country.  It's just not 
good Internet.  You make enough money already.  Be a good netizen.  It pays 
more in the long run and that's all you're really after for your shareholders 
anyway, right?

On 2013-09-08, at 11:54 AM, Derek Andrew derek.and...@usask.ca wrote:

 The topic of Canadian network sovereignty has been part of the Canadian
 conscience since the failure of CANNET back in the 1970s.
 
 Canadians citizens, on Canadian soil, already supply feeds directly to the
 NSA. Rerouting Internet traffic would make no difference.
 
 
 
 
 
 
 
 On Sat, Sep 7, 2013 at 3:08 PM, Paul Ferguson fergdawgs...@mykolab.comwrote:
 
 
 A Canadian ISP colleague of mine suggested that the NANOG constituency
 might be interested in this, given some recent 'revelations', so I
 forward it here for you perusal.
 
 
 
 Preliminary analysis of more than 25,000 traceroutes reveals a
 phenomenon we call ‘boomerang routing’ whereby Canadian-to-Canadian
 internet transmissions are routinely routed through the United States.
 Canadian originated transmissions that travel to a Canadian destination
 via a U.S. switching centre or carrier are subject to U.S. law -
 including the USA Patriot Act and FISAA. As a result, these
 transmissions expose Canadians to potential U.S. surveillance activities
 – a violation of Canadian network sovereignty.
 
 
 http://lawprofessors.typepad.com/media_law_prof_blog/2013/09/routing-internet-transmission-across-the-canada-us-border-and-us-surveillance-activities.html
 
 Cheers,
 
 - ferg
 
 
 --
 Paul Ferguson
 Vice President, Threat Intelligence
 Internet Identity, Tacoma, Washington  USA
 IID -- Connect and Collaborate -- www.internetidentity.com
 
 
 
 
 -- 
 Copyright 2013 Derek Andrew (excluding quotations)
 
 +1 306 966 4808
 Information and Communications Technology
 University of Saskatchewan
 Peterson 120; 54 Innovation Boulevard
 Saskatoon,Saskatchewan,Canada. S7N 2V3
 Timezone GMT-6
 
 Typed but not read.

Recommendations for dynamic imix traffic generators

[Jason Lixfeld]

Hi folks,

I'm trying to put together a test bench to soak some CPE equipment with an imix 
of eyeball traffic.  I'm wondering if anyone has any recommendations on 
open-source platforms that might be able to accomplish this.

I'd like to simulate traffic conditions that various tiers of Internet users 
might create from behind these CPEs - Casual user, business user, gamer, heavy 
users, netflix client, Apple TV client, a combination of any of the 
aforementioned, etc.

In a perfect world, I'd love for these traffic patterns to be dynamic; various 
pps/bps/fps/nat cps rates, various intervals of each and durations of each 
instead of just continually puking out the same set of packets in an endless 
loop.

I think it would be important for this traffic generator to be intelligent 
enough to determine whether or not the tests it's performing are successful or 
not; that is, be cognizant of errors in the tests that would translate into 
what a user would perceive to be a broken web page or a slow loading web page 
or a video freezing or a game to lock up, all of which might be attributed to a 
timeout on a DNS lookup or packet-loss or a bad NAT stack, etc.

If anyone has any ideas or experiences they can share on this type of setup, 
I'd love some feedback or advice.

icir.org has a list of tools which I'm making my way through as well.  Not sure 
what is useful for what it I'm trying to do, but I digress.

Thanks in advance.

Re: Ciena 6200 clue?

[Jason Lixfeld]

Hi,

So just for completeness - the box does support a default gateway and it was 
pretty simple to figure out once we were able to connect to it over the Web UI. 
 The professional services tech who installed this stuff basically copied data 
off of a spreadsheet and didn't really have any notion of how the thing really 
worked so he didn't really have any answers.

On 2013-07-02, at 7:30 PM, Jason Lixfeld ja...@lixfeld.ca wrote:

 So I've got a bunch of Ciena 6200 kit in, with some of their professional 
 services folks onsite, helping with the initial setup.  I know nothing of 
 this kit, other than from what I'm being told, it's pretty bleeding edge, so 
 much so that not even many people at Ciena know how to use it.
 
 The SE who's onsite is apparently claiming that there is no provision to set 
 a default gateway on the management interface.  This seems odd to me.  What 
 is more odd is that we have to buy a manual for it.  There isn't an 
 electronic version available, even.
 
 I've created an account on their portal, so when that gets approved, I'll see 
 what sort of documentation I can find, but off the top of anyone's head, does 
 anyone know how to do this default gateway thing on the management interface? 
  It's apparently been IP'd properly, so that much is working...
 
 Thanks in advance.  Sorry for the lack of content otherwise.

Ciena 6200 clue?

[Jason Lixfeld]

So I've got a bunch of Ciena 6200 kit in, with some of their professional 
services folks onsite, helping with the initial setup.  I know nothing of this 
kit, other than from what I'm being told, it's pretty bleeding edge, so much so 
that not even many people at Ciena know how to use it.

The SE who's onsite is apparently claiming that there is no provision to set a 
default gateway on the management interface.  This seems odd to me.  What is 
more odd is that we have to buy a manual for it.  There isn't an electronic 
version available, even.

I've created an account on their portal, so when that gets approved, I'll see 
what sort of documentation I can find, but off the top of anyone's head, does 
anyone know how to do this default gateway thing on the management interface?  
It's apparently been IP'd properly, so that much is working...

Thanks in advance.  Sorry for the lack of content otherwise.

AS1239 AS701 IP Transit sales folks

[Jason Lixfeld]

If there are any IP Transit sales folks[1] listening form Sprint or Verizon, 
please drop me a line off-list.

Thanks.

[1] No resellers, please.

Re: BCP38 tester?

[Jason Lixfeld]

On 2013-03-31, at 10:48 AM, Jay Ashworth j...@baylink.com wrote:

 Is there a program which users can run on an end-site workstation which
 would test whether they are being some link which is doing BCP38, or some
 related type of source-address ingress filtering?
 
 I'm hoping for something that could be downloaded by users and run, and
 try to forge a few packets to somewhere useful, which could be logged 
 somehow in conjunction with some unforged packets containing a traceroute, 
 so we could build up a database of leaky networks.
 
 On a related topic, while I know GRC Research's Steve Gibson is a bit of
 a polarizing personality, he does have a fairly sizable consumer audience,
 and might be a great distribution venue for such a thing.
 
 Or, perhaps, is there someone on here from Ookla?
 
 Patrick?  Could Akamai be persuaded to take an interest in this as a 
 research project?


From my perspective, 99% of end-users probably don't understand (or care) that 
their provider might be responsible for initiating or precipitating a DDoS 
attacks, period.  Most network operators are probably either too inexperienced 
to understand or too lazy to care.

I believe that most everyone has a CPE of some sort, whether their service is 
resi or commercial.  So, what about shifting the focus to the CPE 
manufacturers?  They bend to technology and/or market pressures by bringing 
things like NAT, Firewalls, DLNA, UPnP, IPv6 (heh), PPPoE, RFC1483, etc. to 
their respective products in to satisfy technology limitations or security 
concerns or whatever.  Why can't they help the cause by implementing some sort 
of RFC'ified BCP38 thing?

Re: BCP38 tester?

[Jason Lixfeld]

On 2013-03-31, at 9:43 PM, Peter Baldridge petebaldri...@gmail.com wrote:

 I can assume that If you are spoofing packets, resetting passwords on cpe and 
 replacing the box would be trivial.  So it's questionable how useful this is. 
  It seems like it just adds cost to for customers that can't spoof a packet 
 to save their lives.

Maybe it's useful for the people who have no idea that their computers are 
infected by bots that spoof packets.

 On Mar 31, 2013 6:37 PM, Jason Lixfeld ja...@lixfeld.ca wrote:
 
 On 2013-03-31, at 10:48 AM, Jay Ashworth j...@baylink.com wrote:
 
  Is there a program which users can run on an end-site workstation which
  would test whether they are being some link which is doing BCP38, or some
  related type of source-address ingress filtering?
 
  I'm hoping for something that could be downloaded by users and run, and
  try to forge a few packets to somewhere useful, which could be logged
  somehow in conjunction with some unforged packets containing a traceroute,
  so we could build up a database of leaky networks.
 
  On a related topic, while I know GRC Research's Steve Gibson is a bit of
  a polarizing personality, he does have a fairly sizable consumer audience,
  and might be a great distribution venue for such a thing.
 
  Or, perhaps, is there someone on here from Ookla?
 
  Patrick?  Could Akamai be persuaded to take an interest in this as a
  research project?
 
 
 From my perspective, 99% of end-users probably don't understand (or care) 
 that their provider might be responsible for initiating or precipitating a 
 DDoS attacks, period.  Most network operators are probably either too 
 inexperienced to understand or too lazy to care.
 
 I believe that most everyone has a CPE of some sort, whether their service is 
 resi or commercial.  So, what about shifting the focus to the CPE 
 manufacturers?  They bend to technology and/or market pressures by bringing 
 things like NAT, Firewalls, DLNA, UPnP, IPv6 (heh), PPPoE, RFC1483, etc. to 
 their respective products in to satisfy technology limitations or security 
 concerns or whatever.  Why can't they help the cause by implementing some 
 sort of RFC'ified BCP38 thing?

ATT AM Director for New York

[Jason Lixfeld]

To any ATT sales folks listening, my account manager has not been very good at 
following up on multiple requests to quote increases in my IP transit commits.  
If any account manager directors or other tiers who these AMs might report to 
could contact me, I'd love to give you their name so you can light a fire under 
their ass.

Thanks in advance.

Re: L3 East cost maint / fiber 05FEB2012 maintenance

[Jason Lixfeld]

I got notification of their maintenance window, albeit with  24 hours notice.  
Notice came in at 11:00GMT-5 yesterday, maintenance was scheduled for 
00:00GMT-5 this morning.

That said, the notice said that the maintenance was in Phoenix but I got a 
notice about my IPT circuit at 60 Hudson which I found confusing.

Based on my logs, our BGP session with them went down at 03:06GMT-5 and back up 
at 03:15GMT-5.  Down again at 03:37GMT-5 until 04:20GMT-5.  A third time at 
06:41GMT-5 and back at 06:45GMT-5.

Traffic graphs tell a bit of a different story.  Just before 05:00GMT-5, our 
outbound traffic to Level 3 dropped substantially.  About that time, I started 
getting reports about issues to Level 3 destinations.  Traces seemed to 
indicate a black hole condition within Level 3's network in NYC, seemingly at, 
or just past csw3.NewYork1.Level3.net.  Stuff seemed to correct itself by about 
06:45GMT-5, but due to Level 3 sending only about 180k routes.  About 20 
minutes later, the table was back to ~431K and all's been fine since.

On 2013-02-05, at 10:39 AM, Josh Reynolds ess...@gmail.com wrote:

 I know a lot of you are out of the office right now, but does anybody have
 any info on what happened with L3 this morning? They went into a 5 hour
 maintenance window with expected downtime of about 30 minutes while they
 upgraded something like *40* of their core routers (their words), but
 also did this during some fiber work and completely cut off several of
 their east coast peers for the entirety of the 5 hour window.
 
 If anybody has any more info on this, on a NOC contact for them on the East
 Coast for future issues, you can hit me off off-list if you don't feel
 comfortable replying with that info here.
 
 Thanks, and I hope hope you guys are enjoying Orlando.
 
 -- 
 *Josh Reynolds*
 ess...@gmail.com - (270) 302-3552

Validation of FCS

[Jason Lixfeld]

Hi all, 

I'm trying to confirm (or debunk) my current understanding of FCS errors.  An 
FCS error is a layer 2 error.  In Ethernet spake, the 4 bytes of FCS data 
within each Ethernet frame is validated by a CRC check, which is done by the 
device receiving said frame.  If the CRC check fails, an FCS error is reported 
by that receiving device.

If that understanding is true and presuming a circuit was made up of many 
layer 2 devices between the A and Z side of said circuit, it would be 
impossible for a CRC error somewhere along the path of that circuit to register 
on the receiving device of either the A or Z side.  Perhaps in simpler terms, a 
CRC error is a localized thing and would never be forwarded from one device to 
another.

Is that fair and/or accurate?

Thanks in advance.

Re: Validation of FCS

[Jason Lixfeld]

On 2012-12-19, at 10:02 AM, Saku Ytti s...@ytti.fi wrote:

 On (2012-12-19 09:53 -0500), Jason Lixfeld wrote:
 
 Perhaps in simpler terms, a CRC error is a localized thing and would
 never be forwarded from one device to another.
 
 It would be forwarded in cut-through switching.

... until the bad frame reached the first store-and-forward switch (or most any 
router) which would log the FCS error, correct?

Procera Networks contact

[Jason Lixfeld]

If anyone has a contact at Procera Networks who can answer some technical 
questions about their product, could you please pass it along?  The suggested 
methods at www. have so far gone unanswered.

Thanks in advance.

Rate shaping in Active E FTTx networks

[Jason Lixfeld]

Hi all,

I'm trying to gauge what operators are doing to handle per-subscriber Internet 
access PIR bandwidth in Active E FTTx networks.  

I presume operators would want to limit the each subscriber to a certain PIR, 
but within that limit, do things like perform preferential treatment of 
interactive services like steaming video or Skype, etc., ahead of 
non-interactive services like FTP.

My impression is that a subscriber's physical access in these networks is 
exponentially larger than their allocated amount of Internet access.  This 
would leave ample room on the physical access access for other services like 
Voice and IPTV that might run on separate VLANs than the Internet access VLAN. 
That said, I doubt there's really that much of a concern about allocating PIR 
on these other service VLANs.

So in terms of PIR for Internet access, is there some magic box that sits 
between the various subscriber aggregation points and the core, which takes 
care of shaping the subscriber's Internet access PIR, while making sure that 
the any preferential treatment of interactive services is performed.

Is that a lot to ask for one box?  The ridiculously deep buffers required in 
order to shape to PIR vs. police to it (because policing to a PIR is just plain 
ugly) and the requirements to perform any sort of preferential packet treatment 
above and beyond that seem like quite a lot to ask of one box.  Am I wrong?

Who might make a box like this, if it exists?  And if not, what are folks using 
the achieve these results?

Thanks in advance for any insights..

AS209/CenturyLink NOC email?

[Jason Lixfeld]

Anyone from AS209/CentryLink around to troubleshoot some routing weirdness?  If 
not, anyone have a NOC email address for them?  Google-fu and RADB searches 
came up empty.

Thanks in advance.

Re: Network Traffic Collection

[Jason Lixfeld]

Splunk is an amazing tool and did an awesome thing and introduced a free 
license in 4.3.

I'm using it at two sites now and I'm loving it!

On 2012-02-23, at 3:34 PM, Mike Lyon wrote:

 Random thought, anyone ever used Splunk for this kind of thing?
 
 -mike
 
 Sent from my iPhone
 
 On Feb 23, 2012, at 10:30, Suresh Rajagopalan sraj...@gmail.com wrote:
 
 On Thu, Feb 23, 2012 at 12:19 PM, Maverick myeaddr...@gmail.com wrote:
 I want to be able to see information like how much traffic an ip send
 over a period of time, what machines it talked to etc from this
 perspective it should be IP based but I would really like to know how
 other people do it.
 
 
 
 Run argus on a span port.
 
 -Suresh
 
 

Re: Overall Netflix bandwidth usage numbers on a network?

[Jason Lixfeld]

On 2011-12-12, at 4:22 PM, Simon Lockhart si...@slimey.org wrote:

 I guess most (i.e. those
 which aren't Akamai) are more concerned with making money than with delivering
 a good service to the end user.

Really?  I always thought that higher profits and buying transit were mutually 
exclusive relative to higher profits and openly peering.

So what you are saying is that one stands to make more by paying upstreams for 
bit swapping?  How's that work?

If the argument is that the opex required for maintaining peering relationships 
is too expensive relative to the direct and indirect cost of buying bandwidth, 
I love to be edumacated on how that math actually works because it makes 
absolutely no sense to me.

--

Sent from my mobile device

Re: Recommendation for customer monitoring network tool/portal for a large ISP

[Jason Lixfeld]

We've just deployed Intermapper to do all of our device polling, link status 
and topology mapping.  Works very well and looks real pretty.

For graphing, we use cacti with the Discovery and Autom8 plugins.

For SNMP trap parsing, we use SNMPTT.

We're currently evaluating Splunk to eat the SNMP trap and syslog data from our 
gear and do cool stuff with it.

Last on my list of tools to try is Cisco NCM as a replacement for RANCID.  
RANCID is amazing, but when we have hundreds of devices with exactly the same 
base configs on them, something a little more sophisticated than RANCID is 
required to keep all of those configs in sync.

On 2011-10-27, at 4:45 AM, Alex Nderitu wrote:

 Hello,
 What solutions do you guys in the fixed network business/ISPs use to provide 
 customer portals for network KPI reporting to customers in a fixed network on 
 real time basis. The KPI in question are network availability, utilization, 
 memory/cpu of managed routers/firewall, jitter, packet loss etc in a multi 
 vendor environment.
 
 
 What would you recommend especially in the licensed/supported options and not 
 the free ones like Zabbix, Cacti, MRTG etc. This solution should scale well 
 for hundreds of thousand of clients.
 
 We have been using Orion NPM and it pretty much does the job but would wish 
 to move to something more scalable for SP environment.
 
 Regards,
 Alex.
 
 
 

Re: events

[Jason Lixfeld]

On 2011-09-30, at 2:13 PM, Brandon Kim wrote:

 I've been happy with my basic ManageEngine's syslog, but I may be looking at 
 Solarwinds too...

I've just installed the Splunk eval myself, but I'm curious about your 
ManageEngine experiences.  I don't have any interest in using ManageEngine as 
an NMS; I have a couple of tools that I use for that already.  Can you use 
ManageEngine's syslog without having to set it up to monitor all of your 
devices first?  Have you looked at the TRAP support in ManageEngine?

FTTH CPE landscape

[Jason Lixfeld]

This isn't necessarily operational content, so I apologize in advance for the 
noise and thus encourage off-list replies (and/or flames).

I figure the NANOG demographic might be able to point me in the right direction 
seeing as how far reaching into the industry the readership is.

I'm doing research on potential FTTH CPE vendors and I'd like to poke around 
for some potential vendors to see who I've missed.

The feature wish list more or less looks like so:

- Small, wall-mount'ish form factor
- 6-8 wire speed 10/100/1000 LAN ports
- Generic consumer grade NAT/Firewall
- Fixed BX WAN port
- 1-2 POTS ports with SIP UA
- TR-69 support for full CPE configuration (User features/configuration and SP 
features/configuration)
- No Wifi (or the ability to disable it from the SP provisioning side)
- DHCP client
- 802.1q on LAN and WAN ports
- Multicast
- -48v input
- Per VLAN egress shaping/policing over WAN port
- DHCP option 82 support

If anyone has something like this in the field or knows of a vendor who can 
meet these requirements in some fashion by product line or custom build, please 
drop me a line.

Also, if anyone knows of any NANOG'esque FTTH lists, I'd welcome a subscribe 
URL.

Thanks in advance.

Re: FTTH CPE landscape

[Jason Lixfeld]

Nope, Ethernet.

--

Sent from my mobile device.

On 2011-08-04, at 6:10 PM, Frank Bulk frnk...@iname.com wrote:

 Are you looking for an xPON ONT?
 
 Frank
 
 -Original Message-
 From: Jason Lixfeld [mailto:ja...@lixfeld.ca] 
 Sent: Thursday, August 04, 2011 9:58 AM
 To: nanog@nanog.org
 Subject: FTTH CPE landscape
 
 This isn't necessarily operational content, so I apologize in advance for
 the noise and thus encourage off-list replies (and/or flames).
 
 I figure the NANOG demographic might be able to point me in the right
 direction seeing as how far reaching into the industry the readership is.
 
 I'm doing research on potential FTTH CPE vendors and I'd like to poke around
 for some potential vendors to see who I've missed.
 
 The feature wish list more or less looks like so:
 
 - Small, wall-mount'ish form factor
 - 6-8 wire speed 10/100/1000 LAN ports
 - Generic consumer grade NAT/Firewall
 - Fixed BX WAN port
 - 1-2 POTS ports with SIP UA
 - TR-69 support for full CPE configuration (User features/configuration and
 SP features/configuration)
 - No Wifi (or the ability to disable it from the SP provisioning side)
 - DHCP client
 - 802.1q on LAN and WAN ports
 - Multicast
 - -48v input
 - Per VLAN egress shaping/policing over WAN port
 - DHCP option 82 support
 
 If anyone has something like this in the field or knows of a vendor who can
 meet these requirements in some fashion by product line or custom build,
 please drop me a line.
 
 Also, if anyone knows of any NANOG'esque FTTH lists, I'd welcome a subscribe
 URL.
 
 Thanks in advance.
 

Re: Verizon Issues? East Coast US

[Jason Lixfeld]

On 2011-08-03, at 3:50 PM, Mike Tancsa wrote:

 On 8/3/2011 3:31 PM, James Smallacombe wrote:
 
 
 However, I AM seeing problems right now as described below...anybody
 aware of any Verizon issues?
 
 I was told by TATA one of their core routers in NY is not reachable. So
 perhaps some inadvertent black hole routing between them / by them.

Do you have a ticket number, Mike?  Seems like they are still blackholing 
traffic.

Community troubleshooting étiquette/BCP (was: L3 Issues)

[Jason Lixfeld]

On 2011-08-01, at 1:48 PM, Jon Lewis wrote:

 Things seem to be moving again.

I happen to have an L3 link out of NYC, but unfortunately I don't have a list 
of on-net L3 prefixes in any of the reportedly affected regions, so I'm unable 
to provide any data from my vantage point up here.  I'm sure others are in my 
position as well.

Is there any sort of etiquette/BCP for reporting issues like this to the 
community?  Something that might specify a method of providing information a 
little more specific than just specifying the affected region(s)?  Maybe a list 
of a few affected hosts/prefixes/URLS/etc?

(incidentally, images.apple.com also resolves to our local Akamai cluster)

Re: Cisco IOS MPLS VPN Bug

[Jason Lixfeld]

On 2011-03-12, at 2:31 AM, Joe Renwick wrote:

 These routers
 are configured as BGP route-reflectors.

...

 Niether
 soft nor hard clears on the BGP neighbors worked, only the config removal.
 Once re-applied life was good.

...

 The bug itself was with the BGP updates sent by the RR.  During the outage
 these updates did not include the Route Target Extended Community required
 by the route-reflector clients which identifies which VRF the route belongs
 too.

...

 Notice the mysterious disappearance of the RT community.

...

 Looking to see if anyone has seen this issue particularly with this version
 of code.  TAC is trying to tell me that this was a bug in a previous version
 but is fixed in the code I am running.

Interesting.  I recently closed off a TAC case on a similar issue, but not an 
identical issue.  In my case, it was 12.2(52)EY on an ME3600 and in my 
particular topology, an ME3600 wasn't announcing a plain ol' BGP community to 
one of it's two RRs.  The extended communities were fine tho.  Also, the 
announcements were being stuffed into two different update groups; the ME that 
was sending the 'good' announcement was announcing updates to update-group 1 
and 2 and the ME that was announcing the 'bad' announcement was announcing 
updates to update-group 1 only.

We didn't spend as much time as you clearly have troubleshooting the issue 
because we caught it before it was customer affecting.  That said, at the time, 
I noticed the same thing; hard clearing the sessions didn't fix it. I didn't 
try to unconfigure the neighbour though; in my case, I was running EY on this 
switch and because the ME3600s are so new and EY1 was available and I knew that 
I'd have to reboot anyway to clear the issue, I decided to upgrade to EY1 and 
that seemed to clear up the problem.

I haven't seen this resurface since.  EY1 was available as soon as we started 
receiving our ME3600s, so as a policy we upgraded every one before it went into 
the field, except I had missed this one in particular.

There were no open bugs pointing to my issue that the TAC engineer could find, 
but if you could pass me the case number, I'd like to give it to my engineer so 
he can see if your issue is somehow related to mine, just manifested in a 
slightly different way.

Any ATT IP Transit sales folks listening?

[Jason Lixfeld]

I'm looking for an ATT IP Transit sales contact.  Email links on the website 
don't seem to work and I was on hold for 30 minutes with an auto-attendant when 
I tried to call.

I'm looking for transit on a 1Gbps access out of TelX @ 60 Hudson.

Thanks in advance.

SFP vs. SFP+

[Jason Lixfeld]

I was asked today what the difference between SFP and SFP+ is.  I did really 
know, so I looked it up and it seems that the SFP spec provides capabilities 
for data rates up to 4.25Gb/s, whereas SFP+ supports up to 10Gb/s.  Naturally, 
this made me wonder whether or not an optic that supported 10GbE always 
conformed to the SFP+ standard inherently, or if there are cases where a 10GbE 
optic might only support the SFP standard, thus having a 4.25Gb/s bottleneck.

Re: Packet over SONET failback

[Jason Lixfeld]

On 2011-02-14, at 6:47 AM, Rob Evans wrote:

 PoS failure detection happens in under 50ms, but what about the failback?  
 Same deal?  I ask because I've got two routers connected to opposite ends of 
 a spare PoS link that I've been playing with and I'm noticing that the 
 failback on the far side seems to be about 15 seconds (assuming the near 
 side failover was initiated with an interface shutdown command and thusly no 
 shut'd to re-enable the link).
 
 I think there are a couple of issues at play here.  First of all,
 SONET/SDH restoration happens at layer 1, whereas it looks like you're
 waiting for a router to reroute.  Your reroute times will be tied to
 recalculation of IGPs.
 
 Secondly, is this with a Cisco?  Try setting pos ais-shut on both
 sides.  Unless you do that, the router won't generate and AIS, and it
 will take the encapsulation timeout (HDLC, PPP) for the interface on
 the other side to go down and signal that to the routing protocols on
 top.

It's becoming clear from the responses that my initial post was lacking a bit 
of info.

It's not actually SONET all the way through.  It's GigE from the router to the 
SONET node, an unprotected OC192 wave to another node, out GigE to the far end 
router.

I didn't mention that because I didn't think it mattered; I assumed the speed 
in which the GigE on the other side came up was protocol agnostic.  I assumed 
that as soon as the far end of the SONET node saw the near end go back up, it 
would turn the laser back on on the far side bringing the interface back up.  I 
assumed also that since it took a whole 15 seconds that it had something to do 
with the SONET side of things.

I'm not necessarily talking about forwarding of packets here, I'm talking 
simply about the time it takes the far side interface to come back up over a 
SONET node; layer 1.  Maybe this has nothing at all to do with SONET, I dunno 
:)   It was gleaned that the next step might be to look at the SONET node and 
see if it's waiting 15 seconds to turn the laser back on or something.

Packet over SONET failback

[Jason Lixfeld]

PoS failure detection happens in under 50ms, but what about the failback?  Same 
deal?  I ask because I've got two routers connected to opposite ends of a spare 
PoS link that I've been playing with and I'm noticing that the failback on the 
far side seems to be about 15 seconds (assuming the near side failover was 
initiated with an interface shutdown command and thusly no shut'd to re-enable 
the link).  Just wanted to know if a higher failback time is a relatively 
normal occurrence and maybe I'm seeing some sort of built-in hold down feature 
working away?

Re: Good MPLS/VPLS book?

[Jason Lixfeld]

While on a MPLS related TAC case recently, I was speaking to an engineer who 
helped vet portions of Cisco Press' MPLS Fundamentals 
(http://www.ciscopress.com/bookstore/product.asp?isbn=1587051974).  He said 
it's one of the best he's come across, but there may perhaps be some bias there 
;)  Not knowing any better, I picked it up and I'm learning quite a bit.  It's 
also seems to be a great reference manual to keep around too.  The Kindle 
version is handy for quick reference from mobile devices too.

On 2010-12-23, at 5:49 PM, Michael Helmeste wrote:

  Does anyone have a favorite book or resource discussing MPLS and all 
 associated Lego blocks (e.g. LDP, TE, VPLS, martini, mBGP et. al.)?
 
  I understand the basics of what MPLS is and how you create a circuit from
 A to B but I'm afraid it still escapes me when trying to figure out how 
 someone would, say, create a multicast capable VPN with 5 edge points.
 
  Any pointers to a good way to reduce my level of ignorance on this subject 
 would be appreciated. Vendor literature doesn't bother me as long as the 
 concepts are there.
 
  Regards,
Michael H.
 
 

Re: Recommendations for Metro-Ethernet Equipment

[Jason Lixfeld]

On 2010-10-20, at 11:24 AM, Eric Merkel wrote:

 Any suggestions, success or horror stories are appreciated. ;)

I've been going through pretty much the same exercise looking for a decent PE 
for almost two years.  Our requirements were for a PE device that had between 
12-24 ports (in a perfect world, mixed mode 10/100/1000 copper + SFP), 10G 
uplinks, EoMPLS, MPLS VPN, DHCP server, port-protect/UNI (or similar) 
capabilities, DC power and a small footprint (1RU)

Of all the ones we looked at (Juniper, Cisco, Extreme, Brocade, MRV, Alcatel) 
initially, MRV was the only contender.  The rest either didn't have a product, 
or their offering didn't meet various points within our criteria.

As such, we bought a bunch of MRVs in early 2009 and after four months of trial 
and error, we yanked every single one out of the network.  From a physical 
perspective, the box was perfect.  Port density was perfect, mixed-mode ports, 
promised a 10G uplink product soon, size was perfect, power was perfect, we 
thought we had it nailed.  Unfortunately there are no words to describe how 
terrible the software was.  The CLI took a little getting used to, which is 
pretty much par for the course when you're dealing with a new vendor, but the 
code itself was just absolutely broken, everywhere.  Duplex issues, LDP 
constantly crashing taking the box with it, OSPF issues, the list went on and 
on.  To their credit, they flew engineers up from the US and they were quite 
committed to making stuff work, but at the end of the day, they just couldn't 
make it go.  We pulled the plug in May 2009 and I haven't heard a thing about 
their product since then, so maybe they've got it all together.

While meeting with Juniper a few months later about a different project, they 
said they had a product that might fit our needs.  The EX4200.  As such, we had 
a few of these loaned to our lab for a few months to put through their paces, 
from a features and interoperability perspective.  They work[1] and they seem 
to work well.  The show stopper was provisioning[1] and size.  The box is 
massive, albeit it is still 1U.

[1] (I'm not a Juniper guy, so my recollection on specific terms and jargon may 
be a bit off kilter) they only support ccc, which makes provisioning an 
absolute nightmare.  From my experience with Cisco and MRV, you only have to 
configure the EoMPLS vc.  On the EX4200, you have to create the LSPs as well.  
To get a ccc working, the JunOS code block was far larger and much more 
involved per vc than the single line Cisco equivalent.  To create the LSPs was, 
I believe, two more equally large sized code blocks.  At the end of the day, it 
was just too involved.  We needed something simpler.

About the same time that we started to evaluate the EX4200, Cisco had pitched 
us on their (then alpha) Whales platform.  It looked promising (MRV still had 
the best form factor) and we expressed our interest in getting a beta unit in 
as soon as we were able to.  This is now known as the ME3600 and ME3800 
platform and we've been testing a beta unit in our lab for the past few months. 
 This is the platform we have chosen.  It's not perfect, but our gripes have 
more to do with form factor (it's 1RU, but it's a bit deeper than what we'd 
like) and port densities (no mixed mode ports) than software or features.  
We've been pretty pleased with it's feature set and performance, but this 
hasn't seen any real world action, so who knows how that will turn out.

If you're asking more about a P router or P/PE hybrid, we've also just ordered 
a few ASR9000s under try-and-buy as P/PEs to close up the chains of ME3600s 
that will start to be deployed in our remote sites.  A Juniper MX would 
certainly work well here too, and it seems to interoperate rather well with the 
ME3600s, so that's certainly an option, but for us, we think it will work more 
in our favor to go with the ASRs in the core, but if not, we'd ship them back 
under the try-and-buy and get Junipers instead.

Hope that helps.