Re: Out of ideas - Comcast issue BGP peering with Tata

[jim deleskie]

I many years ago worked at Tata, responsible for their BGP, they are giving
you the right answer, Comcast has to be the one contacting them, as then
both sides can see what is being sent and received and can resolve this
issue.

-jim

On Fri, Nov 17, 2023 at 10:04 AM Jamie Chetta via NANOG 
wrote:

> I am out of ideas on how to get this fixed.  Long story short I am a
> customer of Comcast and am advertising my own /24 block I own through
> them.  Comcast of course BGP peers with multiple ISPs.  Other ISPs are
> accepting my prefix just fine, except Tata.  This is causing random
> destinations to drop connectivity if Comcast routes it through them.
> Comcast has confirmed they are advertising my block to Tata and that the
> RPKI is good, however when you check the Tata looking glass you can see
> they’re not accepting it.
>
>
>
> I’ve tried escalating within Comcast who refuses to contact Tata as
> they’ve validated the issue is not on their end but they agree with my
> assessment that Tata is not accepting the prefix for some reason.
>
>
>
> I’ve tried multiple email for Tata support (below), but they all circle
> around to a helpdesk who says I do not have a circuit with them so they
> cannot help me.
>
>
>
> Is there anyone from Tata willing to contact me off list to help sort this
> out?  Or anyone with ideas on specifically why other ISPs are accepting my
> route but not Tata?  It would be greatly appreciated.
>
>
>
> Emails I’ve tried
>
> Corporate  Helpdesk corp.helpd...@tatacommunications.com
>
> Tata Communications IP Service Support( AS-6453)
> ipservicesupp...@tatacommunications.com
>
> IPNOC (Tata Communications - AS6453) ip...@tatacommunications.com
>
> l...@as6453.net
>
>
> Response from Tata:
>
> “Acknowledge your email.
>
>
>
> However, since you are not associated with TCL we would not be in a
> position to help you on this.
>
>
>
> Request you to contact comcast for the assistance that you are seeking
> from us.”
>
>
>
> Response from Comcast:
>
> “This was sent back to me as not us. Basically, it’s not a RADB or RPKI
> issue. This is being accepted and re-advertised to TATA but not being
> accepted on their end. But another route that we checked off of that same
> SUR is being advertised the same way and accepted by them off
> pe12.350ecermak.il.ibone as an example of the TATA looking glass.  I would
> suggest that you would probably need to work with other networks as to why
> those that are specific ones are not accepting the block but as previously
> mentioned it’s not a RADB or RPKI issue and as a result not a Comcast
> issue.”
>

Re: TATA Communications

[jim deleskie]

Have you tried NOC not sure who from their actively monitors the list
anymore?  Forwarding to a former colleague.

-jim

On Mon, Dec 12, 2022 at 2:49 PM Norman Jester  wrote:

> Contact me off list... seeing major loss at 64.86.252.65 in your path.
>
> Norman Jester
> 619-319-7055
>

Re: Understanding impact of RPKI and ROA on existing advertisements

[jim deleskie]

I dont think ive every agreed with Owen this much, maybe this is the first
sign the wording is ending further proving his statement :)

On Wed, Nov 2, 2022 at 10:30 PM Owen DeLong via NANOG 
wrote:

> Oh, I’m not ignoring it, I’m just rather underwhelmed by it and given how
> long it took SIDRWG to get RPKI this far,
> not optimistic about any of the rest of the system getting deployed prior
> to IPv6 ubiquity or the end of my time on
> this planet, or even before we manage to destroy the planet, whichever
> comes first.
>
> Owen
>
>
> > On Nov 2, 2022, at 08:30, heasley  wrote:
> >
> > Tue, Nov 01, 2022 at 06:24:50PM -0700, Owen DeLong via NANOG:
> >> RPKI/ROA is a way to cryptographically prove what someone needs to
> prepend if they want to hijack your addresses.
> >
> > Operators should not be deterred by that comment.  Owen seems to be
> ignoring
> > what it does achieve and that this is part of a larger system that is
> still
> > emerging.  See IETF sidrops wg.  In the interim, do your part to improve
> > DFZ hygiene.
> >
> >> Owen
> >>
> >>
> >>> On Oct 28, 2022, at 08:00, Samuel Jackson 
> wrote:
> >>>
> >>> Hello,
> >>> I am new to RPKI/ROA and still learning about RPKI. From all my
> reading on ARIN's documents I am not able to answer some of my questions.
> >>> We have a public ARIN block and advertise smaller subnets from that to
> our ISP's. We do not have any RPKI configs.
> >>> We need to setup ROA's to take another subnet from the ARIN block to
> AWS. Reading ARIN's docs, it seems I need to get setup on their Hosted RPKI
> service after which I can configure ROA's for the networks I am taking to
> AWS.
> >>>
> >>> My question is, will this impact my existing advertisements to my
> ISP's. The current advertisements do not have ROA's.
> >>> Will having RPKI for my ARIN network, without ROA's for the existing
> advertisements impact me?
> >>>
> >>> Thanks for your help.
> >>>
> >>> Ref:
> >>> https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-byoip.html
> >>> https://www.arin.net/resources/manage/rpki/roa_request/
> >>> https://www.arin.net/resources/manage/rpki/hosted/
> >>
>
>

Re: AKAMAI Contact

[jim deleskie]

Seriously search the list people.  Even a little effort on your own.   Same
question a few days ago.

-jim

On Wed, Sep 28, 2022, 3:45 PM Joshua Pool via NANOG  wrote:

> Anyone have a contact for AKAMAI?
>
> Thanks in advance.
>
> Josh
>

Re: Rogers Outage Canada

[jim deleskie]

i cant see BGP taking out SS7.

-jim

On Fri, Jul 8, 2022 at 2:45 PM Snowmobile2004 
wrote:

> According to Cloudflare Radar
> , Rogers
> BGP announcements spiked massively to levels 536,777% higher than normal
> (343,601 vs 64 normally) just minutes before the outage. I would not be
> surprised if this happened to be the culprit.
>
> Regards,
> Josh Green
>
> On Fri, Jul 8, 2022 at 2:19 PM Andrew Paolucci via NANOG 
> wrote:
>
>> In the early hours of the morning around 2-3am my modem got hit with a
>> configuration update that caused a DHCP release that wasn't renewed for
>> about two hours, after rollback the connection was fine for 3 hours before
>> this network wide outage.
>>
>>
>> Maybe a failed night time update was attempted again during office hours,
>> I've heard daytime guys are still WFH and night shift is in building.
>>
>>
>> I expect we'll never get a real explanation. Rogers is notorious for
>> withholding any type of helpful or technical information.
>>
>>
>> Sent from my inoperable Rogers Mobile via emergency eSIM.
>>
>>
>> Regards,
>>
>> Andrew Paolucci
>>  Original Message 
>> On Jul. 8, 2022, 1:48 p.m., Jay Hennigan < j...@west.net> wrote:
>>
>>
>> On 7/8/22 07:44, Robert DeVita wrote: > Does anyone have information on a
>> widespread Rogers outage in Canada. I > have customers with multiple sites
>> down. There's discussion on the Outages mailing list. Seems widespread,
>> affecting all services, mobile, voice, Internet. No cause or ETR posted
>> yet. -- Jay Hennigan - j...@west.net Network Engineering - CCIE #7880 503
>> 897-8550 - WB6RDV
>>
>>
>
> --
> *Josh Green.*
>

Re: 10 Do's + Don'ts for Visiting Québec + Register Now for N85!

[jim deleskie]

Having lived in and continue to spend as much time in Montreal as I can.
This list made be laugh, especially for a group where most of us do a lot
of travel.

Other then no right on red.  Montreal like any other city.  Don't be an ass
and enjoy yourself.



On Thu, May 5, 2022, 9:56 AM Nanog News  wrote:

> *10 Do's + Don'ts for Visiting Québec*
> *NANOG 85 Meeting Will Take Place Jun. 6 - 8 in Montréal*
>
> We are delighted to cross international borders in our mission to grow,
> inspire + profoundly build the Internet of tomorrow!
>
> Montréal is Canada's second-largest city and is known for its melting pot
> of diverse culture, established universities, enthralling art, food,
> history + festivals. It has been called one of the world's "happiest
> locations" as an estimated 45,000 immigrants relocate to the city every
> year.
>
> For those who don't call Québec home, we have prepared a list of cultural
> "Do's and Don'ts" to help you quickly acclimate + thrive in this foreign
> destination.
>
> *READ MORE  *
>
> *Register for NANOG 85 Today!*
>
> Join us in person or virtually for NANOG 85. Don't miss your chance to
> experience hours of ground-breaking industry talks, a legendary keynote
> speaker, opportunities for networking + more.
>
> *REGISTER NOW *
>
>
>

Re: Opinions on Arista for BGP?

[jim deleskie]

I did an eval for some folks last Aug on Arista and 2 other vendors, one of
the others decided they didn't want to play the 3rd did.

Of the 3 Arista performed better/best.  The test plan was shared with all 3
vendors prior to testing and it definitely push all this to and then past
their published limits.  Arista was the only one in 2 days I didn't break.
Use case big fast simple L3 BGP router.

-jim

On Thu, Mar 31, 2022 at 10:11 AM David Hubbard <
dhubb...@dino.hostasaurus.com> wrote:

> Hi all, would love to get any current opinions (on or off list) on the
> stability of Arista’s BGP implementation these days.  Been many years since
> I last looked into it and wasn’t ready for a change yet.  Past many years
> have been IOS XR on NCS5500 platform and Arista everywhere but the edge.
> I’ve been really happy with them in the other roles, so am thinking about
> edge now.  I do like and use XR’s RPL, and prefix/as/community/object sets,
> but we can live without via our own config management if there aren’t easy
> equivalents.  No fancy needs at all, just small web server networks, so
> just need reliable eBGP and internal OSPF/OSPFv3.
>
>
>
> Thanks,
>
>
>
> David
>

Re: IPv6 Only - was Re: Let's Focus on Moving Forward Re: V6 still not supported re: 202203261833.AYC

[jim deleskie]

If then industry still hasn't adopted v6 full in 25 years maybe it's v6
that should be given up it, that it clearly wasn't what customers wanted.
Perhaps we should should have a small group working on the next iteration.

-jim

On Tue, Mar 29, 2022, 5:54 PM Jacques Latour  wrote:

> So, in 25, 50 or 100 years from now, are we still going to be dual stack
> IPv4/IPv6?
>
> When are we going to give up on IPv4?
>
> People can run IPv4 all they want inside their networks for 1000s of years.
>
> What will it take to be IPv6 only?
>
>
>
> 
>
>
>
> *From:* NANOG  *On Behalf
> Of *Owen DeLong via NANOG
> *Sent:* March 29, 2022 3:52 PM
> *To:* Abraham Y. Chen 
> *Cc:* NANOG 
> *Subject:* [EXT] Re: Let's Focus on Moving Forward Re: V6 still not
> supported re: 202203261833.AYC
>
>
>
> Submit an Internet draft, same as any other IP related enhancement gets
> introduced.
>
>
>
> What you’re really complaining about is that it’s been virtually
> impossible to gain consensus to move anything IPv4 related forward in the
> IETF since at least 2015.
>
>
>
> Well… It’s a consensus process. If your idea isn’t getting consensus, then
> perhaps it’s simply that the group you are seeking consensus from doesn’t
> like your idea.
>
>
>
> Your inability to convince the members of the various working groups that
> your idea has merit isn’t necessarily a defect in the IETF process… It
> might simply be a lack of merit in your ideas.
>
>
>
> Owen
>
>
>
>
>
> On Mar 26, 2022, at 15:43 , Abraham Y. Chen  wrote:
>
>
>
> Hi, Justin:
>
>
>
> 1)"... no one is stopping anyone from working on IPv4 ... ":
> After all these discussions, are you still denying this basic issue? For
> example, there has not been any straightforward way to introduce IPv4
> enhancement ideas to IETF since at least 2015. If you know the way, please
> make it public. I am sure that many are eager to learn about it. Thanks.
>
>
>
> Regards,
>
>
>
>
>
> Abe (2022-03-26 18:42)
>
>
>
>
>
>
>
>
>
> On 2022-03-26 11:20, Justin Streiner wrote:
>
> While the Internet is intended to allow the free exchange of information,
> the means of getting that information from place to place is and has to be
> defined by protocols that are implemented in a consistent manner (see: BGP,
> among many other examples).  It's important to separate the ideas from the
> plumbing.
>
>
>
> That said, no one is stopping anyone from working on IPv4, so what
> personal freedoms are being impacted by working toward deploying IPv6, with
> an eye toward sunsetting IPv4 in the future?
>
>
>
> Keep in mind that IPv4 started out as an experiment that found its way
> into wider use.  It's a classic case of a test deployment that suddenly
> mutated into a production service.  Why should we continue to expend effort
> to perpetuate the sins of the past, rather work toward getting v6 into
> wider use?
>
>
>
> Is IPv6 a perfect protocol?  Absolutely not, but it addresses the key pain
> point of IPv4 - address space exhaustion.
>
>
>
> Thank you
>
> jms
>
>
>
> On Sat, Mar 26, 2022 at 9:35 AM Abraham Y. Chen  wrote:
>
>
>
> 3)Re: Ur. Pts. 5) & 6):I believe that there is a philosophic /
> logic baseline that we need to sort out, first. That is, we must keep in
> mind that the Internet community strongly promotes "*personal freedom*".
> Assuming that by stopping others from working on IPv4 will shift their
> energy to IPv6 is totally contradicting such a principle. A project
> attracts contributors by its own merits, not by relying on artificial
> barriers to the competitions. Based on my best understanding, IPv6 failed
> right after the decision of "not emphasizing the backward compatibility
> with IPv4". It broke one of the golden rules in the system engineering
> discipline. After nearly three decades, still evading such fact, but
> defusing IPv6 issues by various tactics is the real impedance to progress,
> not only to IPv4 but also to IPv6.
>
>
>
>
>

Re: Dropping support for the .ru top level domain

[jim deleskie]

Terrible idea on so many levels.

-jim

On Mon, Mar 14, 2022, 12:30 PM Patrick Bryant  wrote:

> I don't like the idea of disrupting any Internet service. But the current
> situation is unprecedented.
>
> The Achilles Heel of general public use of Internet services has always
> been the functionality of DNS.
>
> Unlike Layer 3 disruptions, dropping or disrupting support for the .ru TLD
> can be accomplished without disrupting the Russian population's ability to
> access information and services in the West.
>
> The only countermeasure would be the distribution of Russian national DNS
> zones to a multiplicity of individual DNS resolvers within Russia. Russian
> operators are in fact implementing this countermeasure, but it is a slow
> and arduous process, and it will entail many of the operational
> difficulties that existed with distributing Host files, which DNS was
> implemented to overcome.
>
> The .ru TLD could be globally disrupted by dropping the .ru zone from the
> 13 DNS root servers. This would be the most effective action, but would
> require an authoritative consensus. One level down in DNS delegation are
> the 5 authoritative servers. I will leave it to the imagination of others
> to envision what action that could be taken there...
>
> ru  nameserver = a.dns.ripn.net
> ru  nameserver = b.dns.ripn.net
> ru  nameserver = d.dns.ripn.net
> ru  nameserver = e.dns.ripn.net
> ru  nameserver = f.dns.ripn.net
>
> The impact of any action would take time (days) to propagate.
>
>

Re: The role of Internet governance in sanctions

[jim deleskie]

I respect the people and goals here, but strongly echo Mel's statement.
This is a much larger hammer then mail filtering lists.


-jim

On Thu, Mar 10, 2022, 11:26 AM Mel Beckman  wrote:

> In my view, there is a core problematic statement in this document:
>
> “Military and propaganda agencies and their information infrastructure are
> potential targets of sanctions.”
>
> What is a “propaganda agency”. A political party? An incumbent candidate
> for re-election? The IRS? Anyone the “majority” disagrees with?
>
> Propaganda is in the eye of the beholder, and we’ve seen both sides of the
> political aisle sling this term in recent elections and legislative debates.
>
> I think it is a colossal mistake to weaponize the Internet. The potential
> for unintended consequences is huge, as is the potential for intended,
> politically-driven consequences
>
>  -mel beckman
>
> > On Mar 10, 2022, at 5:03 AM, Randy Bush  wrote:
> >
> > maybe it is just that i am sufficiently anti-authoritarian that i try
> > not to have the hubris to set myself up as the authority.  maybe that
> > in itself is hubris.
> >
> > as i was raised by someone who was a conscious objector in ww2, i can
> > not bring myself to contribute to weapons etc.  so i have donated to
> > folk such as https://razomforukraine.org/ which is focused on medical
> > support.
> >
> > randy
>

Re: Contact request AS 6453

[jim deleskie]

Have you found anyone.  Not there any more but can probably still find
someone for you.

-jim

On Thu, Jan 13, 2022, 10:11 AM Drew Weaver  wrote:

> Does anyone have a contact for AS 6453 or are there any AS 6453 folks on
> list?
>
>
>
> Seeing some routing trouble from their customers to the US.
>
>
>
> Thanks,
>
> -Drew
>
>
>

Re: Redploying most of 127/8 as unicast public

[jim deleskie]

This is actually worse than our collective progress on replacing v4 to
date.

-jim

On Wed, Nov 17, 2021 at 7:31 PM Jay R. Ashworth  wrote:

> This seems like a really bad idea to me; am I really the only one who
> noticed?
>
> https://www.ietf.org/id/draft-schoen-intarea-unicast-127-00.html
>
> That's over a week old and I don't see 3000 comments on it, so maybe it's
> just
> me.  So many things are just me.
>
> [ Hat tip to Lauren Weinstein, whom I stole it from ]
>
> Cheers,
> -- jra
>
> --
> Jay R. Ashworth  Baylink
> j...@baylink.com
> Designer The Things I Think   RFC
> 2100
> Ashworth & Associates   http://www.bcp38.info
> St Petersburg FL USA  BCP38: Ask For It By Name!   +1 727 647
> 1274
>

Re: Disaster Recovery Process

[jim deleskie]

I don't see posting in a DR process thead about thinking to use alternative
entry methods to locked doors and spreading false information.  If do
well.  Mail filters are simple.

-jim

On Tue., Oct. 5, 2021, 7:35 p.m. Niels Bakker, 
wrote:

> * deles...@gmail.com (jim deleskie) [Tue 05 Oct 2021, 19:13 CEST]:
> >World broke.  Crazy $$ per hour down time.  Doors open with a fire axe.
>
> Please stop spreading fake news.
>
> https://twitter.com/MikeIsaac/status/1445196576956162050
> |need to issue a correction: the team dispatched to the Facebook site
> |had issues getting in because of physical security but did not need to
> |use a saw/ grinder.
>
>
> -- Niels.
>

Re: Disaster Recovery Process

[jim deleskie]

World broke.  Crazy $$ per hour down time.  Doors open with a fire axe.
Glass breaks super easy too and much less expensive then adding 15 min to
failure.

-jim

On Tue., Oct. 5, 2021, 7:05 p.m. Jeff Shultz, 
wrote:

> 7. Make sure any access controlled rooms have physical keys that are
> available at need - and aren't secured by the same access control that they
> are to circumvent. .
> 8. Don't make your access control dependent on internet access - always
> have something on the local network  it can fall back to.
>
> That last thing, that apparently their access control failed, locking
> people out when either their outward facing DNS and/or BGP routes went
> goodbye, is perhaps the most astounding thing to me - making your access
> control into an IoT device without (apparently) a quick workaround for a
> failure in the "I" part.
>
> On Tue, Oct 5, 2021 at 6:01 AM Jared Mauch  wrote:
>
>>
>>
>> > On Oct 4, 2021, at 4:53 PM, Jorge Amodio  wrote:
>> >
>> > How come such a large operation does not have an out of bound access in
>> case of emergencies ???
>> >
>> >
>>
>> I mentioned to someone yesterday that most OOB systems _are_ the
>> internet.  It doesn’t always seem like you need things like modems or
>> dial-backup, or access to these services, except when you do it’s
>> critical/essential.
>>
>> A few reminders for people:
>>
>> 1) Program your co-workers into your cell phone
>> 2) Print out an emergency contact sheet
>> 3) Have a backup conference bridge/system that you test
>>   - if zoom/webex/ms are down, where do you go?  Slack?  Google meet?
>> Audio bridge?
>>   - No judgement, but do test the system!
>> 4) Know how to access the office and who is closest.
>>   - What happens if they are in the hospital, sick or on vacation?
>> 5) Complacency is dangerous
>>   - When the tools “just work” you never imagine the tools won’t work.
>> I’m sure the lessons learned will be long internally.
>>   - I hope they share them externally so others can learn.
>> 6) No really, test the backup process.
>>
>>
>>
>> * interlude *
>>
>> Back at my time at 2914 - one reason we all had T1’s at home was largely
>> so we could get in to the network should something bad happen.  My home IP
>> space was in the router ACLs.  Much changed since those early days as this
>> network became more reliable.  We’ve seen large outages in the past 2 years
>> of platforms, carriers, etc.. (the Aug 30th 2020 issue is still firmly in
>> my memory).
>>
>> Plan for the outages and make sure you understand your playbook.  It may
>> be from snow day to all hands on deck.  Test it at least once, and ideally
>> with someone who will challenge a few assumptions (eg: that the cell
>> network will be up)
>>
>> - Jared
>
>
>
> --
> Jeff Shultz
>
>
> Like us on Social Media for News, Promotions, and other information!!
>
> [image:
> https://www.instagram.com/sctc_sctc/]
> 
> 
> 
>
>
>
>
>
>
>
>  This message contains confidential information and is intended only
> for the individual named. If you are not the named addressee you should not
> disseminate, distribute or copy this e-mail. Please notify the sender
> immediately by e-mail if you have received this e-mail by mistake and
> delete this e-mail from your system. E-mail transmission cannot be
> guaranteed to be secure or error-free as information could be intercepted,
> corrupted, lost, destroyed, arrive late or incomplete, or contain viruses.
> The sender therefore does not accept liability for any errors or omissions
> in the contents of this message, which arise as a result of e-mail
> transmission. 
>

Re: S.Korea broadband firm sues Netflix after traffic surge

[jim deleskie]

Having done peering for many $big_boys_club and $small_isps, it always
comes down to politics, $$ and time.  The balance may change but end of day
its those variables and its a painful game some days.  From all sides :(


-jim

On Fri, Oct 1, 2021 at 1:07 PM Laura Smith via NANOG 
wrote:

>
> > The bad news now, is, there are plenty of many, small, local
> > and regional ISP's who are willing to do whatever it takes to
> > work with the content providers. All that's required is some
> > network, a half-decent data centre and an exchange point. Gone
> > are the days where customers clamored to sign up with Big
> > Telco.
>
> Speaking as one of those smaller ISPs willing to do whatever it takes,
> perhaps you could answer me this riddle.
>
> - PoP in one of your "half-decent data centres" ... tick.
> - Connnection to one of your "exchange point" ... tick.
> - $certain_large_cdn present on said "exchange point" ... tick.
>
> And yet .
>
> - $certain_large_cdn publishes routes on route server ? Nope.
> - $certain_large_cdn willing to establish direct peering session ? Nope.
>
> I am well aware of the "big boys club" that operates at most exchanges
> where the large networks see it beneath them to peer with (or publish
> routes for the benefit of) the unwashed masses.
>
> But I struggle to comprehend why $certain_large_cdn would effectively cut
> off their nose to spite their face ?
>

Re: do bgp optimizers think?

[jim deleskie]

Suspect for most th answer is poorly.  This is a conversation I've had with
a few people about how they could be well made

-jim

On Thu., Sep. 9, 2021, 12:45 p.m. Randy Bush,  wrote:

> to control inbound traffic, how do bgp optimizers decide how to tune
> what they announce?  slfow?  exploration?  ouija board?
>
> randy
>

Re: netflow in the core used for surveillance

[jim deleskie]

Randy,

  We all know many folks send their *flow to someone or somewhere.  In
exchange for pretty graphs for intelligence.  I suspect in many cases this
data is then reused in many cases for many purposes.  But let's not
overplay the risk here.  There would be much easier ways for rogue nations,
bad guys/good/in the middle nation to find out about dissidents, activists,
and journos than flow data. I think letting any of those people think ToR
is safe as being a much bigger risk.

-jim

Disclosures for those that don't know.  I've never worked with Team Cymru,
I do know them fairly well and believe them to be the good guys, I do
currently have a relationship with them, I do not currently work for a
large SP that sends them data.  I have worked A LOT with flow data over the
last 20 years, for large SPs, small vendors, and all things in between.

On Wed, Aug 25, 2021 at 6:15 PM Randy Bush  wrote:

> https://www.vice.com/en/article/jg84yy/data-brokers-netflow-data-team-cymru
>
> used to get dissidents, activists, and journos killed
>
> at, comcast, ... zayo, please tell us you do not do this.
>
> randy
>

Re: Cogent x RPKI

[jim deleskie]

It won't get them depeered, nor should it.  I don't currently based much
value in RPKI for BGP.

On Mon., Aug. 9, 2021, 8:43 a.m. Rubens Kuhl,  wrote:

> From a Cogent support ticket:
> "Hello,
>
> Please see the attached LOA.
>
> Regarding the RPKI ROA, for now, we don't create ROA for our prefixes
> nor for prefixes that we assign to our customers and we don't plan to
> do it. Unfortunately, this is not an option."
>
> Someone that poses as a Tier-1 and doesn't even plan to sign their
> announcements ? How much more depeering will make them reconsider ?
>
> Rubens
>

Re: Any2 LAX

[jim deleskie]

Also saw a major traffic drop. There is a Root Cause to be issued early in
the week I'm told.


-jim

On Fri, Jun 11, 2021 at 2:42 PM Siyuan Miao  wrote:

> Yea, it was down but both RS are online and feeding us unreachable
> nexthops during the outage .
>
> On Sat, Jun 12, 2021 at 1:27 AM Seth Mattinen  wrote:
>
>> On 6/11/21 10:16 AM, Jon Lewis wrote:
>> > On Fri, 11 Jun 2021, Seth Mattinen wrote:
>> >
>> >> Did Any2 LAX barf last night between about 1am and 8am Pacific time?
>> >
>> > More like 00:00-7:45 (Pacific time).
>> >
>> > Anyone know what broke, and why the IX was dead for nearly 8 hours?
>> > This is our second recent issue with "an Any2 IX", having dealt with an
>> > IX partition event at Any2 Denver just a few weeks ago.
>> >
>>
>>
>> What I saw was a lot of unreachable nexthops (I'm in LA2) on routes
>> advertised through the route servers. Most of my direct BGP sessions
>> were down, but a handful were still working including the route servers.
>>
>> For example, I was getting routes for AS29791 from the route servers,
>> but nexthop 206.72.211.106 was dead to me. Not to pick on Internap other
>> than a mutual customer called me directly at 1am and wanted to know why
>> things were down.
>>
>> I killed the route server sessions and went back to sleep.
>>
>> Feels like LA1 and LA2 got split, but however the route servers
>> interconnect still worked, which was problematic.
>>
>

Re: DDoS attack with blackmail

[jim deleskie]

While I have no design to engage in over email argument over how much
latency people can actually tolerate, I will simply state that most people
have a very poor understanding of it and how much additional latency is
really introduced by DDoS mitigation.

As for implying that DDoS mitigation companies are complicit or involved in
attacks, while not the first time i heard that crap it's pretty offensive
to those that work long hours for years dealing with the garbage.  If you
honestly believe anyone your dealing with is involved with launching
attacks you clearly have not done your research into potential partners.



On Sat., May 22, 2021, 11:20 a.m. Jean St-Laurent via NANOG, <
nanog@nanog.org> wrote:

> Some industries can’t afford that extra delay by DDoS mitigation vendors.
>
>
>
> The video game industry is one of them and there might be others that
> can’t tolerate these extra ms. Telemedicine, video-conference, fintech, etc.
>
>
>
> As a side note, my former employer in video game was bidding for these
> vendors offering DDoS protection. While bidding, we were hit with abnormal
> patterns. As soon as we chose one vendors those very tricky DDoS patterns
> stopped.
>
> I am not saying they are working on both side, but still the coincidence
> was interesting. In the end, we never used them because they were not able
> to perfectly block the threat without impacting all the others projects.
>
>
>
> I think these mitigators are nice to have as a very last resort. I believe
> what is more important for Network Operators is: to be aware of this, to be
> able to detect it, mitigate it and/or minimize the impact. It’s like magic,
> where did that rabbit go?
>
>
>
> The art of war taught me everything there is to know about DDoS attacks
> even if it was written some 2500 years ago.
>
>
>
> I suspect that the attack that impacted Baldur’s assets was a very easy
> DDoS to detect and block, but can’t confirm.
>
>
>
> @Baldur: do you care to share some metrics?
>
>
>
> Jean
>
>
>
> *From:* NANOG  *On Behalf Of *Jean
> St-Laurent via NANOG
> *Sent:* May 21, 2021 10:52 AM
> *To:* 'Lady Benjamin Cannon of Glencoe, ASCE' ; 'Baldur
> Norddahl' 
> *Cc:* 'NANOG Operators' Group' 
> *Subject:* RE: DDoS attack with blackmail
>
>
>
> I also recommend book Art of War from Sun Tzu.
>
>
>
> All the answers to your questions are in that book.
>
>
>
> Jean
>
>
>
> *From:* NANOG  *On Behalf Of *Lady
> Benjamin Cannon of Glencoe, ASCE
> *Sent:* May 20, 2021 7:18 PM
> *To:* Baldur Norddahl 
> *Cc:* NANOG Operators' Group 
> *Subject:* Re: DDoS attack with blackmail
>
>
>
> 20 years ago I wrote an automatic teardrop attack.  If your IP spammed us
> 5 times, then a script would run, knocking the remote host off the internet
> entirely.
>
>
>
> Later I modified it to launch 1000 teardrop attacks/second…
>
>
>
> Today,  contact the FBI.
>
>
>
> And get a mitigation service above your borders if you can.
>
>
>
>
>
> —L.B.
>
>
>
> Ms. Lady Benjamin PD Cannon of Glencoe, ASCE
>
> 6x7 Networks & 6x7 Telecom, LLC
>
> CEO
>
> l...@6by7.net
>
> "The only fully end-to-end encrypted global telecommunications company in
> the world.”
>
> FCC License KJ6FJJ
>
>
>
>
> On May 20, 2021, at 12:26 PM, Baldur Norddahl 
> wrote:
>
>
>
> Hello
>
>
>
> We got attacked by a group that calls themselves "Fancy Lazarus". They
> want payment in BC to not attack us again. The attack was a volume attack
> to our DNS and URL fetch from our webserver.
>
>
>
> I am interested in any experience in fighting back against these guys.
>
>
>
> Thanks,
>
>
>
> Baldur
>
>
>
>
>

Re: Network / Infrastructure security testing services

[jim deleskie]

Your asking if anyone does it or your offering your services?

-jim

On Tue., Mar. 9, 2021, 3:56 p.m. Nathanael Cariaga, 
wrote:

> Apologies for this shameless plug, but wanted to ask if any folks on this
> list who does network/infrastructure security testing? Please to reach back
> to me off the list.
>
> Thank you for your time.
>
>

Re: "Is BGP safe yet?" test

[jim deleskie]

I remember having this discussion more than 20yrs ago, minus the ARIN bit,
couldn't get every to agree to it it then either :(. We don't need more
rules, we just need to start with basic hygiene. Was a novel idea :)

On Mon., Apr. 20, 2020, 2:41 p.m. Christopher Morrow, <
morrowc.li...@gmail.com> wrote:

> On Mon, Apr 20, 2020 at 12:25 PM Tom Beecher  wrote:
> >
> > Technical people need to make the business case to management for RKPI
> by laying out what it would cost to implement (equipment, resources,
> ongoing opex), and what the savings are to the company from protecting
> themselves against hijacks. By taking this step, I believe RPKI will become
> viewed by non-technical decision makers as a 'Cloudflare initiative'
> instead of a 'good of the internet' initiative, especially by some
> companies who compete with Cloudflare in the CDN space.
>
> you say here: "RPKI"
> but the cloudflare thing is a little bit more nuanced than that, right?
> 'RPKI" is really: "Did you sign ROA for your IP Number Resources?"
> what you do with the RPKI data is the 'more nuanced' part of the webpage.
>1) Do you just sign?
>2) do you sign  and also do Origin Validation(OV) for your peers?
>3) do you just do OV and not sign your own IP Number Resources?
>
> I think CloudFlare (and other folk doing bgp security work) would like
> 'everyone' to:
>   1) sign ROA for their IP number resources
>   2) enable OV on your peerings
>   3) prefix filter all of your peerings
>
> > I believe that will change the calculus and make it a more difficult
> sell for technical people to get resources approved to make it happen.
>
> I don't think that's the case... but I'm sure we'll be proven wrong :)
>
> -chris
>

Re: Disney+ Geolocation issues

[jim deleskie]

Using a TPIA provider here at home in Nova Scotia same issue.

-jim

On Tue., Nov. 12, 2019, 6:29 p.m. Michael Crapse, 
wrote:

> Myself and a few other ISPs are having our eyeballs complain about
> disney+ saying that they're on a VPN. Does anyone have any idea, or who to
> contact regarding this issue?
> This is most likely improper geolocation databases. Anyone have an idea
> who they use?
>
> Mike
>

Re: DOs and DONTs for small ISP

[jim deleskie]

triggered :)


On Tue, Jun 4, 2019 at 11:31 AM Bryan Holloway  wrote:

>
> On 6/4/19 9:20 AM, Mark Tinka wrote:
> >
> >
> > On 3/Jun/19 15:41, Fletcher Kittredge wrote:
> >>
> >> Here is your checklist in descending order of importance:
> >>
> >>  1. market opportunity
> >>  2. finding the right partners (see below)
> >>  3. financial
> >>  4. sales and marketing
> >>  5. organizational capacity and HR
> >>  6. legal, regulatory
> >>  7. capital acquisition
> >>  8. security
> >>  9. ...
> >> 10. ...
> >> 11. ...
> >> 12. technical including equipment selection, routing policy,
> >> filtering, etc
> >>
> >
> > 13. Don't run Mikrotik.
> >
> > I'm kidding... I think :-)...
> >
> > Mark.
>
> 14. Go with K56flex, not X2.
>

Re: modeling residential subscriber bandwidth demand

[jim deleskie]

Louie,

 Its almost like us old guys knew something, and did know everything back
then, the more things have changed the more that they have stayed the same
:)



-jim

On Tue, Apr 2, 2019 at 3:52 PM Louie Lee  wrote:

> +1 Also on this.
>
> From my viewpoint, the game is roughly the same for the last 20+ years.
> You might want to validate that your per-customer bandwidth use across your
> markets is roughly the same for the same service/speeds/product. If you
> have that data over time, then you can extrapolate what each market's
> bandwidth use would be when you lay on a customer growth forecast.
>
> But for something that's simpler and actionable now, yeah, just make sure
> that your upstream and peering(!!) links are not congested. I agree that
> the 50-75% is a good target not only for the lead time to bring up more
> capacity, but also to allow for spikes in traffic for various events
> throughout the year.
>
> Louie
> Google Fiber
>
>
> On Tue, Apr 2, 2019 at 11:36 AM jim deleskie  wrote:
>
>> +1 on this. its been more than 10 years since I've been responsible for a
>> broadband network but have friends that still play in that world and do
>> some very good work on making sure their models are very well managed, with
>> more math than I ever bothered with, That being said, If had used the
>> methods I'd had used back in the 90's they would have fully predicted per
>> sub growth including all the FB/YoutubeNetflix traffic we have today. The
>> "rapid" growth we say in the 90's and the 2000' and even this decade are
>> all magically the same curve, we'd just further up the incline, the
>> question is will it continue another 10+ years, where the growth rate is
>> nearing straight up :)
>>
>> -jim
>>
>> On Tue, Apr 2, 2019 at 3:26 PM Mikael Abrahamsson 
>> wrote:
>>
>>> On Tue, 2 Apr 2019, Tom Ammon wrote:
>>>
>>> > Netflow for historical data is great, but I guess what I am really
>>> > asking is - how do you anticipate the load that your eyeballs are
>>> going
>>> > to bring to your network, especially in the face of transport tweaks
>>> > such as QUIC and TCP BBR?
>>>
>>> I don't see how QUIC and BBR is going to change how much bandwidth is
>>> flowing.
>>>
>>> If you want to make your eyeballs happy then make sure you're not
>>> congesting your upstream links. Aim for max 50-75% utilization in 5
>>> minute
>>> average at peak hour (graph by polling interface counters every 5
>>> minutes). Depending on your growth curve you might need to initiate
>>> upgrades to make sure they're complete before utilization hits 75%.
>>>
>>> If you have thousands of users then typically just look at the
>>> statistics
>>> per user and extrapolate. I don't believe this has fundamentally changed
>>> in the past 20 years, this is still best common practice.
>>>
>>> If you go into the game of running your links full parts of the day then
>>> you're into the game of trying to figure out QoE values which might mean
>>> you spend more time doing that than the upgrade would cost.
>>>
>>> --
>>> Mikael Abrahamssonemail: swm...@swm.pp.se
>>>
>>

Re: modeling residential subscriber bandwidth demand

[jim deleskie]

+1 on this. its been more than 10 years since I've been responsible for a
broadband network but have friends that still play in that world and do
some very good work on making sure their models are very well managed, with
more math than I ever bothered with, That being said, If had used the
methods I'd had used back in the 90's they would have fully predicted per
sub growth including all the FB/YoutubeNetflix traffic we have today. The
"rapid" growth we say in the 90's and the 2000' and even this decade are
all magically the same curve, we'd just further up the incline, the
question is will it continue another 10+ years, where the growth rate is
nearing straight up :)

-jim

On Tue, Apr 2, 2019 at 3:26 PM Mikael Abrahamsson  wrote:

> On Tue, 2 Apr 2019, Tom Ammon wrote:
>
> > Netflow for historical data is great, but I guess what I am really
> > asking is - how do you anticipate the load that your eyeballs are going
> > to bring to your network, especially in the face of transport tweaks
> > such as QUIC and TCP BBR?
>
> I don't see how QUIC and BBR is going to change how much bandwidth is
> flowing.
>
> If you want to make your eyeballs happy then make sure you're not
> congesting your upstream links. Aim for max 50-75% utilization in 5 minute
> average at peak hour (graph by polling interface counters every 5
> minutes). Depending on your growth curve you might need to initiate
> upgrades to make sure they're complete before utilization hits 75%.
>
> If you have thousands of users then typically just look at the statistics
> per user and extrapolate. I don't believe this has fundamentally changed
> in the past 20 years, this is still best common practice.
>
> If you go into the game of running your links full parts of the day then
> you're into the game of trying to figure out QoE values which might mean
> you spend more time doing that than the upgrade would cost.
>
> --
> Mikael Abrahamssonemail: swm...@swm.pp.se
>

Re: Service Provider NetFlow Collectors

[jim deleskie]

Erik,

  Feel free to ping me, I own Mimir Networks, we have a full-service flow
collection/DDoS detection and mitigation system that I'd love to show you.
We built it having been a long time user of other commercial and open
source tools, for very large deployments.  Would be happy to give you a
free trial of the system.

-jim
www.mimirnetworks.com

On Sun, Dec 30, 2018 at 11:30 PM Erik Sundberg 
wrote:

> Hi Nanog….
>
>
>
> We are looking at replacing our Netflow collector. I am wonder what other
> service providers are using to collect netflow data off their Core and Edge
> Routers. Pros/Cons… What to watch out for any info would help.
>
>
>
> We are mainly looking to analyze the netflow data. Bonus if it does ddos
> detection and mitigation.
>
>
>
> We are looking at
>
> ManageEngine Netflow Analyzer
>
> PRTG
>
> Plixer – Scrutinizer
>
> PeakFlow
>
> Kentik
>
> Solarwinds NTA
>
>
>
>
>
> Thanks in advance…
>
>
>
> Erik
>
>
>
> --
>
> CONFIDENTIALITY NOTICE: This e-mail transmission, and any documents, files
> or previous e-mail messages attached to it may contain confidential
> information that is legally privileged. If you are not the intended
> recipient, or a person responsible for delivering it to the intended
> recipient, you are hereby notified that any disclosure, copying,
> distribution or use of any of the information contained in or attached to
> this transmission is STRICTLY PROHIBITED. If you have received this
> transmission in error please notify the sender immediately by replying to
> this e-mail. You must destroy the original transmission and its attachments
> without reading or saving in any manner. Thank you.
>

Re: [OT] Internet in China

[jim deleskie]

Chinese ISP's typically like to run their links very hot.  Don't expect
much different if you change providers.

-jim

On Mon, Jul 23, 2018 at 8:37 AM, Danijel Starman 
wrote:

> Hi,
>
> Can someone suggest a reliable internet provider in China? Are all
> options China Telecom?
>
> Some current links we have in Shanghai are sometimes exhibiting ~40% packet
> loss to Japan/Singapore AWS regions which is not really acceptable.
>
> Off-list replies are welcome too.
>
> Thank you!
>
> --
> *blap*
>

Re: AS PATH limits

[jim deleskie]

Maybe the next best path had, had 562 prepends? :)



On Sat, Sep 30, 2017 at 12:09 PM,  wrote:

> > If you're on cogent, since 22:30 UTC yesterday or so this has been
> happening
> > (or happened).
>
> Still happening here. I count 562 prepends (563 * 262197) in the
> advertisement we receive from Cogent. I see no good reason why we
> should accept that many prepends.
>
> Steinar Haug, Nethelp consulting, sth...@nethelp.no
>

Re: AS PATH limits

[jim deleskie]

In my MUCH younger days, I may have helped abuse the global table via
prepends, but never to that level  :)

On Wed, Sep 20, 2017 at 4:36 PM, Randy Bush  wrote:

> > Below is an example showing an excessive amount of prepending for prefix
> > 185.135.134.0/23 at 2017-09-18 20:20:05 UTC.
>
> and they are probably still wondering why it does not achieve what they
> want.
>
> randy
>

Re: Bell outage

[jim deleskie]

Single fiber cut causes the much impact?

-jim

On Fri, Aug 4, 2017 at 2:59 PM, J  wrote:

> https://www.theglobeandmail.com/news/national/much-of-
> atlantic-canada-loses-cellphone-service-in-widespread-outage/
> article35881182/
>
>
>
> Apparently some fiber cut.  No word on the exact model of construction
> equipment, yet, though.
>
>
>
> :\
>
>
>
>
>  On Fri, 04 Aug 2017 10:14:26 -0500 Krunal Shah ks...@primustel.ca
> wrote 
>
>
>
>
> Does anyone know what is happening with Bell network at East Canada?
>
>
>
> http://canadianoutages.com/status/bell/map/
>
>
>
>
>
> Krunal
>
> 
>
>
>
>  This electronic message contains information from Primus Management ULC
> ("PRIMUS") , which may be legally privileged and confidential. The
> information is intended to be for the use of the individual(s) or entity
> named above. If you are not the intended recipient, be aware that any
> disclosure, copying, distribution or use of the contents of this
> information is prohibited. If you have received this electronic message in
> error, please notify us by telephone or e-mail (to the number or address
> above) immediately. Any views, opinions or advice expressed in this
> electronic message are not necessarily the views, opinions or advice of
> PRIMUS. It is the responsibility of the recipient to ensure that any
> attachments are virus free and PRIMUS bears no responsibility for any loss
> or damage arising in any way from the use thereof.The term "PRIMUS"
> includes its affiliates.
>
>
>
> 
>
>  Pour la version en français de ce message, veuillez voir
>
> http://www.primustel.ca/fr/legal/cs.htm
>
>
>
>
>
>
>

Re: Bell outage

[jim deleskie]

Cell and the internet all down here from Bell and those sharing their
towers, also 911 services. Banking / ATM also impacted, no idea reason
though.


-jim

Mimir Networks
www.mimirnetworks.com

On Fri, Aug 4, 2017 at 12:14 PM, Krunal Shah  wrote:

> Does anyone know what is happening with Bell network at East Canada?
>
> http://canadianoutages.com/status/bell/map/
>
>
> Krunal
> 
>
>  This electronic message contains information from Primus Management ULC
> ("PRIMUS") , which may be legally privileged and confidential. The
> information is intended to be for the use of the individual(s) or entity
> named above. If you are not the intended recipient, be aware that any
> disclosure, copying, distribution or use of the contents of this
> information is prohibited. If you have received this electronic message in
> error, please notify us by telephone or e-mail (to the number or address
> above) immediately. Any views, opinions or advice expressed in this
> electronic message are not necessarily the views, opinions or advice of
> PRIMUS. It is the responsibility of the recipient to ensure that any
> attachments are virus free and PRIMUS bears no responsibility for any loss
> or damage arising in any way from the use thereof.The term "PRIMUS"
> includes its affiliates.
>
> 
>  Pour la version en français de ce message, veuillez voir
> http://www.primustel.ca/fr/legal/cs.htm
>

Re: Long AS Path

[jim deleskie]

I see 5+ prepends as maybe not reason to have your "BGP driving license
revoked" but if I can continue with the concept that you have your BGP
learners permit.
If I think back to when I learned to code or when making ACL's,  we still
used line number and practice would be to give ourselves lots
of space 5 or 10 numbers in case we have to insert something in the middle.
ie I need 2 sets of prepends, I'm still learning this stuff
so I'll go with 5 and 10. We all started somewhere, we all did dumb stuff,
hopefully, we all learned.

12AS hops, I have to go see how they are connected now, maybe someone in
that chain needs to be invited by an IX to a NANOG or GPF or some such,
that can't be super efficient.

-jim

On Thu, Jun 22, 2017 at 3:09 AM, Pierfrancesco Caci  wrote:

> > "Mel" == Mel Beckman  writes:
>
>
> Mel> Why not ask the operator why they are pretending this path?
> Perhaps
> Mel> they have a good explanation that you haven't thought of. Blindly
> Mel> limiting otherwise legal path lengths is not a defensible
> practice, in
> Mel> my opinion.
>
> Mel>  -mel beckman
>
>
> A prepend like that is usually the result of someone using the IOS
> syntax on a XR or Junos router.
>
> Long ago, someone accidentally prepending 255 times hit a bug (or was it
> a too strict bgp implementation? I don't remember) resulting in several
> networks across the globe dropping neighbors. One has to protect against
> these things somehow.
>
> As a data point, here is how many prefixes I see on my network for each
> as-path length, after removing prepends:
>
>
> aspath length   count
> -
> 0:  340
> 1:  47522
> 2:  292879
> 3:  227822
> 4:  58390
> 5:  10217
> 6:  2123
> 7:  638
> 8:  48
> 9:  58
> 11: 20
> 12: 2
>
>
> So, does your customer have a legitimate reason to prepend more than 5
> times? Maybe. I still think that anyone that does should have their BGP
> driving licence revoked, though.
>
> Pf
>
>
>
>
> --
> Pierfrancesco Caci, ik5pvx
>

Re: Rogers Peering Request

[jim deleskie]

Will reach out to some folks I know there. PM me Network, AS etc.

On Thu, Dec 15, 2016 at 3:33 PM, Ryan Gard  wrote:

> Looking for a Rogers contact to get things moving on a peering request.
> Been trying to shout into their ear for well over a month, and haven't
> heard anything back. Further, PeeringDB information seems egregiously
> outdated as the URLs listed no longer are serviceable.
>
> Hoping this is the last ditch effort to wake somebody up in the red tower.
>
> Thanks!
>
> --
> Ryan Gard
>

Re: Canadian National Railway contact

[jim deleskie]

Have a friend that used to work there, will reach out to see if he still
does.

-jim

On Tue, Dec 6, 2016 at 11:48 AM, Andy Ringsmuth  wrote:

> If there happens to be someone here from the Canadian National Railway, or
> if someone knows someone there, could you hit me up off-list?
>
> Attempting to work through an e-mail block from us to them that I’ve been
> unsuccessful remedying so far.
>
> Much appreciated!
>
> 
> Andy Ringsmuth
> a...@newslink.com
> News Link – Manager Travel, Technology & Facilities
> 2201 Winthrop Rd., Lincoln, NE 68502-4158
> (402) 475-6397(402) 304-0083 cellular
>
>

Re: BFD on back-to-back connected BGP-speakers

[jim deleskie]

Hugo,

  I've used this configuration in a past line when I may of had multiple L2
steps between L3 devices.  The only concern we had was around load BFD put
on _some_ endpoint routers, if was handles on the RouteProcessor vs on line
cards.


-jim

On Tue, Nov 29, 2016 at 2:23 PM, Hugo Slabbert  wrote:

> Good morning, nanog,
>
> Is there any/sufficient benefit in adding BFD onto BGP sessions between
> directly-connected routers?  If we have intermediate L2 devices such that
> we can't reliably detect link failures BFD can help us quickly detect peers
> going away even when link remains up, but what about sessions with:
>
> - eBGP with peering to interface addresses (not loopback)
> - no multi-hop
> - direct back-to-back connections (no intermediate devices except patch
>  panels)
>
> Possible failure scenarios where I could see this helping would be fat
> fingering (filters implemented on one or the other side drops traffic from
> the peer) or e.g. something catastrophic that causes the control plane to
> go away without any last gasp to the peer.
>
> Or is adding BFD into the mix in this type of setup getting into
> increasing effort/complexity (an additional protocol) for dimishing returns?
>
> --
> Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
> pgp key: B178313E   | also on Signal
>
>

Re: Spitballing IoT Security

[jim deleskie]

So device is certified,  bug is found 2 years later.  How does this help.
The info to date is last week's issue was patched by the vendor in Sept
2015, I believe is what I read. We know bugs will creep in, (source anyone
that has worked with code forever) Also certification assuming it would
work, in what country, would I need one, per country I sell into?  These
are not the solutions you are looking for ( Jedi word play on purpose)

On Wed, Oct 26, 2016 at 3:53 PM, JORDI PALET MARTINEZ <
jordi.pa...@consulintel.es> wrote:

> Exactly, I was arguing exactly the same with some folks this week during
> the RIPE meeting.
>
> The same way that certifications are needed to avoid radio interferences,
> etc., and if you don’t pass those certifications, you can’t sell the
> products in some countries (or regions in case of EU for example),
> authorities should make sure that those certifications have a broader
> scope, including security and probably some other features to ensure that
> in case something is discovered in the future, they can be updated.
>
> Yes, that means cost, but a few thousand dollars of certification price
> increase, among thousands of millions of devices of the same model being
> manufactured, means a few cents for each unit.
>
> Even if we speak about 1 dollar per each product being sold, it is much
> cheaper than the cost of not doing it and paying for damages, human
> resources, etc., when there is a security breach.
>
> Regards,
> Jordi
>
>
> -Mensaje original-
> De: NANOG  en nombre de Leo Bicknell <
> bickn...@ufp.org>
> Organización: United Federation of Planets
> Responder a: 
> Fecha: miércoles, 26 de octubre de 2016, 19:19
> Para: 
> Asunto: Re: Spitballing IoT Security
>
> In a message written on Wed, Oct 26, 2016 at 08:06:34AM -0400, Rich
> Kulawiec wrote:
> > The makers of IoT devices are falling all over themselves to rush
> products
> > to market as quickly as possible in order to maximize their
> profits.  They
> > have no time for security.  They don't concern themselves with
> privacy
> > implications.  They don't run networks so they don't care about the
> impact
> > their devices may have on them.  They don't care about liability:
> many of
> > them are effectively immune because suing them would mean
> trans-national
> > litigation, which is tedious and expensive.  (And even if they lost:
> > they'd dissolve and reconstitute as another company the next day.)
> > They don't even care about each other -- I'm pretty sure we're
> rapidly
> > approaching the point where toasters will be used to attack garage
> door
> > openers and washing machines.
>
> You are correct.
>
> I believe the answer is to have some sort of test scheme (UL
> Labratories?) for basic security and updateability.  Then federal
> legislation is passed requiring any product being imported into the
> country to be certified, or it is refused.
>
> Now when they rush to market and don't get certified they get $0
> and go out of business.  Products are stopped at the boader, every
> shipment is reviewed by authorities, and there is no cross boarder
> suing issue.
>
> Really it's product safety 101.  UL, the CPSC, NHTSA, DOT and a
> host of others have regulations that if you want to import a product
> for sale it must be safe.  It's not a new or novel concept, pretty
> much every country has some scheme like it.
>
> --
> Leo Bicknell - bickn...@ufp.org
> PGP keys at http://www.ufp.org/~bicknell/
>
>
>
>
> **
> IPv4 is over
> Are you ready for the new Internet ?
> http://www.consulintel.es
> The IPv6 Company
>
> This electronic message contains information which may be privileged or
> confidential. The information is intended to be for the use of the
> individual(s) named above. If you are not the intended recipient be aware
> that any disclosure, copying, distribution or use of the contents of this
> information, including attached files, is prohibited.
>
>
>
>

Re: Death of the Internet, Film at 11

[jim deleskie]

I've heard this crap for 20+ years now.  "attack traffic" is unplanned
traffic.  Build networks to support "random" bursts of garbage is much more
expensive then you will ever get to bill for.  You clearly have no
understanding of the economics of networks.

On Sun, Oct 23, 2016 at 10:39 PM, Keith Medcalf  wrote:

> Why would the provider want to do anything?  They suuport (make money
> from) their cudtomers.  And the more traffic the send/receive, the more
> money the providers make.
>
> Wouldn't surprise me if the providers were selling access to their
> customers networks to the botherders so they could make money from both
> ends.
>
>
>
> ---
> Sent from Samsung Mobile
>
>
>
>  Original message From: "Ronald F.
> Guilmette"  Date:2016-10-23  17:20
> (GMT-07:00) To:  Cc: nanog@nanog.org
> Subject: Re: Death of the Internet, Film at 11 
> 

Re: Death of the Internet, Film at 11

[jim deleskie]

Sure lets sue people because they put too many/bad packets/packets I don't
like on the internet.  Do you think this will really solve the porblem?  Do
you think we'll not just all end up with internet prices like US medical
care prices?

On Sun, Oct 23, 2016 at 4:41 PM,  wrote:

>
> >So once identified, how do you suggest this gets fixed?
>
> Assuming these manufacturers who are culpable carry product liability
> insurance go to their insurance companies and explain the situation.
>
> Better would be someone launching a product liability lawsuit against
> one of them but it's not necessary, ins cos work on projections and
> probabilities as much as being reactive.
>
> The insurance companies will likely re-assess their risk on these
> policies and inform the manufacturers of any adjustment in premiums.
>
> If the premiums are adjusted up significantly the manufacturers will
> sit down with the ins cos and try to determine what needs to be
> improved in their product to bring premiums back down.
>
> Look at what Samsung just went thru with the Note 7. I'd imagine their
> product liability insurance premiums took a big hit. Even if they're
> self-insured they have to treat that as a cost center and make sure
> sufficient money to pay claims is going into that cost center.
>
> It's a button to push, so to speak, and has been successful many times
> in the past (cars, worker exposure to health hazards, etc.)
>
> --
> -Barry Shein
>
> Software Tool & Die| b...@theworld.com |
> http://www.TheWorld.com
> Purveyors to the Trade | Voice: +1 617-STD-WRLD   | 800-THE-WRLD
> The World: Since 1989  | A Public Information Utility | *oo*
>

Re: Death of the Internet, Film at 11

[jim deleskie]

Sure, but now we put it outside the skill level of 99.99% of the people
that don't read and understand this list.

-jim

On Sat, Oct 22, 2016 at 2:09 PM, Luke Guillory <lguill...@reservetele.com>
wrote:

> VPNs can accomplish this without opening ports directly to devices.
>
> Luke
>
>
> *Sent from my iPhone*
>
> On Oct 22, 2016, at 12:06 PM, jim deleskie <deles...@gmail.com> wrote:
>
> It is also likely the desired use case.  In my office I like to be able to
> login when needed when on the road, when the alarm company calls me at 2am
> for a false alarm so I don't have to get someone else out of bed to have
> them dispatched to check on the site.
>
> -jim
>
> On Sat, Oct 22, 2016 at 1:42 PM, Chris Boyd <cb...@gizmopartners.com>
> wrote:
>
>
> On Oct 22, 2016, at 7:34 AM, Mike Hammett <na...@ics-il.net> wrote:
>
>
> "taken all necessary steps to insure that none of the numerous specific
>
> types of CCVT thingies that Krebs and others identified"
>
>
> Serious question... how?
>
>
> Putting them behind a firewall without general Internet access seems to
>
> work for us.  We have a lot of cheap IP cameras in our facility and none of
>
> them can reach the net.  But this is probably a bit beyond the capabilities
>
> of the general home user.
>
>
> —Chris
>
>
>
>
>
> Luke Guillory
> Network Operations Manager
>
>
> <http://www.rtconline.com>
> Tel: 985.536.1212
> Fax: 985.536.0300
> Email: lguill...@reservetele.com
> Web: www.rtconline.com
> Reserve Telecommunications
> 100 RTC Dr
> Reserve, LA 70084
>
>
>
>
>
>
> *Disclaimer:*
> The information transmitted, including attachments, is intended only for
> the person(s) or entity to which it is addressed and may contain
> confidential and/or privileged material which should not disseminate,
> distribute or be copied. Please notify Luke Guillory immediately by
> e-mail if you have received this e-mail by mistake and delete this e-mail
> from your system. E-mail transmission cannot be guaranteed to be secure or
> error-free as information could be intercepted, corrupted, lost, destroyed,
> arrive late or incomplete, or contain viruses. Luke Guillory therefore
> does not accept liability for any errors or omissions in the contents of
> this message, which arise as a result of e-mail transmission.
>
>

Re: Death of the Internet, Film at 11

[jim deleskie]

It is also likely the desired use case.  In my office I like to be able to
login when needed when on the road, when the alarm company calls me at 2am
for a false alarm so I don't have to get someone else out of bed to have
them dispatched to check on the site.

-jim

On Sat, Oct 22, 2016 at 1:42 PM, Chris Boyd  wrote:

>
> > On Oct 22, 2016, at 7:34 AM, Mike Hammett  wrote:
> >
> > "taken all necessary steps to insure that none of the numerous specific
> types of CCVT thingies that Krebs and others identified"
> >
> > Serious question... how?
>
> Putting them behind a firewall without general Internet access seems to
> work for us.  We have a lot of cheap IP cameras in our facility and none of
> them can reach the net.  But this is probably a bit beyond the capabilities
> of the general home user.
>
> —Chris
>
>

Re: Legislative proposal sent to my Congressman

[jim deleskie]

Can we please not get the government ( who's gov ) involved. I fully agree
that it will not only not help, but will make some things worse.  This is
why we can't have nice things.


On Tuesday, October 4, 2016, Anne Mitchell  wrote:

> (Interesting and inarguably well-intentioned, and possibly even sound,
> idea snipped, but noted.)
>
> There are a handful of reasons that this will never happen (well, I'm 98%
> certain it will never happen, nothing is every 100% sure when it comes to
> the law, and legislation)... among them the manufacturer's lobby is much
> more well-girded than is the   'home internet security' lobby;  the
> cyber-security concerns of the Federal government are focussed on other
> things (whether they should be or not, they are);  and for the most part
> legislators are still fairly unsavvy about tech in general, and these
> things make their eyes glaze over.
>
> That said, there are already tort (negligence, etc.) laws and precedents
> under which such manufacturers can be sued, along with things like breach
> of contract between the manufacturer and consumer, and breach of implied
> warranty of fitness for a particular purpose and breach of implied warranty
> of merchantability.
>
> A couple of winning lawsuits against manufacturers under these laws and
> theories - which judges *already understand* - is, I think, not only a more
> likely, but a much faster, route to industry reform.
>
> All that said, much of this faces the same issues that spam lawsuits faced
> - the people who care the most about it are not the ones who can afford to
> finance such lawsuits.
>
> Anne
>
> Anne P. Mitchell,
> Attorney at Law
> Legislative Consultant
> CEO/President, Institute for Social Internet Public Policy
> Member, Cal. Bar Cyberspace Law Committee
> Member, Colorado Cyber Committee
> Member, Asilomar Microcomputer Workshop Committee
> Author: Section 6 of the CAN-SPAM Act of 2003 (the Federal anti-spam law)
> Ret. Professor of Law, Lincoln Law School of San Jose
> Ret. Chair, Asilomar Microcomputer Workshop

Re: Krebs on Security booted off Akamai network after DDoS attack proves pricey

[jim deleskie]

Brandon,

 Sorry you don't understand how multinational companies and peering agreements 
work, nor any of the relationships my past networks would of had with akamai.  
But be confident in the fact none of your concerns would of been an issue and 
it certainly wasn't because decisions were made with out all aspects being 
taken into play

-jim

  Original Message  
From:bran...@rd.bbc.co.uk
Sent:September 25, 2016 3:16 PM
To:cb.li...@gmail.com; deles...@gmail.com
Cc:nanog@nanog.org; j...@aharp.iorc.depaul.edu
Subject:Re: Krebs on Security booted off Akamai network after DDoS attack 
proves pricey

> From: jim deleskie <deles...@gmail.com>
> Sorry but you are mistaken. I've worked at Sr. levels for several LARGE and
> medium sized networks.  What does it cost and what do we make doing it,
> over rules what is "good for the internet" every time it came up.

"nice network you have there, shame if something were to happen to it"

one day they may be the target themselves then they can explain to
shareholders their part in enabling so much business disruption

Sadly it seems there will always be an exploding Pinto on the internet

Perhaps Akamai could present them with a bill for unwanted traffic
as they're monetising ddos they may as well charge both sides and
having dropped Krebs due to the disruption a court may agree damages
too.

brandon

Re: Krebs on Security booted off Akamai network after DDoS attack proves pricey

[jim deleskie]

Sorry but you are mistaken. I've worked at Sr. levels for several LARGE and
medium sized networks.  What does it cost and what do we make doing it,
over rules what is "good for the internet" every time it came up.

On Sun, Sep 25, 2016 at 2:27 PM, Ca By  wrote:

> On Sunday, September 25, 2016, John Kristoff  wrote:
>
> > On Sun, 25 Sep 2016 14:36:18 +
> > Ca By > wrote:
> >
> > > As long as their is one spoof capable network on the net, the problem
> > will
> > > not be solved.
> >
> > This is not strictly true.  If it could be determined where a large
> > bulk of the spoofing came from, public pressure could be applied.  This
> > may not have been the issue in this case, but in many amplification and
> > reflection attacks, the originating spoof-enabled networks were from a
> > limited set of networks.  De-peering, service termination, shaming, etc
> > could have an effect.
> >
> > John
> >
>
> Ok, sorry for the not being exact. I am trying to be practical.
>
> My point is, a lot of access networks will respond to public pressure if
> the data is exposed on the offending real ips of the iot crap, and they
> will enforce their AUP.
>
> We have seen comcast do just that, on this list a few months back. That
> path has legs.
>
> Google also blocks service to certain hacked networks as well, we have seen
> that on this list too. That is an interesting angle in the krebs case. Will
> google block service to folks sharing ip with the iot  ddos mess ?
>

Re: Krebs on Security booted off Akamai network after DDoS attack proves pricey

[jim deleskie]

Not at all.  I refered to AUP's as a way people remove you from a service
when you use more of it then you are paying for.

On Fri, Sep 23, 2016 at 3:58 PM, Marcin Cieslak <sa...@saper.info> wrote:

> On Fri, 23 Sep 2016, jim deleskie wrote:
>
> > They were hosting him for free, and like insurance, I can assure you if
> you
> > are consistently using a service, and not covering the costs of that
> > service you won't be a client for long.  This is the basis for AUP/client
> > contracts and have been going back to the days when we all offered only
> > dialup internet.
>
> Does being a victim of a DDoS constitute a breach of AUP?
>
> Marcin Cieślak
>

Re: Krebs on Security booted off Akamai network after DDoS attack proves pricey

[jim deleskie]

They were hosting him for free, and like insurance, I can assure you if you
are consistently using a service, and not covering the costs of that
service you won't be a client for long.  This is the basis for AUP/client
contracts and have been going back to the days when we all offered only
dialup internet.

On Fri, Sep 23, 2016 at 3:01 PM, Mike Hammett  wrote:

> I believe the article says they were being hosted for free.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
> Midwest-IX
> http://www.midwest-ix.com
>
> - Original Message -
>
> From: "Grant Ridder" 
> To: nanog@nanog.org
> Sent: Friday, September 23, 2016 12:58:44 PM
> Subject: Krebs on Security booted off Akamai network after DDoS attack
> proves pricey
>
> Didn't realize Akamai kicked out or disabled customers
> http://www.zdnet.com/article/krebs-on-security-booted-off-
> akamai-network-after-ddos-attack-proves-pricey/
>
> "Security blog Krebs on Security has been taken offline by host Akamai
> Technologies following a DDoS attack which reached 665 Gbps in size."
>
> -Grant
>
>

Re: "Defensive" BGP hijacking?

[jim deleskie]

Redirecting someone's traffic, with out there permission or a court order,
by a court in your jurisdiction, not a lot different then the "bad guys"
themselves.




On Sun, Sep 11, 2016 at 5:54 PM, Hugo Slabbert  wrote:

> Hopefully this is operational enough, though obviously leaning more
> towards the policy side of things:
>
> What does nanog think about a DDoS scrubber hijacking a network "for
> defensive purposes"?
>
> http://krebsonsecurity.com/2016/09/alleged-vdos-proprietors-arrested-in-
> israel/
>
> "For about six hours, we were seeing attacks of more than 200 Gbps hitting
> us,” Townsend explained. “What we were doing was for defensive purposes. We
> were simply trying to get them to stop and to gather as much information as
> possible about the botnet they were using and report that to the proper
> authorities.”
>
> --
> Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
> pgp key: B178313E   | also on Signal

Re: NFV Solution Evaluation Methodology

[jim deleskie]

I struggled with this whole SDN/NVF/insert marketing term for a while at
first, until I sat down and actually though about.  When I strip away all
the foo, what I'm left with is breaking things down to pieces and and
putting logo blocks together in a way that best suits what I'm doing.  It
is really going back to the way things were a long time ago in the days of
12/2400 baud models and 56k frame relay.  It doesn't help vendors vendors
that want to sell you over priced foo for features you don't really need.
It lets you, if you have clue build your own right bits. It will see some
vendors evolve, new vendors of their brand of foo appear and some vendors
die, but end of day, its no different then most of were doing back in the
"good ol days"

-jim

On Wed, Aug 3, 2016 at 11:27 AM, Christopher Morrow  wrote:

> On Wed, Aug 3, 2016 at 8:20 AM, Ca By  wrote:
>
> >
> >
> > On Wednesday, August 3, 2016, Randy Bush  wrote:
> >
> >> > but, NFV isn't necessarily 'cloud'... It CAN BE taking purpose built
> >> > appliance garbage that can't scale in a cost effective manner and
> >> > replacing it with some software solution on 'many' commodity
> >> > unix-like-hosts that can scale horizontally.
> >>
> >> my main worry about nfv is when they need more forwarding horsepower
> >> than the household appliance  has, and the data plan is is moved
> >>
> >
> this is a scaling problem, and one which points to the need to not do 'all
> of one thing' ('all nfv will solve us!') you may still need other methods
> to load balance or deal with loads which are higher than the nfv
> platform(s) can deal with properly.
>
> In some sense this is the same problem as trying to push too many pps
> through a linecard which you know has a limit lower than line-rate.
>
>
> > out of the control plane and they are not congruent.  we've had too many
> >> lessons debugging this situation (datakit, atm, ...).
> >>
> >>
> seperation of data/control plane ... does require knowledge about what you
> are doing and has clear implications on toolling, troubleshooting, etc.
>
> To some extent this mirrors anycast dns deployment problems. "I made a much
> more complex system, though from the outside perhaps it doesn't appear any
> different." be prepared for interesting times.
>
>
> > Sdn is like authoritarianism and divine creation rolled up into one and
> > sold at 20% premium to easily duped telco types that want to travel to
> > endless conferences
> >
> >
> Sure, you have to know what you are doing/buying... magic doesn't exist in
> this space.
>
>
> >
> >
> >> beyond that, i am not sure i see that much difference whether it's a
> >> YFRV or a SuperMicro.  but i sure wish bird and quagga had solid is-is,
> >> supported communities, ...
> >>
> >> randy
> >>
> >
>

Re: cloudflare hosting a ddos service?

[jim deleskie]

Back in the day didn't we refer to such hosting as bulletproof hosting?

On Tue, Jul 26, 2016 at 11:17 PM, Phil Rosenthal <p...@isprime.com> wrote:

> Plus, it’s good for business!
>
> -Phil
>
> > On Jul 26, 2016, at 10:14 PM, jim deleskie <deles...@gmail.com> wrote:
> >
> > sigh...
> >
> > On Tue, Jul 26, 2016 at 10:55 PM, Patrick W. Gilmore <patr...@ianai.net>
> > wrote:
> >
> >> CloudFlare will claim they are not hosting the problem. They are just
> >> hosting the web page that lets you pay for or points at or otherwise
> >> directs you to the problem.
> >>
> >> The actual source of packets is some other IP address. Therefore, they
> can
> >> keep hosting the web page. It is not sending the actual
> >> [spam|DDoS|hack|etc.], right? So stop asking them to do something about
> it!
> >>
> >> Whether you think that is the proper way to provide service on the
> >> Internet is left as an exercise to the reader.
> >>
> >> --
> >> TTFN,
> >> patrick
> >>
> >>> On Jul 26, 2016, at 9:49 PM, Mike <mike-na...@tiedyenetworks.com>
> wrote:
> >>>
> >>> Hi,
> >>>
> >>>   So vbooter.org's dns and web is hosted by cloudflare?
> >>>
> >>> "Using vBooter you can take down home internet connections, websites
> and
> >> game servers such us Minecraft, XBOX Live, PSN and many more."
> >>>
> >>>   dig -t ns vbooter.org
> >>>
> >>> ; <<>> DiG 9.9.5-3ubuntu0.8-Ubuntu <<>> -t ns vbooter.org
> >>> ;; global options: +cmd
> >>> ;; Got answer:
> >>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62177
> >>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
> >>>
> >>> ;; OPT PSEUDOSECTION:
> >>> ; EDNS: version: 0, flags:; udp: 512
> >>> ;; QUESTION SECTION:
> >>> ;vbooter.org.INNS
> >>>
> >>> ;; ANSWER SECTION:
> >>> vbooter.org.21599INNSrick.ns.cloudflare.com.
> >>> vbooter.org.21599INNSamy.ns.cloudflare.com.
> >>>
> >>> dig -t a www.vbooter.org
> >>>
> >>> ; <<>> DiG 9.9.5-3ubuntu0.8-Ubuntu <<>> -t a www.vbooter.org
> >>> ;; global options: +cmd
> >>> ;; Got answer:
> >>> ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34920
> >>> ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1
> >>>
> >>> ;; OPT PSEUDOSECTION:
> >>> ; EDNS: version: 0, flags:; udp: 512
> >>> ;; QUESTION SECTION:
> >>> ;www.vbooter.org.INA
> >>>
> >>> ;; ANSWER SECTION:
> >>> www.vbooter.org.299INCNAMEvbooter.org.
> >>> vbooter.org.299INA104.28.13.7
> >>> vbooter.org.299INA104.28.12.7
> >>>
> >>>
> >>>   Can anyone from cloudflare answer me why this fits with your business
> >> model?
> >>>
> >>> Mike-
> >>
> >>
>
>

Re: cloudflare hosting a ddos service?

[jim deleskie]

sigh...

On Tue, Jul 26, 2016 at 10:55 PM, Patrick W. Gilmore 
wrote:

> CloudFlare will claim they are not hosting the problem. They are just
> hosting the web page that lets you pay for or points at or otherwise
> directs you to the problem.
>
> The actual source of packets is some other IP address. Therefore, they can
> keep hosting the web page. It is not sending the actual
> [spam|DDoS|hack|etc.], right? So stop asking them to do something about it!
>
> Whether you think that is the proper way to provide service on the
> Internet is left as an exercise to the reader.
>
> --
> TTFN,
> patrick
>
> > On Jul 26, 2016, at 9:49 PM, Mike  wrote:
> >
> > Hi,
> >
> >So vbooter.org's dns and web is hosted by cloudflare?
> >
> > "Using vBooter you can take down home internet connections, websites and
> game servers such us Minecraft, XBOX Live, PSN and many more."
> >
> >dig -t ns vbooter.org
> >
> > ; <<>> DiG 9.9.5-3ubuntu0.8-Ubuntu <<>> -t ns vbooter.org
> > ;; global options: +cmd
> > ;; Got answer:
> > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 62177
> > ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 1
> >
> > ;; OPT PSEUDOSECTION:
> > ; EDNS: version: 0, flags:; udp: 512
> > ;; QUESTION SECTION:
> > ;vbooter.org.INNS
> >
> > ;; ANSWER SECTION:
> > vbooter.org.21599INNSrick.ns.cloudflare.com.
> > vbooter.org.21599INNSamy.ns.cloudflare.com.
> >
> > dig -t a www.vbooter.org
> >
> > ; <<>> DiG 9.9.5-3ubuntu0.8-Ubuntu <<>> -t a www.vbooter.org
> > ;; global options: +cmd
> > ;; Got answer:
> > ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 34920
> > ;; flags: qr rd ra; QUERY: 1, ANSWER: 3, AUTHORITY: 0, ADDITIONAL: 1
> >
> > ;; OPT PSEUDOSECTION:
> > ; EDNS: version: 0, flags:; udp: 512
> > ;; QUESTION SECTION:
> > ;www.vbooter.org.INA
> >
> > ;; ANSWER SECTION:
> > www.vbooter.org.299INCNAMEvbooter.org.
> > vbooter.org.299INA104.28.13.7
> > vbooter.org.299INA104.28.12.7
> >
> >
> >Can anyone from cloudflare answer me why this fits with your business
> model?
> >
> > Mike-
>
>

Re: cross connects and their pound of flesh

[jim deleskie]

I don't buy this.  They sold you one cable before, they sell you cable now.
  Little difference then we moved customers from a T1 to  T3 back in the
90's.  If Colo's can't understand more then 20+ yrs of evolution its hardly
right to blame it on the market.


-jim
Mimir Networks
www.mimirnetworks.com


On Sun, Jun 19, 2016 at 11:07 AM, Mike Hammett  wrote:

> Before 100G, you'd need ten cross connects to move 100G. Now you'd need
> only one. That's a big drop in revenue.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
>
>
> Midwest Internet Exchange
> http://www.midwest-ix.com
>
>
> - Original Message -
>
> From: "Brandon Butterworth" 
> To: br...@pobox.com, d...@temk.in
> Cc: nanog@nanog.org
> Sent: Sunday, June 19, 2016 8:55:57 AM
> Subject: Re: cross connects and their pound of flesh
>
> Dave Temkin  wrote:
> > And as colo operators get freaked out over margin compression on the
> > impending 10->100G conversion (which is happening exponentially faster
> than
> > 100->1G & 1G->10G) they'll need to move those levers of spend around
> > regardless.
>
> If they've based their model on extracting profit proportional
> to technology speed then they've misunderstood Moore's law
>
> brandon
>
>

Re: Netflix VPN detection - actual engineer needed

[jim deleskie]

Damian, I HIGHLY doubt regular folks are running into issues with this, I
suspect its not even geeks in general having issues, I suspect 80% plus of
those having issues spend most of their time complaining about something
related to v6 and the rest of the geeks not loving them/it enough.

-jim

On Sun, Jun 5, 2016 at 6:18 PM, Damian Menscher  wrote:

> On Fri, Jun 3, 2016 at 4:43 PM, Baldur Norddahl  >
> wrote:
>
> > Den 4. jun. 2016 01.26 skrev "Cryptographrix"  >:
> > >
> > > The information I'm getting from Netflix support now is explicitly
> > telling
> > > me to turn off IPv6 - someone might want to stop them before they
> > > completely kill US IPv6 adoption.
> >
> > Not allowing he.net tunnels is not killing ipv6. You just need need
> native
> > ipv6.
> >
>
> This entire thread confuses me.  Are there normal home users who are being
> blocked from Netflix because their ISP forces them through a HE VPN?  Or is
> this massive thread just about a handful of geeks who think IPv6 is cool
> and insist they be allowed to use it despite not having it natively?  I
> could certainly understand ISP concerns that they are receiving user
> complaints because they failed to provide native IPv6 (why not?), but
> whining that you've managed to create a non-standard network setup doesn't
> work with some providers seems a bit silly.
>
> Damian
>

Re: Netflix VPN detection - actual engineer needed

[jim deleskie]

I don't suspect many folks that are outside of this list would likely have
any idea how to set up a v6 tunnel.  Those of us on the list, likely have a
much greater ability to influence v6 adoption or not via day job
deployments then Netflix supporting v6 tunnels or not.

On Fri, Jun 3, 2016 at 8:49 PM, Cryptographrix 
wrote:

> Depends - how many US users have native IPv6 through their ISPs?
>
> If I remember correctly (I can't find the source at the moment), HE.net
> represents something like 70% of IPv6 traffic in the US.
>
> And yeah, not doing that - actually in the middle of an IPv6 project at
> work at the moment that's a bit important to me.
>
>
>
>
> On Fri, Jun 3, 2016 at 7:45 PM Baldur Norddahl 
> wrote:
>
> > Den 4. jun. 2016 01.26 skrev "Cryptographrix"  >:
> > >
> > > The information I'm getting from Netflix support now is explicitly
> > telling
> > > me to turn off IPv6 - someone might want to stop them before they
> > > completely kill US IPv6 adoption.
> >
> > Not allowing he.net tunnels is not killing ipv6. You just need need
> native
> > ipv6.
> >
> > On the other hand it would be nice if Netflix would try the other
> protocol
> > before blocking.
> >
>

Re: BGP FlowSpec

[jim deleskie]

I was going to avoid this thread because I've never been a huge fan of
Flowspec for my own reasons. However having work on /been responsible for
several "Tier 1 and 2" networks and DDoS mitigation services over the last
20 years,  I can say I, nor any of my peers ( in any sense of that word)
 that I have known, have wanted to keep "bad " traffic on our networks so
we can bill for it.  Designing and running a large network is hard enough
with planed growth, without having to manage unplanned spikes on links that
can be  orders of magnitude larger then traffic that "normally" flows
across it.

On top of that any given DDoS attack seldom last long enough to materially
impact 95%ile billing, so carriers don't make anything from it, but have to
do all the work of moving it around.

-jim

On Mon, May 2, 2016 at 6:38 PM, Roland Dobbins  wrote:

> On 2 May 2016, at 20:16, Martin Bacher wrote:
>
>  However, Tier 1s and most probably also some of the Tier 2s may not want
>> to offer it to customers because they are loosing money if less traffic is
>> sent downstream on IP-Transit links.
>>
>
> I will go a step further than Danny's comments and state that this is
> categorically and demonstrably untrue.
>
> Many of the quite large 'Tier-1' and 'Tier-2' (using the old terminology)
> operators on this list offer commercial DDoS mitigation services making use
> of technologies like D/RTBH, S/RTBH, IDMS, et. al. due to customer demand.
> They need these capabilities in order to defend their own properties and
> assets, and they are also offering them to end-customers who want and need
> them.
>
> In point of fact, it's becoming difficult to find one which *doesn't*
> offer this type of service.
>
> There were a couple of situations in the first half of the first decade of
> this millennium where operators took this attitude.  But they changed their
> tunes pretty rapidly once they themselves were impacted, and once they
> started losing customers because they couldn't and wouldn't protect them.
>
> And as Danny notes, these technologies are all tools in the toolbox.  NFV
> and 'SDN' have tremendous potential to make it a lot easier to bring
> mitigation resources to bear in a dynamic and optimal fashion within single
> spans of administrative control; and there are standards-based efforts
> underway to provide for a higher degree of automation, increased rapidity
> of response, and interoperability in both inter- and intra-network DDoS
> mitigation scenarios.
>
> ---
> Roland Dobbins 
>

Re: Cogent <=> Google Peering issue

[jim deleskie]

They haven't been since at least the mid 90's :)

On Wed, Feb 17, 2016 at 12:50 PM, Nick Hilliard  wrote:

> Todd Underwood wrote:
> > Can you scope "issue" with any facts or data?
>
> are facts or data strictly necessary on the nanog mailing list?
>
> Nick
>
> > T
> > On Feb 17, 2016 11:16, "Fred Hollis"  wrote:
> >
> >> Anyone else aware of it?
> >>
>
>

Re: The IPv6 Travesty that is Cogent's refusal to peer Hurricane Electric - and how to solve it

[jim deleskie]

Was part of my first peering spat, probably 95/96‎ since then many more,
couple even big enough they made nanog/ industry news, end of day they are
all the same. If you need to reach every where have more then one provider,
it's good practice anyway, a single cust or even a bunch of cust are NOT
going to influence peer decisions, so build your network so any 2 sides not
playing not, will not impact you cust's, so at least they don't have reason
to complain to you.

-jim

On Thu, Jan 21, 2016 at 11:42 PM, Matthew D. Hardeman  wrote:

> An excellent point.  Nobody would tolerate this in IPv4 land.  Those
> disputes tended to end in days and weeks (sometimes months), but not years.
>
> That said, as IPv6 is finally gaining traction, I suspect we’ll be seeing
> less tolerance for this behavior.
>
>
> > On Jan 21, 2016, at 8:30 PM, Matthew Kaufman  wrote:
> >
> >
> >
> >> On Jan 21, 2016, at 1:05 PM, Ca By  wrote:
> >>
> >> On Thu, Jan 21, 2016 at 10:52 AM, Brandon Butterworth <
> bran...@rd.bbc.co.uk>
> >> wrote:
> >>
> > On Jan 21, 2016, at 1:07 PM, Matthew D. Hardeman <
> >>> mharde...@ipifony.com> wrote:
> > Since Cogent is clearly the bad actor here (the burden being
> > Cogent's to prove otherwise because HE is publicly on record as
> saying
> > that theyd love to peer with Cogent)
> >>>
> >>> I'd like to peer with all tier 1's, they are thus all bad as
> >>> they won't.
> >>>
> >>> HE decided they want to be transit free for v6 and set out on
> >>> a campaign of providing free tunnels/transit/peering to establish
> >>> this. Cogent, for all their faults, are free to not accept the
> >>> offer.
> >>>
> >>> Can the Cogent bashing stop now, save it for when they do something
> >>> properly bad.
> >>>
> >>> brandon
> >>
> >> Selling a service that is considered internet but does not deliver full
> >> internet access is generally considered properly bad.
> >>
> >> I would not do business with either company, since neither of them
> provide
> >> a full view.
> >>
> >> CB
> >
> > I note that if IPv6 was actually important, neither one could have
> gotten away with it for so long.
> >
> > Matthew Kaufman
> >
> > (Sent from my iPhone)
>
>

Re: Ear protection

[jim deleskie]

Maybe I've always listened to my music to loud and spend the bulk of time
via ssh, but I've never felt a need for hearing protection in a DC, is this
generally an issue for people?

On Wed, Sep 23, 2015 at 8:08 AM, Alex Rubenstein  wrote:

> Why not just build a Datacenter that is quiet?
>
> On Sep 23, 2015 05:34, Nick Hilliard  wrote:
> What are people using for ear protection for datacenters these days?  I'm
> down to my last couple of corded 3M 1110:
>
> http://www.shop3m.com/3m-corded-earplugs-hearing-conservation-1110.html
>
> These work reasonably well in practice, with a rated nominal noise
> reduction rate of 29dB.  Some people find them uncomfortable, but they work
> well for me.
>
> There are other ear plugs with rated NRR of up to 32-33dB.  Anyone have any
> opinions on what brands work well for them?
>
> Nick
>

Re: NetFlow - path from Routers to Collector

[jim deleskie]

Adding VRFs/VLAN's/anything else to separate the traffic to reduce fate
sharing is only adding complexity that will likely result in operator
errors.  While many of us have clue, even when we don't agree on the
solutions, there are many more out there typing at routers at 2am, when
even the simplest of configs will mix someone up and cause an out.  The
stats prove out these types of errors are more likely to cause an outage
then DDoS or anything else.  Now if we could only build and sell devices to
stop operator error.

On Wed, Sep 2, 2015 at 1:11 PM, Roland Dobbins  wrote:

>
> On 2 Sep 2015, at 22:26, Mark Tinka wrote:
>
> When the line card congests, it doesn't care that one bit was part of a
>> VRF and the other doesn't. It all goes kaboom (even with the best of QoS
>> intentions).
>>
>
> You don't necessarily have to put everything on the same fiber, interface,
> the same ASIC cluster, the same LC-CPU/-NPU, the same linecard, etc.
>
> Fat-fingers in the global table or the Internet VRF or whatever won't
> cause problems in the management VRF, unless via route-leaking policies
> which allow them to do so or the kind of routing-table explosion which
> takes down a linecard or the whole box.  A hardware casualty or software
> fault which takes down a linecard may not take down the whole box.  And so
> forth.
>
> iACLs are simpler, don't have to be updated so frequently to account for
> moves/adds/changes of the management infrastructure.  It's easier to apply
> QoS policies to reserve bandwidth for telemetry and other management-plane
> traffic, etc.  And so forth.
>
> All this is highly variable and situationally-specific, but logical
> separation of management-plane traffic from production data-plane traffic
> is in general desirable, even as it's running on (at least some of) the
> same hardware.  It isn't as good as true physical separation, but there's
> no sense in making the perfect the enemy of the merely good.
>
> ---
> Roland Dobbins 
>

Re: NetFlow - path from Routers to Collector

[jim deleskie]

I've not read every reply, but let me add my voice as some who has worked
on and ran SEVERAL large networks, in no case in the last long number of
years have I had access to an OOB network that was sized to carry anything
in large volume, and in fact like many others replied on a robust number of
path at that us many the networks inband.  These networks survived many
"large" DDoS attacks and far more fat finger incidents then I like to think
about.  I don't think I've even worked with a client network as far as I
can remember that had a nailed up / robust OOB network for data collection
or other purposes.

-jim

On Tue, Sep 1, 2015 at 8:30 PM, Avi Freedman  wrote:

> (Jared wrote):
>
> 
>
> > Most people I've seen have little data or insight into their
> > networks, or don't have the level that they would desire as
> > tools are expensive or impossible to justify due to capital costs.
> > Tossing in a recurring opex cost of DC XC fee  + transport + XC fee +
> > redundant aggregation often doesn't have the ROI you infer here.
> > I've put together some models in this area.  It seems to me the
> > DC/real estate companies involved could make a lot (more) money by
> > offering an OOB service that is 10Mb/s flat-rate for the same as an XC
> > fee and compete with their customers.
>
> Equinix does have a very aggressively priced 10Mb/s flat-rate OOB (single
> IP only but that's not that hard to work around) for essentially XC
> pricing.  It's been stable but not something you'd rely on for 100%
> packet delivery to some other point on the Internet (so more for
> reaching a per-pop OOB than for making a coherent OOB network with
> a bunch of monitoring running 24x7).
>
> Still, it's a good value for what it is.
>
> 
>
> > - Jared
>
> Avi Freedman
> CEO, Kentik
> avi at kentik dot com
>
>

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[jim deleskie]

There is more to it, then just being tired of it, it take, $$ and time and
bodies to build a network, even in 1 country.  Its not something everyone
can do.  I suspect the game and transit networks, will continue long
after most of us are no long playing


On Sat, Aug 15, 2015 at 5:35 PM, Mark Tinka mark.ti...@seacom.mu wrote:



 On 15/Aug/15 19:32, jim deleskie wrote:

  In my 20+ yrs now of playing this game, everyone has had a turn
 thinking
  their content/eyeballs are special and should get free peering.

 That's why those tired of playing the game build their own networks to
 take out the middleman, for better or worse.

 Mark.

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[jim deleskie]

In my 20+ yrs now of playing this game, everyone has had a turn thinking
their content/eyeballs are special and should get free peering.

On Sat, Aug 15, 2015 at 1:59 PM, Mike Hammett na...@ics-il.net wrote:

 Arrogance is the only reason I can think of why the incumbents think that
 way. I'd be surprised if any competitive providers (regardless of their
 market dominance) would expect free peering.




 -
 Mike Hammett
 Intelligent Computing Solutions
 http://www.ics-il.com



 Midwest Internet Exchange
 http://www.midwest-ix.com


 - Original Message -

 From: Owen DeLong o...@delong.com
 To: Matthew Huff mh...@ox.com
 Cc: nanog@nanog.org
 Sent: Saturday, August 15, 2015 11:44:57 AM
 Subject: Re: net neutrality peering dispute between CenturyTel/Qwest and
 Cogent in Dallas

 This issue isn’t limited to Cogent.

 There is this bizarre belief by the larger eyeball networks (and CC, VZ,
 and TW are the worst offenders, pretty much in that order) that they are
 entitled to be paid by both the content provider _AND_ the eyeball user for
 carrying bits between the two.

 In a healthy market, the eyeball providers would face competition and the
 content providers would simply ignore these demands and the eyeballs would
 buy from other eyeball providers.

 Unfortunately, especially in the US, we don’t have a healthy market. In
 the best of circumstances, we have oligopolies and in the worst places, we
 have effective (or even actual) monopolies.

 For example, in the area where I live, the claim you will hear is that
 there is competition. With my usage patterns, that’s a choice between
 Comcast (up to 30/7 $100/mo), ATT DSL (1.5M/384k $40/mo+) and wireless (Up
 to 30/15 $500+/month).

 I’m not in some rural backwater or even some second-tier metro. I’m within
 10 miles of the former MAE West and also within 10 miles of Equinix SV1 (11
 Great Oaks). There’s major fiber bundles within 2 miles of my house. I’m
 near US101 and Capitol Expressway in San Jose.

 The reason that things are this way, IMHO, is because we have allowed
 “facilities based carriers” to leverage the monopoly on physical
 infrastructure into a monopoly for services over that infrastructure.

 The most viable solution, IMHO, is to require a separation between
 physical infrastructure providers and those that provide services over that
 infrastructure. Breaking the tight coupling between the two and requiring
 physical infrastructure providers to lease facilities to operators on an
 equal footing for all operators will reduce the barriers to competition in
 the operator space. It will also make limited competition in the facilities
 space possible, though unlikely.

 This model exists to some extent in a few areas that have municipal
 residential fiber services, and in most of those localities, it is working
 well.

 That’s one of the reasons that the incumbent facilities based carriers
 have lobbied so hard to get laws in states where a city has done this that
 prevent other cities from following suit.

 Fortunately, one of the big gains in recent FCC rulings is that these laws
 are likely to be rendered null and void.

 Unfortunately, there is so much vested interest in the status quo that
 achieving this sort of separation is unlikely without a really strong grass
 roots movement. Sadly, the average sound-bite oriented citizen doesn’t know
 (or want to learn) enough to facilitate such a grass-roots movement, so if
 we want to build such a future, we have a long slog of public education and
 recruitment ahead of us.

 In the mean time, we’ll get to continue to watch companies like CC, VZ, TW
 screw over their customers and the content providers their customers want
 to reach for the sake of extorting extra money from both sides of the
 transaction.

 Owen

  On Aug 15, 2015, at 06:40 , Matthew Huff mh...@ox.com wrote:
 
  It's only partially about net neutrality. Cogent provides cheap
 bandwidth for content providers, and sends a lot of traffic to eyeball
 networks. In the past, peering partners expected symmetrical load sharing.
 Cogent feels that eyeball networks should be happy to carry their traffic
 since the customers want their services, the eyeball networks want Cogent
 to pay them extra. When there is congestion, neither side wants to upgrade
 their peeing until this is resolved, so they haven't. This has been going
 on for at least 5 years, and happens all over the cogent peering map.
 
  Depending on what protocol you are using, it can be an issue or not. Our
 end users on eyeball networks had difficulty maintaining VPN connections.
 We had to drop our Cogent upstream and work with our remaining upstream
 provides to traffic engineer around Cogent. YMMV.
 
 
 
  
  Matthew Huff | 1 Manhattanville Rd
  Director of Operations | Purchase, NY 10577
  OTA Management LLC | Phone: 914-460-4039
  aim: matthewbhuff | Fax: 914-694-5669
 
  -Original Message-
  From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of 

Re: DDOS Simulation

[jim deleskie]

If anyone offers to test your DDoS devices across a network that you do
not 100% own, you are risking legal issues.

If they offer to test it across your own network, make sure you have in
writing from you upper management that they understand the risk and approve
it.

If you choose to do it anyway then you are taking a LARGE risk.


Testing should be in your lab and even then you should understand 100% what
is happing to avoid leaking attack traffic into the internet.

-jim

On Tue, Jul 28, 2015 at 2:24 PM, Ryan Pugatch r...@lp0.org wrote:

 Hi Dovid,

 I recommend checking out NimbusDDOS. http://www.nimbusddos.com/

 I know that they have done exactly this for several notable customers,
 and also provide insights into impacts (they don't just blindly run the
 attacks for you, they provide intelligence behind what's happening to
 help you make sense of what is going on.)

 Contact me off list if you want me to set up an intro.

 Ryan


 On Mon, Jul 27, 2015, at 11:32 AM, Dovid Bender wrote:
  Hi All,
 
  We are looking into a few different DDOS solutions for a client. We need
  a
  LEGITIMATE company that can simulate some DDOS attacks (the generic +
  specific to the clients business). Anyone have any recommendations?
 
  Regards,
 
  Dovid

Re: ARIN just subdivided their last /17, /18, /19, /20, /21 and /22. Down to only /23s and /24s now. : ipv6

[jim deleskie]

I'd give it another 20 yrs of v4, v6 addressing and all those letters are
to hard for us old folk, we'll find ways to make it make it work :)

On Sat, Jun 27, 2015 at 11:54 AM, Mikael Abrahamsson swm...@swm.pp.se
wrote:

 On Sat, 27 Jun 2015, Rafael Possamai wrote:

  How long do you think it will take to completely get rid of IPv4? Or is
 it even going to happen at all?


 I believe somewhere around 2018-2025 a lot of ISPs, hosting providers etc
 will start to treat IPv4 as a second rate citizen and for the people still
 single-stacked to IPv4 by then, the Internet experience is going to become
 so bad that they'll beg to get IPv6 and the ones not providing it will feel
 severe business impact of not doing IPv6.

 Mobile providers will be the first huge ones to go IPv6 only to the
 devices, which will mean that from your mobile device, IPv4 will most
 likely work worse than IPv6. Then it's downhill from there.

 --
 Mikael Abrahamssonemail: swm...@swm.pp.se

Re: World's Fastest Internet™ in Canadaland

[jim deleskie]

Its mostly marketing, a number of years ago I worked for a cable co, we
knew if we increased BW X we'd see a Y speed increase in usage.  We also
has done the math on several future generations of upgrades, so we'd know
if phone company increases to A we'd move to B.  I know the guy that did
the math for us then, he still sits in that job so I assume he still does
similar I suspect any cable so worth their salt does the same.



On Fri, Jun 26, 2015 at 3:39 PM, Rafael Possamai raf...@gav.ufsc.br wrote:

 How does one fully utilize a gigabit link for home use? For a single person
 it is overkill. Similar to the concept of price elasticity in economics,
 going from 50mbps to 1gbps doesn't necessarily increase your average
 transfer rate, at least I don't think it would for me. Anyone care to
 comment? Just really curious, as to me it's more of a marketing push than
 anything else, even though gigabit to the home sounds really cool.



 On Fri, Jun 26, 2015 at 1:13 PM, Eric Dugas edu...@zerofail.com wrote:

  Nice try Bell.. So-Net did it two years ago, 2Gbps FTTH in Japan.
 
  Article: http://bgr.com/2013/06/13/so-net-nuro-2gbps-fiber-service/
 
  If you read Japanese: http://www.nuro.jp/hikari/
 
  Eric
 
  -Original Message-
  From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Hank Disuko
  Sent: June 26, 2015 2:04 PM
  To: NANOG
  Subject: World's Fastest Internet™ in Canadaland
 
  Bell Canada is apparently gearing up to provide the good people of
 Toronto
  with the World's Fastest Internet™.
 
 
 http://www.thestar.com/news/city_hall/2015/06/25/bell-canada-to-give-toronto-worlds-fastest-internet.html
 
 
 

Re: Open letter to Level3 concerning the global routing issues on June 12th

[jim deleskie]

People from Big telcom should never reply to mailing lists from work
addresses unless specifically allowed, which I suspect TATA doesn't either,
based on some direct, buy old knowledge :)

Filtering has been a community issue since my days @ MCI being AS3561,
often discussed not often enough acted one, I suspect the topic has come up
at every large NSP I've worked at.  Frequently someone complains its
hard to fix, or router X makes it hard to fix, or customer Y won;t agree,
and not enough people stand up to force fix the issues.  I've did a preso
on it ( while working at TATA) with some other smart folks but for all
the usual reasons it died on the vine.  I don't blame (3) for this but our
community as a whole.  Many people/networks have to not do the right
thing(tm) for a failure like this to happen.


-jim

On Fri, Jun 12, 2015 at 12:43 PM, Utkarsh Gosain 
utkarsh.gos...@tatacommunications.com wrote:

 Hi Martin
 I am not a spokesperson on behalf of L3 but I have worked for big telcos
 my whole career and my recommendation is to raise a trouble ticket if any
 one on the forum is their customer and is affected.
 I don’t think Engineers at NOC are authorized to reply to forums at any of
 the major telcos especially regarding outages unless someone raise a
 trouble ticket and seeks an RCA of the issue one on one with them.


 Utkarsh Gosain
 Global Acc Director
 Tata Communications


 -Original Message-
 From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Martin Millnert
 Sent: Friday, June 12, 2015 11:33 AM
 To: NANOG
 Subject: Open letter to Level3 concerning the global routing issues on
 June 12th

 Dear Level3,

 The Internet is a cooperative effort, and it works well only when its
 participants take constructive actions to address errors and remedy
 problems.
 Your position as a major Internet Carrier bestows upon you a certain
 degree of responsibility for the correct operation of the Internet all
 across (and beyond) the planet. You have many customers. Customers will
 always occasionally make mistakes. You as a major Internet Carrier have a
 responsibility to limit, not amplify, your customers' mistakes.
 Other major carriers implement technical measures that severely limits the
 damages from customer mistakes from having global impact.
 Other major carriers also implement operational procedures in addition to
 technical measures.
 In combination, these measures drastically reduce the outage-hours as a
 result of customer configuration errors.

 At 08:44 UTC on Friday 12th of June, one of your transit customers,
 Telekom Malaysia (AS4788) began announcing the full Internet table back to
 you, which you accepted and propagated to your peers and customers, causing
 global outages for close to 3 hours.
 [ https://twitter.com/DynResearch/status/609340592036970496 ] During this
 3 hour window, it appears (from your own service outage
 reports) that you did nothing to stop the global Internet outage, but that
 Telekom Malaysia themselves eventually resolved it. This lack of action on
 your end, and your disregard for the correct operation of the global
 Internet is astonishing. These mistakes do not need to happen.
 AS4788 under normal circumstances announces ~1900 IPv4 prefixes to the
 Internet. You accepted multiple hundred thousand prefixes from them - a max
 prefix setting would have severely limited the damage. We expect that these
 are your practices as well, but they failed. When they do, it should not
 take ~3 hours to shut down the session(s).

 Many operators, in despair, turned down their peering sessions with you
 once it was clear you were causing the outages and no immediate fix was in
 sight. This improved the situation for some - but not all did. Had you
 deployed proper IRR-filtering to filter the bad announcements the impact
 would've been far less critical.

 As a direct consequence of your ~3 hours of inaction, as a local example,
 Swedish payment terminals were experiencing problems all over the country.
 The Swedish economy was directly affected by your inaction.
 There were queues when I was buying lunch! Imagine the food rage. The
 situation was probably similar at other places around the globe where
 people were awake.

 Operators around the planet are curious:
   - Did Level3 not detect or understand that it was causing global
 Internet outages for ~3 hours?
   - If Level3 did in fact detect or understand it was causing global
 Internet outages, why did it not properly and immediately remedy the
 situation?
   - What is Level3 going to do to address these questions and begin work
 on restoring its credibility as a carrier?

 We all understand that mistakes do happen (in applying customer interface
 templates, etc.). However the Internet is all too pervasive in everyday
 life today for anything but swift action by carriers to remedy breakage
 after the fact. It is absolutely not sufficient to let a customer spend 3
 hours to detect and fix a situation like this one. 

Re: Open letter to Level3 concerning the global routing issues on June 12th

[jim deleskie]

Todd,

  One of my few work regrets is we where not able to move this forward.
There was/is lots of value in it.

Agree'd on the posting.

-jim

On Fri, Jun 12, 2015 at 2:36 PM, Todd Underwood toddun...@gmail.com wrote:

 i remember that presentation!

 https://www.nanog.org/meetings/abstract?id=459

 :-)

 On Fri, Jun 12, 2015 at 11:53 AM, jim deleskie deles...@gmail.com wrote:

 People from Big telcom should never reply to mailing lists from work
 addresses unless specifically allowed, which I suspect TATA doesn't
 either,
 based on some direct, buy old knowledge :)


 indeed, people from big companies who post on mailing lists at all will be
 called out as official representatives of their company no matter what
 address they use, from recent experience.

 it's probably far better for everyone in such a situation to simply never
 post anything.  :-/

 t

Re: eBay is looking for network heavies...

[jim deleskie]

There is a good reason there aren't LOTS of good neteng in the 30-35 or
under 30 range with lots of experience.  Its call the hell we went though
for a while after 2000 working in this industry.  Many of us lost jobs and
couldn't find new ones.  I know talented folks that had to go to delivering
pizzas ( not to slag pizza delivery folks) to support themselves and their
families. Some folks ended up leaving the industry because of it and I'm
sure lots of people choose to no get into the field seeing no jobs.  This
type of event causes a whole that takes a long time correct.

On Thu, Jun 11, 2015 at 1:46 AM, Alex White-Robinson ale...@gmail.com
wrote:

 Matthew Petach mpet...@netflight.com wrote:

  On a slightly different note, however--while it's good to
  have an appreciation of the past and how we got here,
  I think it's wise to also recognize we as an industry
  have some challenges bringing new blood in--and
  treating it too much like a sacred priesthood with
  cabalistic knowledge and initiation rites isn't going
  to help us bring new engineers into the field to
  take over for us crusty old farts when our eyes
  give out and we can't type into our 9600 baud
  serial consoles anymore.
 
  Matt
  CCOF #1999322002 [0]

 I've seen very little attention paid to junior talent in the last few
 years, and know a few people who would have been talented engineers that
 never got a chance to show it.
 They moved into other industries because of the lack of junior roles.

 I know very few people in network engineering that are under thirty, and
 not that many under thirty five.


 On Thu, Jun 11, 2015 at 2:01 PM, Matthew Petach mpet...@netflight.com
 wrote:

  On Sun, Jun 7, 2015 at 7:57 PM, Jay Ashworth j...@baylink.com wrote:
  [...]
  
   And this... is NANOG!
 
  Needs more ellipses and capitalization...more like
 
 
  This...IS...NANOG!!!
 
  building up to a nice crescendo roar as you kick the
  hapless interviewee backwards down the deep, dark well
 
 
  On a slightly different note, however--while it's good to
  have an appreciation of the past and how we got here,
  I think it's wise to also recognize we as an industry
  have some challenges bringing new blood in--and
  treating it too much like a sacred priesthood with
  cabalistic knowledge and initiation rites isn't going
  to help us bring new engineers into the field to
  take over for us crusty old farts when our eyes
  give out and we can't type into our 9600 baud
  serial consoles anymore.
 
  Matt
  CCOF #1999322002 [0]
 
 
 
 
  [0] Certified Crufty Old Fart
 

Re: eBay is looking for network heavies...

[jim deleskie]

I remember you asking me who Jon was :)  I have since added to my list of
interview questions... sad but the number of people with clue is declining
not increasing.


On Sat, Jun 6, 2015 at 3:13 AM, Joe Hamelin j...@nethead.com wrote:

 Back in 2000 at Amazon, HR somehow decided to have me do the phone
 interviews for neteng.  I'd go through questions on routing and what not,
 then at the end I would ask questions like, Who was Jon Postel?  Who is
 Larry Wall?  Who is Paul Vixie? What are layers 8  9? Explain the RTFM
 protocol.  What is NANOG?  Those answers (or long silences) told me more
 about the candidate than most of the technical questions.

 --
 Joe Hamelin, W7COM, Tulalip, WA, 360-474-7474

Re: eBay is looking for network heavies...

[jim deleskie]

Based on the number of certified people I've interviewed over the last
20yr, my default view lines up with Jared's 100%

On Fri, Jun 5, 2015 at 10:38 PM, Mike Hale eyeronic.des...@gmail.com
wrote:

 We need a pool on what percentage of readers just googled traceroute.
 On Jun 5, 2015 6:28 PM, na...@cdl.asgaard.org wrote:

  On 5 Jun 2015, at 17:45, Łukasz Bromirski wrote:
 
   On 06 Jun 2015, at 02:26, Jared Mauch ja...@puck.nether.net wrote:
 
 
   On Jun 5, 2015, at 7:13 PM, John Fraizer j...@op-sec.us wrote:
 
  Head of line for CCIE / JNCIE but knowledge and experience trumps a
  piece
  of paper every time!
 
 
  Can you please put these at the back of the line?  My experience is
 that
  the cisco certification (at least) is evidence of the absence of actual
  troubleshooting skills.  (or my standards of what defines “expert” are
  different than the rest of the world).
 
 
  Jared, don’t generalize.
 
  True - there are people that are ‘paper’ CCIE/JNCIEs - but let’s not
  start a rant unless you've met tens of CCIEs/JNCIEs and all of them
  didn’t know a jack. About troubleshooting.
 
 
  't
 
  We had one CCIE at a previous job who just didn't click no matter how
  much we tried to train on the architecture.  Eventually in one backbone
  event, he kept saying that the problem couldn't be with a given router
  because traceroute worked.  When it was pointed out that the potential
  fault wouldn't cause traceroute to fail, we got a very puzzled look.  We
  then asked him to explain how traceroute worked.  He spectacularly
 failed.
 
  It became a tongue-in-cheek interview question.  What was boggling was
 the
  number of *IE's that failed trying to explain traceroute's mechanics.
 
  My test, as crass as it is.  If your CV headlines with a JCIE/CCIE, I am
  pretty certain that you have very little real-world experience.  If it's
 a
  footnote somewhere, that's ok.
 
  Christopher
 
 
 
  —
  CCIE #15929 RS/SP, CCDE #2012::17
  (not that I’d know anything about troubleshooting of course)
 
 
 
  --
  李柯睿
  Avt tace, avt loqvere meliora silentio
  Check my PGP key here: http://www.asgaard.org/cdl/cdl.asc
  Current vCard here: http://www.asgaard.org/cdl/cdl.vcf
  keybase: https://keybase.io/liljenstolpe
 

Re: [SECURITY] Application layer attacks/DDoS attacks

[jim deleskie]

Keith,

  I agree, we can't even get everyone including some LARGE ( I'll avoid
Tier's because people get stupid around that too) networks to filter
customers based on assigned netblocks.

-jim

On Mon, May 25, 2015 at 9:44 AM, Keith Medcalf kmedc...@dessus.com wrote:


 Without a concomitant increase in trustworthy, assigning greater levels
 of trust is fools endeavour.  Whatever this trusted network initiative is,
 I take that  it was designed by fools or government (the two are usually
 indistinguishable) for the purpose of creating utterly untrustworthy
 networks.

  -Original Message-
  From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Ramy Hashish
  Sent: Sunday, 24 May, 2015 22:49
  To: morrowc.li...@gmail.com; nanog@nanog.org
  Subject: Re: [SECURITY] Application layer attacks/DDoS attacks
 
  The idea of restricting access to a certain content during an attack on
  the
  trusted networks only will make all interested ISPs be more trusted
 
  Ramy
 
  On Mon, May 25, 2015 at 5:01 AM, Christopher Morrow
  morrowc.li...@gmail.com
   wrote:
 
   On Sat, May 23, 2015 at 9:12 PM, jim deleskie deles...@gmail.com
  wrote:
However, the trusted network initiative might be a good approach to
   start
influencing operators to apply anti-spoofing mechanisms.
   
  
   explain how you think the 'trusted network initiative' matters in the
   slightest?
  
   -chris
  

Re: [SECURITY] Application layer attacks/DDoS attacks

[jim deleskie]

While I don't think any ISP wants DDoS to make $$, I do based on
experience believe that business cases have to be made for everything.
With the prices pay for BW in most of the world now, ( or the last number
of years) its going to be VERY hard to get anyone to allocated time/$$ or
energy to do anything they don't need to, to get the bit to you.

-jim

On Sat, May 23, 2015 at 6:33 PM, Ramy Hashish ramy.ihash...@gmail.com
wrote:

 Yes Harlan, you are absolutely right, even if this won't stop the
 botnet-based DDoS attacks, but at least will significantly decrease the
 volume/frequency of the volume based attacks.

 On the other side, the DDoS protection now become a business where
 all-tiers ISPs make money of, and those ISPs is the exact place where the
 implementation of anti-spoofing make the best sense, conflict of interests
 now...

 However, the trusted network initiative might be a good approach to start
 influencing operators to apply anti-spoofing mechanisms.

 Salam,

 Ramy
 On 23 May 2015 10:48 pm, Harlan Stenn st...@ntp.org wrote:

 Just to ask, what is the expected effect on DDoS attacks if folks
 implemented BCP38?

 How does the cost of implementing BCP38 compare to the cost of other
 solution attempts?

 H

Re: [SECURITY] Application layer attacks/DDoS attacks

[jim deleskie]

To many pieces to answer on a weekend on NANOG, but those of us that work
in the DDoS space the last number of years have seen huge growth in the
application layer attacks. This does not mean a decrease in volumetric
attack, just that now you have to worry about both and lots of each.  FW's
while they have got better are still not the solution for many reasons.
Moving things to the cloud helps in come cases but not all.  This is an
arms race, the better we protecting the better the bad guys get at
attacking.

-jim

On Sat, May 23, 2015 at 9:56 AM, Ramy Hashish ramy.ihash...@gmail.com
wrote:

 Hello there,

 As a reaction to the increasing demand -from enterprises- over the DDoS
 protection services, a fierce competition between vendors is about to start
 in this playground, big upfront investments started to happen in the tier
 one, tier two and tier three ISPs, IMHO this will have its aggressive
 effect on the volume of the DDoS attacks, and will eventually steer the
 mindset of the enterprises towards hosting the most critical
 applications/services in a well geographically-dispersed cloud and
 increasing the surface area using anycast then relatively decreasing the
 attack volume.

 Back to the DDoS protection, most anti-DDoS vendors are marketing their
 products as application layer attack DDoS defense, I am little bit
 confused; aren't the application firewalls -either integrated in a NGFW
 or a UTM- the responsible for mitigating application layer attacks?

 Thanks,

 Ramy

Re: Cisco/Level3 takedown

[jim deleskie]

Just to add to the noise I think batman wears a black mask/helmet, but
I've never considered it a mask.  I didn't look at the details on this, but
did L3 sink the routes at their border or did they expressly announce the
route to sink it?


-jim

On Thu, Apr 9, 2015 at 3:35 PM, Randy Bush ra...@psg.com wrote:

  Wrong. Batman, for example, wears a black hat.
  vigilantes always wear white hats.

 i stand corrected

Re: Large Ontario DC busted for hosting petabytes of child abuse material

[jim deleskie]

Canadian and US laws are similar.  But I'll leave it up to the lawyers to
figure it all out, happily I'm no where near this, but it being a small
industry here, I suspect I have friends that are dealing with some crap
right now


On Mon, Mar 2, 2015 at 2:03 PM, Mike A mi...@mikea.ath.cx wrote:

 On Mon, Mar 02, 2015 at 05:53:33PM +, Naslund, Steve wrote:
  Don't know who this is but the legalities are pretty clear I think. The
 DC
  is not required to know what data is stored but if the cops can prove
 that
  someone DID know what was stored, that person can be criminally charged.
  IANAL but I have worked with LE on a similar case and that is how it was
  explained to us by the FBI. It will be hard to prove anyone knew however
  since anyone that knew and did not report it committed a crime. Charging
 the
  company will be a stretch unless they can prove that at least one
 corporate
  officer knew. Otherwise the company will fire whichever employee knew and
  say He should have told us.
 
  This is all about who knew what and when.

 True in the USA, I think; but what about Canadian law?

 Popcorn and hyperhumongous drinks time.

 --
 Mike Andrews, W5EGO
 mi...@mikea.ath.cx
 Tired old sysadmin

Re: What is lawful content? [was VZ...]

[deleskie]

I wonder if lawyer sit around all day and argue about CIDR notation

Sent from my BlackBerry 10 smartphone on the Rogers network.
  Original Message  
From: Jim Richardson
Sent: Friday, February 27, 2015 7:26 PM
Cc: NANOG list
Subject: Re: What is lawful content? [was VZ...]

On Fri, Feb 27, 2015 at 2:23 PM, Patrick W. Gilmore patr...@ianai.net wrote:
 I am not a lawyer (in fact, I Am Not An Isp), but my understanding is this is 
 pretty well settled.

 And it is not even weird or esoteric. If the content on the site is against 
 the law in the jurisdiction in question, it is not legal (duh). Otherwise, 
 yes it is, and no ISP gets to decide whether you can see it or not.

Which is the jurisdiction in question ? the originating website? the
ISP? the CDN network's corporate home? my home?

Re: Checkpoint IPS

[jim deleskie]

mh,

 you know that forcing traffic to be symmetrical is evil, and while
backbone traffic and inspection don't play nice, there are very legit
reasons why, in many cases edge traffic must be open for inspection.  I'm
on my way to the office, feel free to ping me if you want to discuss.  Or
maybe I could use it as a reason to come visit  its been a while since
we've had a chance to vis-a-vis :)


-jim

On Thu, Feb 5, 2015 at 8:57 AM, Terry Baranski 
terry.baranski.l...@gmail.com wrote:

 On 5 Feb 2015, at 01:56, Michael Hallgren wrote:
  Le 04/02/2015 17:19, Roland Dobbins a écrit :
 
  Real life limitations?
  https://app.box.com/s/a3oqqlgwe15j8svojvzl
 
  Right ;-) Among many other nice ones, I like:
 
  `` ‘IPS’ devices require artificially-engineered topological symmetry-
  can have a negative impact on resiliency via path diversity.''

 Dang, I thought this quote was from an April 1st RFC when I first read it.

 I hate to be the bearer of bad news, but everything we do is artificial.
 There are no routers in nature, no IP packets, no fiber optics. There is no
 such thing as natural engineering -- engineering is artificial by
 definition.

 So when you're configuring artificially-engineered protocols on your
 artificially-engineered router so that your artificially-engineered network
 can transmit artificially-engineered packets, adding some extra
 artificially-engineered logic to enforce symmetry won't break the bank, I
 promise. And when done properly it has absolutely no impact on resilience
 and path diversity, and will do you all the good in the world from a
 troubleshooting perspective (those of you who operate networks).

 The whole presentation is frankly just odd to me. It looks at one specific
 CND thread (DDoS), and attempts to address it by throwing out the baby with
 the bathwater. It says to eliminate state at all costs, but then at the end
 advocates for reverse proxies -- which are stateful, and which therefore
 create the same problems as firewalls and IPSs.

 The idea of ripping out firewall/IPS devices and replacing them with router
 ACLs is something that, if I were an attacker, I would definitely encourage
 all of my targets to do. Firewalls aren't so much the big issue -- one can
 theoretically use router ACLs for basic L3/L4 blocks, though they scale
 horribly from an OM perspective, are more prone to configuration errors,
 and their manageability is poor. But there's no overstating the usefulness
 of a properly-tuned IPS for attack prevention, and the comment in the brief
 comparing an IPS to [Having] your email client set to alert you to
 incoming
 mail is so bizarre that I wouldn't even know how to counter it.

 (I know you're out there Roland and my intention isn't to get into a big
 thing with you. But the artificial-engineering thing gave me a chuckle.)

 On 5 Feb 2015, at 02:49, Michael Hallgren wrote:
  Le 05/02/2015 08:01, Roland Dobbins a écrit :
 
  The real question is, why 'inspect', at all?
 
  Yes, that's an even more interesting discussion!

 Only if your assets aren't targets. :-)

 -Terry

Re: Facebook down?

[jim deleskie]

From East coast of Canada down as well.


On Wed, Sep 3, 2014 at 4:48 PM, Warren Bailey 
wbai...@satelliteintelligencegroup.com wrote:

 I¹m getting a ton done right now too.. Hasn¹t been working since my first
 attempt about 20 minutes ago.



 On 9/3/14, 12:45 PM, Marshall Eubanks marshall.euba...@gmail.com
 wrote:

 http://www.downforeveryoneorjustme.com/facebook.com
 
 It's not just you! *http://facebook.com* http://facebook.com/ looks
 down
 from here.
 
 Relevant because of the likely increase in productiviity
 
 
 Regards
 
 Marshall Eubanks

Re: [OPINION] Best place in the US for NetAdmins

[jim deleskie]

Rich,

 In principal I agree, and I've said this many times, for years I've
telecommuted myself, mostly effectively.  I'd work much longer hours, but
not always worked as efficiently during all of those hours.  When I started
my own company, with $$ be in short supply like all start ups I I planned
to have as many folks telecommute as possible.  In some cases it worked
out, in others it was a terrible failure.  Maybe it was my hiring choices,
maybe it was being a bad manager but without people in the office it was
harder to tell.  Also with most people under one roof now, I also see the
on going information sharing that isn't as possible with a mostly remote
office.

-jim


On Sat, Jul 26, 2014 at 8:04 AM, Rich Kulawiec r...@gsp.org wrote:

 On Fri, Jul 25, 2014 at 05:35:45PM -0700, Scott Weeks wrote:
  One day, hopefully, telecommuting really takes off [...]

 It often strikes me as incredibly ironic that companies which *would
 not exist* were it not for the Internet are among the most resistant
 to the simple, obvious concept that telecommuting allows them to hire
 the best and brightest regardless of geography.

 Telecommuting should not be a rare exception: it should be the default.
 And corporate headquarters should be as small and inexpensive as
 possible,
 staffed (in person) only by a handful of people -- if even that.  Asking
 net admins to do stupid, wasteful, expensive things like commute 3 hours
 a day and live in areas with ridiculously inflated housing prices is a
 good way to filter *out* the employees one would most like to have.

 ---rsk

Re: Verizon Public Policy on Netflix

[jim deleskie]

So it sounds like your customers want to use the service being sold, but
you can't afford to service them due to the pricing they are being
charged...Sounds like you need to raise prices.  While I haven't worked for
a rural wireless ISP, I have work for wired ISP's in the days of modems,
Large transit networks and MSO's.  If it costs you more to provide service
then you charge for it, your a charity, not a business.

-jim


On Sun, Jul 13, 2014 at 1:09 PM, na...@brettglass.com wrote:

 At 11:39 PM 7/12/2014, Steven Tardy wrote:

 How would 4U of rent and 500W($50) electricity *not* save money?

 Because, on top of that, we'd have huge bandwidth expenses. And Netflix
 would refuse to cover any of that out of the billions in fees it's
 collecting
 from subscribers. We can't raise our prices (that would not only cost us
 customers but be unfair to many of them; it would be forcing the
 non-Netflix
 users to subsidize Netflix). We simply need Netflix to pay at least some
 of its
 freight.

 If your ISP isn't tall enough for Netflix, Akamai has a lower barrier of
 entry.
 Have you let Akamai give you a local cache? why or why not?

 Akamai refused to do so when we approached them. The Akamai rep was rather
 rude
 and dismissive about it; we were too small to be worthy of their attention.

 It's important to note that the growth of rural ISPs is limited by
 population.
 Even if we did not have rapacious cable and telephone monopolies to compete
 with, our size is naturally limited by the number of possible customers.
 Each
 of those customers is every bit as valuable as an urban customer, but
 Netflix
 won't even give us the SAME amount per customer it gives Comcast, much less
 more (it costs more to serve each one). And Netflix is particularly out of
 line
 because it is insisting that we pay huge bandwidth bills for an exclusive
 connection just to it. It is also wasting our existing bandwidth by
 refusing to
 allow caching.

 If Netflix continues on its current course, ALL ISPs -- not just rural
 ones,
 will eventually be forced to rebel. And it will not be pretty.

 Our best hope, unless Netflix changes its ways, is for a competitor to come
 along which has more ISP-friendly practices. Such a competitor could easily
 destroy Netflix via better relations with ISPs... and better performance
 and
 lower costs due to caching at the ISP.

 --Brett Glass

Re: Verizon Public Policy on Netflix

[deleskie]

I've only been 1/2 paying attention, did I miss the sarcasm tag are are 
people really looking for those answers.

-jim

Sent from my BlackBerry 10 smartphone on the Rogers network.
  Original Message  
From: Miles Fidelman
Sent: Saturday, July 12, 2014 6:11 PM
Cc: NANOG
Subject: Re: Verizon Public Policy on Netflix

Joly MacFie wrote:
 Now we're
 so far off in the weeds, I can't even
 see where we started from. ^_^;;



 What I'd like to know is

 1) when does a terminating network become a transit network, and..
 2 )are there, should there, be different peering standards for each, and
 3) if so some kind of functional if not structural separation
 4) by regulation?


Ditto. These questions really get to the nub of the current issues!

Miles Fidelman




-- 
In theory, there is no difference between theory and practice.
In practice, there is.  Yogi Berra

Re: Canada and IPv6 (was: Ars Technica on IPv4 exhaustion)

[jim deleskie]

Those all sounds like legit business questions.

-jim


On Thu, Jun 19, 2014 at 2:45 PM, William F. Maton Sotomayor 
wma...@ottix.net wrote:

 On Wed, 18 Jun 2014, Sadiq Saif wrote:

  On 6/18/2014 14:25, Lee Howard wrote:

 Canada is way behind, just 0.4% deployment.


 Any Canadian ISP folk in here want to shine a light on this dearth of
 residential IPv6 connectivity?

 Is there any progress being made on this front?


 Teksavvy does it (tunnel I believe) if you ask.

 Otherwise it's the usual:

 - 'why do we need this?';
 - 'It costs money to upgrade for something low-demand';
 - 'What's the market?';
 - 'I don't have time';
 - 'Aw gee do I have to??'

 wfms

Re: A simple proposal

[deleskie]

You shouldn't of stopped them I was eagerly ‎ waiting to find out how rtt was 
going to be increased :)

-jim

Sent from my BlackBerry 10 smartphone on the Rogers network.
  Original Message  
From: Suresh Ramasubramanian
Sent: Friday, May 16, 2014 11:26 PM
To: Phil Fagan
Cc: nanog@nanog.org
Subject: Re: A simple proposal

Wow nanog, dissecting the architecture of a sarcastic proposal.

Maybe the joke would have been clearer if Matt had used the phrase a
modest proposal ..

On Saturday, May 17, 2014, Phil Fagan philfa...@gmail.com wrote:

 I agree with Rahul, seems like pointless cycles along the entire path.


 On Thu, May 15, 2014 at 11:35 PM, Rahul Sawarkar 
 srahul...@gmail.comjavascript:;
 wrote:

  You mean consume electricity in cpu cycles on the end devices and all
 the
  network middleboxes in between all over the world/Internet for dud data?
  For what? Just to stop a debate instead of resolving it thought
  intellectual brainstorming? For one thing it will slow down the TCP
  connections as ACKs incur a longer RTT. Then there is the whole question
 of
  managing and lowering power consumption as a green initiative, and
  capacity issues are yet another thing.
 
  ~Rahul
 
 
  On Fri, May 16, 2014 at 10:56 AM, Matthew Petach 
  mpet...@netflight.comjavascript:;
  wrote:
 
   There's been a whole lot of chatter recently
   about whether or not it's sensible to require
   balanced peering ratios when selling heavily
   unbalanced services to customers.
  
   There's a very simple solution, it seems.
   Just have every website, every streaming
   service, every bit of consumable internet
   data have built-in reciprocity.
  
   You want to stream a movie? No problem;
   the video player opens up a second data
   port back to a server next to the streaming
   box; its only purpose is to accept a socket,
   and send all bits received on it to /dev/null.
   The video player sends back an equivalent
   stream of data to what is being received in.
   The server receiving the upstream data stream
   checks the bitrate coming into it from the player,
   and communicates back to the video streaming
   box every few minutes to lower the outbound
   bitrate going to the player to match what the
   inbound bitrate coming from the client is.
   That way, traffic volumes stay nicely balanced,
   and everyone is happy. For extra credit, and
   to deal with multiple layers of NAT in the v4
   world, you could even piggyback on the same
   stream, though that would take just a bit more
   work.
  
   Mobile apps could be programmed the same
   way; you download a certain amount of data,
   an equivalent volume of data is sent back
   upstream to balance it out, and preserve
   the holy ratio. Even web pages could use
   javascript footers to send back upstream an
   equivalent amount of data to what was
   downloaded.
  
   Once and for all, we could put an end to
   the ceaseless bickering about ratios, as
   everyone, everywhere would be forced
   into glorious unity, a perfect 1:1 ratio
   wherever the eye should look.
  
   As far as I can tell, this should solve
   *everyone's* concerns from all sides,
   all in one simple effort.
  
   Matt
  
 
 
 
  --
  ~~
  Regards
  Rahul
 



 --
 Phil Fagan
 Denver, CO
 970-480-7618



-- 
--srs (iPad)

Re: [ PRIVACY Forum ] Critical crypto bug leaves Linux, hundreds of apps open to eavesdropping

[jim deleskie]

Doing some serious adjusting of my tinfoil today over his :)

-jim


On Wed, Mar 5, 2014 at 5:03 PM, Jay Ashworth j...@baylink.com wrote:

 - Original Message -
  From: Leo Bicknell bickn...@ufp.org

  On Mar 4, 2014, at 9:07 PM, Jay Ashworth j...@baylink.com wrote:
 
   Is this the *same* bug that just broke in Apple code last week?
 
  No, the Apple bug was the existence of an /extra/ goto fail;.
 
  The GnuTLS bug was that it was /missing/ a goto fail;.
 
  I'm figuring the same developer worked on both, and just put the line
  in the wrong repository. :)

 Those who speculate that these bugs happened at the behest of the NSA
 would probably agree with you.

 Cheers,
 -- jra
 --
 Jay R. Ashworth  Baylink
 j...@baylink.com
 Designer The Things I Think   RFC
 2100
 Ashworth  Associates   http://www.bcp38.info  2000 Land
 Rover DII
 St Petersburg FL USA  BCP38: Ask For It By Name!   +1 727 647
 1274

Re: Hackers hijack 300, 000-plus wireless routers, make malicious changes | Ars Technica

[jim deleskie]

Why want to swing such a big hammer.  Even blocking those 2 IP's will
isolate your users, and fill your support queue's.

Set up a DNS server locally to reply to those IP's  Your customers stay up
and running and blissfully unaware.

Log the IP's hitting your DNS servers on those IP and have your support
reach out to them in a controlled way, or  reply to any request via DNS
with an internal host that has a web page explaining what is broken and how
they can fix it avoiding  at least some of the calls to your helpdesk.

-jim


On Tue, Mar 4, 2014 at 7:54 AM, Andrew Latham lath...@gmail.com wrote:

 On Tue, Mar 4, 2014 at 5:46 AM, fmm vo...@fakmoymozg.ru wrote:
  On Tue, 04 Mar 2014 09:00:18 +0100, Jay Ashworth j...@baylink.com
 wrote:
 
 
 
 http://arstechnica.com/security/2014/03/hackers-hijack-30-plus-wireless-routers-make-malicious-changes/
 
  Is there any valid reason not to black hole those /32s on the back bone?
 
 
 
  The telltale sign a router has been compromised is DNS settings that
 have
  been changed to 5.45.75.11 and 5.45.76.36. Team Cymru researchers
 contacted
  the provider that hosts those two IP addresses but have yet to receive
 a
  response.
 
 
  you wanted to say blackhole those 5.45.72.0/22 and 5.45.76.0/22,
 aren't
  you?
 
 
  Cheers
 

 Jay is right, it is just the /32s at the moment...  Dropping the /22s
 could cause other sites to be blocked.

 inetnum:5.45.72.0 - 5.45.75.255
 netname:INFERNO-NL-DE
 descr:  
 descr:  * We provide virtual and dedicated servers on this Subnet.
 descr:  *
 descr:  * Those services are self managed by our customers
 descr:  * therefore, we are not using this IP space ourselves
 descr:  * and it could be assigned to various end customers.
 descr:  *
 descr:  * In case of issues related with SPAM, Fraud,
 descr:  * Phishing, DDoS, portscans or others,
 descr:  * feel free to contact us with relevant info
 descr:  * and we will shut down this server: ab...@3nt.com
 descr:  
 country:NL
 admin-c:TNTS-RIPE
 tech-c: TNTS-RIPE
 status: ASSIGNED PA
 mnt-by: MNT-3NT
 mnt-routes: serverius-mnt
 source: RIPE # Filtered




 --
 ~ Andrew lathama Latham lath...@gmail.com http://lathama.net ~

Re: NTP DRDos Blog post

[deleskie]

Re: NSA able to compromise Cisco, Juniper, Huawei switches

[jim deleskie]

There are many ways a backdoor could be used in a properly secured system.
  To think otherwise is a huge mistake.  I can think of several ways, if
tasked and given the resources of a large gov't that I would attack this
problem.  To assume that those tasked and focused only this type of
solution aren't even more capable would be foolhardy.


-jim


On Mon, Dec 30, 2013 at 12:28 PM, Marco Teixeira ad...@marcoteixeira.comwrote:

 Hi all,

 I've been watching this list for a couple weeks now and while risking
 beeing flamed, i just wanted to say that any network professional that puts
 any equipment into production without securing it against the kind of
 issues mentioned so far (cisco/cisco, snmp private, etc) is negligent and
 should be fired on the spot.

 These are not backdoor issues, NSA related, whatever... This is noise.
 Trying to get this thread on track, can the original poster provide any
 proof of this so called ability of the so called inteligence agency beeing
 able to access cisco/juniper, taking into account that management access
 has been correctly configured ?

 Regards

 -Marco


 ---
 Cumprimentos / Best regards

 Marco Teixeira
 email/gtalk/msn: ad...@marcoteixeira.com
 skype: admin-marcoteixeira.com
 ---
 Did you know that Marco Teixeira is an independent,  industry expert,
 senior
 consultant ? His expertise is available for hire.
 ---


 On Mon, Dec 30, 2013 at 4:16 PM, Enno Rey e...@ernw.de wrote:

  On Mon, Dec 30, 2013 at 04:03:07PM +, Dobbins, Roland wrote:
  
   On Dec 30, 2013, at 10:44 PM, valdis.kletni...@vt.edu 
  valdis.kletni...@vt.edu wrote:
  
What percentage of Cisco gear that supports a CALEA lawful intercept
  mode is installed in situations where CALEA doesn't apply, and thus
 there's
  a high likelyhood that said support is misconfigured and abusable without
  being noticed?
  
   AFAIK, it must be explicitly enabled in order to be functional.  It
  isn't the sort of thing which is enabled by default, nor can it be
 enabled
  without making explicit configuration changes.
 
  at least back in 2007 it could be enabled/configured by SNMP RW access
  [see slide 43 of the presentation referenced in this post
 
 http://www.insinuator.net/2013/07/snmp-reflected-amplification-ddos-attacks/
 ]
  so knowing the term private m
  ight be enough to perform the task remotely.
 
  have a good one
 
  Enno
 
 
 
 
  
   ---
   Roland Dobbins rdobb...@arbor.net // http://www.arbornetworks.com
  
   Luck is the residue of opportunity and design.
  
-- John Milton
  
 
 
 
  --
  Enno Rey
 
  ERNW GmbH - Carl-Bosch-Str. 4 - 69115 Heidelberg - www.ernw.de
  Tel. +49 6221 480390 - Fax 6221 419008 - Cell +49 173 6745902
 
  Handelsregister Mannheim: HRB 337135
  Geschaeftsfuehrer: Enno Rey
 
  ===
  Blog: www.insinuator.net || Conference: www.troopers.de
  ===
 
 

Re: The Making of a Router

[jim deleskie]

I've recently pushed a large BSD box to a load of over 300, for more then
an hour, while under test,  some things slowed a little, but she kept on
working!

-jim


On Thu, Dec 26, 2013 at 1:59 PM, Shawn Wilson ag4ve...@gmail.com wrote:

 Totally agree that a routing box should be standalone for tons of reasons.
 Even separating network routing and call routing.

 It used to be that BSD's network stack was much better than Linux's under
 load. I'm not sure if this is still the case - I've never been put in the
 situation where the Linux kernel was at its limits. FWIW

 Jared Mauch ja...@puck.nether.net wrote:
 Have to agree on the below. I've seen too many devices be so integrated
 they do no task well, and can't be rebooted to troubleshoot due to
 everyone using them.
 
 Jared Mauch
 
  On Dec 26, 2013, at 10:55 AM, Andrew D Kirch trel...@trelane.net
 wrote:
 
  Don't put all this in one box.

Re: Someone’s Been Siphoning Data Through a Huge Security Hole in the Internet

[deleskie]

Re: Internet Surveillance and Boomerang Routing: A Call for Canadian Network Sovereignty

[jim deleskie]

Paul,

  I agree this is a problem, but its been a problem since at least 1994 (
my first  exposure ) and I suspect longer, the issue is east we capacity in
Canada is very $$, pushing traffic from Toronto east to points south to get
it to Vancouver is much more cost effective.

-jim


On Sat, Sep 7, 2013 at 6:08 PM, Paul Ferguson fergdawgs...@mykolab.comwrote:


 A Canadian ISP colleague of mine suggested that the NANOG constituency
 might be interested in this, given some recent 'revelations', so I forward
 it here for you perusal.



 Preliminary analysis of more than 25,000 traceroutes reveals a
 phenomenon we call ‘boomerang routing’ whereby Canadian-to-Canadian
 internet transmissions are routinely routed through the United States.
 Canadian originated transmissions that travel to a Canadian destination
 via a U.S. switching centre or carrier are subject to U.S. law -
 including the USA Patriot Act and FISAA. As a result, these
 transmissions expose Canadians to potential U.S. surveillance activities
 – a violation of Canadian network sovereignty.

 http://lawprofessors.typepad.**com/media_law_prof_blog/2013/**
 09/routing-internet-**transmission-across-the-**canada-us-border-and-us-**
 surveillance-activities.htmlhttp://lawprofessors.typepad.com/media_law_prof_blog/2013/09/routing-internet-transmission-across-the-canada-us-border-and-us-surveillance-activities.html

 Cheers,

 - ferg


 --
 Paul Ferguson
 Vice President, Threat Intelligence
 Internet Identity, Tacoma, Washington  USA
 IID -- Connect and Collaborate -- www.internetidentity.com

Re: [Paper] B4: Experience with a Globally-Deployed Software Defined

[jim deleskie]

At iMCI  (pre-Worldcom) we had scripts that would build all our ATM VC's
for a 400node mesh, would take all night to run :)


-jim


On Sat, Aug 17, 2013 at 4:32 PM, Avi Freedman freed...@freedman.net wrote:


 No, people never use *flow controllers* for anything.

 People have been doing SDN since before Google was around.

 OK, so it was horrible expect scripts but it worked.

 Avi

  Unpossible.  I heard that no one really uses sdn for anything.
 
  :)
 
  T

Re: Friday Hosing

[jim deleskie]

I could support any of these services myself, and have guys that work me
that can as well, but none of these are my core business, and my investors
REALLY prefer me focusing on my core business, I suspect most of us have
shareholders, investors, owners that feel the same way.  I struggled with
idea of not running my own boxes for services, but in the end decided that
the trade of various gov't reading my boring office mail was the right
choice for my business.

-jim


On Sun, Jul 14, 2013 at 9:09 PM, Tony Patti t...@swalter.com wrote:

 I think it is (could be) (should be) realistic for many/most businesses.

 TWELVE years ago (press release March 20 2001), Comcast deployed
 Linux-based
 Sun Cobalt Qube appliances as CPE with their business-class Internet
 service,
 these provided firewall security, web caching, optional content filtering,
 an e-mail server, a web server, file and print servers.


 http://www.prnewswire.com/news-releases/comcast-business-communications-hits
 -a-home-run-with-detroits-comerica-park-71752402.html

 You could argue that
 (a) it was not your own server, even though it was CPE, or
 (b) Comcast did not continue to offer these appliances (i.e. that Sun
 cancelled the product line),
 but my point is that it was provided within the economics of the Internet
 Services being purchased, i.e. not cost-prohibitive.

 Tony Patti
 CIO
 S. Walter Packaging Corp.

 -Original Message-
 From: Patrick W. Gilmore [mailto:patr...@ianai.net]
 Sent: Sunday, July 14, 2013 6:23 PM
 To: NANOG list
 Subject: Re: Friday Hosing

 On Jul 12, 2013, at 19:22 , Nick Khamis sym...@gmail.com wrote:

  Set up your own email server, host your own web pages, maintain your
  own cloud, breath your own oxygen FTW.

 That's simply not realistic for many companies and essentially all people
 (to a first approximation).

 --
 TTFN,
 patrick

Re: /25's prefixes announced into global routing table?

[jim deleskie]

I'm not going to even ask or look at who is accepting /26's

-jim


On Mon, Jun 24, 2013 at 2:29 PM, Paul Rolland r...@witbe.net wrote:

 Hello,

 On Fri, 21 Jun 2013 13:56:02 -0600
 Michael McConnell mich...@winkstreaming.com wrote:

  As the IPv4 space get smaller and smaller, does anyone think we'll see a
  time when /25's will be accepted for global BGP prefix announcement. The
  current smallest size is a /24 and generally ok for most people, but the
  crunch gets tighter, routers continue to have more and more ram will it
  always be /24 the smallest size?

 Well, /25 are already in the routing table. I can even find a few /26 !!

 rtr-01.PAR#sh ip b | i /26
 *i193.41.227.128/26
 *i193.41.227.192/26
 *i194.149.243.64/26

 Paul

 --
 TelcoTV Awards 2011 - Witbe winner in Innovation in Test  Measurement

 Paul RollandE-Mail : rol(at)witbe.net
 CTO - Witbe.net SA  Tel. +33 (0)1 47 67 77 77
 Les Collines de l'Arche Fax. +33 (0)1 47 67 77 99
 F-92057 Paris La DefenseRIPE : PR12-RIPE

 LinkedIn : http://www.linkedin.com/in/paulrolland
 Skype: rollandpaul

 I worry about my child and the Internet all the time, even though she's
 too young to have logged on yet. Here's what I worry about. I worry that 10
 or 15 years from now, she will come to me and say 'Daddy, where were you
 when they took freedom of the press away from the Internet?'
 --Mike Godwin, Electronic Frontier Foundation

Re: net neutrality and peering wars continue

[jim deleskie]

Botnets to help with peering ratio's could be a new business model? :)


On Sat, Jun 22, 2013 at 1:00 PM, Christopher Morrow morrowc.li...@gmail.com
 wrote:

 On Sat, Jun 22, 2013 at 9:19 AM, Neil Harris n...@tonal.clara.co.uk
 wrote:
  On 22/06/13 13:08, Matthew Petach wrote:
  That's easily solved by padding the ACK to 1500 bytes as well.
 
  Matt
 
 
  Or indeed by the media player sending large amounts of traffic back to
 the
  CDN via auxiliary HTTP POST requests?

 ah... botnet... how I love thee?

Re: PRISM: NSA/FBI Internet data mining project

[jim deleskie]

Knowing its going on, knowing nothing online is secret != OK with it, it
mealy understand the way things are.

-jim


On Thu, Jun 6, 2013 at 9:16 PM, goe...@anime.net wrote:

 On Thu, 6 Jun 2013, Matthew Petach wrote:

 Much less stress in life that way.  ^_^


 complacency is always the easiest path.

 many abuse@ mailboxes follow the same policy.

 -Dan

Re: What hath god wrought?

[jim deleskie]

Maybe my tinfoil isn't on tight enough, or maybe I give to much credit to a
gov't, or perhaps I'm just feeding the trolls, but I have a very hard time
believing that DHS, launched a DoS from their own machines.


-jim


On Tue, May 21, 2013 at 12:18 PM, David Conrad d...@virtualized.org wrote:

 On May 20, 2013, at 9:56 PM, Jay Farrell jay...@jayfar.com wrote:
  Are you certain it was a DoS attempt?

 And if you were certain, are you certain the folks at DHS were aware their
 machine(s) were engaged in a DoS attack?

 You can find zombies in the oddest places...

 Regards,
 -drc

Re: Google Public DNS having issues.

[jim deleskie]

reachable from eastern canada

On Thu, Feb 7, 2013 at 1:41 PM, Blair Trosper blair.tros...@gmail.com wrote:
 ...seems to be having trouble as reported by Systems Watch:
 https://twitter.com/systemswatch/status/299572918936039424

 Indeed, it's inaccessible to me from Minneapolis, Tampa, SJC, and
 Seattle...both 8.8.8.8 and 8.8.4.4.

 I know it's anycast, so I'm not sure which DCs are affected...

 Blair

Re: NSA and the exchanges

[jim deleskie]

If your talking the NSA I doubt anyone would tell you.  That being
said: it would mean the US gov't breaking Canadian law I suspect.  Now
in Canada it is quite possible that the Canadian Fed gov't monitors
traffic but I would also say no one would tell you because telling you
would also be in violation in wiretap laws.

Best advice, assume they do and hope they don't. :)

-jim

On Wed, Oct 31, 2012 at 3:25 PM, andy lam anwa...@yahoo.com wrote:
 Anyone knows if there's a way to find out how involved NSA monitors 151 front 
 street at Toronto?  NSA allegedly monitors data centres in the US, but does 
 it have the same influence at a building sitting in its neighbor's soil?

 There's something on the web like www.ixmaps.ca that tries to piece it 
 together.  but not sure how helpful the information on there really is?


 feedback welcome.

Re: max-prefix and platform tcam limits: they are things

[jim deleskie]

I know that I should know better then comment on networks others then
my own, ( and I know to never comment on my own publicly :) )


But here goes, 210x the size of normal really?  210% I'd have a hard
time believing. Did anyone else anywhere see a route leak equal to
larger then the entire Internet that day, anywhere else that could of
caused this?

I won't even get into max-prefix and how we've managed this long with
someone people still not setting them.


-jim
On Fri, Oct 5, 2012 at 7:31 PM, Anton Kapela tkap...@gmail.com wrote:
 Submitted without comment:
 http://inside.godaddy.com/inside-story-happened-godaddy-com-sept-10-2012/

 -Tk

Re: max-prefix and platform tcam limits: they are things

[jim deleskie]

Yes that math would work, but if your device can't handle 1x Internet
routing and your running without some serious max-prefix/filters it
says even more about your IP eng team then I'd be willing to comment
on.

-jim

On Fri, Oct 5, 2012 at 9:17 PM,  valdis.kletni...@vt.edu wrote:
 On Fri, 05 Oct 2012 21:05:07 -0300, jim deleskie said:

 But here goes, 210x the size of normal really?  210% I'd have a hard
 time believing. Did anyone else anywhere see a route leak equal to
 larger then the entire Internet that day, anywhere else that could of
 caused this?

 If the device was only expecting 2K or so internal routes, getting hit with
 the 440K routes in the DFZ would be 210x

Re: /. Terabit Ethernet is Dead, for Now

[jim deleskie]

  That problem IMO will only be worse with a 4x speed multiplier over
100G what premium will anyone be willing to spend to have a single
400G pipe over 4 bonded 100G pipes?

-jim

On Thu, Sep 27, 2012 at 10:07 AM, Jared Mauch ja...@puck.nether.net wrote:

 On Sep 27, 2012, at 8:58 AM, Darius Jahandarie djahanda...@gmail.com wrote:

 I recall 40Gbit/s Ethernet being promoted heavily for similar reasons
 as the ones in this article, but then 100Gbit/s being the technology
 that actually ended up in most places. Could this be the same thing
 happening?

 I would say yes, except for the physics involved here.  Getting the signal 
 done optically is the easy part.

 I'm not concerned if the next step after 100 is 400.  It's in the right 
 direction and a fair multiple.  There is also a problem in the 100GbE space 
 where the market pricing hasn't yet reached an amount whereby the economics 
 are close enough to push people beyond N*10G.

 - Jared