Re: Centurylink having a bad morning?

[Chase Christian]

Multiple BGP sessions with Level3 (DIA) started flapping at approx 03:00
Pacific:

Aug 30 03:05:13 rtr02 Rib: %BGP-3-NOTIFICATION: sent to neighbor 4.35.X.Y
(AS 3356) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes
Aug 30 03:05:13 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old
state Established event HoldTime new state Idle
Aug 30 03:07:37 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old
state OpenConfirm event RecvKeepAlive new state Established
Aug 30 03:15:38 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old
state Established event HoldTime new state Idle
Aug 30 03:17:15 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old
state OpenConfirm event RecvKeepAlive new state Established
Aug 30 03:19:55 rtr02 Rib: %BGP-3-NOTIFICATION: sent to neighbor
4.35.X.Y+52091 (proto) 6/7 (Cease/connection collision resolution) 0 bytes
Aug 30 03:20:11 rtr02 Rib: %BGP-3-NOTIFICATION: received from neighbor
4.35.X.Y (AS 3356) 4/0 (Hold Timer Expired Error/Unspecified) 0 bytes
Aug 30 03:20:11 rtr02 Rib: %BGP-5-ADJCHANGE: peer 4.35.X.Y (AS 3356) old
state Established event RecvNotify new state Idle

And incoming traffic from AS3356 and AS209 both dropped to very low volumes.

On Sun, Aug 30, 2020 at 5:58 AM Jason Kuehl  wrote:

> Well, When I tried calling I got a fast busy, so that's nice.
>
> On Sun, Aug 30, 2020 at 8:33 AM David Hubbard <
> dhubb...@dino.hostasaurus.com> wrote:
>
>> Same.  Also, as reported on outages list, what’s even worse is that they
>> appear to be continuing to propagate advertisements from circuits whose
>> sessions have been turned down.  I validated ours still were via a couple
>> looking glass portals.  Down Detector shows nearly every major service
>> provider impacted.
>>
>>
>>
>> They’re not reachable so who knows if they’re even working on it.  I feel
>> like they’ve been cutting heavily on the network ops side in recent years…
>>
>>
>>
>> *From: *NANOG  on
>> behalf of Drew Weaver via NANOG 
>> *Reply-To: *Drew Weaver 
>> *Date: *Sunday, August 30, 2020 at 8:23 AM
>> *To: *"nanog@nanog.org" 
>> *Subject: *Centurylink having a bad morning?
>>
>>
>>
>> Hello,
>>
>>
>>
>> Woke up this morning to a bunch of reports of issues with connectivity
>> had to shut down some Level3/CTL connections to get it to return to normal.
>>
>>
>>
>> As of right now their support portal won’t load:
>> https://www.centurylink.com/business/login/
>>
>>
>>
>> Just wondering what others are seeing.
>>
>>
>>
>
>
> --
> Sincerely,
>
> Jason W Kuehl
> Cell 920-419-8983
> jason.w.ku...@gmail.com
>

Re: Spiffy Netflow tools?

[Chase Christian]

 +1 for ElastiFlow. Couldn't be easier to set up and run. Logstash has
native support for netflow and sflow now via codecs. Kibana is an
easy-to-use dashboard. I trimmed out a bunch of stuff in the ElastiFlow
config that assumed a unidirectional network (like a corporate site).

On Tue, Mar 13, 2018 at 8:48 AM, Luke Guillory 
wrote:

> There is also https://github.com/robcowart/elastiflow which uses the ELK
> stack.
>
>
>
>
>
> Luke Guillory
> Vice President – Technology and Innovation
>
> Tel:985.536.1212
> Fax:985.536.0300
> Email:  lguill...@reservetele.com
>
> Reserve Telecommunications
> 100 RTC Dr
> Reserve, LA 70084
>
> 
> _
>
> Disclaimer:
> The information transmitted, including attachments, is intended only for
> the person(s) or entity to which it is addressed and may contain
> confidential and/or privileged material which should not disseminate,
> distribute or be copied. Please notify Luke Guillory immediately by e-mail
> if you have received this e-mail by mistake and delete this e-mail from
> your system. E-mail transmission cannot be guaranteed to be secure or
> error-free as information could be intercepted, corrupted, lost, destroyed,
> arrive late or incomplete, or contain viruses. Luke Guillory therefore does
> not accept liability for any errors or omissions in the contents of this
> message, which arise as a result of e-mail transmission. .
>
> -Original Message-
> From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Hugo Slabbert
> Sent: Tuesday, March 13, 2018 10:44 AM
> To: Fredrik Korsbäck
> Cc: nanog@nanog.org
> Subject: Re: Spiffy Netflow tools?
>
>
> On Tue 2018-Mar-13 00:50:26 +0100, Fredrik Korsbäck 
> wrote:
> >
> >Kentik is probably top of the foodchain right now.
> >
> >But they are certainly not alone in the biz. Ontop of my head...
> >
> >* Flowmon
> >* Talaia
> >* Arbor Peakflow
> >* Deepfield
> >* Pmacct + supporting toolkit
> >* NFsen/Nfdump/AS-stats
> >* Put kibana/ES infront of any collector
>
> Logstash has a netflow plugin as of 5.x or something
> (https://www.elastic.co/guide/en/logstash/current/netflow-module.html) to
> act as a collector.
>
> A walkthrough:
> http://www.routereflector.com/2017/07/elk-as-a-free-netflow-
> ipfix-collector-and-visualizer/
>
> Using the logstash module setup thing adds a whole bunch of pretty netflow
> graphs and visualizations and such into Kibana for you.
>
> Caveat:
> Supports netflow v5 and v9, but does not indicate support for IPFIX
> explicitly.  It definitely does not support sFlow, though if you really
> want you can stick sflowtool in front of it to translate sFlow->netflow,
> e.g. http://blog.sflow.com/2011/12/sflowtool.html.
>
> >* Solarwinds something something
> >* Different vendor toolkits
> >
> >--
> >hugge
>
> --
> Hugo Slabbert   | email, xmpp/jabber: h...@slabnet.com
> pgp key: B178313E   | also on Signal
>

Re: NAT firewall for IPv6?

[Chase Christian]

The original email was not a serious question, but a joke:

https://twitter.com/SwiftOnSecurity/status/749059605360062464
https://twitter.com/SwiftOnSecurity/status/749062835687174144
https://twitter.com/SwiftOnSecurity/status/749068172460847105



On Tue, Jul 5, 2016 at 1:41 PM, Naslund, Steve  wrote:

> It is all about defense in depth.  The engineers here are speaking to the
> network pieces (the second N in NANOG is network, right :) and we have told
> this person that it is unlikely that v6 in the only vector and I myself
> talked about malware handling on the clients themselves.  From a network
> engineering perspective many of us agreed that the biggest single threat to
> his network was a firewall in an unknown state with an unknown
> administrator password that could be owned by anyone on earth at this
> point.  That single piece threatens the entire network as a whole and is a
> ticking time bomb ready to blow his entire LAN off the Internet if it fails.
>
> He probably does not own the entire environment himself, he is filling in
> for a vacationing network engineer.  So he is working on the network piece
> and is probably not responsible for the anti-malware software on the
> clients (if anyone is, see below).
>
> Our "support" as you call it was a response to this person questions about
> blocking v6 as an attack vector in the first place.  We answered his
> question but then told him that was unlikely to be the problem and what he
> should do about taking back his firewall, securing v6 via the firewall, and
> handling the malware at the client.  Seems solid advise to me so far.
>
> BTW we did not bill him for anything.  He got a lot of free advice from a
> lot of people he could not even begin to afford to employ, so not a bad
> deal for him.  You also have to understand that this gentleman seems to be
> in an educational environment which usually means lots of clients he does
> not have control over so having some kind of network based malware control
> is helpful.  Clients in this type of environment have to defend themselves
> from each other and he will likely have stuff brought in from the outside.
> Good malware detection in the network can help identify clients that
> contain malware and are a threat to other devices.  Fancier network
> gear/IDS/IDP would actually remove offending clients from the network or at
> least segments them into an isolation area.
>
> Let me re-iterate:
>
> 1.  Take back ownership of your firewall and bring it up to
> date including new malware signatures.  If you don't have current support,
> get it...directly so if your consultant bails you are not dead
> meat.  This will ensure that the outside world will not own or control
> stuff inside your network while you put the fires out.  At the very least
> it can help malware infected machines from phoning home to their command
> and control servers which sometimes prevents a lot of damage.
> 2.  Make your v6 rules mirror at least the security level of
> your v4 rules.  Passing v6 unchallenged is unacceptable.  If your firewall
> won't do it replace it with one that will.
> 3.  Ensure all clients under your control have current
> anti-virus/anti-malware detection.  Clients have to defend themselves from
> threats internal to the firewall as well as ones outside.  Don't be hard on
> the outside with a soft chewy center.
> 4.  Never, ever accept anything less than full administrative
> control passwords and accounts from your consultants, before you give them
> final payment.  I actually prefer to lock them out when they complete an
> install until I need them to help with something.  This prevents them from
> holding you hostage or one of their "postal" employees from wiping you out
> as well as preventing them from using your network for experimentation
> without you knowing it.  It is an important part of change control to
> ensure that outsiders cannot modify your configuration without contacting
> you first.  We usually give our consultants highly logged VPN accounts that
> we can disable or enable as needed.
>
> Steven Naslund
> Chicago IL
>
>
>
> >>No while that is also needed, it is very unlikely to fix his issue. The
> issue at hand is that some of their computers have become virus infected.
> >>The fix for that is to upgrade the virus scanner and making sure that
> all software upgrades are done.
>
> >>Someone comes to you and says his Firefox is getting infected through
> IPv6.
> >>If your support is worth anything, you will not take that at face value
> and bill him for a ton work related to IPv6. No, you will go find out what
> the real issue is and solve that. The only thing we know right now is that
> he is >>confused.
> >>
> >>Regards,
> >>
> >>Baldur
>

Re: DDoS auto-mitigation best practices (for eyeball networks)

[Chase Christian]

Most video games utilize peer-to-peer traffic (which is why many require
port forwarding/UPnP), so the attacker has the IP addresses of all of their
peers in their firewall logs. There are even 'gaming routers' that
specialize in gaming this peer-to-peer system for competitive advantages,
such as specifically blocking the IPs of players you don't want to play
against:

https://netduma.com/why/for-gamers/

Once an attacker has identified his target, getting the IP is as simple as
joining/being in an online game with that player.

On Mon, Sep 21, 2015 at 5:00 AM,  wrote:

> 99% of the attacks we see are gaming related -- somehow the other players
> know the IP and then attack our customer for an advantage in the game, or
> retribution.
>
> Most DHCP servers (correctly) give the same IP address if the CPE is
> rebooted.  Ours are one of those. =)
>
> Frank
>
> -Original Message-
> From: Mehmet Akcin [mailto:meh...@akcin.net]
> Sent: Saturday, September 19, 2015 3:10 PM
> To: Frank Bulk 
> Cc: nanog@nanog.org
> Subject: Re: DDoS auto-mitigation best practices (for eyeball networks)
>
> How does he/she become target? How does IP address gets exposed?
>
> I guess simplest way is to reboot modem and hope to get new ip (or call n
> request)
>
> Mehmet
>
> > On Sep 19, 2015, at 12:54, Frank Bulk  wrote:
> >
> > Could the community share some DDoS auto-mitigation best practices for
> > eyeball networks, where the target is a residential broadband subscriber?
> > I'm not asking so much about the customer communication as much as
> > configuration of any thresholds or settings, such as:
> > - minimum traffic volume before responding (for volumetric attacks)
> > - minimum time to wait before responding
> > - filter percentage: 100% of the traffic toward target (or if volumetric,
> > just a certain percentage)?
> > - time before mitigation is automatically removed
> > - and if the attack should recur shortly thereafter, time to respond and
> > remove again
> > - use of an upstream provider(s) mitigation services versus one's own
> > mitigation tools
> > - network placement of mitigation (presumably upstream as possible)
> > - and anything else
> >
> > I ask about best practice for broadband subscribers on eyeball networks
> > because it's different environment than data center and hosting
> environments
> > or when one's network is being used to DDoS a target.
> >
> > Regards,
> >
> > Frank
> >
>
>
>