Re: Oh dear, we've all been made redundant...

[Larry Sheldon]

On 3/24/2016 08:08, Casey Russell wrote:

 >>Just goes to show the vast range of technical issues that can be
 >>readily righted with little more than a good thump with a hammer.

We always referred to that as "percussive maintenance"

Casey Russell
Network Engineer
Kansas Research and Education Network

2029 Becker Drive, Suite 282

Lawrence, KS  66047

(785)856-9820  ext 9809
cruss...@kanren.net 

On Mon, Mar 21, 2016 at 3:19 AM, Wayne Bouchard > wrote:

On Sun, Mar 20, 2016 at 11:00:36PM -0500, Larry Sheldon wrote:
> On 3/19/2016 18:16, Warren Kumari wrote:
> > Found on Staple's website:
> 
>http://www.staples.com/NetReset-Automated-Power-Cycler-for-Modems-and-Routers/product_1985686
> >
> > Fixes all issues, less downtime, less stress...
>
> etc...
> ...
> 
> ...and so forth
> 
> .
> ..and so on.
>
> > Resetting allows equipment to auto-correct issues
>
> Recalls to mind years ago in the Toll testroom where I worked, the
> evenings equipment man (charged with and assigned to the task of
> repairing equipment that had been "patched out" by the day shift) would,
> when he arrived for work each day, retrieve the piece of 2 X 4 from its
> hiding place and whack each bay of relay-rich equipment as he walked in
> the area.
>
> Then, after some coffee and a cigarette, he would go through the
> trouble-ticket collection, retest the item, mark the ticket "NTF" and
> proceed to the next item.

I love that!

Just goes to show the vast range of technical issues that can be
readily righted with little more than a good thump with a hammer.


In a later live, I worked in a computer center housing A computer (1110, 
1100/80, 1100/90).  The UNIVAC CEs had in their kit an tool for locating 
"shock-sensitive" boards--looked like and worked like an "automatic 
centerpunch" with a blunt point.


--
sed quis custodiet ipsos custodes? (Juvenal)

PitIX or pair.com

[Mike Hammett]

Do any of you have any information on PitIX? It looks like it was another IX 
started by a datacenter that didn't pan out. Therefore, anyone have a contact 
at pair.com? An inquiry sent to their web site (well, whatever contact 
information they have on it) has gone unanswered. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 

Re: Oh dear, we've all been made redundant...

[Owen DeLong]

> On Mar 24, 2016, at 06:08 , Casey Russell  wrote:
> 
>>> Just goes to show the vast range of technical issues that can be
>>> readily righted with little more than a good thump with a hammer.
> 
> We always referred to that as "percussive maintenance”

If such a need presents itself and one lacks the requisite hammer, one can also 
resort to “brogan maintenance” in many cases.

Owen

RE: Paging someone at SonicWall/Dell

[Vinny_Abello]

Hi Dan,

Did you manage to get in touch with anyone? If not, I can attempt to broadcast 
to the Dell Networking group internally. There should be Sonicwall people on 
there that can help.

-Vinny

-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Dan Mahoney, System 
Admin
Sent: Monday, March 21, 2016 9:47 PM
To: nanog@nanog.org
Subject: Paging someone at SonicWall/Dell

All,

If anyone has any contacts at Dell/SonicWall that can assist, their latest 
firewalls have a misfeature that have started blocking F-root (ISC, my
Day-job) as a possible botnet responder.

You can reach me, 24/7, at 703-338-2497, or at dmaho...@isc.org.

Needless to say, this is seriously bad. As we ourselves are not Sonicwall 
users, our options here are limited.

I've gotten their usual call center nonsense where they told me "I need to 
contact my system administrator" and wouldn't transfer me further.

I've submitted a "removal" request via the form at 
http://botnet.global.sonicwall.com/change, but that still doesn't help if 
there's a firmware out there doing broken hueristics and mass-DDOSing folks.

If for some reason there actually IS some botnet-related traffic going on 
toward the root servers (the root servers get a LOT of garbage, so this doesn't 
surprise me), we'd like to know that too.

-Dan Mahoney

--

Dan Mahoney
Techie, Sysadmin, WebGeek
Gushi on efnet/undernet IRC
ICQ: 13735144 AIM: LarpGM
Site: http://www.gushi.org
---

Re: Oh dear, we've all been made redundant...

[Casey Russell]

>>Just goes to show the vast range of technical issues that can be
>>readily righted with little more than a good thump with a hammer.

We always referred to that as "percussive maintenance"

Casey Russell
Network Engineer
Kansas Research and Education Network

2029 Becker Drive, Suite 282

Lawrence, KS  66047
(785)856-9820  ext 9809
cruss...@kanren.net

On Mon, Mar 21, 2016 at 3:19 AM, Wayne Bouchard  wrote:

> On Sun, Mar 20, 2016 at 11:00:36PM -0500, Larry Sheldon wrote:
> > On 3/19/2016 18:16, Warren Kumari wrote:
> > > Found on Staple's website:
> > >
> http://www.staples.com/NetReset-Automated-Power-Cycler-for-Modems-and-Routers/product_1985686
> > >
> > > Fixes all issues, less downtime, less stress...
> >
> > etc...
> > ...
> > 
> > ...and so forth
> > 
> > .
> > ..and so on.
> >
> > > Resetting allows equipment to auto-correct issues
> >
> > Recalls to mind years ago in the Toll testroom where I work, the
> > evenings equipment man (charged with and assigned to the task of
> > repairing equipment that had been "patched out" by the day shift) would,
> > when he arrived for work each day, retrieve the piece of 2 X 4 from its
> > hiding place and whack each bay of relay-rich equipment as he walked in
> > the area.
> >
> > Then, after some coffee and a cigarette, he would go through the
> > trouble-ticket collection, retest the item, mark the ticket "NTF" and
> > proceed to the next item.
>
> I love that!
>
> Just goes to show the vast range of technical issues that can be
> readily righted with little more than a good thump with a hammer.
>
> -Wayne
>
> ---
> Wayne Bouchard
> w...@typo.org
> Network Dude
> http://www.typo.org/~web/
>

Re: NTT communications horrible routing, unresponsive NOC

[Jared Mauch]

Hello,

Could you send me some more details in private about who you spoke with 
including email addresses and ticket numbers with our NOC if any?

I'd like to understand what happened here as this is an uncharacteristic 
outcome. I know there was planned work in Seattle last night for a software 
upgrade combined with hardware work, but an operational issue like this should 
have been resolved. 

I'd like to understand the timeline and what broke down if anything. 

Thanks,

Jared Mauch

> On Mar 23, 2016, at 7:39 PM, Paras Jha  wrote:
> 
> Hi all,
> 
> I've been trying to get this issue resolved for the entire day now, but NTT
> has been pretty unreceptive here.
> 
> We're announcing a large prefix for a client across our network, and we
> discovered some insanely high latency.
> 
> After tracking down the issue, we determined it to be something wrong with
> NTT at their Seattle location. We anycast this prefix, but no matter where
> in the world traffic is originating from, it's going to Seattle and then to
> Atlanta. Example: Rotterdam in the Netherlands routes from Europe -> east
> coast -> west coast Seattle -> los angeles -> atlanta. The gist of it is
> that something is seriously messed up at NTT in Seattle.
> 
> We contacted our transit provider to try and carry the issue upstream, and
> what they told us was
> 
> Sorry for delay, I've asked NTT to clear the more specific for this one as
> well.
> The problem seems to be a bug on the NTT side which keeps stale routes in
> the routing table for more specifics ( at random ).
> If you have more routes affected please notify us of the routes and I will
> ask them to clear the routing table for these routes.
> NTT is working on this with their vendor to get this resolved as soon as
> possible.
> 
> 
> I had spoken to a sales rep for NTT a few weeks prior, and they assured me
> that the NOC was top notch, and that all routes were redundant, and they
> guaranteed less than 50ms in the US, and all kinds of marketing. However,
> it looks like it's all marketing - for this entire day this router has been
> causing tons of issues for our clients.
> 
> No-exporting it to NTT does not even solve the problem, as NTT's router in
> Seattle apparently just decides to keep random small prefixes in it,
> causing traffic to go there.
> 
> At a loss as to what to do now, since their NOC isn't receptive. Anyone
> have someone I can contact off-list to get this issue resolved? It's
> especially frustrating because the problem absolutely cannot be resolved on
> our end, even with a no-export since NTT is keeping the routes in their
> router.

Re: NTT communications horrible routing, unresponsive NOC

[James Bensley]

On 23 March 2016 at 23:39, Paras Jha  wrote:
> At a loss as to what to do now, since their NOC isn't receptive. Anyone
> have someone I can contact off-list to get this issue resolved? It's
> especially frustrating because the problem absolutely cannot be resolved on
> our end, even with a no-export since NTT is keeping the routes in their
> router.

It would be helpful to show how you came to these conclusions about
prefix caching and routing issues etc (share the data you gathered).

If you are an NTT customer (it's sounds a bit like you're a customer
of an NTT customer, it's not clear) then this should be account
management 101, you should have an escalations process/matrix, kick
off that process now.

There are NTT dudes on this list so probably they will reach out to
you off list, if not post back requesting an NTT contact.

James.