Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-22 Thread jamie rishaw
Data on June 20 :

  .COM. :
108,985,894 unique domains + the tld.
  - 234,479 NSEC3/RRSIG records,
  - 2,253,400 nameserver entries on 831,088 unique IP addresses.

.. ish.

-jamie

On Fri, Jun 21, 2013 at 5:23 PM, Barry Shein b...@world.std.com wrote:


 I think we need a better measure than number of domains (in this case
 .COM), particularly vs total domains.

 If it was 100 domains it might seem small, unless that list began with
 facebook.com, amazon.com, google.com and g*d forbid theworld.com.

 --
 -Barry Shein

 The World  | b...@theworld.com   |
 http://www.TheWorld.com
 Purveyors to the Trade | Voice: 800-THE-WRLD| Dial-Up: US, PR,
 Canada
 Software Tool  Die| Public Access Internet | SINCE 1989 *oo*



Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread Jimmy Hess
On 6/20/13, Hal Murray hmur...@megapathdsl.net wrote:

 Perhaps we should setup a distributed system for checking things rather than
 another SPOF.  That's distributed both geographically and administratively
 and using several code-bases.
[snip]

I would be in favor of being able to pay two competitive  to be
registrars for a domain,  and assign them two roles:

Registrar Primary
and Registrar Auditor

With the requirement that all changes to the domain be initiated with
my  Primary Registrar,
AND no  major change would be allowed to take effect until validated
by my secondary change Auditor Registrar

Including  changes to NS records, DS records,  contacts,  unlocking,
renewal, deactivation, or transfers.

Essentially, forcing me to submit the same change to both registrars,
but denying either registrar the capability  of  forging authorization
or submitting changes that I had not authorized.

Also (in some measure) protecting me from identity theft, and other
security issues -- since there are now two accounts with two
providers,  possibly with different authentication procedures.

--
-JH



Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread Jimmy Hess
On 6/20/13, valdis.kletni...@vt.edu valdis.kletni...@vt.edu wrote:

 It's relatively small when you consider there's something like 140M .com's

Yeah... I'm in agreement about that's probably what is going on...
It's relatively small, but absolutely large,  and absolute numbers
matter. 5 domains is small, 50k  is not,  even if  Netsol has a 100
billion domains.

If I had 50,000 fingers;  I might think differently.   But the
definition of a large number doesn't change to people,  just because
you also have a massive number of that thing.


The phrase a small number   means an absolutely small number, so it
seems like a really really misleading if not possibly dishonest PR
spin;  they could have said a small proportion or  a relatively
small number, in that case.

--
-JH



RE: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread Kain, Rebecca (.)
I remember when I used to own a small ISP and NetSOL lost 1/3 of the domains. 
 Just lost them. And it wasn't a DDOS, it was their screw up.  It went on for 
days


-Original Message-
From: Hank Nussbacher [mailto:h...@efes.iucc.ac.il] 
Sent: Thursday, June 20, 2013 11:10 PM
To: Richard Golodner
Cc: nanog@nanog.org
Subject: Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

At 17:12 20/06/2013 -0500, Richard Golodner wrote:

 I think you are reading it the wrong way. Mr.Kletnieks never said it
was okay. He just stated that the numbers were trivial when compared to
the rest of potential customers being affected.
 Be cool, Richard Golodner

sarcasm
and Netsol agrees with you:
http://www.networksolutions.com/blog/2013/06/important-update-for-network-solutions-customers-experiencing-website-issues/

a small number of Network Solutions customers were inadvertently affected 
for up to several hours.
/sarcasm

-Hank





Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread Valdis . Kletnieks
On Thu, 20 Jun 2013 23:42:24 -0400, shawn wilson said:

 I think Netsol should be fined. Maybe even a class action suite filed
 against them for lost business. And that's it.

So your contract with NetSol has an SLA guarantee in it, and you can
demonstrate that (a) said SLA has been violated and (b) that NetSol has not
made the contracted restitution?




pgpIcdxHHMFzt.pgp
Description: PGP signature


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread Nicolai
On Thu, Jun 20, 2013 at 05:28:17PM -0400, valdis.kletni...@vt.edu wrote:
 It's relatively small when you consider there's something like 140M .com's

Just FWIW, the current size of .com is roughly 109M domains.  Someday it
will reach 140M but not today.

Nicolai



Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread David Walker
 https://www.networksolutions.com/blog/2013/06/important-update-for-network-solutions-customers-experiencing-website-issues/

Why are they infinitely looping a script on their web server to check
for a cookie?

Are these people insane?



Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread John Levine
Registrar Primary and Registrar Auditor

There are certainly registrars who are more security oriented than
Netsol.  If you haven't followed all of the corporate buying and
selling, Netsol is now part of web.com, so their business is more to
support web hosting than to be a registrar.

I expect that if you put your domain at Markmonitor or CSC corporate
domains, you would not have this problem, and you would pay
accordingly.




Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread Barry Shein

I think we need a better measure than number of domains (in this case
.COM), particularly vs total domains.

If it was 100 domains it might seem small, unless that list began with
facebook.com, amazon.com, google.com and g*d forbid theworld.com.

-- 
-Barry Shein

The World  | b...@theworld.com   | http://www.TheWorld.com
Purveyors to the Trade | Voice: 800-THE-WRLD| Dial-Up: US, PR, Canada
Software Tool  Die| Public Access Internet | SINCE 1989 *oo*



RE: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread John Souvestre
Hi Shawn.

Or you could vote with your feet, and wish then a fine g'day.

John

John Souvestre - New Orleans LA - (504) 454-0899


-Original Message-
From: shawn wilson [mailto:ag4ve...@gmail.com] 
Sent: Thursday, June 20, 2013 10:42 pm
To: Hal Murray
Cc: North American Network Operators Group
Subject: Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

I think ICANN would have to add a delay in where a request was sent out to make 
sure everyone was on the same page and then what happens the couple thousand 
(more)  times a day that someone isn't updated or is misconfigured?

I think Netsol should be fined. Maybe even a class action suite filed against 
them for lost business. And that's it.
On Jun 20, 2013 11:28 PM, Hal Murray hmur...@megapathdsl.net wrote:





RE: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread Frank Bulk
It's 120M if you add the .COM and the .NET's together, both of which NetSol
is responsible for.
http://www.verisigninc.com/en_US/products-and-services/domain-name-services/
registry-products/tld-zone-access/index.xhtml

Frank

-Original Message-
From: Nicolai [mailto:nicolai-na...@chocolatine.org] 
Sent: Friday, June 21, 2013 11:16 AM
To: nanog@nanog.org
Subject: Re: This is a coordinated hacking. (Was Re: Need help in flushing
DNS)

On Thu, Jun 20, 2013 at 05:28:17PM -0400, valdis.kletni...@vt.edu wrote:
 It's relatively small when you consider there's something like 140M .com's

Just FWIW, the current size of .com is roughly 109M domains.  Someday it
will reach 140M but not today.

Nicolai






Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread John Levine
In article 001a01ce6ef9$bf74d4a0$3e5e7de0$@iname.com you write:
It's 120M if you add the .COM and the .NET's together, both of which NetSol
is responsible for.
http://www.verisigninc.com/en_US/products-and-services/domain-name-services/
registry-products/tld-zone-access/index.xhtml

In late breaking news, Verisign spun off Network Solutions in 2003,
and the two companies have been unrelated for the past decade.

These days NetSol is just another registrar.  Since 2011 it has been
part of web hosting company web.com.

R's,
John



Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-21 Thread George Herbert

I know how we got here, but perhaps we can take corporate parentage and how big 
.com is now to -discuss?

What happened with the registry data that caused the outage and what can / 
should be done about it / to prevent it happening again still seem to me to be 
operational topics.


George William Herbert
Sent from my iPhone


This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread jamie rishaw
This is most definitely a coordinated and planned attack.

And by 'attack' I mean hijacking of domain names.

I show as of this morning nearly fifty thousand domain names that appear
suspicious.

I'm tempted to call uscentcom and/or related agencies (which agencies, who
the hell knows, as ICE seems to have some sort of authority over domains
(nearly two hundred fifty of them as I type this in COM alone and another
thirty-some in NET).

Anyone credentialed (credentialed /n/., I know you or know of you,)
wanting data, e-mail me off-list for some TLD goodness.






On Thu, Jun 20, 2013 at 12:29 PM, Phil Fagan philfa...@gmail.com wrote:

 Agree'd in these smaller scenario's I just wonder if in a larger scale
 scenario, whatever that might look like, if its necessary. Whereby many
 organizations who provide services are effected. Perhaps the result of a
 State led campaign topic for another day.




 On Thu, Jun 20, 2013 at 11:25 AM, Paul Ferguson fergdawgs...@gmail.com
 wrote:

  I am betting that Netsol doesn't need any more coordination at the
  moment -- their phones are probably ringing off-the-hook. There are
  still ~400 domains still pointing to the ztomy NS:
 
 
   ;  DiG 9.7.3  @foohost parsonstech.com NS
   ; (1 server found)
   ;; global options: +cmd
   ;; Got answer:
   ;; -HEADER- opcode: QUERY, status: NOERROR, id: 49064
   ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
 
   ;; QUESTION SECTION:
   ;parsonstech.com.INNS
 
   ;; ANSWER SECTION:
   parsonstech.com.172800INNSns2617.ztomy.com.
   parsonstech.com.172800INNSns1617.ztomy.com.
 
   ;; Query time: 286 msec
   ;; SERVER: 127.0.0.1#53(127.0.0.1)
   ;; WHEN: Thu Jun 20 19:16:25 2013
   ;; MSG SIZE  rcvd: 81
 
  - ferg
 
  On Thu, Jun 20, 2013 at 10:13 AM, Phil Fagan philfa...@gmail.com
 wrote:
 
   I should caveat.coordinate the recovery of.
  
  
   On Thu, Jun 20, 2013 at 11:10 AM, Brandon Butterworth
   bran...@rd.bbc.co.ukwrote:
  
Is there an organization that coordinates outages like this amongst
  the
industry?
  
   No, usually they are surprise outages though Anonymous have tried
   coordinating a few
  
   brandon
  
  
  
  
   --
   Phil Fagan
   Denver, CO
   970-480-7618
 
 
 
  --
  Fergie, a.k.a. Paul Ferguson
   fergdawgster(at)gmail.com
 



 --
 Phil Fagan
 Denver, CO
 970-480-7618




-- 
Jamie Rishaw // .com.arpa@j - reverse it. ish.
[Impressive C-level Title Here], arpa / arpa labs


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Jared Mauch
It seems there may be a need for some sort of 'dns-health' check out there that 
can be done in semi-realtime.

I ran a report for someone earlier today on a domain doing an xref against open 
resolver data searching for valid responses vs invalid ones.

Is this of value?  Does it need to be automated?

- Jared

On Jun 20, 2013, at 3:53 PM, jamie rishaw j...@arpa.com wrote:

 This is most definitely a coordinated and planned attack.
 
 And by 'attack' I mean hijacking of domain names.
 
 I show as of this morning nearly fifty thousand domain names that appear
 suspicious.
 
 I'm tempted to call uscentcom and/or related agencies (which agencies, who
 the hell knows, as ICE seems to have some sort of authority over domains
 (nearly two hundred fifty of them as I type this in COM alone and another
 thirty-some in NET).
 
 Anyone credentialed (credentialed /n/., I know you or know of you,)
 wanting data, e-mail me off-list for some TLD goodness.
 
 
 
 
 
 
 On Thu, Jun 20, 2013 at 12:29 PM, Phil Fagan philfa...@gmail.com wrote:
 
 Agree'd in these smaller scenario's I just wonder if in a larger scale
 scenario, whatever that might look like, if its necessary. Whereby many
 organizations who provide services are effected. Perhaps the result of a
 State led campaign topic for another day.
 
 
 
 
 On Thu, Jun 20, 2013 at 11:25 AM, Paul Ferguson fergdawgs...@gmail.com
 wrote:
 
 I am betting that Netsol doesn't need any more coordination at the
 moment -- their phones are probably ringing off-the-hook. There are
 still ~400 domains still pointing to the ztomy NS:
 
 
 ;  DiG 9.7.3  @foohost parsonstech.com NS
 ; (1 server found)
 ;; global options: +cmd
 ;; Got answer:
 ;; -HEADER- opcode: QUERY, status: NOERROR, id: 49064
 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
 
 ;; QUESTION SECTION:
 ;parsonstech.com.INNS
 
 ;; ANSWER SECTION:
 parsonstech.com.172800INNSns2617.ztomy.com.
 parsonstech.com.172800INNSns1617.ztomy.com.
 
 ;; Query time: 286 msec
 ;; SERVER: 127.0.0.1#53(127.0.0.1)
 ;; WHEN: Thu Jun 20 19:16:25 2013
 ;; MSG SIZE  rcvd: 81
 
 - ferg
 
 On Thu, Jun 20, 2013 at 10:13 AM, Phil Fagan philfa...@gmail.com
 wrote:
 
 I should caveat.coordinate the recovery of.
 
 
 On Thu, Jun 20, 2013 at 11:10 AM, Brandon Butterworth
 bran...@rd.bbc.co.ukwrote:
 
 Is there an organization that coordinates outages like this amongst
 the
 industry?
 
 No, usually they are surprise outages though Anonymous have tried
 coordinating a few
 
 brandon
 
 
 
 
 --
 Phil Fagan
 Denver, CO
 970-480-7618
 
 
 
 --
 Fergie, a.k.a. Paul Ferguson
 fergdawgster(at)gmail.com
 
 
 
 
 --
 Phil Fagan
 Denver, CO
 970-480-7618
 
 
 
 
 -- 
 Jamie Rishaw // .com.arpa@j - reverse it. ish.
 [Impressive C-level Title Here], arpa / arpa labs




Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread jamie rishaw
I'm rechecking realtime ns1620/2620 DNS right now and, looking at the
output, I see an odd number of domains (that have changed) with a listed
nameserver of localhost..

Is this some sort of tactic I'm unaware of?


On Thu, Jun 20, 2013 at 2:57 PM, Jared Mauch ja...@puck.nether.net wrote:

 It seems there may be a need for some sort of 'dns-health' check out there
 that can be done in semi-realtime.

 I ran a report for someone earlier today on a domain doing an xref against
 open resolver data searching for valid responses vs invalid ones.

 Is this of value?  Does it need to be automated?

 - Jared

 On Jun 20, 2013, at 3:53 PM, jamie rishaw j...@arpa.com wrote:

  This is most definitely a coordinated and planned attack.
 
  And by 'attack' I mean hijacking of domain names.
 
  I show as of this morning nearly fifty thousand domain names that appear
  suspicious.
 
  I'm tempted to call uscentcom and/or related agencies (which agencies,
 who
  the hell knows, as ICE seems to have some sort of authority over domains
  (nearly two hundred fifty of them as I type this in COM alone and another
  thirty-some in NET).
 
  Anyone credentialed (credentialed /n/., I know you or know of you,)
  wanting data, e-mail me off-list for some TLD goodness.
 
 
 
 
 
 
  On Thu, Jun 20, 2013 at 12:29 PM, Phil Fagan philfa...@gmail.com
 wrote:
 
  Agree'd in these smaller scenario's I just wonder if in a larger scale
  scenario, whatever that might look like, if its necessary. Whereby many
  organizations who provide services are effected. Perhaps the result
 of a
  State led campaign topic for another day.
 
 
 
 
  On Thu, Jun 20, 2013 at 11:25 AM, Paul Ferguson fergdawgs...@gmail.com
  wrote:
 
  I am betting that Netsol doesn't need any more coordination at the
  moment -- their phones are probably ringing off-the-hook. There are
  still ~400 domains still pointing to the ztomy NS:
 
 
  ;  DiG 9.7.3  @foohost parsonstech.com NS
  ; (1 server found)
  ;; global options: +cmd
  ;; Got answer:
  ;; -HEADER- opcode: QUERY, status: NOERROR, id: 49064
  ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
 
  ;; QUESTION SECTION:
  ;parsonstech.com.INNS
 
  ;; ANSWER SECTION:
  parsonstech.com.172800INNSns2617.ztomy.com.
  parsonstech.com.172800INNSns1617.ztomy.com.
 
  ;; Query time: 286 msec
  ;; SERVER: 127.0.0.1#53(127.0.0.1)
  ;; WHEN: Thu Jun 20 19:16:25 2013
  ;; MSG SIZE  rcvd: 81
 
  - ferg
 
  On Thu, Jun 20, 2013 at 10:13 AM, Phil Fagan philfa...@gmail.com
  wrote:
 
  I should caveat.coordinate the recovery of.
 
 
  On Thu, Jun 20, 2013 at 11:10 AM, Brandon Butterworth
  bran...@rd.bbc.co.ukwrote:
 
  Is there an organization that coordinates outages like this amongst
  the
  industry?
 
  No, usually they are surprise outages though Anonymous have tried
  coordinating a few
 
  brandon
 
 
 
 
  --
  Phil Fagan
  Denver, CO
  970-480-7618
 
 
 
  --
  Fergie, a.k.a. Paul Ferguson
  fergdawgster(at)gmail.com
 
 
 
 
  --
  Phil Fagan
  Denver, CO
  970-480-7618
 
 
 
 
  --
  Jamie Rishaw // .com.arpa@j - reverse it. ish.
  [Impressive C-level Title Here], arpa / arpa labs




-- 
Jamie Rishaw // .com.arpa@j - reverse it. ish.
[Impressive C-level Title Here], arpa / arpa labs


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread George Herbert
Poisoning a domain's NS records with localhost will most certainly DOS the
domain, yes.

I have not yet seen the source of this; if anyone has a clue where the
updates are coming from please post the info.

Is there anything about ztomy.com that has been seen that's supicious as in
they might be the origin?  This could be them, or could be a joe-job
against them.  I do not want to point a finger lacking any sort of actual
data dump of the poisoning activity...




On Thu, Jun 20, 2013 at 1:02 PM, jamie rishaw j...@arpa.com wrote:

 I'm rechecking realtime ns1620/2620 DNS right now and, looking at the
 output, I see an odd number of domains (that have changed) with a listed
 nameserver of localhost..

 Is this some sort of tactic I'm unaware of?


 On Thu, Jun 20, 2013 at 2:57 PM, Jared Mauch ja...@puck.nether.net
 wrote:

  It seems there may be a need for some sort of 'dns-health' check out
 there
  that can be done in semi-realtime.
 
  I ran a report for someone earlier today on a domain doing an xref
 against
  open resolver data searching for valid responses vs invalid ones.
 
  Is this of value?  Does it need to be automated?
 
  - Jared
 
  On Jun 20, 2013, at 3:53 PM, jamie rishaw j...@arpa.com wrote:
 
   This is most definitely a coordinated and planned attack.
  
   And by 'attack' I mean hijacking of domain names.
  
   I show as of this morning nearly fifty thousand domain names that
 appear
   suspicious.
  
   I'm tempted to call uscentcom and/or related agencies (which agencies,
  who
   the hell knows, as ICE seems to have some sort of authority over
 domains
   (nearly two hundred fifty of them as I type this in COM alone and
 another
   thirty-some in NET).
  
   Anyone credentialed (credentialed /n/., I know you or know of you,)
   wanting data, e-mail me off-list for some TLD goodness.
  
  
  
  
  
  
   On Thu, Jun 20, 2013 at 12:29 PM, Phil Fagan philfa...@gmail.com
  wrote:
  
   Agree'd in these smaller scenario's I just wonder if in a larger
 scale
   scenario, whatever that might look like, if its necessary. Whereby
 many
   organizations who provide services are effected. Perhaps the result
  of a
   State led campaign topic for another day.
  
  
  
  
   On Thu, Jun 20, 2013 at 11:25 AM, Paul Ferguson 
 fergdawgs...@gmail.com
   wrote:
  
   I am betting that Netsol doesn't need any more coordination at the
   moment -- their phones are probably ringing off-the-hook. There are
   still ~400 domains still pointing to the ztomy NS:
  
  
   ;  DiG 9.7.3  @foohost parsonstech.com NS
   ; (1 server found)
   ;; global options: +cmd
   ;; Got answer:
   ;; -HEADER- opcode: QUERY, status: NOERROR, id: 49064
   ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
  
   ;; QUESTION SECTION:
   ;parsonstech.com.INNS
  
   ;; ANSWER SECTION:
   parsonstech.com.172800INNSns2617.ztomy.com.
   parsonstech.com.172800INNSns1617.ztomy.com.
  
   ;; Query time: 286 msec
   ;; SERVER: 127.0.0.1#53(127.0.0.1)
   ;; WHEN: Thu Jun 20 19:16:25 2013
   ;; MSG SIZE  rcvd: 81
  
   - ferg
  
   On Thu, Jun 20, 2013 at 10:13 AM, Phil Fagan philfa...@gmail.com
   wrote:
  
   I should caveat.coordinate the recovery of.
  
  
   On Thu, Jun 20, 2013 at 11:10 AM, Brandon Butterworth
   bran...@rd.bbc.co.ukwrote:
  
   Is there an organization that coordinates outages like this
 amongst
   the
   industry?
  
   No, usually they are surprise outages though Anonymous have tried
   coordinating a few
  
   brandon
  
  
  
  
   --
   Phil Fagan
   Denver, CO
   970-480-7618
  
  
  
   --
   Fergie, a.k.a. Paul Ferguson
   fergdawgster(at)gmail.com
  
  
  
  
   --
   Phil Fagan
   Denver, CO
   970-480-7618
  
  
  
  
   --
   Jamie Rishaw // .com.arpa@j - reverse it. ish.
   [Impressive C-level Title Here], arpa / arpa labs
 
 


 --
 Jamie Rishaw // .com.arpa@j - reverse it. ish.
 [Impressive C-level Title Here], arpa / arpa labs




-- 
-george william herbert
george.herb...@gmail.com


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread jamie rishaw
It's not poisoning.  They somehow were able to modify the NS records; one
would presume, at the registrar/s.

As far as the logic of the DNS, it is functioning as designed (What's up,
Vix!) - There's another aspect of this that caused this situation.

Any Alexa or similar people on this list (Goog PR, etc)?  I'd love to bulk
submit a domain list for some analytics.  Contact me off list.



On Thu, Jun 20, 2013 at 3:14 PM, George Herbert george.herb...@gmail.comwrote:

 Poisoning a domain's NS records with localhost will most certainly DOS the
 domain, yes.

 I have not yet seen the source of this; if anyone has a clue where the
 updates are coming from please post the info.

 Is there anything about ztomy.com that has been seen that's supicious as
 in they might be the origin?  This could be them, or could be a joe-job
 against them.  I do not want to point a finger lacking any sort of actual
 data dump of the poisoning activity...




 On Thu, Jun 20, 2013 at 1:02 PM, jamie rishaw j...@arpa.com wrote:

 I'm rechecking realtime ns1620/2620 DNS right now and, looking at the
 output, I see an odd number of domains (that have changed) with a listed
 nameserver of localhost..

 Is this some sort of tactic I'm unaware of?


 On Thu, Jun 20, 2013 at 2:57 PM, Jared Mauch ja...@puck.nether.net
 wrote:

  It seems there may be a need for some sort of 'dns-health' check out
 there
  that can be done in semi-realtime.
 
  I ran a report for someone earlier today on a domain doing an xref
 against
  open resolver data searching for valid responses vs invalid ones.
 
  Is this of value?  Does it need to be automated?
 
  - Jared
 
  On Jun 20, 2013, at 3:53 PM, jamie rishaw j...@arpa.com wrote:
 
   This is most definitely a coordinated and planned attack.
  
   And by 'attack' I mean hijacking of domain names.
  
   I show as of this morning nearly fifty thousand domain names that
 appear
   suspicious.
  
   I'm tempted to call uscentcom and/or related agencies (which agencies,
  who
   the hell knows, as ICE seems to have some sort of authority over
 domains
   (nearly two hundred fifty of them as I type this in COM alone and
 another
   thirty-some in NET).
  
   Anyone credentialed (credentialed /n/., I know you or know of you,)
   wanting data, e-mail me off-list for some TLD goodness.
  
  
  
  
  
  
   On Thu, Jun 20, 2013 at 12:29 PM, Phil Fagan philfa...@gmail.com
  wrote:
  
   Agree'd in these smaller scenario's I just wonder if in a larger
 scale
   scenario, whatever that might look like, if its necessary. Whereby
 many
   organizations who provide services are effected. Perhaps the result
  of a
   State led campaign topic for another day.
  
  
  
  
   On Thu, Jun 20, 2013 at 11:25 AM, Paul Ferguson 
 fergdawgs...@gmail.com
   wrote:
  
   I am betting that Netsol doesn't need any more coordination at the
   moment -- their phones are probably ringing off-the-hook. There are
   still ~400 domains still pointing to the ztomy NS:
  
  
   ;  DiG 9.7.3  @foohost parsonstech.com NS
   ; (1 server found)
   ;; global options: +cmd
   ;; Got answer:
   ;; -HEADER- opcode: QUERY, status: NOERROR, id: 49064
   ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
  
   ;; QUESTION SECTION:
   ;parsonstech.com.INNS
  
   ;; ANSWER SECTION:
   parsonstech.com.172800INNSns2617.ztomy.com.
   parsonstech.com.172800INNSns1617.ztomy.com.
  
   ;; Query time: 286 msec
   ;; SERVER: 127.0.0.1#53(127.0.0.1)
   ;; WHEN: Thu Jun 20 19:16:25 2013
   ;; MSG SIZE  rcvd: 81
  
   - ferg
  
   On Thu, Jun 20, 2013 at 10:13 AM, Phil Fagan philfa...@gmail.com
   wrote:
  
   I should caveat.coordinate the recovery of.
  
  
   On Thu, Jun 20, 2013 at 11:10 AM, Brandon Butterworth
   bran...@rd.bbc.co.ukwrote:
  
   Is there an organization that coordinates outages like this
 amongst
   the
   industry?
  
   No, usually they are surprise outages though Anonymous have tried
   coordinating a few
  
   brandon
  
  
  
  
   --
   Phil Fagan
   Denver, CO
   970-480-7618
  
  
  
   --
   Fergie, a.k.a. Paul Ferguson
   fergdawgster(at)gmail.com
  
  
  
  
   --
   Phil Fagan
   Denver, CO
   970-480-7618
  
  
  
  
   --
   Jamie Rishaw // .com.arpa@j - reverse it. ish.
   [Impressive C-level Title Here], arpa / arpa labs
 
 


 --
 Jamie Rishaw // .com.arpa@j - reverse it. ish.
 [Impressive C-level Title Here], arpa / arpa labs




 --
 -george william herbert
 george.herb...@gmail.com




-- 
Jamie Rishaw // .com.arpa@j - reverse it. ish.
[Impressive C-level Title Here], arpa / arpa labs


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Andrew Fried
Not so easy and straightforward to do.  You'll find that a lot of the
big names out there frequently tweak DNS, which will result in a
non-stop stream of alerts.

Andy

Andrew Fried
andrew.fr...@gmail.com

On 6/20/13 3:57 PM, Jared Mauch wrote:
 It seems there may be a need for some sort of 'dns-health' check out there 
 that can be done in semi-realtime.
 
 I ran a report for someone earlier today on a domain doing an xref against 
 open resolver data searching for valid responses vs invalid ones.
 
 Is this of value?  Does it need to be automated?
 
 - Jared
 
 On Jun 20, 2013, at 3:53 PM, jamie rishaw j...@arpa.com wrote:
 
 This is most definitely a coordinated and planned attack.

 And by 'attack' I mean hijacking of domain names.

 I show as of this morning nearly fifty thousand domain names that appear
 suspicious.

 I'm tempted to call uscentcom and/or related agencies (which agencies, who
 the hell knows, as ICE seems to have some sort of authority over domains
 (nearly two hundred fifty of them as I type this in COM alone and another
 thirty-some in NET).

 Anyone credentialed (credentialed /n/., I know you or know of you,)
 wanting data, e-mail me off-list for some TLD goodness.






 On Thu, Jun 20, 2013 at 12:29 PM, Phil Fagan philfa...@gmail.com wrote:

 Agree'd in these smaller scenario's I just wonder if in a larger scale
 scenario, whatever that might look like, if its necessary. Whereby many
 organizations who provide services are effected. Perhaps the result of a
 State led campaign topic for another day.




 On Thu, Jun 20, 2013 at 11:25 AM, Paul Ferguson fergdawgs...@gmail.com
 wrote:

 I am betting that Netsol doesn't need any more coordination at the
 moment -- their phones are probably ringing off-the-hook. There are
 still ~400 domains still pointing to the ztomy NS:


 ;  DiG 9.7.3  @foohost parsonstech.com NS
 ; (1 server found)
 ;; global options: +cmd
 ;; Got answer:
 ;; -HEADER- opcode: QUERY, status: NOERROR, id: 49064
 ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0

 ;; QUESTION SECTION:
 ;parsonstech.com.INNS

 ;; ANSWER SECTION:
 parsonstech.com.172800INNSns2617.ztomy.com.
 parsonstech.com.172800INNSns1617.ztomy.com.

 ;; Query time: 286 msec
 ;; SERVER: 127.0.0.1#53(127.0.0.1)
 ;; WHEN: Thu Jun 20 19:16:25 2013
 ;; MSG SIZE  rcvd: 81

 - ferg

 On Thu, Jun 20, 2013 at 10:13 AM, Phil Fagan philfa...@gmail.com
 wrote:

 I should caveat.coordinate the recovery of.


 On Thu, Jun 20, 2013 at 11:10 AM, Brandon Butterworth
 bran...@rd.bbc.co.ukwrote:

 Is there an organization that coordinates outages like this amongst
 the
 industry?

 No, usually they are surprise outages though Anonymous have tried
 coordinating a few

 brandon




 --
 Phil Fagan
 Denver, CO
 970-480-7618



 --
 Fergie, a.k.a. Paul Ferguson
 fergdawgster(at)gmail.com




 --
 Phil Fagan
 Denver, CO
 970-480-7618




 -- 
 Jamie Rishaw // .com.arpa@j - reverse it. ish.
 [Impressive C-level Title Here], arpa / arpa labs
 
 



Fwd: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread jamie rishaw
Wait, wait.

whois doesnt jive with dns.

.. Conspiracy Theory Hat On :

- Did someone gain access to the COM dispersion zone, or parts thereof?
- Did someone figure out how to [ insert theory here ] ?

I'm looking at domains that were solidly pointing at ztomy at 2:30AM (that
are 'recovered'  to other nameservers) that show no updates in `whois`
records.

Curiouser and curiouser.

Paul?

-- Forwarded message --
From: jamie rishaw j...@arpa.com
Date: Thu, Jun 20, 2013 at 3:21 PM
Subject: Re: This is a coordinated hacking. (Was Re: Need help in flushing
DNS)
To: George Herbert george.herb...@gmail.com
Cc: Jared Mauch ja...@puck.nether.net, NANOG nanog@nanog.org


It's not poisoning.  They somehow were able to modify the NS records; one
would presume, at the registrar/s.

As far as the logic of the DNS, it is functioning as designed (What's up,
Vix!) - There's another aspect of this that caused this situation.

Any Alexa or similar people on this list (Goog PR, etc)?  I'd love to bulk
submit a domain list for some analytics.  Contact me off list.



On Thu, Jun 20, 2013 at 3:14 PM, George Herbert george.herb...@gmail.comwrote:

 Poisoning a domain's NS records with localhost will most certainly DOS the
 domain, yes.

 I have not yet seen the source of this; if anyone has a clue where the
 updates are coming from please post the info.

 Is there anything about ztomy.com that has been seen that's supicious as
 in they might be the origin?  This could be them, or could be a joe-job
 against them.  I do not want to point a finger lacking any sort of actual
 data dump of the poisoning activity...




 On Thu, Jun 20, 2013 at 1:02 PM, jamie rishaw j...@arpa.com wrote:

 I'm rechecking realtime ns1620/2620 DNS right now and, looking at the
 output, I see an odd number of domains (that have changed) with a listed
 nameserver of localhost..

 Is this some sort of tactic I'm unaware of?


 On Thu, Jun 20, 2013 at 2:57 PM, Jared Mauch ja...@puck.nether.net
 wrote:

  It seems there may be a need for some sort of 'dns-health' check out
 there
  that can be done in semi-realtime.
 
  I ran a report for someone earlier today on a domain doing an xref
 against
  open resolver data searching for valid responses vs invalid ones.
 
  Is this of value?  Does it need to be automated?
 
  - Jared
 
  On Jun 20, 2013, at 3:53 PM, jamie rishaw j...@arpa.com wrote:
 
   This is most definitely a coordinated and planned attack.
  
   And by 'attack' I mean hijacking of domain names.
  
   I show as of this morning nearly fifty thousand domain names that
 appear
   suspicious.
  
   I'm tempted to call uscentcom and/or related agencies (which agencies,
  who
   the hell knows, as ICE seems to have some sort of authority over
 domains
   (nearly two hundred fifty of them as I type this in COM alone and
 another
   thirty-some in NET).
  
   Anyone credentialed (credentialed /n/., I know you or know of you,)
   wanting data, e-mail me off-list for some TLD goodness.
  
  
  
  
  
  
   On Thu, Jun 20, 2013 at 12:29 PM, Phil Fagan philfa...@gmail.com
  wrote:
  
   Agree'd in these smaller scenario's I just wonder if in a larger
 scale
   scenario, whatever that might look like, if its necessary. Whereby
 many
   organizations who provide services are effected. Perhaps the result
  of a
   State led campaign topic for another day.
  
  
  
  
   On Thu, Jun 20, 2013 at 11:25 AM, Paul Ferguson 
 fergdawgs...@gmail.com
   wrote:
  
   I am betting that Netsol doesn't need any more coordination at the
   moment -- their phones are probably ringing off-the-hook. There are
   still ~400 domains still pointing to the ztomy NS:
  
  
   ;  DiG 9.7.3  @foohost parsonstech.com NS
   ; (1 server found)
   ;; global options: +cmd
   ;; Got answer:
   ;; -HEADER- opcode: QUERY, status: NOERROR, id: 49064
   ;; flags: qr rd ra; QUERY: 1, ANSWER: 2, AUTHORITY: 0, ADDITIONAL: 0
  
   ;; QUESTION SECTION:
   ;parsonstech.com.INNS
  
   ;; ANSWER SECTION:
   parsonstech.com.172800INNSns2617.ztomy.com.
   parsonstech.com.172800INNSns1617.ztomy.com.
  
   ;; Query time: 286 msec
   ;; SERVER: 127.0.0.1#53(127.0.0.1)
   ;; WHEN: Thu Jun 20 19:16:25 2013
   ;; MSG SIZE  rcvd: 81
  
   - ferg
  
   On Thu, Jun 20, 2013 at 10:13 AM, Phil Fagan philfa...@gmail.com
   wrote:
  
   I should caveat.coordinate the recovery of.
  
  
   On Thu, Jun 20, 2013 at 11:10 AM, Brandon Butterworth
   bran...@rd.bbc.co.ukwrote:
  
   Is there an organization that coordinates outages like this
 amongst
   the
   industry?
  
   No, usually they are surprise outages though Anonymous have tried
   coordinating a few
  
   brandon
  
  
  
  
   --
   Phil Fagan
   Denver, CO
   970-480-7618
  
  
  
   --
   Fergie, a.k.a. Paul Ferguson
   fergdawgster(at)gmail.com
  
  
  
  
   --
   Phil Fagan
   Denver, CO
   970-480-7618
  
  
  



 --
 -george william herbert
 george.herb...@gmail.com



Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Jimmy Hess
On 6/20/13, jamie rishaw j...@arpa.com wrote:
 It's not poisoning.  They somehow were able to modify the NS records; one
 would presume, at the registrar/s.

https://www.networksolutions.com/blog/2013/06/important-update-for-network-solutions-customers-experiencing-website-issues/

--
-JH



Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Jeff Shultz

On 6/20/2013 1:46 PM, Jimmy Hess wrote:

On 6/20/13, jamie rishaw j...@arpa.com wrote:

It's not poisoning.  They somehow were able to modify the NS records; one
would presume, at the registrar/s.


https://www.networksolutions.com/blog/2013/06/important-update-for-network-solutions-customers-experiencing-website-issues/

--
-JH



small number of Network Solutions customers

They must be staffed with physicists, astronomers, or economists I 
don't know anyone else that would consider nearly fifty thousand (from 
a previous post by Phil Fagan) to be a small number.


--
Jeff Shultz





Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Carsten Bormann
Wild speculation:

netsol says this is a human error incurred during DDOS mitigation.
ztomy.com is a wild-card DNS provider that seems to use prolexic.
Now imagine someone at netsol or its DDOS service providers
fat-fingered their DDOS-averting routing in such a way that netsol
DNS traffic arrived at ztomy.com instead of a netsol server.
The ztomy.com server would know how to answer the queries...

I have no data to base this speculation on.

Grüße, Carsten




RE: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Gabor Tokaji
Hello everyone, I'm new here.
+1 to this theory. I've been watching what's happening since 3am Eastern, 
because a domain of mine (of the many at NetSol) was a victim of this event.

-Gabor

-Original Message-
From: Carsten Bormann [mailto:c...@tzi.org] 
Sent: Thursday, June 20, 2013 5:11 PM
To: NANOG list
Subject: Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

Wild speculation:

netsol says this is a human error incurred during DDOS mitigation.
ztomy.com is a wild-card DNS provider that seems to use prolexic.
Now imagine someone at netsol or its DDOS service providers fat-fingered their 
DDOS-averting routing in such a way that netsol DNS traffic arrived at 
ztomy.com instead of a netsol server.
The ztomy.com server would know how to answer the queries...

I have no data to base this speculation on.

Grüße, Carsten





Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Valdis . Kletnieks
On Thu, 20 Jun 2013 14:08:18 -0700, Jeff Shultz said:

 small number of Network Solutions customers

 They must be staffed with physicists, astronomers, or economists I
 don't know anyone else that would consider nearly fifty thousand (from
 a previous post by Phil Fagan) to be a small number.

It's relatively small when you consider there's something like 140M .com's



pgpA4dQRKUb7v.pgp
Description: PGP signature


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread RijilV
On 20 June 2013 14:28, valdis.kletni...@vt.edu wrote:

 On Thu, 20 Jun 2013 14:08:18 -0700, Jeff Shultz said:

  small number of Network Solutions customers
 
  They must be staffed with physicists, astronomers, or economists I
  don't know anyone else that would consider nearly fifty thousand (from
  a previous post by Phil Fagan) to be a small number.

 It's relatively small when you consider there's something like 140M .com's


So it's okay to screw over nearly fifty thousand customer domains because
there are 140M .com's?  When talking about inadvertently effecting that
many folks I don't think it is appropriate to trivialize the customer
impact by calling it small when you're talking about a handful of large
websites that aren't somehow magically shared over those 140M .coms.  Also
it is untrue to limit it to only the websites given how many other things
folks are likely to be using DNS for...

.r'


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Randy Bush
 So it's okay to screw over nearly fifty thousand customer domains because
 there are 140M .com's?

luckily, none of the rest of us make mistakes



Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Ryan - Lists
I don't think he was saying that at all. Just stating that from a pure numbers 
standpoint 50k/140mil is a small percentage.

OTOH, I agree to your point - Network Solutions definitely downplayed this in 
their release. Curiously so.

Sent from my iPhone

On Jun 20, 2013, at 5:42 PM, RijilV rij...@riji.lv wrote:

 On 20 June 2013 14:28, valdis.kletni...@vt.edu wrote:
 
 On Thu, 20 Jun 2013 14:08:18 -0700, Jeff Shultz said:
 
 small number of Network Solutions customers
 
 They must be staffed with physicists, astronomers, or economists I
 don't know anyone else that would consider nearly fifty thousand (from
 a previous post by Phil Fagan) to be a small number.
 
 It's relatively small when you consider there's something like 140M .com's
 So it's okay to screw over nearly fifty thousand customer domains because
 there are 140M .com's?  When talking about inadvertently effecting that
 many folks I don't think it is appropriate to trivialize the customer
 impact by calling it small when you're talking about a handful of large
 websites that aren't somehow magically shared over those 140M .coms.  Also
 it is untrue to limit it to only the websites given how many other things
 folks are likely to be using DNS for...
 
 .r'



Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Bryan Irvine
On Thu, Jun 20, 2013 at 2:49 PM, Randy Bush ra...@psg.com wrote:

  So it's okay to screw over nearly fifty thousand customer domains
 because
  there are 140M .com's?

 luckily, none of the rest of us make mistakes


Ages ago I responded on a Cisco list where the topic was biggest screwup
you've made.  I posted that I once forgot the implicit deny in an ACL and
accidentally blocked all traffic between 4 locations in 2 states for a
company I was working for. Downtime was a very brutal 60 seconds. Someone
very insightful responded with anyone who hasn't done similar is lying
about the 10 years on their resume.  So the real question would be, why
wasn't there someone who has already done this in the past working on this
zone? ;)

-B


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Richard Golodner
On Thu, 2013-06-20 at 14:42 -0700, RijilV wrote:
 On 20 June 2013 14:28, valdis.kletni...@vt.edu wrote:
 
  On Thu, 20 Jun 2013 14:08:18 -0700, Jeff Shultz said:
 
   small number of Network Solutions customers
  
   They must be staffed with physicists, astronomers, or economists I
   don't know anyone else that would consider nearly fifty thousand (from
   a previous post by Phil Fagan) to be a small number.
 
  It's relatively small when you consider there's something like 140M .com's
 
 
 So it's okay to screw over nearly fifty thousand customer domains because
 there are 140M .com's?  When talking about inadvertently effecting that
 many folks I don't think it is appropriate to trivialize the customer
 impact by calling it small when you're talking about a handful of large
 websites that aren't somehow magically shared over those 140M .coms.  Also
 it is untrue to limit it to only the websites given how many other things
 folks are likely to be using DNS for...
 
 .r'
 

I think you are reading it the wrong way. Mr.Kletnieks never said it
was okay. He just stated that the numbers were trivial when compared to
the rest of potential customers being affected.
Be cool, Richard Golodner




Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Randy Bush
netsol screwed up.  they screwed up bigtime.  they are shoveling kitty
litter over it as fast as they can, and they have a professional kitty
litter, aka pr, department.

but none of this is surprising.

and dnssec did not save us.  is there anything which could have?

randy




Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread George Herbert
At the DNS Servers or service provider level, one can (and I often do) have 
redundant providers.

At the registrar level?  ...

Not with our current infrastructure, as far as I know how.

The Internet:  Discovering new SPOF since 1969!


George William Herbert
Sent from my iPhone

On Jun 20, 2013, at 3:28 PM, Randy Bush ra...@psg.com wrote:

 netsol screwed up.  they screwed up bigtime.  they are shoveling kitty
 litter over it as fast as they can, and they have a professional kitty
 litter, aka pr, department.
 
 but none of this is surprising.
 
 and dnssec did not save us.  is there anything which could have?
 
 randy
 
 



Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Phil Fagan
at what point is the Internet a piece of infrastructure whereby we
actually need a way to watch this thing holistically as it is one system
and not just a bunch of inter-jointed systems? Who's job is it to do
nothing but ensure that the state of DNS and other services is running as
it shouldwho's the clearing house here.


On Thu, Jun 20, 2013 at 4:28 PM, Randy Bush ra...@psg.com wrote:

 netsol screwed up.  they screwed up bigtime.  they are shoveling kitty
 litter over it as fast as they can, and they have a professional kitty
 litter, aka pr, department.

 but none of this is surprising.

 and dnssec did not save us.  is there anything which could have?

 randy





-- 
Phil Fagan
Denver, CO
970-480-7618


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread jamie rishaw
No.

The ztomy nameservers appeared in this morning's master .COM zonefile as
/authoritative/ for the number of domains I mentioned.

It is a clear change from just a couple of days ago, when the listed
nameservers were nowhere to be seen.

I have solid data to back this up, straight from Verisign GRS (Verisign),
the authoritative registry for .COM, .NET and others.

j



On Thu, Jun 20, 2013 at 4:10 PM, Carsten Bormann c...@tzi.org wrote:

 Wild speculation:

 netsol says this is a human error incurred during DDOS mitigation.
 ztomy.com is a wild-card DNS provider that seems to use prolexic.
 Now imagine someone at netsol or its DDOS service providers
 fat-fingered their DDOS-averting routing in such a way that netsol
 DNS traffic arrived at ztomy.com instead of a netsol server.
 The ztomy.com server would know how to answer the queries...

 I have no data to base this speculation on.

 Grüße, Carsten





-- 
Jamie Rishaw // .com.arpa@j - reverse it. ish.
[Impressive C-level Title Here], arpa / arpa labs


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Fred Reimer
I, for one, would not be in favor of an authoritarian rule over DNS, or
any other Internet system, to ensure that the state of [the] service[s]
is running as it should.  I suppose one could view such an authoritarian
rule over (sub) systems to be a good thing, as in there is someone to
complain to when things don't work, but recent events show that it is also
easily abused.  I much rather prefer the current cooperative
administration of the Internet.

Thanks,

Fred Reimer


On 6/20/13 6:39 PM, Phil Fagan philfa...@gmail.com wrote:

at what point is the Internet a piece of infrastructure whereby we
actually need a way to watch this thing holistically as it is one system
and not just a bunch of inter-jointed systems? Who's job is it to do
nothing but ensure that the state of DNS and other services is running as
it shouldwho's the clearing house here.


On Thu, Jun 20, 2013 at 4:28 PM, Randy Bush ra...@psg.com wrote:

 netsol screwed up.  they screwed up bigtime.  they are shoveling kitty
 litter over it as fast as they can, and they have a professional kitty
 litter, aka pr, department.

 but none of this is surprising.

 and dnssec did not save us.  is there anything which could have?

 randy





-- 
Phil Fagan
Denver, CO
970-480-7618




Fwd: Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Timothy Morizot
On Jun 20, 2013 5:31 PM, Randy Bush ra...@psg.com wrote:
 and dnssec did not save us.  is there anything which could have?

Hmmm. DNSSEC wouldn't have prevented an outage. But from everything I've
seen reported, had the zones been signed, validating recursive resolvers
(comcast, google, much of federal government, mine) would have returned
servfail and would not have cached the bad nameservers in their good cache.

Users would have simply failed to connect instead of being sent to the
wrong page and recovery would have been quicker and easier. From my
perspective as someone responsible for DNS at a fairly large enterprise,
that would have been preferable.

But then, the zones for which I'm responsible are signed.

YMMV,

Scott


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Jimmy Hess
On 6/20/13, Randy Bush ra...@psg.com wrote:
 netsol screwed up.  they screwed up bigtime.  they are shoveling kitty
 litter over it as fast as they can, and they have a professional kitty
 litter, aka pr, department.
 but none of this is surprising.
 and dnssec did not save us.  is there anything which could have?

What's puzzling is  the How the heck did they do that?

The registrar doesn't maintain the .COM database that contains the
list of nameservers
they had to submit changes to all those records.

So, why weren't there security controls to make sure that the
registrar could not submit changes without appropriate authorization
from the Administrative/Tech contact?


 randy
--
-JH



Re: Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Rubens Kuhl
On Thu, Jun 20, 2013 at 8:41 PM, Timothy Morizot tmori...@gmail.com wrote:

 On Jun 20, 2013 5:31 PM, Randy Bush ra...@psg.com wrote:
  and dnssec did not save us.  is there anything which could have?

 Hmmm. DNSSEC wouldn't have prevented an outage. But from everything I've
 seen reported, had the zones been signed, validating recursive resolvers
 (comcast, google, much of federal government, mine) would have returned
 servfail and would not have cached the bad nameservers in their good cache.

 Users would have simply failed to connect instead of being sent to the
 wrong page and recovery would have been quicker and easier. From my
 perspective as someone responsible for DNS at a fairly large enterprise,
 that would have been preferable.

 But then, the zones for which I'm responsible are signed.


In this case of registrar compromise, DS record could have been changed
alongside NS records, so DNSSEC would only have been a early warning,
because uncoordinated DS change disrupts service. As soon as previous
timeouts played out, new DS/NS pairs would be considered as trustworthy as
the old ones.


Rubens


Re: Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Timothy Morizot
On Jun 20, 2013 7:30 PM, Rubens Kuhl rube...@gmail.com wrote:
 In this case of registrar compromise, DS record could have been changed
 alongside NS records, so DNSSEC would only have been a early warning,
 because uncoordinated DS change disrupts service. As soon as previous
 timeouts played out, new DS/NS pairs would be considered as trustworthy as
 the old ones.

Since DS records typically have a ttl of 24 hours, that protection should
not be underestimated even in the case of registrar compromise.

However, everything released so far indicates this was a netsol error and
not a compromise. And it was an error corrected fairly quickly from what I
can tell. The impact was prolonged because the bad nameservers were cached
in resolvers across the Internet.

Of course, very few details have actually been released, so that
construction could be wrong. But even in the worst case DNSSEC would have
provided some mitigation for a time.


Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Hank Nussbacher

At 07:28 21/06/2013 +0900, Randy Bush wrote:

netsol screwed up.  they screwed up bigtime.  they are shoveling kitty
litter over it as fast as they can, and they have a professional kitty
litter, aka pr, department.


They are too busy adding new revenue:
http://www.streetinsider.com/Corporate+News/NetSol+%28NTWK%29+Enters+$10M+Agreement+for+Financial+Suite+Implementation/8434663.html

-Hank




Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Hank Nussbacher

At 17:12 20/06/2013 -0500, Richard Golodner wrote:


I think you are reading it the wrong way. Mr.Kletnieks never said it
was okay. He just stated that the numbers were trivial when compared to
the rest of potential customers being affected.
Be cool, Richard Golodner


sarcasm
and Netsol agrees with you:
http://www.networksolutions.com/blog/2013/06/important-update-for-network-solutions-customers-experiencing-website-issues/

a small number of Network Solutions customers were inadvertently affected 
for up to several hours.

/sarcasm

-Hank




Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Hal Murray

 at what point is the Internet a piece of infrastructure whereby we
 actually need a way to watch this thing holistically as it is one system and
 not just a bunch of inter-jointed systems? Who's job is it to do nothing but
 ensure that the state of DNS and other services is running as it
 shouldwho's the clearing house here.

 The Internet:  Discovering new SPOF since 1969! 
:)  Thanks.

Perhaps we should setup a distributed system for checking things rather than 
another SPOF.  That's distributed both geographically and administratively 
and using several code-bases.

In this context, I'd expect lots of false alarms due to people changing their 
DNS servers but forgetting to inform their monitoring setup (either internal 
or outsourced).

How would you check/verify that the communication path from the monitoring 
agency to the right people in your NOC was working correctly?


-- 
These are my opinions.  I hate spam.






Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread shawn wilson
I think ICANN would have to add a delay in where a request was sent out to
make sure everyone was on the same page and then what happens the couple
thousand (more)  times a day that someone isn't updated or is
misconfigured?

I think Netsol should be fined. Maybe even a class action suite filed
against them for lost business. And that's it.
On Jun 20, 2013 11:28 PM, Hal Murray hmur...@megapathdsl.net wrote:


  at what point is the Internet a piece of infrastructure whereby we
  actually need a way to watch this thing holistically as it is one system
 and
  not just a bunch of inter-jointed systems? Who's job is it to do nothing
 but
  ensure that the state of DNS and other services is running as it
  shouldwho's the clearing house here.

  The Internet:  Discovering new SPOF since 1969!
 :)  Thanks.

 Perhaps we should setup a distributed system for checking things rather
 than
 another SPOF.  That's distributed both geographically and administratively
 and using several code-bases.

 In this context, I'd expect lots of false alarms due to people changing
 their
 DNS servers but forgetting to inform their monitoring setup (either
 internal
 or outsourced).

 How would you check/verify that the communication path from the monitoring
 agency to the right people in your NOC was working correctly?


 --
 These are my opinions.  I hate spam.







Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)

2013-06-20 Thread Valdis . Kletnieks
On Thu, 20 Jun 2013 20:25:24 -0700, Hal Murray said:

 How would you check/verify that the communication path from the monitoring
 agency to the right people in your NOC was working correctly?

Remember to consider the possible impact of a false-positive report over
an unauthenticated channel. Because if it's possible, somebody will try it,
just because they just want to watch stuff burn. :)


pgpvQasT4FmSG.pgp
Description: PGP signature