Data on June 20 :
.COM. :
108,985,894 unique domains + the tld.
- 234,479 NSEC3/RRSIG records,
- 2,253,400 nameserver entries on 831,088 unique IP addresses.
.. ish.
-jamie
On Fri, Jun 21, 2013 at 5:23 PM, Barry Shein b...@world.std.com wrote:
I think we need a better measure than
On 6/20/13, Hal Murray hmur...@megapathdsl.net wrote:
Perhaps we should setup a distributed system for checking things rather than
another SPOF. That's distributed both geographically and administratively
and using several code-bases.
[snip]
I would be in favor of being able to pay two
On 6/20/13, valdis.kletni...@vt.edu valdis.kletni...@vt.edu wrote:
It's relatively small when you consider there's something like 140M .com's
Yeah... I'm in agreement about that's probably what is going on...
It's relatively small, but absolutely large, and absolute numbers
matter. 5 domains
Golodner
Cc: nanog@nanog.org
Subject: Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)
At 17:12 20/06/2013 -0500, Richard Golodner wrote:
I think you are reading it the wrong way. Mr.Kletnieks never said it
was okay. He just stated that the numbers were trivial when
On Thu, 20 Jun 2013 23:42:24 -0400, shawn wilson said:
I think Netsol should be fined. Maybe even a class action suite filed
against them for lost business. And that's it.
So your contract with NetSol has an SLA guarantee in it, and you can
demonstrate that (a) said SLA has been violated and
On Thu, Jun 20, 2013 at 05:28:17PM -0400, valdis.kletni...@vt.edu wrote:
It's relatively small when you consider there's something like 140M .com's
Just FWIW, the current size of .com is roughly 109M domains. Someday it
will reach 140M but not today.
Nicolai
https://www.networksolutions.com/blog/2013/06/important-update-for-network-solutions-customers-experiencing-website-issues/
Why are they infinitely looping a script on their web server to check
for a cookie?
Are these people insane?
Registrar Primary and Registrar Auditor
There are certainly registrars who are more security oriented than
Netsol. If you haven't followed all of the corporate buying and
selling, Netsol is now part of web.com, so their business is more to
support web hosting than to be a registrar.
I expect
I think we need a better measure than number of domains (in this case
.COM), particularly vs total domains.
If it was 100 domains it might seem small, unless that list began with
facebook.com, amazon.com, google.com and g*d forbid theworld.com.
--
-Barry Shein
The World |
Operators Group
Subject: Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)
I think ICANN would have to add a delay in where a request was sent out to make
sure everyone was on the same page and then what happens the couple thousand
(more) times a day that someone isn't updated
Hi,
Do we know which DNS server started leaking the poisoned entry?
Being new to this, i still dont understand how could a hacker gain access
to the DNS server and corrupt the entry there? Wouldnt it require special
admin rights, etc. to log in?
Glen
On Thu, Jun 20, 2013 at 11:32 AM, Paul
Not sure of some of the underlying details of the mechanics right now.
http://news.softpedia.com/news/LinkedIn-Outage-Caused-by-DDOS-Attack-on-Network-Solutions-362473.shtml
- ferg
On Fri, Jun 21, 2013 at 5:22 PM, Glen Kent glen.k...@gmail.com wrote:
Hi,
Do we know which DNS server started
The indications and claim are that the root cause was registrar internal
goof, not hostile action against name servers.
The story is not yet detailed enough to add up; getting from point A to
point B requires steps that so far don't really make sense. A more
detailed explanation is hopefully to
...@chocolatine.org]
Sent: Friday, June 21, 2013 11:16 AM
To: nanog@nanog.org
Subject: Re: This is a coordinated hacking. (Was Re: Need help in flushing
DNS)
On Thu, Jun 20, 2013 at 05:28:17PM -0400, valdis.kletni...@vt.edu wrote:
It's relatively small when you consider there's something like 140M .com's
In article 001a01ce6ef9$bf74d4a0$3e5e7de0$@iname.com you write:
It's 120M if you add the .COM and the .NET's together, both of which NetSol
is responsible for.
http://www.verisigninc.com/en_US/products-and-services/domain-name-services/
registry-products/tld-zone-access/index.xhtml
In late
I know how we got here, but perhaps we can take corporate parentage and how big
.com is now to -discuss?
What happened with the registry data that caused the outage and what can /
should be done about it / to prevent it happening again still seem to me to be
operational topics.
George
The only apparent link is registration thru network solutions
On Wed, Jun 19, 2013 at 10:49 PM, Alex Buie alex.b...@frozenfeline.netwrote:
Anyone have news/explanation about what's happening/happened?
On Wed, Jun 19, 2013 at 10:34 PM, Paul Ferguson fergdawgs...@gmail.com
wrote:
Sure
Hanlon's razor? Misconfiguration. Perhaps not done in malice, but I
have no idea where the poison leaked in, or why. :-)
- ferg
On Wed, Jun 19, 2013 at 10:49 PM, Alex Buie alex.b...@frozenfeline.net wrote:
Anyone have news/explanation about what's happening/happened?
On Wed, Jun 19, 2013 at
On 6/20/13, Paul Ferguson fergdawgs...@gmail.com wrote:
On Wed, Jun 19, 2013 at 10:44 PM, Tom Paseka t...@cloudflare.com wrote:
On Wed, Jun 19, 2013 at 10:32 PM, Patrick W. Gilmore
I think ztomy.com smells really bad for some reason, looks like
100% advertising;
sure doesn't appear to be a
On Jun 19, 2013, at 11:23 PM, Jimmy Hess mysi...@gmail.com wrote:
On 6/20/13, Paul Ferguson fergdawgs...@gmail.com wrote:
On Wed, Jun 19, 2013 at 10:44 PM, Tom Paseka t...@cloudflare.com wrote:
On Wed, Jun 19, 2013 at 10:32 PM, Patrick W. Gilmore
I think ztomy.com smells really bad for some
.-- My secret spy satellite informs me that at 2013-06-19 10:34 PM Paul
Ferguson wrote:
; DiG 9.7.3 @localhost yelp.com A
SNIP
;; ANSWER SECTION:
yelp.com. 300 IN A 204.11.56.20
Interesting to see that traffic to this IP addresses is going through
prolexic...
I guess they're considering
I have no knowledge of any DDoS -related activity involving Yelp! and
Prolexic. Even if there is one, the fact that their DNS records have
been poisoned has not direct relationship to any current DDoS (there
isn't one that I am aware of).
- ferg
On Thu, Jun 20, 2013 at 12:31 AM, Andree Toonk
.-- My secret spy satellite informs me that at 2013-06-20 12:31 AM
Andree Toonk wrote:
.-- My secret spy satellite informs me that at 2013-06-19 10:34 PM Paul
Ferguson wrote:
; DiG 9.7.3 @localhost yelp.com A
SNIP
;; ANSWER SECTION:
yelp.com. 300 IN A 204.11.56.20
Interesting to
Hi,
.-- My secret spy satellite informs me that at 2013-06-20 12:38 AM Paul
Ferguson wrote:
I have no knowledge of any DDoS -related activity involving Yelp! and
Prolexic. Even if there is one, the fact that their DNS records have
been poisoned has not direct relationship to any current DDoS
I have domains that are *not* expired, which are being affected by this.
Domains are hosted via Dynect, and are resolving into this 204.11.56.0/24 range
across the globe.
Dynect management portal was down until minutes ago as well.
- Charles
On Jun 20, 2013, at 12:45 AM, David Conrad
Smileyface aside, I'm disappointed to see operators simply flushing caches
and not performing at the least a dumpdb for possible future forensic
analysis.
This is what I call the Windows solution, - 'Oh, just reboot, and it'll
work'.
We're better than that.
(Aren't we?)
On Thu, Jun 20, 2013
I am not speaking officially, but the evidence so far is that this was not
DNS poisoning, but domain name hijacking. My colleagues will have more to
say later today.
On Thu, Jun 20, 2013 at 1:19 AM, John Levine jo...@iecc.com wrote:
Reaching out to DNS operators around the globe. Linkedin.com
Subject: Re: Need help in flushing DNS
On 6/20/13, Paul Ferguson fergdawgs...@gmail.com wrote:
On Wed, Jun 19, 2013 at 10:44 PM, Tom Paseka t...@cloudflare.com wrote:
On Wed, Jun 19, 2013 at 10:32 PM, Patrick W. Gilmore
I think ztomy.com smells really bad for some reason, looks like
100
ng/http://techcrunch.com/2013/06/19/linkedin-outage-due-to-possible-dns-hijacking/
Frank
-Original Message-
From: Jimmy Hess [mailto:mysi...@gmail.com]
Sent: Thursday, June 20, 2013 1:23 AM
To: Paul Ferguson
Cc: NANOG list
Subject: Re: Need help in flushing DNS
On 6/20/13, Paul
/http://techcrunch.com/2013/06/19/linkedin-outage-due-to-possible-dns-hijacking/
Frank
-Original Message-
From: Jimmy Hess [mailto:mysi...@gmail.com]
Sent: Thursday, June 20, 2013 1:23 AM
To: Paul Ferguson
Cc: NANOG list
Subject: Re: Need help in flushing DNS
On 6/20/13, Paul
, June 20, 2013 1:23 AM
To: Paul Ferguson
Cc: NANOG list
Subject: Re: Need help in flushing DNS
On 6/20/13, Paul Ferguson fergdawgs...@gmail.com wrote:
On Wed, Jun 19, 2013 at 10:44 PM, Tom Paseka t...@cloudflare.com
wrote:
On Wed, Jun 19, 2013 at 10:32 PM, Patrick W. Gilmore
I
/
Frank
-Original Message-
From: Jimmy Hess [mailto:mysi...@gmail.com]
Sent: Thursday, June 20, 2013 1:23 AM
To: Paul Ferguson
Cc: NANOG list
Subject: Re: Need help in flushing DNS
On 6/20/13, Paul Ferguson fergdawgs...@gmail.com wrote:
On Wed, Jun 19, 2013 at 10:44 PM, Tom
: Need help in flushing DNS
On 6/20/13, Paul Ferguson fergdawgs...@gmail.com wrote:
On Wed, Jun 19, 2013 at 10:44 PM, Tom Paseka t...@cloudflare.com
wrote:
On Wed, Jun 19, 2013 at 10:32 PM, Patrick W. Gilmore
I think ztomy.com smells really bad for some reason, looks like
100
* philfa...@gmail.com (Phil Fagan) [Thu 20 Jun 2013, 17:50 CEST]:
Is there an organization that coordinates outages like this amongst
the industry?
No; all outages on the Internet happen independently from each other
and are not coordinated to (not) coincide in any way.
-- Niels.
http://www.networksolutions.com/blog/2013/06/important-update-for-network-solutions-customers-experiencing-website-issues/
- Jared
On Jun 19, 2013, at 11:42 PM, Zaid Ali Kahn z...@zaidali.com wrote:
Reaching out to DNS operators around the globe. Linkedin.com has had some
issues with DNS and
Is there an organization that coordinates outages like this amongst the
industry?
No, usually they are surprise outages though Anonymous have tried
coordinating a few
brandon
I should caveat.coordinate the recovery of.
On Thu, Jun 20, 2013 at 11:10 AM, Brandon Butterworth
bran...@rd.bbc.co.ukwrote:
Is there an organization that coordinates outages like this amongst the
industry?
No, usually they are surprise outages though Anonymous have tried
I am betting that Netsol doesn't need any more coordination at the
moment -- their phones are probably ringing off-the-hook. There are
still ~400 domains still pointing to the ztomy NS:
; DiG 9.7.3 @foohost parsonstech.com NS
; (1 server found)
;; global options: +cmd
;; Got answer:
;;
Agree'd in these smaller scenario's I just wonder if in a larger scale
scenario, whatever that might look like, if its necessary. Whereby many
organizations who provide services are effected. Perhaps the result of a
State led campaign topic for another day.
On Thu, Jun 20, 2013 at 11:25
This is most definitely a coordinated and planned attack.
And by 'attack' I mean hijacking of domain names.
I show as of this morning nearly fifty thousand domain names that appear
suspicious.
I'm tempted to call uscentcom and/or related agencies (which agencies, who
the hell knows, as ICE
It seems there may be a need for some sort of 'dns-health' check out there that
can be done in semi-realtime.
I ran a report for someone earlier today on a domain doing an xref against open
resolver data searching for valid responses vs invalid ones.
Is this of value? Does it need to be
I'm rechecking realtime ns1620/2620 DNS right now and, looking at the
output, I see an odd number of domains (that have changed) with a listed
nameserver of localhost..
Is this some sort of tactic I'm unaware of?
On Thu, Jun 20, 2013 at 2:57 PM, Jared Mauch ja...@puck.nether.net wrote:
It
Poisoning a domain's NS records with localhost will most certainly DOS the
domain, yes.
I have not yet seen the source of this; if anyone has a clue where the
updates are coming from please post the info.
Is there anything about ztomy.com that has been seen that's supicious as in
they might be
It's not poisoning. They somehow were able to modify the NS records; one
would presume, at the registrar/s.
As far as the logic of the DNS, it is functioning as designed (What's up,
Vix!) - There's another aspect of this that caused this situation.
Any Alexa or similar people on this list (Goog
Not so easy and straightforward to do. You'll find that a lot of the
big names out there frequently tweak DNS, which will result in a
non-stop stream of alerts.
Andy
Andrew Fried
andrew.fr...@gmail.com
On 6/20/13 3:57 PM, Jared Mauch wrote:
It seems there may be a need for some sort of
' to other nameservers) that show no updates in `whois`
records.
Curiouser and curiouser.
Paul?
-- Forwarded message --
From: jamie rishaw j...@arpa.com
Date: Thu, Jun 20, 2013 at 3:21 PM
Subject: Re: This is a coordinated hacking. (Was Re: Need help in flushing
DNS)
To: George Herbert
On 6/20/13, jamie rishaw j...@arpa.com wrote:
It's not poisoning. They somehow were able to modify the NS records; one
would presume, at the registrar/s.
https://www.networksolutions.com/blog/2013/06/important-update-for-network-solutions-customers-experiencing-website-issues/
--
-JH
On 6/20/2013 1:46 PM, Jimmy Hess wrote:
On 6/20/13, jamie rishaw j...@arpa.com wrote:
It's not poisoning. They somehow were able to modify the NS records; one
would presume, at the registrar/s.
Wild speculation:
netsol says this is a human error incurred during DDOS mitigation.
ztomy.com is a wild-card DNS provider that seems to use prolexic.
Now imagine someone at netsol or its DDOS service providers
fat-fingered their DDOS-averting routing in such a way that netsol
DNS traffic arrived
PM
To: NANOG list
Subject: Re: This is a coordinated hacking. (Was Re: Need help in flushing DNS)
Wild speculation:
netsol says this is a human error incurred during DDOS mitigation.
ztomy.com is a wild-card DNS provider that seems to use prolexic.
Now imagine someone at netsol or its DDOS service
On Thu, 20 Jun 2013 14:08:18 -0700, Jeff Shultz said:
small number of Network Solutions customers
They must be staffed with physicists, astronomers, or economists I
don't know anyone else that would consider nearly fifty thousand (from
a previous post by Phil Fagan) to be a small number.
On 20 June 2013 14:28, valdis.kletni...@vt.edu wrote:
On Thu, 20 Jun 2013 14:08:18 -0700, Jeff Shultz said:
small number of Network Solutions customers
They must be staffed with physicists, astronomers, or economists I
don't know anyone else that would consider nearly fifty thousand
So it's okay to screw over nearly fifty thousand customer domains because
there are 140M .com's?
luckily, none of the rest of us make mistakes
I don't think he was saying that at all. Just stating that from a pure numbers
standpoint 50k/140mil is a small percentage.
OTOH, I agree to your point - Network Solutions definitely downplayed this in
their release. Curiously so.
Sent from my iPhone
On Jun 20, 2013, at 5:42 PM, RijilV
On Thu, Jun 20, 2013 at 2:49 PM, Randy Bush ra...@psg.com wrote:
So it's okay to screw over nearly fifty thousand customer domains
because
there are 140M .com's?
luckily, none of the rest of us make mistakes
Ages ago I responded on a Cisco list where the topic was biggest screwup
you've
On Thu, 2013-06-20 at 14:42 -0700, RijilV wrote:
On 20 June 2013 14:28, valdis.kletni...@vt.edu wrote:
On Thu, 20 Jun 2013 14:08:18 -0700, Jeff Shultz said:
small number of Network Solutions customers
They must be staffed with physicists, astronomers, or economists I
don't
netsol screwed up. they screwed up bigtime. they are shoveling kitty
litter over it as fast as they can, and they have a professional kitty
litter, aka pr, department.
but none of this is surprising.
and dnssec did not save us. is there anything which could have?
randy
At the DNS Servers or service provider level, one can (and I often do) have
redundant providers.
At the registrar level? ...
Not with our current infrastructure, as far as I know how.
The Internet: Discovering new SPOF since 1969!
George William Herbert
Sent from my iPhone
On Jun 20,
at what point is the Internet a piece of infrastructure whereby we
actually need a way to watch this thing holistically as it is one system
and not just a bunch of inter-jointed systems? Who's job is it to do
nothing but ensure that the state of DNS and other services is running as
it
No.
The ztomy nameservers appeared in this morning's master .COM zonefile as
/authoritative/ for the number of domains I mentioned.
It is a clear change from just a couple of days ago, when the listed
nameservers were nowhere to be seen.
I have solid data to back this up, straight from Verisign
I, for one, would not be in favor of an authoritarian rule over DNS, or
any other Internet system, to ensure that the state of [the] service[s]
is running as it should. I suppose one could view such an authoritarian
rule over (sub) systems to be a good thing, as in there is someone to
complain to
On Jun 20, 2013 5:31 PM, Randy Bush ra...@psg.com wrote:
and dnssec did not save us. is there anything which could have?
Hmmm. DNSSEC wouldn't have prevented an outage. But from everything I've
seen reported, had the zones been signed, validating recursive resolvers
(comcast, google, much of
On 6/20/13, Randy Bush ra...@psg.com wrote:
netsol screwed up. they screwed up bigtime. they are shoveling kitty
litter over it as fast as they can, and they have a professional kitty
litter, aka pr, department.
but none of this is surprising.
and dnssec did not save us. is there anything
On Thu, Jun 20, 2013 at 8:41 PM, Timothy Morizot tmori...@gmail.com wrote:
On Jun 20, 2013 5:31 PM, Randy Bush ra...@psg.com wrote:
and dnssec did not save us. is there anything which could have?
Hmmm. DNSSEC wouldn't have prevented an outage. But from everything I've
seen reported, had
On Jun 20, 2013 7:30 PM, Rubens Kuhl rube...@gmail.com wrote:
In this case of registrar compromise, DS record could have been changed
alongside NS records, so DNSSEC would only have been a early warning,
because uncoordinated DS change disrupts service. As soon as previous
timeouts played out,
At 07:28 21/06/2013 +0900, Randy Bush wrote:
netsol screwed up. they screwed up bigtime. they are shoveling kitty
litter over it as fast as they can, and they have a professional kitty
litter, aka pr, department.
They are too busy adding new revenue:
At 17:12 20/06/2013 -0500, Richard Golodner wrote:
I think you are reading it the wrong way. Mr.Kletnieks never said it
was okay. He just stated that the numbers were trivial when compared to
the rest of potential customers being affected.
Be cool, Richard Golodner
sarcasm
and
at what point is the Internet a piece of infrastructure whereby we
actually need a way to watch this thing holistically as it is one system and
not just a bunch of inter-jointed systems? Who's job is it to do nothing but
ensure that the state of DNS and other services is running as it
I think ICANN would have to add a delay in where a request was sent out to
make sure everyone was on the same page and then what happens the couple
thousand (more) times a day that someone isn't updated or is
misconfigured?
I think Netsol should be fined. Maybe even a class action suite filed
On Thu, 20 Jun 2013 20:25:24 -0700, Hal Murray said:
How would you check/verify that the communication path from the monitoring
agency to the right people in your NOC was working correctly?
Remember to consider the possible impact of a false-positive report over
an unauthenticated channel.
Reaching out to DNS operators around the globe. Linkedin.com has had some
issues with DNS and would like DNS operators to flush their DNS. If you see
www.linkedin.com resolving NS to ns1617.ztomy.com or ns2617.ztomy.com then
please flush your DNS.
Any other info please reach out to me
Reaching out to DNS operators around the globe. Linkedin.com has had some
issues with DNS
and would like DNS operators to flush their DNS. If you see www.linkedin.com
resolving NS to
ns1617.ztomy.com or ns2617.ztomy.com then please flush your DNS.
Any other info please reach out to me off-list.
Yelp is evidently also affected
On Wed, Jun 19, 2013 at 10:19 PM, John Levine jo...@iecc.com wrote:
Reaching out to DNS operators around the globe. Linkedin.com has had some
issues with DNS
and would like DNS operators to flush their DNS. If you see
www.linkedin.com resolving NS to
On Jun 20, 2013, at 01:30 , Grant Ridder shortdudey...@gmail.com wrote:
Yelp is evidently also affected
Not from here.
If the NS or www points to 204.11.56.0/24 for a production domain/hostname,
that's bad. Yelp seems to be resolving normally for me.
--
TTFN,
patrick
On Wed, Jun 19, 2013
Sure enough:
; DiG 9.7.3 @localhost yelp.com A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; -HEADER- opcode: QUERY, status: NOERROR, id: 53267
;; flags: qr rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 0
;; QUESTION SECTION:
;yelp.com. IN A
;; ANSWER
On Wed, Jun 19, 2013 at 10:32 PM, Patrick W. Gilmore patr...@ianai.netwrote:
On Jun 20, 2013, at 01:30 , Grant Ridder shortdudey...@gmail.com wrote:
Yelp is evidently also affected
Not from here.
Patrick:
$ dig NS yelp.com @8.8.8.8 +short
ns1620.ztomy.com.
ns2620.ztomy.com.
Some DNS
On Wed, Jun 19, 2013 at 10:44 PM, Tom Paseka t...@cloudflare.com wrote:
On Wed, Jun 19, 2013 at 10:32 PM, Patrick W. Gilmore patr...@ianai.netwrote:
On Jun 20, 2013, at 01:30 , Grant Ridder shortdudey...@gmail.com wrote:
Yelp is evidently also affected
Not from here.
Patrick:
$ dig
Anyone have news/explanation about what's happening/happened?
On Wed, Jun 19, 2013 at 10:34 PM, Paul Ferguson fergdawgs...@gmail.comwrote:
Sure enough:
; DiG 9.7.3 @localhost yelp.com A
; (1 server found)
;; global options: +cmd
;; Got answer:
;; -HEADER- opcode: QUERY, status:
78 matches
Mail list logo
