Charles,
I am completely missing the statement:
'We have been ignorant by having just two nameservers and placing them in
the same location/datacenter, therefore any domain that was registered with
dns1.tucows.com and dns2.tucows.com was offline and unresolvable. We will
bring up an additional secondary at a complete different physical and
network location shortly"
Just my $0.02 ...
Taco Scargo
----- Original Message -----
From: "Charles Daminato" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>
Sent: Tuesday, May 29, 2001 19:36
Subject: OpenSRS Update - 05/29/01
> Greetings,
>
> As you have likely noticed, Tucows has experienced extended periods of
> intermittent outages affecting a broad range of our systems over the
> last 24 hours. Please be assured that we now believe that we have
> rectified the problem causing the degraded service.
>
> It is important to note that the following systems were not affected as
> a result of these outages:
>
> - All international systems including the Tucows mirror network.
> - All registry systems operated by LibertyRMS including .info
> etc.
>
> The following systems were affected as a result of these outages:
>
> - OpenSRS
> - Domain Direct
> - www.tucows.com main site
> - tucows.com and opensrs.org email
>
> For your benefit, the following is a rundown of the specific issues that
> we encountered, including the time period during which service was
> affected and the final resolution of the problem.
>
> - Yesterday afternoon, all services located in our primary
> datacenter were subject to intermittent network connectivity
> problems. A Cisco 6500 series switch appeared to be at the
> center of this issue and was quickly brought back within
> normal operating parameters.
>
> - Beginning shortly before 2300 EDT May 28 2001 (0300 UTC May
> 29 2001), we began to notice significantly degraded
> performance from all systems located within our primary
> datacenter, followed quickly by complete failure. Further
> investigation indicated that the core switching gear had
> failed. While system redundancy is in place, the erratic
> behaviour was sufficient enough to cause a core failure.
> Working in conjunction with Cisco, our network operations
> team determined that complete replacement of the gear in
> question was the most expedient solution. Replacement
> hardware was shipped out from Nashville at roughly 0300 EDT
> May 29 (0700 UTC) and arrived promptly at 0800 EDT (1200
> UTC). The equipment was brought online at approximately 1230
> EDT (1530 UTC) and the system was fully operational at 1300
> EDT (1700 UTC)
>
> Tucows has not yet determined the cause of the problem, but
> our staff has narrowed it down to either repeated failures of
> our Cisco gear, or a denial of service attack.
>
> It is important to note that while we are completely satisfied that this
> is an isolated issue, we are continuing our investigation until we can
> ensure that the steps taken to solve the problem are lasting in nature.
>
> We sincerely apologize for the inconvenience that this matter may have
> caused you. Please be assured that we are working with all relevant
> parties to ensure that we can continue to provide you with the highest
> possible level of service going forward.
>
> Sincerely,
>
> Charles Daminato
> TUCOWS Product Manager
> OpenSRS - Special Operations
>
