Send Outages-discussion mailing list submissions to
outages-discussion@outages.org
To subscribe or unsubscribe via the World Wide Web, visit
https://puck.nether.net/mailman/listinfo/outages-discussion
or, via email, send a message with subject or body 'help' to
outages-discussion-requ...@outages.org
You can reach the person managing the list at
outages-discussion-ow...@outages.org
When replying, please edit your Subject line so it is more specific
than "Re: Contents of Outages-discussion digest..."
Today's Topics:
1. Re: [EXTERNAL] Re: [outages] Major Verizon FiOS Outage
VA/MD/NJ/PA 11/12/24 12:28am EST - Ongoing (Ross Tajvar)
2. Re: [outages] Major Verizon FiOS Outage VA/MD/NJ/PA 11/12/24
12:28am EST - Ongoing (Charles Sprickman)
3. Re: [EXTERNAL] Re: [outages] Major Verizon FiOS Outage
VA/MD/NJ/PA 11/12/24 12:28am EST - Ongoing (Peter Beckman)
----------------------------------------------------------------------
Message: 1
Date: Tue, 12 Nov 2024 01:59:08 -0500
From: Ross Tajvar <r...@tajvar.io>
To: "Chapman, Brad (NBCUniversal)" <brad.chap...@nbcuni.com>
Cc: Kenneth Padgett <kenn...@itlifesaver.com>, Cary Wiedemann
<carywiedem...@gmail.com>, Outages Discussion
<outages-discussion@outages.org>
Subject: Re: [Outages-discussion] [EXTERNAL] Re: [outages] Major
Verizon FiOS Outage VA/MD/NJ/PA 11/12/24 12:28am EST - Ongoing
Message-ID:
<CA+FDdDRLfVsWV=4-npip8eyhofgxnnxjhokutb_qn7sjkaj...@mail.gmail.com>
Content-Type: text/plain; charset="utf-8"
I'm on fios and was/am affected by this outage. Traceroutes were stopping
(I switched to my VZW hotspot so I'm not in a position to check again) at
my local firewall so, while I didn't directly try to ping the upstream BNG,
I'm pretty sure it was unreachable. My ONT remained connected though. So
not sure what's going on here but I don't think it's as simple as a
backhaul failure.
On Tue, Nov 12, 2024 at 1:55?AM Chapman, Brad (NBCUniversal) via
Outages-discussion <outages-discussion@outages.org> wrote:
> (this email was sent to outages-discussion)
>
> Downdetector is showing a wide range of services and applications with
> outage spikes, and the maps corresponding to the user reports are *mostly*
> concentrated on the eastern Seaboard.
>
> I'm gonna put $20 on "backbone provider failure," specifically Verizon /
> GTE.
>
> Did someone trip over some fiber optic cables? Has there been a loss of
> power at a major communications hub?
>
> -Brad
>
> ?Sent from my iPhone
>
> On Nov 11, 2024, at 10:16?PM, Kenneth Padgett via Outages <
> outa...@outages.org> wrote:
>
> ?
> Confirmed. We?re seeing sites all over these areas offline.
>
> KP
>
> On Tue, Nov 12, 2024 at 01:07 Cary Wiedemann via Outages <
> outa...@outages.org> wrote:
>
>> Major multi-state Verizon FiOS outage began at 12:28am EST and is still
>> ongoing.
>>
>> Virginia, DC, Maryland, New Jersey, all affected.
>> Eastern Pennsylvania (Philadelphia region) is affected, Western
>> Pennsylvania (Pittsburgh) not impacted.
>> New York City is not impacted.
>>
>> Last few Verizon FiOS events were far more regional, this one is
>> widespread.
>>
>> Verizon Wireless seems unaffected so far.
>>
>> Traceroutes die before hitting the ALTER.NET
>> <https://urldefense.com/v3/__http://ALTER.NET__;!!PIZeeW5wscynRQ!tV7PM7WRGJI4oUUHxju5EdmiEHIH79yI9mtgjN2da_HnUHribTEdCzJGLNRi78Qow4LzHvvorX0m48NcqXg$>
>> core.
>>
>> Hopefully unannounced maintenance given the timing.
>>
>> - Cary
>>
>>
>> _______________________________________________
>> Outages mailing list
>> outa...@outages.org
>> https://puck.nether.net/mailman/listinfo/outages
>> <https://urldefense.com/v3/__https://puck.nether.net/mailman/listinfo/outages__;!!PIZeeW5wscynRQ!tV7PM7WRGJI4oUUHxju5EdmiEHIH79yI9mtgjN2da_HnUHribTEdCzJGLNRi78Qow4LzHvvorX0mYA-pDBA$>
>>
> _______________________________________________
> Outages mailing list
> outa...@outages.org
>
> https://urldefense.com/v3/__https://puck.nether.net/mailman/listinfo/outages__;!!PIZeeW5wscynRQ!tV7PM7WRGJI4oUUHxju5EdmiEHIH79yI9mtgjN2da_HnUHribTEdCzJGLNRi78Qow4LzHvvorX0mYA-pDBA$
>
> _______________________________________________
> Outages-discussion mailing list
> Outages-discussion@outages.org
> https://puck.nether.net/mailman/listinfo/outages-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://puck.nether.net/pipermail/outages-discussion/attachments/20241112/cef8b21c/attachment-0001.htm>
------------------------------
Message: 2
Date: Tue, 12 Nov 2024 12:28:05 -0500
From: Charles Sprickman <sp...@bway.net>
To: Cary Wiedemann <carywiedem...@gmail.com>
Cc: Outages Discussion <outages-discussion@outages.org>
Subject: Re: [Outages-discussion] [outages] Major Verizon FiOS Outage
VA/MD/NJ/PA 11/12/24 12:28am EST - Ongoing
Message-ID: <4ede41e3-dcf6-4b00-89b6-58c23eb61...@bway.net>
Content-Type: text/plain; charset="utf-8"
FWIW, *not* seeing any issues at all here in northern NJ
Also you all know that VZ makes absolutely no announcement of maintenance
windows on their fios stuff. A few times my "oh, all of NJ has been down for a
half hour" would be later referred to as "maintenance" if I actually followed
through with support.
>From last night when people were posting about this, trace to Quad 9, which
>for me gets served out of Vienna somewhere:
spork@SporkBook-BigBoy ~ % traceroute6 2620:fe::fe
traceroute6 to 2620:fe::fe (2620:fe::fe) from
2600:4040:a9d0:beaa:80de:e52a:2d18:3b2a, 64 hops max, 28 byte packets
1 xxx 22.542 ms 2.410 ms 2.401 ms
2 2600:4040:a9d0::1 3.078 ms 3.388 ms 3.894 ms
3 2600:4000:1:224::6c 5.585 ms 8.693 ms
2600:4000:1:224::6a 8.629 ms
4 * * *
5 *
nyk-b13-link.ip.twelve99.net 9.289 ms 4.761 ms
6 nyk-bb2-v6.ip.twelve99.net 8.732 ms 6.801 ms *
7 rest-bb1-v6.ip.twelve99.net 15.707 ms 12.167 ms 11.748 ms
8 0.et-0-0-1.bsr2.dc13.packet.net 12.485 ms 13.668 ms 11.499 ms
9 * * *
10 * * *
(this is normal, they block)
spork@SporkBook-BigBoy Downloads % traceroute 9.9.9.9
traceroute to 9.9.9.9 (9.9.9.9), 64 hops max, 40 byte packets
1 lo0-100.nwrknj-vfttp-312.verizon-gni.net (108.53.194.1) 15.663 ms 6.862
ms 10.507 ms
2 100.41.27.128 (100.41.27.128) 9.141 ms
100.41.27.126 (100.41.27.126) 5.889 ms
100.41.27.128 (100.41.27.128) 6.804 ms
3 * * *
4 * * nyk-b13-link.ip.twelve99.net (80.239.192.36) 13.776 ms
5 * * nyk-bb2-link.ip.twelve99.net (62.115.135.162) 6.767 ms
6 ash-bb2-link.ip.twelve99.net (62.115.136.201) 13.462 ms 12.438 ms
rest-bb1-link.ip.twelve99.net (62.115.139.35) 19.026 ms
7 rest-b2-link.ip.twelve99.net (62.115.123.41) 14.233 ms 13.928 ms
rest-b2-link.ip.twelve99.net (62.115.121.216) 11.905 ms
8 equinix-ic-361185.ip.twelve99-cust.net (62.115.155.127) 14.370 ms 12.081
ms 10.673 ms
9 0.et-0-0-19.bsr1.dc13.packet.net (147.28.130.61) 16.492 ms
0.et-0-0-1.bsr2.dc13.packet.net (198.16.4.99) 72.518 ms
0.et-0-0-19.bsr1.dc13.packet.net (147.28.130.61) 13.230 ms
10 * * *
11 * * *
12 paz1.qiad4 (147.75.61.242) 15.997 ms 13.685 ms 11.622 ms
13 * * *
14 * * *
(also normal)
I'm curious about those who saw an outage in NJ, what area were you? I'm in
Morris county.
Charles
> On Nov 12, 2024, at 1:07?AM, Cary Wiedemann via Outages <outa...@outages.org>
> wrote:
>
> Major multi-state Verizon FiOS outage began at 12:28am EST and is still
> ongoing.
>
> Virginia, DC, Maryland, New Jersey, all affected.
> Eastern Pennsylvania (Philadelphia region) is affected, Western Pennsylvania
> (Pittsburgh) not impacted.
> New York City is not impacted.
>
> Last few Verizon FiOS events were far more regional, this one is widespread.
>
> Verizon Wireless seems unaffected so far.
>
> Traceroutes die before hitting the ALTER.NET <http://alter.net/> core.
>
> Hopefully unannounced maintenance given the timing.
>
> - Cary
>
>
> _______________________________________________
> Outages mailing list
> outa...@outages.org
> https://puck.nether.net/mailman/listinfo/outages
-------------- next part --------------
An HTML attachment was scrubbed...
URL:
<https://puck.nether.net/pipermail/outages-discussion/attachments/20241112/87bd0b15/attachment-0001.htm>
------------------------------
Message: 3
Date: Wed, 13 Nov 2024 17:21:24 -0500
From: Peter Beckman <beck...@angryox.com>
To: Ross Tajvar <r...@tajvar.io>
Cc: "Chapman, Brad (NBCUniversal)" <brad.chap...@nbcuni.com>, Kenneth
Padgett <kenn...@itlifesaver.com>, Cary Wiedemann
<carywiedem...@gmail.com>, Outages Discussion
<outages-discussion@outages.org>
Subject: Re: [Outages-discussion] [EXTERNAL] Re: [outages] Major
Verizon FiOS Outage VA/MD/NJ/PA 11/12/24 12:28am EST - Ongoing
Message-ID: <dc9bb4aa-a038-c949-afc2-f60b8e931...@angryox.com>
Content-Type: text/plain; charset="utf-8"; Format="flowed"
Has anyone heard the RFO/COE?
On Tue, 12 Nov 2024, Ross Tajvar via Outages-discussion wrote:
> I'm on fios and was/am affected by this outage. Traceroutes were stopping
> (I switched to my VZW hotspot so I'm not in a position to check again) at
> my local firewall so, while I didn't directly try to ping the upstream BNG,
> I'm pretty sure it was unreachable. My ONT remained connected though. So
> not sure what's going on here but I don't think it's as simple as a
> backhaul failure.
>
> On Tue, Nov 12, 2024 at 1:55?AM Chapman, Brad (NBCUniversal) via
> Outages-discussion <outages-discussion@outages.org> wrote:
>
>> (this email was sent to outages-discussion)
>>
>> Downdetector is showing a wide range of services and applications with
>> outage spikes, and the maps corresponding to the user reports are *mostly*
>> concentrated on the eastern Seaboard.
>>
>> I'm gonna put $20 on "backbone provider failure," specifically Verizon /
>> GTE.
>>
>> Did someone trip over some fiber optic cables? Has there been a loss of
>> power at a major communications hub?
>>
>> -Brad
>>
>> ?Sent from my iPhone
>>
>> On Nov 11, 2024, at 10:16?PM, Kenneth Padgett via Outages <
>> outa...@outages.org> wrote:
>>
>> ?
>> Confirmed. We?re seeing sites all over these areas offline.
>>
>> KP
>>
>> On Tue, Nov 12, 2024 at 01:07 Cary Wiedemann via Outages <
>> outa...@outages.org> wrote:
>>
>>> Major multi-state Verizon FiOS outage began at 12:28am EST and is still
>>> ongoing.
>>>
>>> Virginia, DC, Maryland, New Jersey, all affected.
>>> Eastern Pennsylvania (Philadelphia region) is affected, Western
>>> Pennsylvania (Pittsburgh) not impacted.
>>> New York City is not impacted.
>>>
>>> Last few Verizon FiOS events were far more regional, this one is
>>> widespread.
>>>
>>> Verizon Wireless seems unaffected so far.
>>>
>>> Traceroutes die before hitting the ALTER.NET
>>> <https://urldefense.com/v3/__http://ALTER.NET__;!!PIZeeW5wscynRQ!tV7PM7WRGJI4oUUHxju5EdmiEHIH79yI9mtgjN2da_HnUHribTEdCzJGLNRi78Qow4LzHvvorX0m48NcqXg$>
>>> core.
>>>
>>> Hopefully unannounced maintenance given the timing.
>>>
>>> - Cary
>>>
>>>
>>> _______________________________________________
>>> Outages mailing list
>>> outa...@outages.org
>>> https://puck.nether.net/mailman/listinfo/outages
>>> <https://urldefense.com/v3/__https://puck.nether.net/mailman/listinfo/outages__;!!PIZeeW5wscynRQ!tV7PM7WRGJI4oUUHxju5EdmiEHIH79yI9mtgjN2da_HnUHribTEdCzJGLNRi78Qow4LzHvvorX0mYA-pDBA$>
>>>
>> _______________________________________________
>> Outages mailing list
>> outa...@outages.org
>>
>> https://urldefense.com/v3/__https://puck.nether.net/mailman/listinfo/outages__;!!PIZeeW5wscynRQ!tV7PM7WRGJI4oUUHxju5EdmiEHIH79yI9mtgjN2da_HnUHribTEdCzJGLNRi78Qow4LzHvvorX0mYA-pDBA$
>>
>> _______________________________________________
>> Outages-discussion mailing list
>> Outages-discussion@outages.org
>> https://puck.nether.net/mailman/listinfo/outages-discussion
>>
>
---------------------------------------------------------------------------
Peter Beckman Internet Guy
beck...@angryox.com https://www.angryox.com/
---------------------------------------------------------------------------
-------------- next part --------------
_______________________________________________
Outages-discussion mailing list
Outages-discussion@outages.org
https://puck.nether.net/mailman/listinfo/outages-discussion
------------------------------
Subject: Digest Footer
_______________________________________________
Outages-discussion mailing list
Outages-discussion@outages.org
https://puck.nether.net/mailman/listinfo/outages-discussion
------------------------------
End of Outages-discussion Digest, Vol 172, Issue 2
**************************************************
