>From status.mimecast.com: May 3, 2022 4:28PM UTC [Monitoring] We can confirm that email processing and all Mimecast services have been restored for our US-hosted customers. All services should now be functioning normally and processing queues are approaching normal levels as email is retried on the normal SMTP schedules. We continue to monitor and will provide updates here.
May 3, 2022 3:58PM UTC [Identified] We can confirm we experienced a major power outage in one of our US data centers that impacted all power supply sources, including the backup generators, which in turn caused a cascading issue and resulted in degraded performance. Midway through our Disaster Recovery Process, the power was restored which complicated our ability to recover. As of now, services are available and we are processing email but customers may still experience delays until the backlog clears. We sincerely apologize for the extended outage and the disruption caused. - Casimir Couvillion - Texon LP - Manager, IT Operations and Helpdesk - [email protected] -----Original Message----- From: Outages <[email protected]> On Behalf Of Mike Lewinski via Outages Sent: Tuesday, May 3, 2022 11:49 AM To: outages ([email protected]) <[email protected]> Subject: [outages] Mimecast SMTP outage I'm unable to reach the SMTP service on us-smtp-inbound-1.mimecast.com or us-smtp-inbound-2.mimecast.com from multiple markets: Dallas, Denver, Phoenix. I can reach them from Atlanta and Seattle. They are reachable on at least one address in Los Angeles, unreachable on others. Dallas: $ telnet us-smtp-inbound-1.mimecast.com 25 Trying 205.139.110.221... Trying 207.211.30.221... Trying 207.211.30.141... Trying 205.139.110.141... Trying 205.139.110.242... Trying 207.211.30.242... telnet: Unable to connect to remote host: Connection timed out Denver: $ telnet us-smtp-inbound-1.mimecast.com 25 Trying 207.211.30.221... Trying 207.211.30.242... Trying 207.211.30.141... Trying 205.139.110.242... Trying 205.139.110.221... Trying 205.139.110.141... telnet: Unable to connect to remote host: Connection timed out Finally found one host in LA that worked after two failures: $ telnet us-smtp-inbound-1.mimecast.com 25 Trying 205.139.110.221... Trying 205.139.110.141... Trying 207.211.30.242... Connected to us-smtp-inbound-1.mimecast.com. Escape character is '^]'. 220 us-smtp-1.mimecast.com ESMTP; Tue, 03 May 2022 12:38:11 -0400 QUIT 221 Service closing transmission channel [uajtgwEROGqfo99gP4hQyg.us397] Connection closed by foreign host. $ mtr -r -c3 -w 205.139.110.221 Start: 2022-05-03T10:43:06-0600 HOST: los-angeles Loss% Snt Last Avg Best Wrst StDev 1.|-- LA624-C6504-1-VL624.massivenetworks.net 0.0% 3 0.6 0.5 0.5 0.6 0.1 2.|-- 199.45.152.18 0.0% 3 0.4 0.4 0.4 0.4 0.0 3.|-- xe-10-0-0.edge1.LosAngeles9.Level3.net 0.0% 3 4.7 2.0 0.6 4.7 2.4 4.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0 5.|-- MIMECAST-SE.ear1.NewYork6.Level3.net 0.0% 3 68.8 68.8 68.7 68.8 0.1 6.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0 $ mtr -r -c3 -w 207.211.30.242 Start: 2022-05-03T10:47:11-0600 HOST: los-angeles Loss% Snt Last Avg Best Wrst StDev 1.|-- LA624-C6504-1-VL624.massivenetworks.net 0.0% 3 0.6 0.5 0.4 0.6 0.1 2.|-- 199.45.152.18 0.0% 3 0.4 0.4 0.4 0.5 0.1 3.|-- et-0-0-6-2.cr4-lax2.ip4.gtt.net 0.0% 3 0.4 0.5 0.4 0.6 0.1 4.|-- ae23.cr3-lax1.ip4.gtt.net 0.0% 3 1.3 1.3 1.1 1.3 0.1 5.|-- ae-3.r01.lsanca07.us.bb.gin.ntt.net 0.0% 3 1.3 1.2 1.2 1.3 0.1 6.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0 7.|-- ae-3.r24.snjsca04.us.bb.gin.ntt.net 33.3% 3 15.4 13.7 12.1 15.4 2.4 8.|-- ae-8.r23.chcgil09.us.bb.gin.ntt.net 0.0% 3 57.3 58.0 57.3 58.9 0.8 9.|-- ae-0.r22.chcgil09.us.bb.gin.ntt.net 0.0% 3 64.4 61.5 58.9 64.4 2.8 10.|-- ae-6.r21.nwrknj03.us.bb.gin.ntt.net 0.0% 3 69.7 71.2 69.4 74.6 2.9 11.|-- ae-11.r01.nycmny17.us.bb.gin.ntt.net 0.0% 3 70.2 70.0 69.8 70.2 0.2 12.|-- ce-0-8-0-3-200.r01.nycmny17.us.ce.gin.ntt.net 0.0% 3 74.9 87.4 74.9 112.2 21.5 13.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0 $ Phoenix: $ telnet us-smtp-inbound-1.mimecast.com 25 Trying 207.211.30.141... Trying 207.211.30.221... Trying 205.139.110.141... Trying 205.139.110.242... Trying 207.211.30.242... Trying 205.139.110.221... telnet: Unable to connect to remote host: Connection timed out $ Seattle: $ telnet us-smtp-inbound-1.mimecast.com 25 Trying 207.211.30.141... Connected to us-smtp-inbound-1.mimecast.com. Escape character is '^]'. 220 us-smtp-1.mimecast.com ESMTP; Tue, 03 May 2022 12:34:04 -0400 QUIT 221 Service closing transmission channel [0EM41sKhMWizUnOQs3cnYQ.us479] Connection closed by foreign host. $ mtr -r -c3 -w us-smtp-inbound-1.mimecast.com Start: 2022-05-03T10:37:23-0600 HOST: seattle Loss% Snt Last Avg Best Wrst StDev 1.|-- 204.144.179.129 0.0% 3 0.4 0.4 0.4 0.5 0.0 2.|-- 9-1-4.ear1.Seattle3.Level3.net 0.0% 3 0.5 0.6 0.5 0.7 0.1 3.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0 4.|-- MIMECAST-SE.ear1.NewYork6.Level3.net 0.0% 3 76.6 76.6 76.5 76.7 0.1 5.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0 $ Atlanta: $ telnet us-smtp-inbound-1.mimecast.com 25 Trying 207.211.30.242... Connected to us-smtp-inbound-1.mimecast.com. Escape character is '^]'. 220 us-smtp-1.mimecast.com ESMTP; Tue, 03 May 2022 12:08:11 -0400 QUIT 221 Service closing transmission channel [M0wFRdZ-NhONlAOBhS22Sg.us227] Connection closed by foreign host. $ mtr 207.211.30.242 -w -r -c 3 Start: Tue May 3 10:17:58 2022 HOST: atlanta Loss% Snt Last Avg Best Wrst StDev 1.|-- Atl55-C6504-1-VL412.massivenetworks.net 0.0% 3 0.3 0.4 0.3 0.7 0.0 2.|-- 6-2-2.ear3.Atlanta2.Level3.net 0.0% 3 0.3 0.3 0.3 0.3 0.0 3.|-- 4.69.217.206 0.0% 3 17.9 7.3 1.5 17.9 9.1 4.|-- NTT-level3-100G.Atlanta2.Level3.net 0.0% 3 1.1 1.0 0.8 1.1 0.0 5.|-- ae-4.r23.atlnga05.us.bb.gin.ntt.net 0.0% 3 1.3 2.6 0.5 6.0 2.9 6.|-- ae-4.r25.asbnva02.us.bb.gin.ntt.net 0.0% 3 13.0 13.6 13.0 14.2 0.0 7.|-- ae-4.r21.nwrknj03.us.bb.gin.ntt.net 33.3% 3 18.9 18.8 18.7 18.9 0.0 8.|-- ae-11.r01.nycmny17.us.bb.gin.ntt.net 0.0% 3 19.3 19.2 19.0 19.3 0.0 9.|-- ce-0-8-0-3-200.r01.nycmny17.us.ce.gin.ntt.net 0.0% 3 24.8 24.7 24.5 24.8 0.0 10.|-- ??? 100.0 3 0.0 0.0 0.0 0.0 0.0
_______________________________________________ Outages mailing list [email protected] https://puck.nether.net/mailman/listinfo/outages
