Re: verizon fios, northeast, routing issues?
FYI: Funny thing, this morning, the packet loss rate has dropped to between 0 and 10% (from closer to 40%). Route seems to the same. One wonders if some piece of equipment has been updated. Miles Fidelman Miles Fidelman wrote: Ok folks, Thanks for the info about uunet. But that doesn't address: 3. The intermittent, high delays (factor of 10) jump out (also, when running ping tests, there seem to be intermittent periods of long sequences of timeouts) or, that, for about 4 years now, gamers seem to be reporting really poor performance across FIOS in the Northeast - tied to rather high packet loss rates. Note that those packet losses seems to be bursts of 8-10 lost packets every 10 packets or so, and the really high delays on some of the traceroutes seem to indicate that it's happening somewhere in the middle of the path, not at my end. And, come to think of it, that might explain some of the horrid performance of the FIOS channel guide. Any thoughts? Anyone here from Verizon Northeast FIOS operations who might have a comment? Thanks, Miles Fidelman Miles Fidelman wrote: Any Verizon folks here? I've been having some rather weird network issues lately - just reading email via IMAP, from home. Over a 1gig FIOS connection to a machine in a nearby Tierpoint data center that has LOTS of good connectivity. I just tried some traceroutes, and got some interesting results: These originate on a machine connected to a 1gig FIOS feed, and end at a machine, located in a Tierpoint datacenter, about 10 miles from here. traceroute to ntcorp.com (207.154.13.58), 64 hops max, 52 byte packets 1 * fios_quantum_gateway (192.168.1.1) 3.530 ms 2.822 ms 2 * * * 3 100.41.27.110 (100.41.27.110) 14.970 ms 5.323 ms 6.306 ms 4 0.csi1.bstnmafr-mse01-bb-su1.alter.net (140.222.10.32) 11.069 ms 8.477 ms 17.097 ms 5 * * * 6 0.ae1.br1.bos30.alter.net (140.222.236.253) 17.121 ms 19.027 ms 0.ae2.br1.bos30.alter.net (140.222.236.255) 19.795 ms 7 * * * 8 colo4-dalla.bear1.boston1.level3.net (4.53.61.86) 2205.648 ms 8.331 ms 13.161 ms 9 static-33-65-203-66.axsne.net (66.203.65.33) 16.951 ms 13.791 ms static-145-65-203-66.axsne.net (66.203.65.145) 21.503 ms 10 server1.ntcorp.com (207.154.13.58) 17.872 ms 15.902 ms 14.415 ms Several things jump out: 1. alter.net is not a common path between here & there - usually a lower grade connection, when other backbones aren't working right 2. origin - alter.net - level.3 - endpoint is just bizarre, one would think that the regional FIOS network has a direct connection to level.3 (it also seems kind of odd that the packets are flowing from Acton MA, to Boston, and back out to Marlboro MA - there's an awful lot of fiber running along Rt. 495, and the networks are fairly dense around here) 3. The intermittent, high delays (factor of 10) jump out (also, when running ping tests, there seem to be intermittent periods of long sequences of timeouts) All in all it's really mucking with both streaming services, and simply posting emails (SMTP timeouts). All of which leads me to wonder if there's something mucked up with Verizon's routing tables (or a particular network interface). Any insights (or fixes) to be had? Thanks, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. Yogi Berra Theory is when you know everything but nothing works. Practice is when everything works but no one knows why. In our lab, theory and practice are combined: nothing works and no one knows why. ... unknown
Re: verizon fios, northeast, routing issues?
Ok folks, Thanks for the info about uunet. But that doesn't address: 3. The intermittent, high delays (factor of 10) jump out (also, when running ping tests, there seem to be intermittent periods of long sequences of timeouts) or, that, for about 4 years now, gamers seem to be reporting really poor performance across FIOS in the Northeast - tied to rather high packet loss rates. Note that those packet losses seems to be bursts of 8-10 lost packets every 10 packets or so, and the really high delays on some of the traceroutes seem to indicate that it's happening somewhere in the middle of the path, not at my end. And, come to think of it, that might explain some of the horrid performance of the FIOS channel guide. Any thoughts? Anyone here from Verizon Northeast FIOS operations who might have a comment? Thanks, Miles Fidelman Miles Fidelman wrote: Any Verizon folks here? I've been having some rather weird network issues lately - just reading email via IMAP, from home. Over a 1gig FIOS connection to a machine in a nearby Tierpoint data center that has LOTS of good connectivity. I just tried some traceroutes, and got some interesting results: These originate on a machine connected to a 1gig FIOS feed, and end at a machine, located in a Tierpoint datacenter, about 10 miles from here. traceroute to ntcorp.com (207.154.13.58), 64 hops max, 52 byte packets 1 * fios_quantum_gateway (192.168.1.1) 3.530 ms 2.822 ms 2 * * * 3 100.41.27.110 (100.41.27.110) 14.970 ms 5.323 ms 6.306 ms 4 0.csi1.bstnmafr-mse01-bb-su1.alter.net (140.222.10.32) 11.069 ms 8.477 ms 17.097 ms 5 * * * 6 0.ae1.br1.bos30.alter.net (140.222.236.253) 17.121 ms 19.027 ms 0.ae2.br1.bos30.alter.net (140.222.236.255) 19.795 ms 7 * * * 8 colo4-dalla.bear1.boston1.level3.net (4.53.61.86) 2205.648 ms 8.331 ms 13.161 ms 9 static-33-65-203-66.axsne.net (66.203.65.33) 16.951 ms 13.791 ms static-145-65-203-66.axsne.net (66.203.65.145) 21.503 ms 10 server1.ntcorp.com (207.154.13.58) 17.872 ms 15.902 ms 14.415 ms Several things jump out: 1. alter.net is not a common path between here & there - usually a lower grade connection, when other backbones aren't working right 2. origin - alter.net - level.3 - endpoint is just bizarre, one would think that the regional FIOS network has a direct connection to level.3 (it also seems kind of odd that the packets are flowing from Acton MA, to Boston, and back out to Marlboro MA - there's an awful lot of fiber running along Rt. 495, and the networks are fairly dense around here) 3. The intermittent, high delays (factor of 10) jump out (also, when running ping tests, there seem to be intermittent periods of long sequences of timeouts) All in all it's really mucking with both streaming services, and simply posting emails (SMTP timeouts). All of which leads me to wonder if there's something mucked up with Verizon's routing tables (or a particular network interface). Any insights (or fixes) to be had? Thanks, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. Yogi Berra Theory is when you know everything but nothing works. Practice is when everything works but no one knows why. In our lab, theory and practice are combined: nothing works and no one knows why. ... unknown
Re: verizon fios, northeast, routing issues?
alter.net is just the legacy RDNS for things in AS701 (uunet). Nothing weird there. https://en.wikipedia.org/wiki/UUNET On Sat, Oct 9, 2021 at 1:46 PM Miles Fidelman wrote: > Any Verizon folks here? > > I've been having some rather weird network issues lately - just reading > email via IMAP, from home. Over a 1gig FIOS connection to a machine in > a nearby Tierpoint data center that has LOTS of good connectivity. > > I just tried some traceroutes, and got some interesting results: > > These originate on a machine connected to a 1gig FIOS feed, and end at a > machine, located in a Tierpoint datacenter, about 10 miles from here. > > traceroute to ntcorp.com (207.154.13.58), 64 hops max, 52 byte packets > 1 * fios_quantum_gateway (192.168.1.1) 3.530 ms 2.822 ms > 2 * * * > 3 100.41.27.110 (100.41.27.110) 14.970 ms 5.323 ms 6.306 ms > 4 0.csi1.bstnmafr-mse01-bb-su1.alter.net (140.222.10.32) 11.069 ms > 8.477 ms 17.097 ms > 5 * * * > 6 0.ae1.br1.bos30.alter.net (140.222.236.253) 17.121 ms 19.027 ms > 0.ae2.br1.bos30.alter.net (140.222.236.255) 19.795 ms > 7 * * * > 8 colo4-dalla.bear1.boston1.level3.net (4.53.61.86) 2205.648 ms > 8.331 ms 13.161 ms > 9 static-33-65-203-66.axsne.net (66.203.65.33) 16.951 ms 13.791 ms > static-145-65-203-66.axsne.net (66.203.65.145) 21.503 ms > 10 server1.ntcorp.com (207.154.13.58) 17.872 ms 15.902 ms 14.415 ms > > Several things jump out: > > 1. alter.net is not a common path between here & there - usually a lower > grade connection, when other backbones aren't working right > > 2. origin - alter.net - level.3 - endpoint is just bizarre, one would > think that the regional FIOS network has a direct connection to level.3 > (it also seems kind of odd that the packets are flowing from Acton MA, > to Boston, and back out to Marlboro MA - there's an awful lot of fiber > running along Rt. 495, and the networks are fairly dense around here) > > 3. The intermittent, high delays (factor of 10) jump out (also, when > running ping tests, there seem to be intermittent periods of long > sequences of timeouts) > > All in all it's really mucking with both streaming services, and simply > posting emails (SMTP timeouts). > > All of which leads me to wonder if there's something mucked up with > Verizon's routing tables (or a particular network interface). > > Any insights (or fixes) to be had? > > Thanks, > > Miles Fidelman > > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. Yogi Berra > > Theory is when you know everything but nothing works. > Practice is when everything works but no one knows why. > In our lab, theory and practice are combined: > nothing works and no one knows why. ... unknown > >
Re: verizon fios, northeast, routing issues?
> On Sat, Oct 9, 2021, 13:45 Miles Fidelman > > > 2. origin - alter.net - level.3 - endpoint is just bizarre, one would > think that the regional FIOS network has a direct connection to level.3 No. Former verizon-gni backbone (where FiOS sits) takes transit solely from VZB (now UUNET), this has been the case for many many years, but most people get confused when interpretting traceroutes due to mpls no-ttl-propagate on VZ networks. > (it also seems kind of odd that the packets are flowing from Acton MA, > to Boston, and back out to Marlboro MA - there's an awful lot of fiber > running along Rt. 495, and the networks are fairly dense around here) Level3-VZB-px is at 300 Bent Street, Cambridge, MA, where both parties have peering routers installed (bear1.Boston1.Level3.net and BR1.BOS30.ALTER.NET). It makes perfect sense for interconnection to occur in Cambridge/Boston, rather than maintaining peering routers out in the suburb within the same metro. None of these are contributory factors to the issues you're describing. James
Re: verizon fios, northeast, routing issues?
On Sat, Oct 9, 2021, 13:45 Miles Fidelman wrote: > Any Verizon folks here? > > > > I've been having some rather weird network issues lately - just reading > email via IMAP, from home. Over a 1gig FIOS connection to a machine in > a nearby Tierpoint data center that has LOTS of good connectivity. > > I just tried some traceroutes, and got some interesting results: > > These originate on a machine connected to a 1gig FIOS feed, and end at a > machine, located in a Tierpoint datacenter, about 10 miles from here. > > traceroute to ntcorp.com (207.154.13.58), 64 hops max, 52 byte packets > 1 * fios_quantum_gateway (192.168.1.1) 3.530 ms 2.822 ms > 2 * * * > 3 100.41.27.110 (100.41.27.110) 14.970 ms 5.323 ms 6.306 ms > 4 0.csi1.bstnmafr-mse01-bb-su1.alter.net (140.222.10.32) 11.069 ms > 8.477 ms 17.097 ms > 5 * * * > 6 0.ae1.br1.bos30.alter.net (140.222.236.253) 17.121 ms 19.027 ms > 0.ae2.br1.bos30.alter.net (140.222.236.255) 19.795 ms > 7 * * * > 8 colo4-dalla.bear1.boston1.level3.net (4.53.61.86) 2205.648 ms > 8.331 ms 13.161 ms > 9 static-33-65-203-66.axsne.net (66.203.65.33) 16.951 ms 13.791 ms > static-145-65-203-66.axsne.net (66.203.65.145) 21.503 ms > 10 server1.ntcorp.com (207.154.13.58) 17.872 ms 15.902 ms 14.415 ms > > Several things jump out: > > 1. alter.net is not a common path between here & there - usually a lower > grade connection, when other backbones aren't working right > > > Alternet is the domain used by legacy uunet equipment/ips, or was that > domain "forever". > > > > 2. origin - alter.net - level.3 - endpoint is just bizarre, one would > > Why? "Br" is the role name used for sfp peer interconnect devices on > uunet's network. > > think that the regional FIOS network has a direct connection to level.3 > (it also seems kind of odd that the packets are flowing from Acton MA, > to Boston, and back out to Marlboro MA - there's an awful lot of fiber > running along Rt. 495, and the networks are fairly dense around here) > > 3. The intermittent, high delays (factor of 10) jump out (also, when > running ping tests, there seem to be intermittent periods of long > sequences of timeouts) > > All in all it's really mucking with both streaming services, and simply > posting emails (SMTP timeouts). > > All of which leads me to wonder if there's something mucked up with > Verizon's routing tables (or a particular network interface). > > Any insights (or fixes) to be had? > > Thanks, > > Miles Fidelman > > > > -- > In theory, there is no difference between theory and practice. > In practice, there is. Yogi Berra > > Theory is when you know everything but nothing works. > Practice is when everything works but no one knows why. > In our lab, theory and practice are combined: > nothing works and no one knows why. ... unknown > >
verizon fios, northeast, routing issues?
Any Verizon folks here? I've been having some rather weird network issues lately - just reading email via IMAP, from home. Over a 1gig FIOS connection to a machine in a nearby Tierpoint data center that has LOTS of good connectivity. I just tried some traceroutes, and got some interesting results: These originate on a machine connected to a 1gig FIOS feed, and end at a machine, located in a Tierpoint datacenter, about 10 miles from here. traceroute to ntcorp.com (207.154.13.58), 64 hops max, 52 byte packets 1 * fios_quantum_gateway (192.168.1.1) 3.530 ms 2.822 ms 2 * * * 3 100.41.27.110 (100.41.27.110) 14.970 ms 5.323 ms 6.306 ms 4 0.csi1.bstnmafr-mse01-bb-su1.alter.net (140.222.10.32) 11.069 ms 8.477 ms 17.097 ms 5 * * * 6 0.ae1.br1.bos30.alter.net (140.222.236.253) 17.121 ms 19.027 ms 0.ae2.br1.bos30.alter.net (140.222.236.255) 19.795 ms 7 * * * 8 colo4-dalla.bear1.boston1.level3.net (4.53.61.86) 2205.648 ms 8.331 ms 13.161 ms 9 static-33-65-203-66.axsne.net (66.203.65.33) 16.951 ms 13.791 ms static-145-65-203-66.axsne.net (66.203.65.145) 21.503 ms 10 server1.ntcorp.com (207.154.13.58) 17.872 ms 15.902 ms 14.415 ms Several things jump out: 1. alter.net is not a common path between here & there - usually a lower grade connection, when other backbones aren't working right 2. origin - alter.net - level.3 - endpoint is just bizarre, one would think that the regional FIOS network has a direct connection to level.3 (it also seems kind of odd that the packets are flowing from Acton MA, to Boston, and back out to Marlboro MA - there's an awful lot of fiber running along Rt. 495, and the networks are fairly dense around here) 3. The intermittent, high delays (factor of 10) jump out (also, when running ping tests, there seem to be intermittent periods of long sequences of timeouts) All in all it's really mucking with both streaming services, and simply posting emails (SMTP timeouts). All of which leads me to wonder if there's something mucked up with Verizon's routing tables (or a particular network interface). Any insights (or fixes) to be had? Thanks, Miles Fidelman -- In theory, there is no difference between theory and practice. In practice, there is. Yogi Berra Theory is when you know everything but nothing works. Practice is when everything works but no one knows why. In our lab, theory and practice are combined: nothing works and no one knows why. ... unknown
