Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Mark Tinka]

On 16/Aug/15 00:50, Harry McGregor wrote:

>
>  
>
> Before this happens (ie when hell freezes over), I would like to see
> new home communities deploying fiber networks as part of the building
> of the "master plan" of the community.   That way the home owners
> association can go out for bid every year or few years for a service
> provider to operate the fiber network. 

This is exactly what communities in South Africa are starting to do.

In major suburbs around Johannesburg, neighborhoods are getting their
residents together and putting out bids for service providers to build
and operate FTTH networks on their behalf. In other parts of the
country, the local city or municipal councils are also getting involved.

In other neighborhoods, new and lean service providers are, by their own
volition, pulling fibre into various neighbors and deploying GPON FTTH
access nodes in homes regardless of whether you want a service or not.
If you want the service, ring them up and someone will come set you up.
You only pay for the setup fee, as the ONU remains theirs. This is what
happened in my neighborhood, which means I'm now sitting on a 25M/25M
FTTH service for about US$70/month. Big difference from when I had a
384k/3.2M ADSL service for about the same price.

The idea is that folk are taking matters into their own hands, as while
there is a national broadband strategy, its actual implementation is a
far cry.

Mark.

Re: Drops in Core

[alif.terranson (IMAP)]

Sean Donelan Opined Thusly,

"Generally I don't believe ISPs that claim 100% uptime or 0% packet loss."

I have seen a "perfect score" on a Keynote evaluation, while we had a rather 
nasty outage outside a preststed maintenance window.  When we objected that the 
outages was a planned maintenance that somehow escaped publication, it was 
ignored by Keynote - how generous!  I doubt anyone here would deny the 
difficulty of maintaining a five 9's record, going above and beyond to claim no 
packet loss just tells you who they really are...

--
Yours,
/s/

Alif

Sent from my 100% uptime, packet loss-free digital carrier pigeon.

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Harry McGregor]

On 08/15/2015 09:44 AM, Owen DeLong wrote:

The most viable solution, IMHO, is to require a separation between physical 
infrastructure providers and those that provide services over that 
infrastructure. Breaking the tight coupling between the two and requiring 
physical infrastructure providers to lease facilities to operators on an equal 
footing for all operators will reduce the barriers to competition in the 
operator space. It will also make limited competition in the facilities space 
possible, though unlikely.

This model exists to some extent in a few areas that have municipal residential 
fiber services, and in most of those localities, it is working well.

That’s one of the reasons that the incumbent facilities based carriers have 
lobbied so hard to get laws in states where a city has done this that prevent 
other cities from following suit.

Fortunately, one of the big gains in recent FCC rulings is that these laws are 
likely to be rendered null and void.

Unfortunately, there is so much vested interest in the status quo that 
achieving this sort of separation is unlikely without a really strong grass 
roots movement. Sadly, the average sound-bite oriented citizen doesn’t know (or 
want to learn) enough to facilitate such a grass-roots movement, so if we want 
to build such a future, we have a long slog of public education and recruitment 
ahead of us.

In the mean time, we’ll get to continue to watch companies like CC, VZ, TW 
screw over their customers and the content providers their customers want to 
reach for the sake of extorting extra money from both sides of the transaction.

Owen


I have talked about this idea for years, but most places seem to have a 
difficult time understanding the difference between layer 0, layer 2, 
and layer 3 networks.


IMHO the should be one residential fiber network (either passive or 
active, depending on the deployment and the physical layout of the 
area), and it should be run by an "essential" utility, such as the 
city/county water department, or if necessary the local electric company 
(I far prefer the water department).  The access would be near 
universal, and the layer 0 and layer 2 network fees would be part of the 
"water" bill.  Apartment complexes may have to be serviced with G.fast 
or other technologies to make the deployment faster and easier.


Getting IP bandwidth, technical support, voice service, and video 
service would be a competitive service provider model, with local ISPs, 
and large Cable COs and TelCOs competing on top of this physical 
network.  You could even have providers that specialize in low income 
"life line" services, such as 5Mbit of IP bandwidth and local voice 
service with e911.


Historically services that have huge sunk costs, and high build out 
costs have been a natural monopoly and regulated.  You would not think 
of trying to build out a competitive water or sewer network, and most 
building codes prevent the installation of a septic system if a sewer 
connection is at all possible.   Why we are not going this way for a 
high cost of build out network (last mile) is beyond me.


Before this happens (ie when hell freezes over), I would like to see new 
home communities deploying fiber networks as part of the building of the 
"master plan" of the community.   That way the home owners association 
can go out for bid every year or few years for a service provider to 
operate the fiber network.   Around here (souther AZ) new communities 
tend to either alliance with CenturyLink, Cox, or Comcast depending on 
the location, and they DO NOT bring in the other providers.  If a 
builder goes with Cox, you can NOT get a CenturyLink (ILEC) landline or 
DSL, if a builder goes with CenturyLink, Cox will not run anything into 
the community.


-Harry

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Mark Tinka]

On 15/Aug/15 22:45, jim deleskie wrote:

> There is more to it, then just being tired of it, it take, $$ and time
> and bodies to build a network, even in 1 country.  Its not something
> everyone can do.  I suspect the "game" and transit networks, will
> continue long after most of us are no long "playing"

I do not disagree.

Mark.

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[jim deleskie]

There is more to it, then just being tired of it, it take, $$ and time and
bodies to build a network, even in 1 country.  Its not something everyone
can do.  I suspect the "game" and transit networks, will continue long
after most of us are no long "playing"


On Sat, Aug 15, 2015 at 5:35 PM, Mark Tinka  wrote:

>
>
> On 15/Aug/15 19:32, jim deleskie wrote:
>
> > In my 20+ yrs now of playing this game, "everyone" has had a turn
> thinking
> > their content/eyeballs are special and should get free "peering".
>
> That's why those tired of playing the game build their own networks to
> take out the middleman, for better or worse.
>
> Mark.
>

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Mark Tinka]

On 15/Aug/15 22:01, Owen DeLong wrote:

>
> IMHO, there’s only one yes answer here… If enough of the eyeball/content
> providers are able to cooperate and peer with each other directly, you might
> see a significant impact (reduction in need) on transit providers as their 
> entire
> business would become largely irrelevant.

This will work in a single market.

I've thought about this before too - when you start to cross nations or
continents, transit providers became a necessity; the eyeball networks
are typically not geared up to handle international or trans-continental
communications on their own.

The solution would be content providers deploying in each country to
remove the need for transit, but they still have to feed those clusters
somehow.

Ultimately, the big content players build and run their own networks,
completely bypassing the transit providers and peering with the eyeball
(and all) networks wherever they pitch tent. As it were, not all of them
have this muscle.

Mark.

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Mark Tinka]

On 15/Aug/15 19:32, jim deleskie wrote:

> In my 20+ yrs now of playing this game, "everyone" has had a turn thinking
> their content/eyeballs are special and should get free "peering".

That's why those tired of playing the game build their own networks to
take out the middleman, for better or worse.

Mark.

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Owen DeLong]

Let me turn that on its head…

I don’t think anyone’s eyeballs are special.
I don’t think anyone’s content is special.

I think everyone should get free peering with any network whose customers 
expect to be able to reach that other network’s customers.

Ignoring for a moment the idea of maximizing effective avarice, think how
much better it would be for eyeballs and content providers alike if they
could all just peer directly settlement free and/or pay a single layer of
transit providers all of whom peered with each other
for free.

Time and time again we have repeatedly proven that increased interconnect
density and promiscuous settlement free peering reduce
costs, improve performance, and generally make the internet better for
all concerned.

Now, ask yourself… If everyone followed that model, would it actually reduce
the viability of any of the businesses in question?

IMHO, there’s only one yes answer here… If enough of the eyeball/content
providers are able to cooperate and peer with each other directly, you might
see a significant impact (reduction in need) on transit providers as their 
entire
business would become largely irrelevant.

That’s called cutting out the middle man. In almost every industry that has been
able to do so, it’s been considered a really good thing for everyone except the
middle man who rarely gets much sympathy.

Owen

> On Aug 15, 2015, at 10:32 , jim deleskie  wrote:
> 
> In my 20+ yrs now of playing this game, "everyone" has had a turn thinking
> their content/eyeballs are special and should get free "peering".
> 
> On Sat, Aug 15, 2015 at 1:59 PM, Mike Hammett  wrote:
> 
>> Arrogance is the only reason I can think of why the incumbents think that
>> way. I'd be surprised if any competitive providers (regardless of their
>> market dominance) would expect free peering.
>> 
>> 
>> 
>> 
>> -
>> Mike Hammett
>> Intelligent Computing Solutions
>> http://www.ics-il.com
>> 
>> 
>> 
>> Midwest Internet Exchange
>> http://www.midwest-ix.com
>> 
>> 
>> - Original Message -
>> 
>> From: "Owen DeLong" 
>> To: "Matthew Huff" 
>> Cc: nanog@nanog.org
>> Sent: Saturday, August 15, 2015 11:44:57 AM
>> Subject: Re: net neutrality peering dispute between CenturyTel/Qwest and
>> Cogent in Dallas
>> 
>> This issue isn’t limited to Cogent.
>> 
>> There is this bizarre belief by the larger eyeball networks (and CC, VZ,
>> and TW are the worst offenders, pretty much in that order) that they are
>> entitled to be paid by both the content provider _AND_ the eyeball user for
>> carrying bits between the two.
>> 
>> In a healthy market, the eyeball providers would face competition and the
>> content providers would simply ignore these demands and the eyeballs would
>> buy from other eyeball providers.
>> 
>> Unfortunately, especially in the US, we don’t have a healthy market. In
>> the best of circumstances, we have oligopolies and in the worst places, we
>> have effective (or even actual) monopolies.
>> 
>> For example, in the area where I live, the claim you will hear is that
>> there is competition. With my usage patterns, that’s a choice between
>> Comcast (up to 30/7 $100/mo), AT&T DSL (1.5M/384k $40/mo+) and wireless (Up
>> to 30/15 $500+/month).
>> 
>> I’m not in some rural backwater or even some second-tier metro. I’m within
>> 10 miles of the former MAE West and also within 10 miles of Equinix SV1 (11
>> Great Oaks). There’s major fiber bundles within 2 miles of my house. I’m
>> near US101 and Capitol Expressway in San Jose.
>> 
>> The reason that things are this way, IMHO, is because we have allowed
>> “facilities based carriers” to leverage the monopoly on physical
>> infrastructure into a monopoly for services over that infrastructure.
>> 
>> The most viable solution, IMHO, is to require a separation between
>> physical infrastructure providers and those that provide services over that
>> infrastructure. Breaking the tight coupling between the two and requiring
>> physical infrastructure providers to lease facilities to operators on an
>> equal footing for all operators will reduce the barriers to competition in
>> the operator space. It will also make limited competition in the facilities
>> space possible, though unlikely.
>> 
>> This model exists to some extent in a few areas that have municipal
>> residential fiber services, and in most of those localities, it is working
>> well.
>> 
>> That’s one of the reasons that the incumbent facilities based carriers
>> have lobbied so hard to get laws in states where a city has done this that
>> prevent other cities from following suit.
>> 
>> Fortunately, one of the big gains in recent FCC rulings is that these laws
>> are likely to be rendered null and void.
>> 
>> Unfortunately, there is so much vested interest in the status quo that
>> achieving this sort of separation is unlikely without a really strong grass
>> roots movement. Sadly, the average sound-bite oriented citizen doesn’t know
>> (or want to learn) enough to facilitat

Re: Drops in Core

[Sean Donelan]

On Sat, 15 Aug 2015, Glen Kent wrote:

bets are off on whether it will get dropped or not. However, the key point
is that the core usually does not drop too many packets - the probability
of drops are highest in the access side.

Is this correct?


1. TCP (and most other IP protocols) depends on, and forces packet 
congestion and drops.  Packet drops alone are not necessarily a 
measure of network quality.  Other than some laboratory conditions,

there must always be some congestion somewhere.

2. Packet queuing and drops are most likely at network transition points. 
Usually speed or latency transition points, but also network 
administration transition points.


3. Packet queuing and drops are less likely between network transition 
points, i.e. across the same network (LAN, WAN, ISP, etc).


That's why some ISPs claim they have 0% packet loss on their network. They 
don't include network transition points in their statistics; but have 
worse end-to-end performance than another network which includes 0.1%

packet drops in their reported statistics.

Generally I don't believe ISPs that claim 100% uptime or 0% packet loss.

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Matthew Petach]

I dunno, Jim, that sounds almost like you might
think the inevitable outcome will be an "everyone
pays" model of settlements, the way telcos do
it.  Unfortunately, in that model, the only winners
are the transit networks in the middle, because
no accounting department is going to want to
keep track of settlements for 4,000 other ASNs
that you peer with; their demand will be "reduce
the number of invoices, aggregate through 2 or
3 providers so we only have a small number of
invoices to reconcile."
I can see where you're coming from, but I'm not
sure I like the destination.  :(

Matt


On Sat, Aug 15, 2015 at 10:32 AM, jim deleskie  wrote:
> In my 20+ yrs now of playing this game, "everyone" has had a turn thinking
> their content/eyeballs are special and should get free "peering".
>
> On Sat, Aug 15, 2015 at 1:59 PM, Mike Hammett  wrote:
>
>> Arrogance is the only reason I can think of why the incumbents think that
>> way. I'd be surprised if any competitive providers (regardless of their
>> market dominance) would expect free peering.
>>
>>
>>
>>
>> -
>> Mike Hammett
>> Intelligent Computing Solutions
>> http://www.ics-il.com
>>
>>
>>
>> Midwest Internet Exchange
>> http://www.midwest-ix.com
>>
>>
>> - Original Message -
>>
>> From: "Owen DeLong" 
>> To: "Matthew Huff" 
>> Cc: nanog@nanog.org
>> Sent: Saturday, August 15, 2015 11:44:57 AM
>> Subject: Re: net neutrality peering dispute between CenturyTel/Qwest and
>> Cogent in Dallas
>>
>> This issue isn’t limited to Cogent.
>>
>> There is this bizarre belief by the larger eyeball networks (and CC, VZ,
>> and TW are the worst offenders, pretty much in that order) that they are
>> entitled to be paid by both the content provider _AND_ the eyeball user for
>> carrying bits between the two.
>>
>> In a healthy market, the eyeball providers would face competition and the
>> content providers would simply ignore these demands and the eyeballs would
>> buy from other eyeball providers.
>>
>> Unfortunately, especially in the US, we don’t have a healthy market. In
>> the best of circumstances, we have oligopolies and in the worst places, we
>> have effective (or even actual) monopolies.
>>
>> For example, in the area where I live, the claim you will hear is that
>> there is competition. With my usage patterns, that’s a choice between
>> Comcast (up to 30/7 $100/mo), AT&T DSL (1.5M/384k $40/mo+) and wireless (Up
>> to 30/15 $500+/month).
>>
>> I’m not in some rural backwater or even some second-tier metro. I’m within
>> 10 miles of the former MAE West and also within 10 miles of Equinix SV1 (11
>> Great Oaks). There’s major fiber bundles within 2 miles of my house. I’m
>> near US101 and Capitol Expressway in San Jose.
>>
>> The reason that things are this way, IMHO, is because we have allowed
>> “facilities based carriers” to leverage the monopoly on physical
>> infrastructure into a monopoly for services over that infrastructure.
>>
>> The most viable solution, IMHO, is to require a separation between
>> physical infrastructure providers and those that provide services over that
>> infrastructure. Breaking the tight coupling between the two and requiring
>> physical infrastructure providers to lease facilities to operators on an
>> equal footing for all operators will reduce the barriers to competition in
>> the operator space. It will also make limited competition in the facilities
>> space possible, though unlikely.
>>
>> This model exists to some extent in a few areas that have municipal
>> residential fiber services, and in most of those localities, it is working
>> well.
>>
>> That’s one of the reasons that the incumbent facilities based carriers
>> have lobbied so hard to get laws in states where a city has done this that
>> prevent other cities from following suit.
>>
>> Fortunately, one of the big gains in recent FCC rulings is that these laws
>> are likely to be rendered null and void.
>>
>> Unfortunately, there is so much vested interest in the status quo that
>> achieving this sort of separation is unlikely without a really strong grass
>> roots movement. Sadly, the average sound-bite oriented citizen doesn’t know
>> (or want to learn) enough to facilitate such a grass-roots movement, so if
>> we want to build such a future, we have a long slog of public education and
>> recruitment ahead of us.
>>
>> In the mean time, we’ll get to continue to watch companies like CC, VZ, TW
>> screw over their customers and the content providers their customers want
>> to reach for the sake of extorting extra money from both sides of the
>> transaction.
>>
>> Owen
>>
>> > On Aug 15, 2015, at 06:40 , Matthew Huff  wrote:
>> >
>> > It's only partially about net neutrality. Cogent provides cheap
>> bandwidth for content providers, and sends a lot of traffic to eyeball
>> networks. In the past, peering partners expected symmetrical load sharing.
>> Cogent feels that eyeball networks should be happy to carry their traffic
>> since the customers want their services

Re: Drops in Core

[Matthew Petach]

Quite the inverse, I'd say; most of the capacity
headaches center around the handoff between
networks, and most of the congestion points
I come across are with private peering links
where one party or the other is unwilling or
unable to augment capacity.  The first and
last mile are fine, but the handoff between
the networks is where congestion and drops
occur.
As others have noted, this will vary greatly
depending on the network in question--so
asking a broad community like this is going
to yield a broad range of answers.  You
aren't going to find one single answer, you'll
find a probability curve that represents the
answers from many people running different
networks.
You'll find the location of packet drops tends
to shift depending on where companies are
willing to spend money; some companies
will spend money on the access layer to
ensure no drops happen there, but are
less willing to pay for capacity upgrades
at peering handoffs.  Other networks will
short-change their access, but maintain a
well-connected peering edge.

So--short answer is there is no one answer
to your question.  Collect the different answers,
plot the curve, and decide where along the
curve you want *your* network to land,
and build accordingly.  Nobody has infinite
money, so nobody builds to a level to ensure
zero loss probability to every destination around
the planet.

Matt



On Sat, Aug 15, 2015 at 9:47 AM, Glen Kent  wrote:
> Hi,
>
> Is it fair to say that most traffic drops happen in the access layers, or
> the first and the last miles, and the % of packet drops in the core are
> minimal? So, if the packet has made it past the first mile and has
> "entered" the core then chances are high that the packet will safely get
> across till the exit in the core. Sure once it gets off the core, then all
> bets are off on whether it will get dropped or not. However, the key point
> is that the core usually does not drop too many packets - the probability
> of drops are highest in the access side.
>
> Is this correct?
>
> Glen
>

Re: Drops in Core

[William Herrin]

On Sat, Aug 15, 2015 at 1:31 PM, Glen Kent  wrote:
> Is there a paper or a presentation that discusses the drops in the core?

Hi Glen,

Probably, but I don't know where to point you.


> If i were to break the total path into three legs -- the first, middle and
> the last, then are you saying that the probability of packet loss is perhaps
> 1/3 in each leg (because the packet passes through different IXes). That
> sounds too aggressive for the middle mile. Dont you think so?

Break it in to five legs:

1. Your immediate last mile
2. The set of networks you directly or indirectly pay to transmit and
receive packets
3. The border link between your networks and the remote user's networks
4. The set of networks the remote user directly or indirectly pays to
transmit and receive packets
5. The remote user's immediate last mile

In some cases, your packets meet on a network which both you and the
remote user pay for. In those cases, leg 3 does not exist. However,
those cases are less common than the one where neither of you pays the
same networks.

Legs 1 and 5 are often over noisy copper wire suspended from a street pole.
Leg 3 is routinely under-provisioned (too little bandwidth for the
traffic demand).
Legs 2 and 4 rarely exhibit loss for long.

Regards,
Bill Herrin


-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: Drops in Core

[Job Snijders]

On Sat, Aug 15, 2015 at 11:01:56PM +0530, Glen Kent wrote:
> Is there a paper or a presentation that discusses the drops in the core?
> 
> If i were to break the total path into three legs -- the first, middle
> and the last, then are you saying that the probability of packet loss
> is perhaps 1/3 in each leg (because the packet passes through
> different IXes).

It is unlikely packets pass through an IXP more then once.

Kind regards,

Job

Re: Drops in Core

[Rafael Possamai]

That was just an example, that list has to be completed on a specific
network or scenario, it changes dramatically. Imagine you were to create a
list for a DoD network instead of public peering based network, it would
change dramatically.



On Sat, Aug 15, 2015 at 12:28 PM, Glen Kent  wrote:

> Why do you say that Layer 1 issues in the last mile would be very high?
> How is it any different from the first mile?
>
> On Sat, Aug 15, 2015 at 10:56 PM, Rafael Possamai 
> wrote:
>
>> Hi Glen,
>>
>> If you first list the causes of a dropped packet, then you can figure out
>> how likely they are at different points in time (first\last\peer\etc) by
>> making some assumptions.
>>
>> Here's an **example**:
>>
>> *Cause | Location | Likelihood*
>> Congestion | Last mile | Low
>> Congestion | First mile | Low
>> Congestion | Peering | Medium
>> Layer 1 | First mile | Low
>> Layer 1 | Core | Low
>> Layer 1 | Last mile | High
>>
>> You can even go as far as drawing a cause and effect diagram for each
>> location. Then you can collect real world data and fine tune your
>> assumptions.
>>
>>
>> Rafael
>>
>>
>> On Sat, Aug 15, 2015 at 11:47 AM, Glen Kent  wrote:
>>
>>> Hi,
>>>
>>> Is it fair to say that most traffic drops happen in the access layers, or
>>> the first and the last miles, and the % of packet drops in the core are
>>> minimal? So, if the packet has made it past the first mile and has
>>> "entered" the core then chances are high that the packet will safely get
>>> across till the exit in the core. Sure once it gets off the core, then
>>> all
>>> bets are off on whether it will get dropped or not. However, the key
>>> point
>>> is that the core usually does not drop too many packets - the probability
>>> of drops are highest in the access side.
>>>
>>> Is this correct?
>>>
>>> Glen
>>>
>>
>>
>

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[jim deleskie]

In my 20+ yrs now of playing this game, "everyone" has had a turn thinking
their content/eyeballs are special and should get free "peering".

On Sat, Aug 15, 2015 at 1:59 PM, Mike Hammett  wrote:

> Arrogance is the only reason I can think of why the incumbents think that
> way. I'd be surprised if any competitive providers (regardless of their
> market dominance) would expect free peering.
>
>
>
>
> -
> Mike Hammett
> Intelligent Computing Solutions
> http://www.ics-il.com
>
>
>
> Midwest Internet Exchange
> http://www.midwest-ix.com
>
>
> - Original Message -
>
> From: "Owen DeLong" 
> To: "Matthew Huff" 
> Cc: nanog@nanog.org
> Sent: Saturday, August 15, 2015 11:44:57 AM
> Subject: Re: net neutrality peering dispute between CenturyTel/Qwest and
> Cogent in Dallas
>
> This issue isn’t limited to Cogent.
>
> There is this bizarre belief by the larger eyeball networks (and CC, VZ,
> and TW are the worst offenders, pretty much in that order) that they are
> entitled to be paid by both the content provider _AND_ the eyeball user for
> carrying bits between the two.
>
> In a healthy market, the eyeball providers would face competition and the
> content providers would simply ignore these demands and the eyeballs would
> buy from other eyeball providers.
>
> Unfortunately, especially in the US, we don’t have a healthy market. In
> the best of circumstances, we have oligopolies and in the worst places, we
> have effective (or even actual) monopolies.
>
> For example, in the area where I live, the claim you will hear is that
> there is competition. With my usage patterns, that’s a choice between
> Comcast (up to 30/7 $100/mo), AT&T DSL (1.5M/384k $40/mo+) and wireless (Up
> to 30/15 $500+/month).
>
> I’m not in some rural backwater or even some second-tier metro. I’m within
> 10 miles of the former MAE West and also within 10 miles of Equinix SV1 (11
> Great Oaks). There’s major fiber bundles within 2 miles of my house. I’m
> near US101 and Capitol Expressway in San Jose.
>
> The reason that things are this way, IMHO, is because we have allowed
> “facilities based carriers” to leverage the monopoly on physical
> infrastructure into a monopoly for services over that infrastructure.
>
> The most viable solution, IMHO, is to require a separation between
> physical infrastructure providers and those that provide services over that
> infrastructure. Breaking the tight coupling between the two and requiring
> physical infrastructure providers to lease facilities to operators on an
> equal footing for all operators will reduce the barriers to competition in
> the operator space. It will also make limited competition in the facilities
> space possible, though unlikely.
>
> This model exists to some extent in a few areas that have municipal
> residential fiber services, and in most of those localities, it is working
> well.
>
> That’s one of the reasons that the incumbent facilities based carriers
> have lobbied so hard to get laws in states where a city has done this that
> prevent other cities from following suit.
>
> Fortunately, one of the big gains in recent FCC rulings is that these laws
> are likely to be rendered null and void.
>
> Unfortunately, there is so much vested interest in the status quo that
> achieving this sort of separation is unlikely without a really strong grass
> roots movement. Sadly, the average sound-bite oriented citizen doesn’t know
> (or want to learn) enough to facilitate such a grass-roots movement, so if
> we want to build such a future, we have a long slog of public education and
> recruitment ahead of us.
>
> In the mean time, we’ll get to continue to watch companies like CC, VZ, TW
> screw over their customers and the content providers their customers want
> to reach for the sake of extorting extra money from both sides of the
> transaction.
>
> Owen
>
> > On Aug 15, 2015, at 06:40 , Matthew Huff  wrote:
> >
> > It's only partially about net neutrality. Cogent provides cheap
> bandwidth for content providers, and sends a lot of traffic to eyeball
> networks. In the past, peering partners expected symmetrical load sharing.
> Cogent feels that eyeball networks should be happy to carry their traffic
> since the customers want their services, the eyeball networks want Cogent
> to pay them extra. When there is congestion, neither side wants to upgrade
> their peeing until this is resolved, so they haven't. This has been going
> on for at least 5 years, and happens all over the cogent peering map.
> >
> > Depending on what protocol you are using, it can be an issue or not. Our
> end users on eyeball networks had difficulty maintaining VPN connections.
> We had to drop our Cogent upstream and work with our remaining upstream
> provides to traffic engineer around Cogent. YMMV.
> >
> >
> >
> > 
> > Matthew Huff | 1 Manhattanville Rd
> > Director of Operations | Purchase, NY 10577
> > OTA Management LLC | Phone: 914-460-4039
> > aim: matthewbhuff | Fax: 914-694-5669
> >
> > 

Re: Drops in Core

[Glen Kent]

Is there a paper or a presentation that discusses the drops in the core?

If i were to break the total path into three legs -- the first, middle and
the last, then are you saying that the probability of packet loss is
perhaps 1/3 in each leg (because the packet passes through different IXes).
That sounds too aggressive for the middle mile. Dont you think so?

On Sat, Aug 15, 2015 at 10:51 PM, Owen DeLong  wrote:

> I would say that the probability of a packet drop at any particular peering
> point is less than the probability at one of the two edges.
>
> However, given that most packets are likely to traverse multiple peering
> points between the two edges, the probability of a packet drop along
> the way at one of the several peering points overall is roughly equal
> to the probability of a drop at one of the two edges.
>
> YMMV.
>
> Owen
>
> > On Aug 15, 2015, at 10:07 , Glen Kent  wrote:
> >
> > Hi Bill,
> >
> > Just making sure that i get your point:
> >
> > Youre saying that the probability of packet drop at peering points would
> > roughly match that at the edge. Is it? I thought that most core switches
> > have minimal buffering and really do cut-through forwarding. The idea is
> > that the traffic that they receive is already shaped by the upstream
> > routers.
> >
> > Glen
> >
> >
> >
> > On Sat, Aug 15, 2015 at 10:33 PM, William Herrin  wrote:
> >
> >> On Sat, Aug 15, 2015 at 12:47 PM, Glen Kent 
> wrote:
> >>> Is it fair to say that most traffic drops happen in the access layers,
> or
> >>> the first and the last miles, and the % of packet drops in the core are
> >>> minimal? So, if the packet has made it past the first mile and has
> >>> "entered" the core then chances are high that the packet will safely
> get
> >>> across till the exit in the core.
> >>
> >> Hi Glen,
> >>
> >> I would expect congestion loss at enough peering points (center of the
> >> core) to put it in the same league as noisy cable at the edge.
> >>
> >> Regards,
> >> Bill Herrin
> >>
> >>
> >>
> >> --
> >> William Herrin  her...@dirtside.com  b...@herrin.us
> >> Owner, Dirtside Systems . Web: 
> >>
>
>

Re: Drops in Core

[William Herrin]

On Sat, Aug 15, 2015 at 1:21 PM, Owen DeLong  wrote:
> I would say that the probability of a packet drop at any particular peering
> point is less than the probability at one of the two edges.
>
> However, given that most packets are likely to traverse multiple peering
> points between the two edges, the probability of a packet drop along
> the way at one of the several peering points overall is roughly equal
> to the probability of a drop at one of the two edges.

Hi Owen,

Generally speaking there are zero or one settlement free peering
points in the active path between any two edges. Not always, but close
enough to it to discount the exceptions.

Speaking for my own experience, I almost never see loss on my Verizon
FiOS edge but see loss at the various Verizon borders with other
networks -all the time-.  They keep pitching me on upgrading to 75/75
but that isn't the upgrade I need.

Regards,
Bill Herrin


-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: Drops in Core

[Rafael Possamai]

Hi Glen,

If you first list the causes of a dropped packet, then you can figure out
how likely they are at different points in time (first\last\peer\etc) by
making some assumptions.

Here's an **example**:

*Cause | Location | Likelihood*
Congestion | Last mile | Low
Congestion | First mile | Low
Congestion | Peering | Medium
Layer 1 | First mile | Low
Layer 1 | Core | Low
Layer 1 | Last mile | High

You can even go as far as drawing a cause and effect diagram for each
location. Then you can collect real world data and fine tune your
assumptions.


Rafael


On Sat, Aug 15, 2015 at 11:47 AM, Glen Kent  wrote:

> Hi,
>
> Is it fair to say that most traffic drops happen in the access layers, or
> the first and the last miles, and the % of packet drops in the core are
> minimal? So, if the packet has made it past the first mile and has
> "entered" the core then chances are high that the packet will safely get
> across till the exit in the core. Sure once it gets off the core, then all
> bets are off on whether it will get dropped or not. However, the key point
> is that the core usually does not drop too many packets - the probability
> of drops are highest in the access side.
>
> Is this correct?
>
> Glen
>

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Mike Hammett]

I think we're on the same side, just saying it differently substituting greed 
for arrogance. 

Additionally, the last mile providers are acting no differently than a carrier 
would, getting paid on both sides... only carriers are typically balanced 
ratios where as last mile\first mile are not. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Owen DeLong"  
To: "Mike Hammett"  
Cc: nanog@nanog.org 
Sent: Saturday, August 15, 2015 12:18:04 PM 
Subject: Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent 
in Dallas 

Your reply implies that your understanding does not match my intended meaning. 

(IOW, Perhaps you did not receive what I intended to transmit) 

I’m saying that the incumbents in an act of unreasonable greed are demanding 
money for peering from providers with a lot of content providers while also 
collecting money from their direct customers for the sake of delivering that 
same content. 

It would be like me standing between you and a hotdog stand and demanding that 
you give me 1.5x the price of the hotdog and then demanding that the hotdog 
stand sell me the hotdog to give to you for 0.5x the listed price. 

In the more functional physical world, you simply walk around me and buy the 
hotdog for 1x the listed price and the only one who loses is the guy standing 
in the middle. 

In the case of the incumbent facilities based carriers, they’ve managed to 
build a wall in front of the hot dog stand and a wall in front of you such that 
your view is limited to the window that they have to open and so is the hot dog 
vendor. Thus, you have no choice but to give them the extra 50% for the hot dog 
and the hot dog vendor has no choice but to give them half of the listed price 
as a “delivery charge”. 

Admittedly, the fractions are not as I described, but the basic principle is 
exactly as I have described it. 

Owen 

> On Aug 15, 2015, at 09:59 , Mike Hammett  wrote: 
> 
> Arrogance is the only reason I can think of why the incumbents think that 
> way. I'd be surprised if any competitive providers (regardless of their 
> market dominance) would expect free peering. 
> 
> 
> 
> 
> - 
> Mike Hammett 
> Intelligent Computing Solutions 
> http://www.ics-il.com 
> 
> 
> 
> Midwest Internet Exchange 
> http://www.midwest-ix.com 
> 
> 
> - Original Message - 
> 
> From: "Owen DeLong"  
> To: "Matthew Huff"  
> Cc: nanog@nanog.org 
> Sent: Saturday, August 15, 2015 11:44:57 AM 
> Subject: Re: net neutrality peering dispute between CenturyTel/Qwest and 
> Cogent in Dallas 
> 
> This issue isn’t limited to Cogent. 
> 
> There is this bizarre belief by the larger eyeball networks (and CC, VZ, and 
> TW are the worst offenders, pretty much in that order) that they are entitled 
> to be paid by both the content provider _AND_ the eyeball user for carrying 
> bits between the two. 
> 
> In a healthy market, the eyeball providers would face competition and the 
> content providers would simply ignore these demands and the eyeballs would 
> buy from other eyeball providers. 
> 
> Unfortunately, especially in the US, we don’t have a healthy market. In the 
> best of circumstances, we have oligopolies and in the worst places, we have 
> effective (or even actual) monopolies. 
> 
> For example, in the area where I live, the claim you will hear is that there 
> is competition. With my usage patterns, that’s a choice between Comcast (up 
> to 30/7 $100/mo), AT&T DSL (1.5M/384k $40/mo+) and wireless (Up to 30/15 
> $500+/month). 
> 
> I’m not in some rural backwater or even some second-tier metro. I’m within 10 
> miles of the former MAE West and also within 10 miles of Equinix SV1 (11 
> Great Oaks). There’s major fiber bundles within 2 miles of my house. I’m near 
> US101 and Capitol Expressway in San Jose. 
> 
> The reason that things are this way, IMHO, is because we have allowed 
> “facilities based carriers” to leverage the monopoly on physical 
> infrastructure into a monopoly for services over that infrastructure. 
> 
> The most viable solution, IMHO, is to require a separation between physical 
> infrastructure providers and those that provide services over that 
> infrastructure. Breaking the tight coupling between the two and requiring 
> physical infrastructure providers to lease facilities to operators on an 
> equal footing for all operators will reduce the barriers to competition in 
> the operator space. It will also make limited competition in the facilities 
> space possible, though unlikely. 
> 
> This model exists to some extent in a few areas that have municipal 
> residential fiber services, and in most of those localities, it is working 
> well. 
> 
> That’s one of the reasons that the incumbent facilities based carriers have 
> lobbied so hard to get laws in states where a city has done this that prevent 
> other cities from following

Re: Drops in Core

[Owen DeLong]

I would say that the probability of a packet drop at any particular peering
point is less than the probability at one of the two edges.

However, given that most packets are likely to traverse multiple peering
points between the two edges, the probability of a packet drop along
the way at one of the several peering points overall is roughly equal
to the probability of a drop at one of the two edges.

YMMV.

Owen

> On Aug 15, 2015, at 10:07 , Glen Kent  wrote:
> 
> Hi Bill,
> 
> Just making sure that i get your point:
> 
> Youre saying that the probability of packet drop at peering points would
> roughly match that at the edge. Is it? I thought that most core switches
> have minimal buffering and really do cut-through forwarding. The idea is
> that the traffic that they receive is already shaped by the upstream
> routers.
> 
> Glen
> 
> 
> 
> On Sat, Aug 15, 2015 at 10:33 PM, William Herrin  wrote:
> 
>> On Sat, Aug 15, 2015 at 12:47 PM, Glen Kent  wrote:
>>> Is it fair to say that most traffic drops happen in the access layers, or
>>> the first and the last miles, and the % of packet drops in the core are
>>> minimal? So, if the packet has made it past the first mile and has
>>> "entered" the core then chances are high that the packet will safely get
>>> across till the exit in the core.
>> 
>> Hi Glen,
>> 
>> I would expect congestion loss at enough peering points (center of the
>> core) to put it in the same league as noisy cable at the edge.
>> 
>> Regards,
>> Bill Herrin
>> 
>> 
>> 
>> --
>> William Herrin  her...@dirtside.com  b...@herrin.us
>> Owner, Dirtside Systems . Web: 
>> 

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Owen DeLong]

Your reply implies that your understanding does not match my intended meaning.

(IOW, Perhaps you did not receive what I intended to transmit)

I’m saying that the incumbents in an act of unreasonable greed are demanding 
money for peering from providers with a lot of content providers while also 
collecting money from their direct customers for the sake of delivering that 
same content.

It would be like me standing between you and a hotdog stand and demanding that 
you give me 1.5x the price of the hotdog and then demanding that the hotdog 
stand sell me the hotdog to give to you for 0.5x the listed price.

In the more functional physical world, you simply walk around me and buy the 
hotdog for 1x the listed price and the only one who loses is the guy standing 
in the middle.

In the case of the incumbent facilities based carriers, they’ve managed to 
build a wall in front of the hot dog stand and a wall in front of you such that 
your view is limited to the window that they have to open and so is the hot dog 
vendor. Thus, you have no choice but to give them the extra 50% for the hot dog 
and the hot dog vendor has no choice but to give them half of the listed price 
as a “delivery charge”.

Admittedly, the fractions are not as I described, but the basic principle is 
exactly as I have described it.

Owen

> On Aug 15, 2015, at 09:59 , Mike Hammett  wrote:
> 
> Arrogance is the only reason I can think of why the incumbents think that 
> way. I'd be surprised if any competitive providers (regardless of their 
> market dominance) would expect free peering. 
> 
> 
> 
> 
> - 
> Mike Hammett 
> Intelligent Computing Solutions 
> http://www.ics-il.com 
> 
> 
> 
> Midwest Internet Exchange 
> http://www.midwest-ix.com 
> 
> 
> - Original Message -
> 
> From: "Owen DeLong"  
> To: "Matthew Huff"  
> Cc: nanog@nanog.org 
> Sent: Saturday, August 15, 2015 11:44:57 AM 
> Subject: Re: net neutrality peering dispute between CenturyTel/Qwest and 
> Cogent in Dallas 
> 
> This issue isn’t limited to Cogent. 
> 
> There is this bizarre belief by the larger eyeball networks (and CC, VZ, and 
> TW are the worst offenders, pretty much in that order) that they are entitled 
> to be paid by both the content provider _AND_ the eyeball user for carrying 
> bits between the two. 
> 
> In a healthy market, the eyeball providers would face competition and the 
> content providers would simply ignore these demands and the eyeballs would 
> buy from other eyeball providers. 
> 
> Unfortunately, especially in the US, we don’t have a healthy market. In the 
> best of circumstances, we have oligopolies and in the worst places, we have 
> effective (or even actual) monopolies. 
> 
> For example, in the area where I live, the claim you will hear is that there 
> is competition. With my usage patterns, that’s a choice between Comcast (up 
> to 30/7 $100/mo), AT&T DSL (1.5M/384k $40/mo+) and wireless (Up to 30/15 
> $500+/month). 
> 
> I’m not in some rural backwater or even some second-tier metro. I’m within 10 
> miles of the former MAE West and also within 10 miles of Equinix SV1 (11 
> Great Oaks). There’s major fiber bundles within 2 miles of my house. I’m near 
> US101 and Capitol Expressway in San Jose. 
> 
> The reason that things are this way, IMHO, is because we have allowed 
> “facilities based carriers” to leverage the monopoly on physical 
> infrastructure into a monopoly for services over that infrastructure. 
> 
> The most viable solution, IMHO, is to require a separation between physical 
> infrastructure providers and those that provide services over that 
> infrastructure. Breaking the tight coupling between the two and requiring 
> physical infrastructure providers to lease facilities to operators on an 
> equal footing for all operators will reduce the barriers to competition in 
> the operator space. It will also make limited competition in the facilities 
> space possible, though unlikely. 
> 
> This model exists to some extent in a few areas that have municipal 
> residential fiber services, and in most of those localities, it is working 
> well. 
> 
> That’s one of the reasons that the incumbent facilities based carriers have 
> lobbied so hard to get laws in states where a city has done this that prevent 
> other cities from following suit. 
> 
> Fortunately, one of the big gains in recent FCC rulings is that these laws 
> are likely to be rendered null and void. 
> 
> Unfortunately, there is so much vested interest in the status quo that 
> achieving this sort of separation is unlikely without a really strong grass 
> roots movement. Sadly, the average sound-bite oriented citizen doesn’t know 
> (or want to learn) enough to facilitate such a grass-roots movement, so if we 
> want to build such a future, we have a long slog of public education and 
> recruitment ahead of us. 
> 
> In the mean time, we’ll get to continue to watch companies like CC, VZ, TW 
> screw over their customers and the content p

Re: Drops in Core

[William Herrin]

On Sat, Aug 15, 2015 at 1:07 PM, Glen Kent  wrote:
> Youre saying that the probability of packet drop at peering points would
> roughly match that at the edge. Is it? I thought that most core switches
> have minimal buffering and really do cut-through forwarding. The idea is
> that the traffic that they receive is already shaped by the upstream
> routers.

Hi Glen,

It a capacity question. Several core networks [cough Verizon cough]
intentionally under-provision the "settlement-free" peering links to
other core networks. You can't cut-through when the destination
interface already has a queue of packets waiting to be sent.

Regards,
Bill Herrin



-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: Data Center operations mail list?

[Martin Hannigan]

On Tue, Aug 11, 2015 at 3:27 PM, Simon Lockhart  wrote:

> On Tue Aug 11, 2015 at 01:35:28pm -0400, Jay Ashworth wrote:
> > Absolutely feel free to use it; I haven't seen a single message on it
> in...
> > well, it was 3 years ago I was in datacenters regularly, so I'm goin with
> > "3 years".  :-)
>
> There's a message there now... :)
>
> Rather than fragmenting further, I'd suggest building up demand first on
> existing infrastructure. If it gets to the size of NANOG and needing a
> support organisation around it, then it can split off then...
>
>
>

A few of us have been operating and supporting a data center track at the
last 4 or 5 NANOG meetings. The sessions are well attended and the topics
are getting more interesting e.g. real estate, energy management and
interconnection focused.

There is reasonable demand for a forum.  It might need a little marketing
to get a list with traction going.

Best,

-M<

Re: Drops in Core

[Glen Kent]

Hi Bill,

Just making sure that i get your point:

Youre saying that the probability of packet drop at peering points would
roughly match that at the edge. Is it? I thought that most core switches
have minimal buffering and really do cut-through forwarding. The idea is
that the traffic that they receive is already shaped by the upstream
routers.

Glen



On Sat, Aug 15, 2015 at 10:33 PM, William Herrin  wrote:

> On Sat, Aug 15, 2015 at 12:47 PM, Glen Kent  wrote:
> > Is it fair to say that most traffic drops happen in the access layers, or
> > the first and the last miles, and the % of packet drops in the core are
> > minimal? So, if the packet has made it past the first mile and has
> > "entered" the core then chances are high that the packet will safely get
> > across till the exit in the core.
>
> Hi Glen,
>
> I would expect congestion loss at enough peering points (center of the
> core) to put it in the same league as noisy cable at the edge.
>
> Regards,
> Bill Herrin
>
>
>
> --
> William Herrin  her...@dirtside.com  b...@herrin.us
> Owner, Dirtside Systems . Web: 
>

Re: Drops in Core

[William Herrin]

On Sat, Aug 15, 2015 at 12:47 PM, Glen Kent  wrote:
> Is it fair to say that most traffic drops happen in the access layers, or
> the first and the last miles, and the % of packet drops in the core are
> minimal? So, if the packet has made it past the first mile and has
> "entered" the core then chances are high that the packet will safely get
> across till the exit in the core.

Hi Glen,

I would expect congestion loss at enough peering points (center of the
core) to put it in the same league as noisy cable at the edge.

Regards,
Bill Herrin



-- 
William Herrin  her...@dirtside.com  b...@herrin.us
Owner, Dirtside Systems . Web: 

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Mike Hammett]

Arrogance is the only reason I can think of why the incumbents think that way. 
I'd be surprised if any competitive providers (regardless of their market 
dominance) would expect free peering. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Owen DeLong"  
To: "Matthew Huff"  
Cc: nanog@nanog.org 
Sent: Saturday, August 15, 2015 11:44:57 AM 
Subject: Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent 
in Dallas 

This issue isn’t limited to Cogent. 

There is this bizarre belief by the larger eyeball networks (and CC, VZ, and TW 
are the worst offenders, pretty much in that order) that they are entitled to 
be paid by both the content provider _AND_ the eyeball user for carrying bits 
between the two. 

In a healthy market, the eyeball providers would face competition and the 
content providers would simply ignore these demands and the eyeballs would buy 
from other eyeball providers. 

Unfortunately, especially in the US, we don’t have a healthy market. In the 
best of circumstances, we have oligopolies and in the worst places, we have 
effective (or even actual) monopolies. 

For example, in the area where I live, the claim you will hear is that there is 
competition. With my usage patterns, that’s a choice between Comcast (up to 
30/7 $100/mo), AT&T DSL (1.5M/384k $40/mo+) and wireless (Up to 30/15 
$500+/month). 

I’m not in some rural backwater or even some second-tier metro. I’m within 10 
miles of the former MAE West and also within 10 miles of Equinix SV1 (11 Great 
Oaks). There’s major fiber bundles within 2 miles of my house. I’m near US101 
and Capitol Expressway in San Jose. 

The reason that things are this way, IMHO, is because we have allowed 
“facilities based carriers” to leverage the monopoly on physical infrastructure 
into a monopoly for services over that infrastructure. 

The most viable solution, IMHO, is to require a separation between physical 
infrastructure providers and those that provide services over that 
infrastructure. Breaking the tight coupling between the two and requiring 
physical infrastructure providers to lease facilities to operators on an equal 
footing for all operators will reduce the barriers to competition in the 
operator space. It will also make limited competition in the facilities space 
possible, though unlikely. 

This model exists to some extent in a few areas that have municipal residential 
fiber services, and in most of those localities, it is working well. 

That’s one of the reasons that the incumbent facilities based carriers have 
lobbied so hard to get laws in states where a city has done this that prevent 
other cities from following suit. 

Fortunately, one of the big gains in recent FCC rulings is that these laws are 
likely to be rendered null and void. 

Unfortunately, there is so much vested interest in the status quo that 
achieving this sort of separation is unlikely without a really strong grass 
roots movement. Sadly, the average sound-bite oriented citizen doesn’t know (or 
want to learn) enough to facilitate such a grass-roots movement, so if we want 
to build such a future, we have a long slog of public education and recruitment 
ahead of us. 

In the mean time, we’ll get to continue to watch companies like CC, VZ, TW 
screw over their customers and the content providers their customers want to 
reach for the sake of extorting extra money from both sides of the transaction. 

Owen 

> On Aug 15, 2015, at 06:40 , Matthew Huff  wrote: 
> 
> It's only partially about net neutrality. Cogent provides cheap bandwidth for 
> content providers, and sends a lot of traffic to eyeball networks. In the 
> past, peering partners expected symmetrical load sharing. Cogent feels that 
> eyeball networks should be happy to carry their traffic since the customers 
> want their services, the eyeball networks want Cogent to pay them extra. When 
> there is congestion, neither side wants to upgrade their peeing until this is 
> resolved, so they haven't. This has been going on for at least 5 years, and 
> happens all over the cogent peering map. 
> 
> Depending on what protocol you are using, it can be an issue or not. Our end 
> users on eyeball networks had difficulty maintaining VPN connections. We had 
> to drop our Cogent upstream and work with our remaining upstream provides to 
> traffic engineer around Cogent. YMMV. 
> 
> 
> 
>  
> Matthew Huff | 1 Manhattanville Rd 
> Director of Operations | Purchase, NY 10577 
> OTA Management LLC | Phone: 914-460-4039 
> aim: matthewbhuff | Fax: 914-694-5669 
> 
> -Original Message- 
> From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Jordan Hamilton 
> Sent: Friday, August 14, 2015 5:31 PM 
> To: nanog@nanog.org 
> Subject: net neutrality peering dispute between CenturyTel/Qwest and Cogent 
> in Dallas 
> 
> I have several customers that are

Re: Drops in Core

[Mike Hammett]

I'd guess first\last\peering. 




- 
Mike Hammett 
Intelligent Computing Solutions 
http://www.ics-il.com 



Midwest Internet Exchange 
http://www.midwest-ix.com 


- Original Message -

From: "Glen Kent"  
To: nanog@nanog.org 
Sent: Saturday, August 15, 2015 11:47:31 AM 
Subject: Drops in Core 

Hi, 

Is it fair to say that most traffic drops happen in the access layers, or 
the first and the last miles, and the % of packet drops in the core are 
minimal? So, if the packet has made it past the first mile and has 
"entered" the core then chances are high that the packet will safely get 
across till the exit in the core. Sure once it gets off the core, then all 
bets are off on whether it will get dropped or not. However, the key point 
is that the core usually does not drop too many packets - the probability 
of drops are highest in the access side. 

Is this correct? 

Glen 

Drops in Core

[Glen Kent]

Hi,

Is it fair to say that most traffic drops happen in the access layers, or
the first and the last miles, and the % of packet drops in the core are
minimal? So, if the packet has made it past the first mile and has
"entered" the core then chances are high that the packet will safely get
across till the exit in the core. Sure once it gets off the core, then all
bets are off on whether it will get dropped or not. However, the key point
is that the core usually does not drop too many packets - the probability
of drops are highest in the access side.

Is this correct?

Glen

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Owen DeLong]

This issue isn’t limited to Cogent.

There is this bizarre belief by the larger eyeball networks (and CC, VZ, and TW 
are the worst offenders, pretty much in that order) that they are entitled to 
be paid by both the content provider _AND_ the eyeball user for carrying bits 
between the two.

In a healthy market, the eyeball providers would face competition and the 
content providers would simply ignore these demands and the eyeballs would buy 
from other eyeball providers.

Unfortunately, especially in the US, we don’t have a healthy market. In the 
best of circumstances, we have oligopolies and in the worst places, we have 
effective (or even actual) monopolies.

For example, in the area where I live, the claim you will hear is that there is 
competition. With my usage patterns, that’s a choice  between Comcast (up to 
30/7 $100/mo), AT&T DSL (1.5M/384k $40/mo+) and wireless (Up to 30/15 
$500+/month).

I’m not in some rural backwater or even some second-tier metro. I’m within 10 
miles of the former MAE West and also within 10 miles of Equinix SV1 (11 Great 
Oaks). There’s major fiber bundles within 2 miles of my house. I’m near US101 
and Capitol Expressway in San Jose.

The reason that things are this way, IMHO, is because we have allowed 
“facilities based carriers” to leverage the monopoly on physical infrastructure 
into a monopoly for services over that infrastructure.

The most viable solution, IMHO, is to require a separation between physical 
infrastructure providers and those that provide services over that 
infrastructure. Breaking the tight coupling between the two and requiring 
physical infrastructure providers to lease facilities to operators on an equal 
footing for all operators will reduce the barriers to competition in the 
operator space. It will also make limited competition in the facilities space 
possible, though unlikely.

This model exists to some extent in a few areas that have municipal residential 
fiber services, and in most of those localities, it is working well.

That’s one of the reasons that the incumbent facilities based carriers have 
lobbied so hard to get laws in states where a city has done this that prevent 
other cities from following suit.

Fortunately, one of the big gains in recent FCC rulings is that these laws are 
likely to be rendered null and void.

Unfortunately, there is so much vested interest in the status quo that 
achieving this sort of separation is unlikely without a really strong grass 
roots movement. Sadly, the average sound-bite oriented citizen doesn’t know (or 
want to learn) enough to facilitate such a grass-roots movement, so if we want 
to build such a future, we have a long slog of public education and recruitment 
ahead of us.

In the mean time, we’ll get to continue to watch companies like CC, VZ, TW 
screw over their customers and the content providers their customers want to 
reach for the sake of extorting extra money from both sides of the transaction.

Owen

> On Aug 15, 2015, at 06:40 , Matthew Huff  wrote:
> 
> It's only partially about net neutrality. Cogent provides cheap bandwidth for 
> content providers, and sends a lot of traffic to eyeball networks. In the 
> past, peering partners expected symmetrical load sharing. Cogent feels that 
> eyeball networks should be happy to carry their traffic since the customers 
> want their services, the eyeball networks want Cogent to pay them extra. When 
> there is congestion, neither side wants to upgrade their peeing until this is 
> resolved, so they haven't. This has been going on for at least 5 years, and 
> happens all over the cogent peering map.
> 
> Depending on what protocol you are using, it can be an issue or not. Our end 
> users on eyeball networks had difficulty maintaining VPN connections. We had 
> to drop our Cogent upstream and work with our remaining upstream provides to 
> traffic engineer around Cogent. YMMV.
> 
> 
> 
> 
> Matthew Huff | 1 Manhattanville Rd
> Director of Operations   | Purchase, NY 10577
> OTA Management LLC   | Phone: 914-460-4039
> aim: matthewbhuff| Fax:   914-694-5669
> 
> -Original Message-
> From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Jordan Hamilton
> Sent: Friday, August 14, 2015 5:31 PM
> To: nanog@nanog.org
> Subject: net neutrality peering dispute between CenturyTel/Qwest and Cogent 
> in Dallas 
> 
> I have several customers that are having packet loss issues, the packet loss 
> appears to be associated with a Cogent router interface of 38.104.86.222.  My 
> upstream provider is telling me that the packet loss is being caused by a net 
> neutrality peering dispute between CenturyTel/Quest and Cogent in Dallas.  I 
> did some quick googling to see if I could come up with any articles or 
> something like that I could provide to my customers and did not see anything. 
>  Anyone know any details?
> 
> Thanks
> 
> Jordan Hamilton
> Senior Telecommunications Engineer
> 
> Empire District E

Re: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Stephen Satchell]

On 08/15/2015 06:40 AM, Matthew Huff wrote:

neither side wants to upgrade their peeing


Oh, the irony of this typo of "peering"...

RE: net neutrality peering dispute between CenturyTel/Qwest and Cogent in Dallas

[Matthew Huff]

It's only partially about net neutrality. Cogent provides cheap bandwidth for 
content providers, and sends a lot of traffic to eyeball networks. In the past, 
peering partners expected symmetrical load sharing. Cogent feels that eyeball 
networks should be happy to carry their traffic since the customers want their 
services, the eyeball networks want Cogent to pay them extra. When there is 
congestion, neither side wants to upgrade their peeing until this is resolved, 
so they haven't. This has been going on for at least 5 years, and happens all 
over the cogent peering map.

Depending on what protocol you are using, it can be an issue or not. Our end 
users on eyeball networks had difficulty maintaining VPN connections. We had to 
drop our Cogent upstream and work with our remaining upstream provides to 
traffic engineer around Cogent. YMMV.




Matthew Huff | 1 Manhattanville Rd
Director of Operations   | Purchase, NY 10577
OTA Management LLC   | Phone: 914-460-4039
aim: matthewbhuff    | Fax:   914-694-5669

-Original Message-
From: NANOG [mailto:nanog-boun...@nanog.org] On Behalf Of Jordan Hamilton
Sent: Friday, August 14, 2015 5:31 PM
To: nanog@nanog.org
Subject: net neutrality peering dispute between CenturyTel/Qwest and Cogent in 
Dallas 

I have several customers that are having packet loss issues, the packet loss 
appears to be associated with a Cogent router interface of 38.104.86.222.  My 
upstream provider is telling me that the packet loss is being caused by a net 
neutrality peering dispute between CenturyTel/Quest and Cogent in Dallas.  I 
did some quick googling to see if I could come up with any articles or 
something like that I could provide to my customers and did not see anything.  
Anyone know any details?

Thanks

Jordan Hamilton
Senior Telecommunications Engineer

Empire District Electric Co.
720 Schifferdecker
PO Box 127
Joplin, MO 64802

Ph:  417-625-4223
Cell:  417-388-3351


--
Note: To protect against computer viruses, e-mail programs may prevent sending 
or receiving certain types of file attachments.  Check your e-mail security 
settings to determine how attachments are handled.

--
This e-mail and any files transmitted with it are the property of THE EMPIRE 
DISTRICT ELECTRIC COMPANY, are confidential, and are intended solely for the 
use of the individual or entity to whom this email is addressed. If you are not 
one of the named recipients or otherwise have reason to believe that you have 
received this message in error, please delete this message immediately from 
your computer and contact the sender by telephone at (417)-625-5100.
Any other use, retention, dissemination, forwarding, printing or copying of 
this email is strictly prohibited.

Re: Experience on Wanguard for 'anti' DDOS solutions

[marcel.durega...@yahoo.fr]

One thing which is not so obvious is to reduce false positive.
This is hard when you have a mix of traffic profiles/patterns within 
your network, with customers in differents domains (scientists, 
financials, video addicted, torrent addicted, etc...) with different 
bandwidth.


a)
Does anybody tried to separate ip range by traffic profile to apply 
specific rule/profile per ip allocation?


puts all financials clients into range X/X and define rule Z
puts all scientists clients into range Y/Y and apply rule Q
etc

Does this help ?

b)
One other method could be to classify customers by their bandwidth.

profile 1. from 10-100M
profile 2. 100-500M
profile 3. 500M-1000M
profile 4. >1000M

Like this you do not mix big BW with small BW customer, and do not get 
alerted when client from profile 4 start to download at 1G.


Any experience ?

My guess is that solution b is better than a. Not so easy to classify 
traffic pattern per group of client.


Thank, best regards.
- Marcel



On 13.08.2015 06:42, Ramy Hashish wrote:

Hello Fabien,

And why don't you use A10 for both detection and mitigation?

Thanks,

Ramy

On Wed, Aug 12, 2015 at 6:42 PM, Fabien Delmotte  wrote:


Hello

My 2 cents
You can use Wanguard for the detection and A10 for the mitigation, you
have just to play with the API.

Regards

Fabien


Le 12 août 2015 à 16:28, Ramy Hashish  a écrit

:





Date: Tue, 11 Aug 2015 08:14:54 +0200
From: "marcel.durega...@yahoo.fr" 
To: nanog@nanog.org
Subject: Re: Experience on Wanguard for 'anti' DDOS solutions
Message-ID: <55c992de.3020...@yahoo.fr>
Content-Type: text/plain; charset=windows-1252; format=flowed

anybody from this impressive list ?:

https://www.andrisoft.com/company/customers

-- Marcel




Anybody here compared Wanguard's performance with the DDoS vendors in the
market (Arbor, Radware, NSFocus, A10, RioRey, Staminus, F5 ..)?

Another question, have anybody from the reviewers tested the false
positives of the box, or experienced any false positive incidents?

Thanks,

Ramy